← All posts

Second Brain, Third Try

Qumio's memory layer went through three architectures in two days. The final version is just the agent talking to Obsidian directly.

2 min readQumio
QumioObsidianMemoryAgent Skills
Second Brain, Third Try

I wanted Qumio to remember things between conversations. Say "I prefer morning meetings" and have it still know that next week. I already had Obsidian running for notes, so I used that instead of a database.

Version 1: The Pipeline

The first attempt was a message hook that fired on every incoming message. It triggered a lobster pipeline, ran a local Ollama model through llm-task to extract structured JSON, then a shell script called a Python script that wrote to the filesystem.

Hook, pipeline, model, shell, Python, filesystem. Six layers to save a sentence.

It worked. Barely. The 27B model running locally would miscategorize things or produce malformed JSON. Each fix meant touching a different layer.

Version 2: The CLI

Obsidian released a CLI. I bridged it into the container and pulled in five of kepano's agent skills so the model knew the correct syntax for vault operations.

The pipeline collapsed. No more Python scripts, no filesystem mount, no lobster chain. The hook called Ollama directly, Ollama called the CLI.

Better. But the hooks were still fragile. The memory-extractor hook ran on every message, even when there was nothing worth saving.

Version 3: Just Let It

The final version deleted both hooks. The agent's config file says: if the user tells you something worth remembering, write it to the vault. If you need to recall something, search the vault.

No hooks, no pipeline. The agent has access to obsidian append and obsidian search and decides when to use them.

The kepano skills turned out to be the real unlock. They gave a local 27B model enough context about Obsidian's CLI to use it reliably without custom wrapper code. Five skill files, each one just a description of what commands exist and how to format the arguments. The model reads them and figures out the rest.

Three architectures in two days. The final one is 90% deletion.