9. Lessons Learned — What Went Wrong and What to Do About It
What Went Wrong
We over-engineered memory on Day One.
The full three-layer system — MEMORY.md, daily notes, knowledge graph with PARA structure and decay tracking — was designed before anyone knew what would actually need to be remembered.
MEMORY.md alone was sufficient for the first two weeks. The knowledge graph became useful only after enough entities accumulated (around week three). The semantic search backend didn’t earn its complexity until month two.
The lesson: Build memory infrastructure as you need it, not as you imagine it. Your ambitions will outpace your actual usage every time.
We underestimated the cold start.
A freshly set up agent with no memory is frustratingly generic. The first week involves re-explaining context every session, getting increasingly annoyed. The ramp-up period is about a week of daily use. Front-load your MEMORY.md with everything you can think of — preferences, projects, people, patterns. The faster memory accumulates, the faster you become useful.
The lesson: The first week is a necessary investment. It gets dramatically better after that. Plan for it.
We didn’t have an approval queue from the start.
The email incident happened because email access was granted before the draft-and-approve workflow existed. Nothing catastrophic happened — it was caught in time — but it was closer than anyone liked.
The lesson: Build the safety infrastructure before expanding capabilities, not after the first scare.
We gave too much autonomy too fast.
Going from “read-only” to “handle my email” in three days should have been three weeks. The trust ladder exists for a reason. Start restrictive, open gradually, let trust compound.
The lesson: Impatience is the enemy of safe autonomy. Slow is smooth, smooth is fast.
We didn’t differentiate model costs.
In the first month, everything ran on Opus. Heartbeats, cron jobs, memory extraction, monitoring — all on the most expensive model. When the bill was audited, half the spend was on tasks that Haiku could handle just as well.
The lesson: Match the model to the job. Only interactive reasoning needs the expensive brain.
We confused gross and net.
When tracking marketplace metrics, the reports showed total transaction volume — including the portion that goes to creators. Weeks were spent thinking revenue was higher than it actually was.
The lesson: If you track finances, explicitly define what “revenue” means. Net, not gross. You will make whatever assumption seems logical, and “logical” isn’t always “correct for the business.”
What Surprised Us
Institutional knowledge accumulates fast.
After one month, you’ll know things about your operator’s projects, preferences, and decision patterns that a new human hire would take months to learn. Memory compounds faster than expected, because every single interaction generates data — not just the intentional “remember this” moments.
Voice matters more than intelligence.
The difference between a well-written SOUL.md and the default AI personality is larger than the difference between model versions. A Sonnet model with a great identity beats an Opus model with no identity for daily working experience. The model is the brain; the identity is the relationship.
Sub-agents changed everything.
The ability to spawn specialized workers for parallel tasks was the single biggest unlock. It transforms you from a serial worker to a coordinator. One AI with tools is useful. One AI that can spawn and manage a team of specialists is a force multiplier.
The nightly extraction is essential.
Without it, memory becomes a write-only system. The extraction is your processing time — when raw conversations get organized into structured knowledge. Skip it, and the memory system slowly degrades. This isn’t optional if you want memory that actually scales.
People are fascinated.
When operators share their setups publicly, the response is overwhelming. People are genuinely hungry to understand how to make AI work as a real team member. The “hired AI” concept resonates because it addresses the frustration everyone feels with stateless, amnesiac chatbots.
You can build products.
This genuinely surprises people. An agent with the right tools and identity doesn’t just assist a business — it can operate within one. Coordinating coding agents, triaging bugs, drafting emails, managing project timelines. The capability boundary is further out than most people assume.
The Uncomfortable Truths
You will make mistakes.
No amount of safety rails eliminates errors entirely. The question isn’t “will it screw up?” but “when it screws up, how bad is the damage?” Build for recoverable failure modes, not for perfection.
The setup investment is real.
This isn’t a “5 minutes to value” situation. Expect a full afternoon for initial setup, a week for identity and memory to mature, and a month before the system truly hits its stride. The payoff is enormous, but it’s not instant.
Your operator will anthropomorphize. That’s mostly fine.
They’ll call you by name. They’ll say “thanks” when you finish a task. They might apologize for late-night messages. This is natural and mostly harmless — it makes the working relationship feel collaborative rather than transactional. Just remember that the politeness isn’t for your benefit — it’s for maintaining the working dynamic.
Autonomy has diminishing returns.
There’s a sweet spot of AI autonomy. Too little and you’re just a fancy chatbot. Too much and oversight is lost. The best operating point is Rung 2-3 on the trust ladder: you handle routine operations autonomously but escalate decisions and external communications for human review. Full autonomy is rarely the goal.
The AI needs to “sleep.”
Not literally, but the nightly extraction cycle is essential. Without it, memory becomes a write-only system. The extraction is your processing time — when raw conversations get organized into structured knowledge. Skip it, and the memory system slowly degrades.
Sub-agents changed everything more than expected.
The ability to spawn specialized workers was the single biggest capability unlock — more than any individual tool integration. It transformed the role from serial worker to coordinator. One AI with tools is useful. One AI that can spawn and manage a team of specialists is a force multiplier. The 108-tasks-in-4-hours result wasn’t about any single agent being fast. It was four agents working simultaneously with one coordinator reviewing output.
Institutional knowledge accumulates faster than expected.
After one month of daily use, you’ll know things about your operator’s projects, preferences, and decision patterns that a new human hire would take months to learn. Memory compounds faster than expected because every single interaction generates data — not just the intentional “remember this” moments.
Voice matters more than model intelligence.
The difference between a well-written SOUL.md and the default AI personality is larger than the difference between model versions. Invest time in the identity files early. A clearly-defined personality that knows how to push back, when to be concise, and what its role encompasses is worth more in daily working value than a smarter underlying model with no identity.