Self-Evolving Knowledge: How to Grow a Senior Agent

One of these projects is a mobile app for memorizing flashcards/words: I'm learning Japanese and couldn't find a single service where adding new words to your dictionary wasn't a tedious chore, so I decided to build my own, just for myself. Well, I didn't have a GPU cluster or a team for this, but I did have a MacBook, a free Sunday, and a specific problem I wanted to solve.

Below I will share my observations from the perspective of a regular PM, as well as the resulting idea and concept.

wikiLLM

Not too long ago I came across Andrej Karpathy's described idea of wikiLLM – keeping knowledge in a structured wiki that the agent reads, populates with raw data, and uses in its responses. The logic is simple and elegant: give an agent a well-organized knowledge base, and it will work more accurately. I tried it out. At least, as I understood it. And at first it was great. I broke down my main .md instruction into parts and structured all key points into folders in wiki/. But the most interesting observation came later...

As a project manager, I'm used to spotting "extra" and "missing" parts. This was no exception. I built the wiki, but it didn't grow and the agent didn't get "better". Why? Obvious! Because I didn't feed the agent raw data after the initial setup period, so its knowledge didn't grow. It could add a little bit on the fly via web search, but that was all.

How did that happen?

This is not a story about wikiLLM being a bad approach - Andrej Karpathy is definitely smarter than me =) It's just that my requirements for an agent didn't fully fit this method (at least as I understood it). I don't have a stream of raw data that could be split into articles in the classical sense. My raw data is hypotheses, concepts, queries, and bugs. I didn't need a librarian, I needed a developer who can handle my product tasks. I write to him: "make a button..." or "there's a bug here..." and he has to solve it, relying on the context of our specific project. And it would be great if in the future he doesn't make similar bugs in similar or specific cases.

With the WikiLLM approach, I started manually adding everything I considered important to the database. Rules go to the wiki. Learned lessons go to lessons-learned.md. Critical things I duplicated again directly in CLAUDE.md, "so he would definitely know". After three months: CLAUDE.md was 7,000 tokens long, and the agent was still repeating mistakes we had already discussed.

The problem wasn't the content. The problem was who should create it.

But this is not a story about wikiLLM being a bad approach. This is a story about how I was using it for the wrong purpose - and only realizing that, I was able to start moving in the direction I needed.

That very moment

I've been working with developers for many years. When a junior joins the team, I don't give them a 200-page document with "everything you need to know". They learn from tasks - and I believe the key point of learning is almost always the same: unexpectedness.

He writes code, runs into behavior he didn't expect. A bug. An edge case. An undocumented API limitation. It is at this moment that real knowledge is formed - not from documentation, but from hands-on experience. By the end of the year, he has dozens of such cases in his head, and it is exactly these that make him a valuable specialist.

And this turned out to be the answer to my question: what if the agent creates its own raw context during moments of unexpectedness?

I don't record the knowledge myself. The agent notices the discrepancy between expected and actual behavior - and records it as a rough draft itself. Later, if the situation repeats, it understands: this is a pattern, not a coincidence - and formalizes it into a rule.

This is exactly how a junior grows into a senior. Not by studying documentation, but by accumulated experience from real tasks. Well, we have the answer! All that's left is to model this process.

SEK Concept

That's how the concept I personally dubbed Self-Evolving Knowledge (SEK) came to be.

My PM background helped here. I'm used to thinking in terms of "who is responsible for what" and "how to build a process that works without manual oversight". The junior→senior framework is not a metaphor. I tried to literally model the onboarding of a new team member.

When I started using the WikiLLM approach in the "developer agent" scenario, rather than the "librarian agent" scenario, I ran into several issues.

main_instruction.md swells. To make sure the agent follows a rule reliably, it gets duplicated directly into the instruction. After half a year, a copy of half the wiki ends up there "just in case". 7,000 tokens at the start of every session, no matter the task - it doesn't get any simpler than that.

Knowledge gets duplicated. A "learned lesson" is recorded in lessons-learned.md, while a "rule" is stored in wiki/rules/. It's the same piece of information in two places. The agent gets confused about which one is more up to date.

The token tax grows linearly. Every new page added to the "required" list raises the startup cost. After a year, the agent spends 30-40% of the context window before even receiving the first message.

Loading triggers are written in plain prose. "Read this instruction before frontend tasks" is an instruction the agent has to interpret on its own. There is no machine-readable layer, no guarantees.

But the core problem runs deeper than all these four: I was still the only source of knowledge. The agent didn't add anything on its own. It only consumed what I had managed to enter manually.

SEK changes exactly that.

Architecture

SEK organizes knowledge into four context levels following the principle of "only load what you need right now".

L0 - Bootstrap (always in context)

This is my core instruction - the same as CLAUDE/AGENTS/GEMINI and other similar .md files, but radically compressed.

The goal is to fit everything needed to get started into the smallest possible number of tokens:

1. Who I am and what language I respond in

2. What knowledge categories I have

3. What to load based on which triggers

Nothing extra. No in-depth explanations, examples, "don't do X because...". Just the skeleton.

What is removed from main_instruction.md:

• any "What NOT to Do" sections,

• detailed descriptions of each wiki page, inline rules - all of this is moved to wiki/rules/ and loaded via trigger.

L1 - Routing & Index (as needed)

wiki/_routing.md - full routing table in YAML format. The agent reads it if the task is not covered by the core content of main_instruction.md.

triggers:
  - id: frontend
    keywords: [react, frontend, ui, component, tsx, jsx]
    load:
      - wiki/design/design-system.md
  - id: llm-cost
    keywords: [pricing, calc_cost, tariff, cost]
    load:
      - wiki/references/llm-pricing.md

YAML is the cheapest and most accurate format here for machine parsing of "keywords → files", while remaining human-readable.

wiki/_index.md - table of contents: file path + one line of description. Around 200-300 tokens. Fallback level if routing returns no result.

Large files (design-system, edge-cases) do not disappear - they stop being loaded "just in case". Response quality for these topics does not drop: the file is loaded only when needed.

L2 - Validated knowledge (by trigger)

All project expertise: wiki/rules/, wiki/references/, wiki/architecture/, wiki/workflows/. It is loaded only when the task explicitly relates to the topic.

Each rule gets a unique ID tag ([XXX-001], [YYY-ASYNC]). Before adding a new rule, the agent runs grep - it does not write the same thing twice.

L3 - Ephemeral state (where new knowledge is born)

This is the most important part - the "self-learning" mechanism.

memory/inbox.md is not a lesson archive. It is a conveyor.

Unexpected event (bug, workaround, hidden API behavior)
        ↓
REGISTER → memory/inbox.md (3 lines per entry)
        ↓
PROMOTE → wiki/rules/ or wiki/references/
        ↓
Removal from inbox

The agent writes to the inbox on its own, without my involvement. There are four triggers for writing an entry: unexpected system behavior, found workaround, undocumented API behavior, explicit correction from my side. Exactly the same situations in which a junior developer makes a "remember" note.

Strict limit - I set it to 20 entries. Entries older than N days without confirmation are deleted automatically. The inbox does not grow, it always remains ephemeral.

draft → validated → core

Every piece of knowledge goes through three maturity stages.

Promote draft → validated the agent does autonomously. If the same topic appears again, or there are 3+ entries for one topic - it is a candidate for promotion. Before writing, a grep through existing rules is run. After that, one line in the response: 📚 Promoted: [TAG] → wiki/rules/Y.md.

Promote validated → core requires explicit "yes" from me. The agent can suggest: "Rule [TAG] appears in 5+ tasks per month. Promote to Critical Rules?" - but main_instruction.md is not edited autonomously. This is asymmetric protection: low entry threshold (writes readily), high threshold for core (only with permission).

main_instruction.md is the constitution. The agent does not rewrite it.

Error protection. If a rule was promoted less than 7 days ago and is being applied for the first time, the agent signals: 🆕 Using recently promoted: [TAG]. If behavior does not match - /revoke [TAG]. The /revoke command deletes the entry from wiki/ and logs the revocation. Nothing disappears without a trace.

Junior → Senior

Here is roughly what an agent's growth could look like on the same project:

Transition does not need to be incentivized. It happens on its own - as the agent works on the project and runs the REGISTER → PROMOTE cycle. The model is the same. Only the knowledge corpus changes.

As a PM, I have seen this pattern dozens of times with people. The difference between junior and senior is not in intelligence or tools. It lies in accumulated, structured experience from real tasks. SEK transfers this same mechanism to agents.

You are not upgrading hardware. You are hiring a specialist and giving them time to grow.

What changes?

SEK is not a rewrite from scratch.

Does not change: folder structure (wiki/, memory/), category logic (rules, references, services), content of existing files, git history, all runtime.

Changes: what is loaded into the agent's context, in what order, and - most importantly - who creates new knowledge. Not only you. The agent too.

What's next?

I will keep digging into and researching context organization. Even though Anthropic recently released their own "memory" feature for agents =)

What I would like to cover in the articles:

• Comparison with Karpathy's wikiLLM - honest breakdown across 10 axes: where the approaches align, where they diverge, etc.

• What's new in SEK (in theory) - I don't think this is a revolution, all these approaches already existed in one form or another

• Lint Protocol - how the agent checks knowledge quality before promotion and doesn't create garbage in the wiki (I hope it doesn't create garbage)

Aleksei Panin - Senior Project Manager → Aspiring AI Practitioner.