Reasoning models broke my prompt engineering. I’m retraining for a year

Tuesday, 1 AM. I’m sitting here writing a prompt to extract the required fields from 40 PDFs of official regulatory acts into JSON format. It’s a routine task, and I have a proven template ready for it: an expanded CoT setup, three few-shot examples, and the role prompt of "senior financial analyst with 15 years of experience". It used to work like clockwork.

I feed it into GPT-5.5 with high reasoning enabled. I get garbage. Half the fields are incorrect, the format is broken, and the output includes the expanded chain of thought I never asked for.

I figure it’s just a glitch. I run it again. Same result.

I delete the entire prompt. I write a new one from scratch, super simple: "extract fields X, Y, Z from the attached text as JSON, no explanations". Ten lines long. I run it.

It works.

I sit staring at the monitor for ten full minutes. I just threw three years of accumulated prompt engineering arsenal straight in the trash. And the minimal prompt did a better job.

So what actually happened here

I started digging into it. And here’s what I found.

In 2025–2026, a wave of reasoning models hit the market: the o-series, Opus 4.7 in high-thinking mode, GPT-5.5 with high reasoning enabled, Gemini 3 with thinking mode. The fundamental difference from older LLMs is this: the model runs its own chain-of-thought process inside the response, no prompt required. No "think step by step" nudge. No need for my custom prompt frameworks.

Before, I was basically an instructor, teaching the model how to think. Now it thinks on its own, without me. And that "without me" part is what threw me for a loop.

Half of my old prompt techniques are now useless or even actively harmful. Especially the ones where I was directly interfering with the model’s reasoning process.

What started breaking first

My favorite lengthy CoT prompt performed the worst. It was the kind that goes: "First analyze the task, then list the key entities, then build a hypothesis, then test it for edge cases, then provide the answer". On GPT-4, this used to add 10 to 15 percentage points to accuracy on complex tasks, I measured that myself.

On reasoning models, that same prompt now drops accuracy by 5 to 7 percentage points. That's because the model already does roughly the same thing internally by default. My prompt adds a second layer of reasoning on top of that, which conflicts with the model's native process.

Next to fall by the wayside was the emotional role-play prompt: "You are a brilliant programmer with 20 years of experience, you love elegant solutions". It used to work, I had no idea why but it did. Now when the model sees that kind of opening, it latches onto the tone rather than the quality instruction. It starts writing like a LeetCode forum hero, pompously, full of exclamation marks. That one flopped too, so I rewrote it.

A little less obvious, but it also stopped working: the heavy few-shot prompt for logic tasks. I'd include 6 or 7 examples of the exact same problem to teach the model how to solve that entire class of problems. That used to be standard practice. Now the model already knows that class of problems anyway, and the extra examples make it default to copying instead of generalizing.

And then there's the whole genre of "wordy intro explaining how important the task is". I used to think that motivated the model. Now I realize I'm just burning tokens on it.

What, by contrast, has actually gotten better

This is where it gets interesting. The most boring techniques, which I used to spend just five minutes on in my prompts, have now turned out to be almost the most important ones.

First and foremost is the output contract. Be specific about what you want as the end result. Not «give me a good report», but «a 5-row table, with these specific columns, no introductory or concluding explanations». A reasoning model will work out perfectly how to arrive at the answer. It shouldn't have to guess what I consider the correct answer to be.

Right now I catch myself spending half of my new prompt on the output: what format it should be in, how long it should be, what it definitely shouldn't include. Earlier I used to spend just two lines on this, now it's half the entire prompt.

Second: use system prompts instead of user prompts. CLAUDE.md, the system message in the API, instructions for a subagent. Put stable, consistent working rules there. The user prompt only retains the specific task. This separation used to seem purely aesthetic. Now it's structural. The system layer almost completely dictates behavior, while the user prompt is just the day-to-day request.

If you have «you are a helpful assistant, respond in Russian, do not give medical advice» written in every user prompt, that content does not belong in the user prompt. It belongs in the system prompt.

Third, and this one is a personal lifesaver for me: constraints. Things that are not allowed. «Do not use third-party libraries», «do not modify files outside /src», «do not propose solutions with downtime longer than 5 minutes». Older models would sometimes brush off «do not do Y» and do Y anyway. Reasoning models respect negative constraints surprisingly faithfully.

Just don't include more than 3-4 constraints at a time. If you overload it with too many, they turn into background noise.

Fourth — few-shot is still necessary, but not for the purpose you might assume. One example to demonstrate the format. Two if the format is particularly complex. No more than that. A reasoning model learns logic from a single example, while extra examples push it toward copying instead of generalizing. I used to provide six examples to teach it a task class. Now I give one example to show the expected answer format.

Not "how to solve the problem", but "how to format the solution". It sounds trivial, but it has a huge impact on quality.

Fifth — personas work, but not all of them. "You are a security auditor looking for vulnerabilities in this code" works. "You are a brilliant programmer with twenty years of experience" does not work. The difference is simple: function versus compliment. The first is a constraint on perspective, which is useful. The second is a motivational opening, which is garbage.

Sixth, this is purely technical — structured output. JSON Schema, XML with tags, markdown with fixed formatting. If the output needs to be parsed, it's a must-have. Previously, you could write "output as JSON" and the model would often make mistakes, adding comments before and after the code block and breaking quotation marks. Now, structured output via an API parameter or an explicit prompt contract with XML works consistently.

If your parser crashes on model outputs, this is almost always fixed by switching to structured output, rather than doing yet another rewrite of the user prompt.

A dedicated set for each task

I’d like to compile all this into a single table. It’s much faster to reread it in six months when I’ve forgotten everything again.

Right now the most common mistake I spot both in my own work and in my clients' is trying to write a single prompt for every use case. The "universal system prompt: you are a helpful assistant who can do everything" approach doesn't work. One task type gets one dedicated template. In my repositories, I don't have just one CLAUDE.md file — I have several: a general one, one for subsystems, and a separate one for tests. That's exactly how they are structured and written.

When prompt engineering is not needed at all

Sometimes it isn't needed at all.

Parsing JSON, writing a unit test for a pure function, translating from Russian to English — a reasoning model can handle these tasks with a minimal prompt. Any added complexity here is unnecessary overhead. It wastes tokens, slows down processing, and can even degrade output quality.

Rule: Start with the bare minimum. Only make it more complex if the bare minimum can’t get the job done. Not the other way around.

And on a separate note: sometimes a bad response isn’t a prompt problem. It’s a problem of me not even properly knowing what I want myself. No prompt can fix that. You need to clarify your request first, then reach for the tool.

Where this is all heading

I think next year we’ll see models that write their own prompts tailored to the specific task. There are already meta-prompting tools out there right now, where one model optimizes the prompt for another. Once this becomes standard practice, the familiar role of the "prompt engineer" will shrink down to the level of the "task setter".

And then the core skill won’t be "how to write a prompt", but "how to phrase a question correctly". In other words, it’s the thing that was always the most important, just without the noisy hype layer on top.

If you have a couple of favorite go-to prompts full of examples and detailed CoT right now, try shortening them. Not by 10%. Cut them by 70 to 80%. Compare the results. After I did this exercise, some of my prompts shrunk from 2000 tokens down to 150. The quality was either the same, or better.

All in all, I’m still relearning the ropes myself. In six months, half of the new techniques I’m using now might be obsolete too, and I’ll be sitting at my kitchen table late at night again with that same exact look on my face.

By the way, it would be interesting to hear whose workflows for working with models have also fallen apart over the past few months. I get the feeling I’m not the only one.