TLDR:
Opus 4.7 is lazier than Opus 4.6, and it seems to react to prompting instructions meaningfully differently per Anthropic directly.
That means any carefully curated workflows you’ve crafted to teach Opus 4.5/4.6 what context/documents/references to load at the right time may not work nearly as well for Opus 4.7. Incomplete context inevitably leads to rapid performance degradation and hallucination city, which is probably why many think Opus 4.7 is such a downgrade.
This type of drift is probably going to be inevitable with model advancements and harness updates, because behavioral consistency is just too multi-faceted for model providers to optimize around -- especially as model and harness development velocity continue to accelerate. In other words: we really cannot take backwards compatibility as a given.
Key takeaway: If you aren’t actively logging and benchmarking model adherence in the context of your specific workflows, you absolutely need to start doing so regularly ASAP, because this will definitely not be the last time this sort of issue happens with a new model launch. I’d argue this is also going to become increasingly likely for seemingly insignificant harness updates, as well.
Bigger picture: Context engineering is still a very weird, very volatile frontier. Every model reacts slightly differently to different prompts, harnesses, context engineering techniques, and more. For my money, this is probably the biggest barrier to greater and more rapid societal adoption of AI.
The launch of Opus 4.7 this week has thrown a lot of the Claude userbase into disarray: is this a downgrade from Opus 4.6 despite the purported benchmark improvements? I mean, just take a look at some of the random Reddit threads I pulled into the image above from just the past 24 hours. This isn’t even an inflammatory cherry-picked subset: there is not a single positive headline about Opus 4.7’s performance in my entire feed across platforms right now. And this is coming hot off the heels of many, many complaints about Opus 4.6’s performance apparently degrading without any transparency over the past couple of weeks. Oof.
I can’t speak to every issue, and I can’t pretend to have a crystal ball to help explain why you, specifically, might be having problems if you are. But as I read more and more examples of frustrations people are having, I’m seeing at least one common thread and likely explanation for why this rollout has been such a mess -- and why this definitely will not be the last time it happens.
Let’s get back to basics for a second. I’ve spent most of the past month teaching fundamental AI principles to a wide variety of researcher audiences, and there’s one key lesson that is essential here:
Modern AI’s usefulness lives and dies on context
Pretty much every modern AI workflow relies heavily on getting the model to read and reference useful guiding information before it acts. In other words, is it loading the right contextual references (local files, websites, code documentation, etc.) at the right time to do your requested work as expected/desired? This is how we fight slop: Give AI an index of known-useful information it needs to begin with, and then let it search over when to surface key references based on the task at hand. This is the core design challenge for all context engineering, all agent orchestration systems, and basically any well-written Skill with progressive disclosure. To put a fine point on it: AI systems getting better at dynamically and intelligently curating their reference files while working is, in my view, the main reason why AI capabilities have been exploding since Opus 4.5 in November. It is the paradigm shift enabling everything crazy this year.
But without the right context loaded at the right time, most LLM assistants very ungracefully degrade into overconfident hallucination machines -- worse than functionally useless, they can become actively dangerous. Because they often don’t know when they’re doing it (though this is an active area of development), you probably won’t know either, and that can lead to many, many, many problems down the road well after it’s particularly easy to address.
As a simple illustrative example: you might ask Claude Code to help you change something in a given code file or document that ideally requires some good contextual understanding of how that change will need to percolate across multiple files. Claude has generally been good about doing some careful spelunking for relevant keywords across files, reading relevant design/style documents, tracing through some prior Git commits to identify recent related changes, and then coming up with a pretty comprehensive plan about what a proper edit will need to look like to address the task at hand and integrate carefully into the rest of the codebase.
Think about how many files it needs to read to do a basic task like this as well as you’d want. Think about how many lines of information it needs to examine in each of those files. What if it does those searches and decides: “You know what, let me just read the single relevant line in this file and then move on -- that’s probably all I need.” Or what if it decides, “Well, I don’t want to read too many of these, so let me just pick a few and move on.” Or what if it just… doesn’t actually read any of the files at all? What will the edits it proposes look like, then?
Ask yourself right now: How would you notice the difference in these scenarios as this is happening? And would you have any way of recognizing it before you move onto the next task?
It’s not dumber, it’s lazier
This style of degraded reference file searching/reading is probably what’s happening in at least ~80% of the issues I’m seeing people report with Opus 4.7 (and, likely in a similar way, reported 4.6 degradation over the past couple of weeks). It’s not that the model is outright dumber than Opus 4.6/4.5 -- it’s that it appears to be dumber because it’s doing a very bad job of choosing to load the right context/info/documents/references in the same way past models have. And because it’s not loading the proper info, it instead resorts to confidently responding with pure hallucination, or using only incomplete contextual information. In those circumstances, it’s not about intelligence but about needed context.
The root of the problem is then the model being some version of lazier, not dumber per se -- which aligns with what we know about Anthropic’s current heavy bottlenecks on their compute resources. They have every incentive to try and throttle users where ever possible (”token efficiency strategies”), and this is leading to major systems degradation because Opus 4.7 just isn’t loading the things it should be as comprehensively as it should be. It’s also the case that it’s simply responding to progressive disclosure and context engineering directions differently.
You don’t actually have to take my word for any of this: this is all explained directly in Anthropic’s recent guidance on how to adapt to Opus 4.7. This sort of public blog post is notable because Anthropic is explicitly telling us that the Opus 4.7 update has introduced breaking changes in a way I don’t think they’ve done for any past release. Put a different way, backwards compatibility is no longer a feature of Opus 4.7: workflows you made for Opus 4.6 and Opus 4.5 may not work well anymore, and it’s your job to figure out how to fix things going forward.
If you take nothing else away from the launch fiasco, it should be the crystal clear reminder that these sorts of model effort and context engineering drifts are likely inevitable as frontier models continue to develop. It is impossible for model providers to guarantee a new model will adhere as faithfully to your context engineering instructions the way you happened to write them for a prior model, especially because the ways people are using these models are becoming so much more diverse. In fact, even basic harness version updates (e.g., updating Claude Code from version 2.1.87 to 2.1.90) have similar potential to affect model adherence to context engineering instructions (e.g., if they change how certain core Tools like Read() or WebSearch() or Agent() are written or are specified in the System Message). Combine that with increasing pressure to launch models and harness versions with greater velocity, plus the use of said new models and harnesses to help build the next ones… And yeah, this is a formula for similar breaking changes all over the place from now on.
You should invest in better logging and monitoring systems ASAP
Because these sorts of updates are inevitable -- and frankly the whole dang point of AI technological advancement right now -- you are setting yourself up for a WHOLE WORLD of frustration in the next few months and beyond if you’re not explicitly designing for potential changes in models’ adherence to your context engineering pipelines.
What does that mean, practically? First and foremost, it means that you should have extremely detailed ways of tracking:
Exactly what tools are my agents and subagents using, and when?
Exactly what files are being read/written via these tools?
Exactly how much of each file is being read/written via these tools?
What is the actual sequence of each file being read/written via these tools?
Exactly how reliable and consistent are these tool patterns from run to run?
If you can’t answer these questions off-the-cuff, you’ve got work to do but don’t need to panic too much. Thankfully, many harnesses like Claude Code already track these via session transcripts automatically, which is an extremely helpful starting point. Just ask Claude Code to tell you where the .jsonl files are stored and tell it you should set up a system to start archiving and auditing them formally. I would also highly recommend changing your “showThinkingSummaries” to True in settings.json, alongside setting your /config verbose output on, to get better visibility into why Claude is making some of its context loading and tool calling decisions as it works.
But it’s one thing to track them and another thing entirely to actively monitor them: You need to have some ability to assess the extent to which your model is actually doing what you want, the way you want, according to some kind of opinionated stance on what optimal behavior actually looks like. One way to do this is by having a baseline set of prompts you can run on a regular basis with ground-truth correct sequences of tool calls, file reads, and file writes that you personally approve of. With that in hand, you can then run the benchmark prompts on any given day -- say, when a new model drops, or ideally something like weekly for your own peace of mind -- and have a clear sense of the extent to which the models are behaving according to your context engineering protocols or not. This is how we track degradation for your specific use-cases in a more coherent and robust way than just the vibes you often see across Reddit and X (e.g., “Oh Opus 4.7 really doesn’t get the carwash puzzle” over and over again). Don’t forget to run each such prompt multiple times, because the probabilistic/non-deterministic nature of LLMs means one test success doesn’t actually imply another later on.
To be clear, formulating these adherence checks will be extremely idiosyncratic and case-by-case based on exactly what your system looks like and what it’s supposed to do, and it becomes unfortunately non-trivial the more complex your orchestration system is. Relying on the standard tool-calling benchmarks that model providers tout is simply not a sufficient substitute for knowing how well these models will actually perform in your specific contexts.
For DAAF, as an example, I have a couple of benchmarking prompts that I run through every new version to check into how various files are read, how tasks are handed off, and what reference files are read by various subagents, across different styles of workflows I’ve built into the system. I recently updated the logging system in v2.0.0 to capture every agent and subagent transcript for any given project run, and spent a significant amount of time over the past couple of weeks using these logs to chart out every agent call, file read, and tool call from a sample project for my own edification and as an educational tool.
But while I’ve been doing these sorts of spot-checks manually since DAAF launch, you can certainly bet I’m going to be spending a lot more time over the next week or two formalizing these systems into a repeatable, quantifiable benchmark, and I think you should, too. Software developers already know that having a test battery suite is basically non-negotiable for system reliability, but making an analog for agentic AI systems is quite a different animal because it’s rarely going to be as simple as “Did this throw an error or not.” Working with your own AI to come up with scalable monitoring systems is a very good idea to make this task more tractable, but beware: Your checks will probably be better off NOT relying on LLM-as-judge or any qualitative support from the LLMs themselves, given the possible contamination/contagion effects of system degradation due to model/harness changes also causing LLM-as-judge monitoring system degradation.
While very unsexy work, this will save you enormous headaches in the long run. I would also argue such work will be amazingly valuable for three additional reasons:
You are going to keep improving and changing and adjusting the way your context engineering systems work. You should absolutely have a clear sense of how these changes affect downstream adherence to your system beyond just a handful of informal checks.
Given compute availability and cost pressures, every model provider is going to be surfacing more options and levers that users can pull to conditionally change model effort levels and other parameters. We already saw this with Opus 4.6 having multiple effort levels, and now with Opus 4.7’s Adaptive Thinking option. An adherence testing suite will let you explore the actual ramifications of these potential cost-saving options for your system, specifically.
This is also a way to give yourself a clearer sense of model/provider optionality and prevent vendor lock-in. Systems like this will allow you to better understand whether and when, say, open-source and local models start to hit sufficient capability that you can start offloading certain elements of your pipelines out to these instead of paying for the private frontier models (a major accessibility milestone I’m excited about for DAAF!). It also will help you better understand when you should start seriously considering competitors (e.g., Codex v Gemini CLI v Claude Code) and keeping your options open.
All said, it simply isn’t safe to assume that one model version will follow your instructions the same way as any other model version, so we need to be diagnosing and tracking this potential source of system performance drift diligently. Reliability and trust are going to be some of the most important currencies in this next phase of AI building, so time here is likely going to pay dividends in ways we can’t yet fully understand.
Using AI well is still just way weirder and more esoteric than most people realize right now
Last month, I wrote about how weird the current frontier of AI capability is right now. I’m finding the three-part mental model I proposed there more and more useful as time goes on:
I recommend reading the full article if you haven’t already, but the TLDR is that the quality of output from any AI right now is a combined function of (1) base model capability, (2) the specific frameworks and tooling it has available to it (e.g., why Claude Code is so much more capable than basic web interfaces), and (3) actual user prompting skill. I presented it as a sort of multiplicative function -- being sufficiently bad on any one of those fronts is enough to tank the whole operation, while putting all three together at a high level unlocks some really high heights on the developing frontier for AI capability. The other big idea I try to get across is that we’re really still just figuring out what it means to prompt models well, and how to make them use advanced Tools reliably while operating within complex orchestration frameworks. In other words, those latter two values of the model are shifting particularly fast in a way that’s generally not visible to the average person (likely only tracking the overall frontier model benchmarks alone).
This whole Opus 4.7 fiasco makes it clear that there are two more complications we need to incorporate into this model:
Base Model AI Capability can’t really be thought of as a singular value now that model providers are going to allow for varying token/thinking budgets and “effort” levels. It’s not as simple as “Opus 4.7 is better than Opus 4.6” when Opus 4.6’s extra-high thinking/effort model might far outplay Opus 4.7’s default middle thinking/effort mode. And exactly how those performance battles work out may be highly context-specific.
There are enormously weird interaction effects happening across these parameters, as well. That is: different orchestration frameworks and tools and harnesses will actually work better and worse with different models -- there’s not actually some linear measure from bad-to-good for framework/tool/harness quality. Similarly, what makes a good user prompt for one model may not be so for another model.
These two complications really reinforce my broad takeaway from the full post: it is still just way too weird and way too complicated and way too idiosyncratic to try and get the full squeeze of capabilities out of AI right now. AI isn’t actually getting simpler -- if anything, the Opus 4.7 launch makes clear that things continue to get only weirder as the months march on by. As long as this remains the case, I think it’s just not feasible for the vast majority of the working population to use AI tools well and reliably at scale, and so I have serious doubts about the speed with which AI can genuinely upend any particular profession, field, or society writ large as a result. That doesn’t diminish my other core recommendation from the post, though: It is still an enormously valuable time to get your hands dirty and start learning these intuitions and complexities yourself. Hands-on experimentation is the only way to begin surmounting this skill curve; the best time to start was six months ago, the second best time is now. See my full post linked above on the mental model for fuller recommendations on what that can look like to get started.
That’s it from me for now. I’ll be excited to learn more about what other issues rise to the surface from this launch, but until then: good luck out there!
Great piece! It will be fascinating to see what happens to the field when a decent open source reasoning model can be run on consumer(ish) hardware. I wonder whether the frontier labs will suddenly value performance stability…