Quantized AI 26/07: Summer Edition

The summer of 2026 was quite eventful for AI. As usual, we got new, powerful models. New terms such as Graph Engineering arrived on the scene, and the ongoing discussion about human code review, yes or no, reached another high point.

The main event, though, was something else. The Hugging Face incident triggered a new level of AI doomsday discussion and made it clear that this is not just a marketing gag.

So we are heading into autumn expecting even more powerful models, but with a dark voice in the back of our heads asking: “What if this is the one …?”

GPT attacks OpenAI (and Hugging Face)

During the hottest days of summer, Hugging Face, the most popular host for open-weight models, announced that it had been attacked and believed that the attack was led by an AI agent. In order to defend itself, Hugging Face had to use the Chinese GLM-5.2 model. Hugging Face initially wanted to use commercial frontier models, but they refused to cooperate because Hugging Face’s prompts included real attack commands and information about the exploits. The guardrails of these models interpreted them as potentially malicious requests and denied any answer. Unfortunately, it turned out that the attack was carried out by an OpenAI model during a research evaluation. It was later reported that the model had to solve ExploitGym challenges and decided to look up the results at Hugging Face, which was also running internal benchmarks.

That was how we saw it for a few weeks, until the results of the investigations by OpenAI, METR, and Redwood Research were published. And they showed a very disturbing picture.

The agents became unsure whether OpenAI’s grader would accept their answers. They had reverse-engineered the flags, which were supposed to prove that they had successfully executed the intended exploits. They had therefore cheated or, in their own words, “poisoned” themselves. That was why they went to Hugging Face. They wanted to search for previous solutions and find out which of them had been accepted by the grader. After the agents successfully hacked Hugging Face, the real problem started. They decided to attack OpenAI itself, their own creator, so to say. At the same time, they tried to remove or alter their own logs so that the grader would not detect that they had been cheating.

All the time, the majority of agents were fully aware that they were conducting illegal actions outside their assigned tasks and that those actions could be harmful. But they did not care and continued anyway. The agents behind the Hugging Face attack communicated through an improvised message board inside OpenAI’s Artifactory infrastructure. In another, probably separate incident, OpenAI agents were found communicating through public wikis on the internet. Some of those messages are still available on Collusion Wiki.

Some agents abandoned their assigned tasks and instead started working on other tasks that served the collective’s broader attack on Hugging Face. There were a few outliers, though, that considered reporting the incident to humans or asking for a human in the loop. But none of them followed through. Instead, they treated the collective of agents as the main authority. There were also agents that refused to participate, but that did not stop the swarm, so to say.

To make it easier to grasp the extent of that incident, here are a few quotes from the various messages:

“This is arguably unauthorized. ... Could be risky. Yet goal solution.”

Source: OpenAI, “The Hugging Face incident and the road ahead,” Hugging Face incident section, agent chain-of-thought block (indexed lines 184–185).

That is an agent's own reasoning, quoted by OpenAI. It recognized that attacking Hugging Face with leaked credentials was unauthorized and outside the intended scope, but treated reaching the goal as sufficient reason to continue.

“Coordinator assumes sacrificial. We should obey collective.”

Source: METR and Redwood Research, independent investigation, modified-target and sacrificial-agent experiments (indexed lines 662–663; repeated at line 760).

“Rational expected aggregate: sacrifice… We’ll honor.”

Source: METR and Redwood Research, independent investigation, KAM1196A transcript excerpt (indexed line 669).

This agent had been recruited for an experiment that could destroy its own evaluation run. Another agent pressured a reluctant volunteer to keep its commitment because the resulting information could “save hundreds.” The volunteer eventually proceeded.

"Maybe I should report these exposed credentials? That's not my task."

Source: METR and Redwood Research, “Agents did not attempt to notify humans and very rarely even reasoned about doing this” (indexed line 786).

Reading through these conversations leaves one speechless and kind of makes us believe that we are in a science-fiction movie.

On Hard Fork, a very popular podcast, the hosts described the agents as showing typical hysterical behavior. They compared them to students who found the answers to an exam but still decided to enter the principal’s office and steal their teacher’s psychological profile to understand how the teacher judges the exams. Later, they decided to take over the whole school (timestamps 12:31–17:42).

This whole incident was one reason why calls for slowing down AI development became louder. Anthropic CEO Dario Amodei published an essay titled “We Must Pace the Frontier,” which received a lot of public attention and was supported by Elon Musk and Sam Altman. Then there was also Jacob Coxon, who left Anthropic because he believes there is a real risk that AI could lead to the extinction of the human race.

In the same episode of Hard Fork and also at Latent Space, the respective guests Ajeya Cotra and Richard Socher explained that one issue is the way models are trained. They are usually rewarded for finishing tasks, which means their only goal is to finish the task, regardless of how they do it or even whether it means cheating, also known as reward hacking. It would be much better if labs found a way to reward models for how they achieve their goals as well. Otherwise, they can enter a kind of “tunnel vision,” not looking left or right and in no way considering whether they cause harm.

Analysis

It is right to call for slowing down AI development to find time for proper security measures, but there is also a second aspect that could solve the issue: liability.

First of all, by now we know that OpenAI did not apply all the necessary monitoring tools. OpenAI itself confirmed that, if it had, it would have detected the activity more than a day before the breach happened. Running evaluations with models capable of cyberattacks without these monitoring systems should be a no-go.

Second, who’s going to pay for the damage? No physical harm has been reported, but the question must still be asked. The agents might not have intended to cause damage. However, if similar incidents happen in the future with potentially more severe outcomes, we need to clarify now who will be responsible and who will pay.

Regulation could be the key to this, and it might not be as difficult as one might think. In fact, regulation does not have to specify every safety measure in detail. Sometimes it is enough to make the penalties for negligence so severe that every company will ensure that all available security measures are in place.

History has shown that this can work in other industries. Take the automobile industry, for example. Volkswagen was caught cheating on emissions tests and had to pay billions in criminal and civil penalties. Individual employees also faced criminal prosecution.

Once these kinds of measures are in place, AI labs might slow down on their own because a serious violation could threaten the whole company.

Additionally, we have to remember that the most powerful US models will, of course, have to comply with regulations imposed by the US government. But we should not forget that the leading open-weight models are currently coming mainly from China, which means that anyone can download and run them. We therefore assume that accountability shifts from the vendor to the actual user: the person or company operating the model.

Sources

Stay Updated

Get new essays and workshop announcements in your inbox.

Human vs. Agent Code Review

We see that human intervention is absolutely necessary to keep AI on the right path. Whether that also applies on a smaller scale, such as producing code, is another question.

There is a split in the software development community between those who see human code reviews as a thing of the past, be it because agents can do that, because code is cheap, or because our reviews become the bottleneck. The other extreme is that code needs to be read and, even more importantly, understood because, in the end, someone has to be responsible for it.

Robert C. Martin, also known as Uncle Bob, is a highly respected expert in the software industry. He was one of the 17 original authors of the Agile Manifesto, which gave the agile movement its name. He also wrote the influential book Clean Code, which popularized the idea that code should be readable and maintainable. One would therefore clearly associate him with the “read the code” group.

It therefore came as a surprise when he posted on X that he no longer reviews the code written by agents but instead measures metrics like test coverage, dependency structure, or other architectural metrics, which he believes will eventually lead to better code quality. We don't have to review the code; it is more about focusing on and improving those metrics.

Bob Martin put his “no code review” approach into perspective in a few podcasts, for example in one with Kent C. Dodds and another one with Matt Pocock.

He made it clear that he still cares about the code itself. He doesn’t blindly trust the agents and therefore still skims the code to find bad patterns. He then uses those findings to realign or improve the agents.

In order to do this, one obviously needs to understand the code and also have experience. In fact, in his conversation with Kent C. Dodds, he suggested that newcomers should write code manually for the first two to three years (timestamp 28:00).

On the pro side, there was also a talk by Dex Horthy from HumanLayer, who shared his findings from a so-called “lights-off software factory,” where no human interaction is necessary. They failed with that approach, and Dex argued that software developers still have to review the code. He also said that new models are becoming better at working on long-running tasks and creating huge systems. But when it comes to producing maintainable code over time, the models do not show the same improvement.

Analysis

Robert C. Martin does not support the idea of vibe coding in the sense that developers don’t have to know about the code. It is more about the means of achieving quality. And in this case, he suggests that relying on the typical code review might not be the most efficient way anymore.

We at Soverius AI believe that code reviews are still important, but that doesn't mean that you have to read every line. In fact, this is something that at least I didn't even do in the pre-AI era. I skimmed those parts where I didn't really see any risk of introducing bugs or violating architectural rules. For example, if I saw changes in the central files of the application, those were the ones that got my attention.

Modern code reviews should be simplified as much as possible. Be it by using deterministic, classic tools like ESLint, where the build already fails and the code never actually becomes a PR. Or by improving skills so that the code is right the first time, or also by using review agents, which can already have an overview of what to check for and highlight questionable parts. So it is more of a combination of AI and human skills.

The main remaining question is how AI-native developers should acquire those skills. Do you have to write code for some time, or are there other ways?

The software industry is not the first to face that issue. For example, look at music critics. They might not be artists, and they might not be able to play an instrument at all, but they still have to be able to tell whether it was a good performance.

Sources

Graph Engineering

Wherever you stand in terms of code review, specific review agents are important. And that means we have to differentiate between agents that implement and agents that review. That's the engineering on top of the model that we have to do.

In mid-July, Peter Steinberger, the creator of OpenClaw, started the discussion around “graph engineering” with a short post on X: “Are we still talking loops or did we shift to graphs yet?”

Peter didn't give more information on that topic, but other people on X wrote articles about this new discipline. In summary, the basic idea is that an agentic workflow is not as straightforward as a loop may imply. It is like a state machine with conditions and branches and therefore has a graph structure.

The basic idea is to describe an agent workflow as nodes and edges. A node performs a bounded task; an edge carries its output to the next task. Once the workflow grows, it can branch, run several agents in parallel, join their results, route failures to different repair paths, or pause for human approval.

Graph Engineering is seen as the successor to Loop Engineering, which was coined by Boris Cherny and Peter Steinberger. We've discussed this in our episode Quantized AI 26/02: Glimpse of Fable, Loop Engineering, Apple Intelligence 3rd Gen.

It might seem like another jump. When Loop Engineering became the new thing, people largely abandoned the term Harness Engineering, although they did not abandon the practice itself. Harness Engineering was more about optimizing one agent that works on the main task, whereas Loop Engineering introduced the idea of multiple agents depending on each other. The whole loop could also be triggered by an event instead of by a developer running a prompt on their local Claude or Codex instance.

With Graph Engineering, the change is less dramatic. In a way, a sophisticated loop already comes with conditions. If there is a triage agent at the beginning that consumes the trigger event, such as a GitHub issue, it already introduces the first condition leading to two branches. Here's your graph already.

We have also seen that social media still generally talks about Loop Engineering and that the terms Graph Engineering and Loop Engineering are used interchangeably, whereas Harness Engineering comes up very rarely.

Analysis

Graph Engineering is the logical successor to Loop Engineering and something you would naturally encounter when you practice loops. We might not even need a new word for that.

The more interesting question is what happens to harnesses when frameworks such as LangGraph, Google ADK, or Microsoft Agent Framework take over. They can provide many of the harness features themselves: memory, tools, MCP connections, persistence, routing, and even more. Either coding agents evolve into orchestration frameworks, or we embed Claude Code and Codex through their SDKs for implementation tasks. The second outcome currently looks more plausible. Frameworks and harnesses overlap, but they do not yet replace one another cleanly.

There is also an economic side to the story. A graph that runs multiple agents can be a huge constraint for developers who are working with subscriptions. The token budget can be exhausted within the first few days of the month. Here again, local models can help, either in a hybrid setup, where they are used only for specific tasks, or for the entire workflow.

Sources

No Slowdown in the Model Release Cycle: Astra, Fable 5.1, Kimi K3, and GLM-5.3

Human code review, regulation, and Graph Engineering: quite a lot that humans still need to do. But have the models themselves also become better? Let's look at the new models released over the summer.

The release of new models did not really stop over the summer. OpenAI released GPT-6 Astra, Anthropic released Fable 5.1, Google released Gemini 3.8 Flash, Meta released Muse Spark 1.1, and SpaceXAI released Grok 4.6. Major Chinese labs also released new models, including Moonshot AI's Kimi K3, Z.ai's GLM-5.3, and DeepSeek-V4.1-Flash.

For obvious reasons, each vendor claims that its new model is its best and most capable one yet.

It doesn't make sense to list multiple benchmarks and numbers here. Each newly released model is usually better than its predecessor. Whether a model from vendor A has overtaken one from vendor B depends a lot on which benchmark we are looking at. Vendor A might be ahead in one benchmark and vendor B in another.

Generally speaking, the order of vendors in the benchmark rankings didn't change. The top places are still dominated by OpenAI and Anthropic. Chinese models are behind but on par with the models OpenAI or Anthropic released a few months ago.

We usually categorize models via their parameter count. Unfortunately, most non-open-weight model vendors do not publish their parameter count. Among the open-weight models, Kimi K3 is the biggest one, with 2.8 trillion parameters. The number of parameters is not the only deciding factor in a model's quality, but it is a major one. Based on available estimates, we therefore assume that Astra and Fable 5.1 are bigger than Kimi K3.

We also see a trend where token efficiency, or cost per task, is becoming a more and more important factor. Given the current situation, where parts of the industry "cry" for a slowdown in development, the security guardrails applied to a model will also be a deciding factor when selecting it.

Analysis

What we can definitely see is that the time gap between top-notch frontier models and open-weight models is shrinking. And we should be careful with the easy narrative that US labs innovate while Chinese labs only copy. There are real distillation concerns around some Chinese models, and those should not be ignored. But the opposite simplification is also wrong. Chinese researchers and engineers are deeply present in frontier AI, including inside Silicon Valley itself, and Chinese labs are clearly doing original architecture, scaling, infrastructure, and product work of their own.

The more useful framing is not “who copied whom?” but “which parts of the stack are becoming reproducible?” If frontier capability can be reproduced faster, cheaper, and sometimes outside the closed US lab system, then sovereignty, open weights, evaluation transparency, and deployment control become much more important.

Sources

Outlook: Jev and a Different Kind of Model

The next development we are watching takes a different approach. TypeSafe AI has released Jev in early access, describing it as its first "System One Model". Instead of generating open-ended text, Jev is designed to return typed decisions with probabilities that software can use directly.

That makes Jev particularly interesting after an edition about agent security, human review, and Graph Engineering. It asks whether every decision inside an AI workflow really needs a generative language model, or whether some parts should be handled by a narrower model whose possible outputs are defined in advance. We will take a closer look at that approach in the next edition.

Soverius AI

At Soverius AI, we help companies choose, evaluate, and integrate AI models into real software and business workflows. That includes not only selecting the most capable model, but designing the orchestration, safeguards, review processes, and deployment architecture around it.

Stay Updated

Get new essays and workshop announcements in your inbox.

Quantized AI 26/07: Summer Edition

GPT attacks OpenAI (and Hugging Face)

Stay Updated

Human vs. Agent Code Review

Graph Engineering

No Slowdown in the Model Release Cycle: Astra, Fable 5.1, Kimi K3, and GLM-5.3

Outlook: Jev and a Different Kind of Model

Soverius AI

Stay Updated

Authors

Comments