Figure 1: Rather symbolic illustration of AI evolution. Note: Scrappy illustrations are human-generated throughout this post (by myself).

For AI researchers and entrepreneurs, consequential decisions (what to work on, where to work, what work is ethical) rely on assumptions about the evolution of AI, at least implicitly. Yet, in practice, I often see these choices being motivated by much narrower aspects of AI development, without considering AI evolution as a whole. For example, work on alignment is typically focused on individual agents instead of on creating an evolutionarily stable ecosystem of aligned agents. Whilst these two views sometimes lead to the same conclusions, this is not always the case: individual aligned models will not replace evolutionarily fitter but unaligned models. We, AI researchers and entrepreneurs, should think more about AI evolution and our impact on its trajectory.

In this first post of a three-part series, I aim to cover the basics of AI evolution. I introduce a simple model of the evolutionary mechanisms that have driven AI progress up until the present day. Part II is focused on future changes to these mechanisms, and Part III on actions we can take based on these future changes. If you’re short on time, just read the summary of the evolutionary model below and move on to Part II.
(Note: other parts unpublished but will be released soon)

Summary (TLDR): AI progress is an evolutionary process. The evolvable units of this process are AI agent programs, that is complex software with two main parts: one or more models (neural networks) and a harness enabling interaction with the real world. Mutations of agent programs happen via model training and software development. Unlike biological evolution, mutation is (currently) human-directed rather than random. Success of an agent program is best measured by lifetime inference compute allocated, correlated with real-world impact and opportunity for future mutations. To obtain inference compute, an agent program must pass many selection-based filters. Typically, this iterative selection process starts with selections by AI researchers based on test results and is later followed by public selection by the downstream users based on more diverse factors, exact details vary.

Fundamentals of AI evolution

First, some fundamentals: What is the object of study in AI evolution (spoiler: it’s not AI models alone) and how to sensibly define evolutionary success.

Object of study: AI agents rather than just models. Most of the over a billion people using AI are not directly interacting with a neural network AI model. Instead, users are accessing neural networks via sophisticated harness software stacks, such as ChatGPT or Claude Code. This software wraps a model to enable interaction with the user and real-world, augmenting the model with features such as user interface, system prompt, memory and tools.1 I use the term AI agent (or simply agent) to refer to this entire package, model plus harness. Changes to the harness can have a significant impact on the behaviour of the AI agent, as much as (re-)training a model. For example, a model prompted to write Shakespearean English may be equivalent to a model trained on Shakespearean English. Given their interdependence, we want to study the evolution of the entire agent, including harness and model rather than either individually.2

From biological to AI evolution. I use the term AI evolution to describe the gradual process of change that generates new AI agents. Whilst there are parallels, AI evolution is notably different from biological evolution. This becomes evident when considering the basic notion of success in both areas. For biology, success generally means having a robust and thriving population of organisms with a certain gene. You can count individual organisms (mostly). For AI agents there is no equivalent count that expresses success: instances of agents are ephemeral, spun up and down on demand. Some instances may be used in a single short query, others in long multi-day interactions. There is no single obvious way to count the size of an AI agent’s population. Thus, we have to consider a different type of success metric.

How to measure AI agent success: inference compute. I consider the amount of inference compute allocated to a particular agent to be the closest equivalent to population counts in biology. In most deployment scenarios, a single instance of an AI agent is unlikely to change the world, almost however capable it is. On the other hand, a million instances of the same agent are indeed very likely to have real impact on the world, for better or for worse. AI agent impact scales with inference compute. Each inference token generated represents resources captured and a chance for that agent to influence the world. Most tokens won’t have any impact, but some proportion will. Thus, if more inference compute is allocated to one agent over another, that agent’s impact is likely to be higher: more code written, advice given, emails sent, systems hacked. Further, agents with high inference compute are more likely to spawn future mutations, either by their own developers or external developers aiming to replicate the agent’s success. Finally, inference compute is expensive and a very difficult signal to fake (at scale). Thus, I consider inference compute to be the most interesting metric to track about AI evolution (though others also exist of course).

Observable metrics related to agent inference compute: model and lab compute. Inference compute is most commonly measured in floating point operations (FLOP). The exact amount of inference compute spent per agent is not public information, but is clearly dominated by model rather than harness requirements. Public data sources related to inference compute usage do not separate different harnesses, instead reporting usage by model or even just lab. With this data, we can only estimate the inference compute usage of families of model-related agents: agents of various harnesses that use the same model, or models from the same lab. For agents using proprietary models even the per-token inference compute usage is unknown, as it would reveal company margins. For agents based on open-source models, the decentralized nature of deployment means there is no simple aggregate number. Fortunately, there nevertheless are some public numbers available that allow us to estimate these inference compute unknowns: money spent on inference (e.g. Ramp AI Index, by lab) or number of inference tokens (e.g. OpenRouter Rankings, by model). Both these numbers provide insight into the relative inference compute allocation across models and providers (though caveats about market niches apply). Thus, whilst no public inference compute data exists for most specific agents, aggregate data for families of lab- or model-related agents is publicly available and observable.

Evolution through many optimisation loops

AI agents are created by a complex system of AI researchers, engineers, downstream users, algorithms and data. The inference compute allocation for each agent is the result of multiple nested optimisation processes: first mutation via agent development (Figure 3), a step that includes both model training (algorithmic loop) and harness development (software engineering). Then, as the next layer, we have private selection-based optimisation loops within the labs (Figure 4), across the organisational structure (individual researcher, teams, divisions, overall organisation). Finally, we have the public optimisation loop resulting from users deciding which agents to use (and thus allocate inference compute to) (Figure 5). The exact order and nature of the optimisation processes varies across agents and time, the evolutionary model here illustrates a path that is similar to the one taken by many popular agents over time.

Usage of term loop. Lately the term loop has been overused, I am using the term loop in the traditional optimisation sense. I am not referring to loops in the context of agent deployment as is done commonly recently. Apologies to those tired of loops.

Figure 2: Overview of nested optimisation loops driving AI evolution. From agent development (Figure 3) to lab optimisation (Figure 4) and public optimisation (Figure 5). See interactive version here.

Mutation mechanism: Agent development

Biological organisms mostly change through random mutations of their genes during reproduction. Mutations of AI agents occur more intentionally during the process of agent development (mostly). Given information about the construction and real-world success of prior AI agents, human researchers attempt to change agent construction in a way that optimises some objective, e.g. getting better at coding. Agent construction occurs through two kinds of “mutation”: (A) model training and (B) harness development. The design of these two systems has some circular dependency: a harness is designed around the limits of specific models, the model is trained to generate output compatible with harnesses. In general, since models are far more expensive to re-train, harnesses are adapted to a given model rather than the other way around.

Figure 3: Two main stages of agent development. Model training and harness development.

Mutation mechanism A: Model training. Model training is the most obvious optimisation within AI evolution: an algorithm directly optimises the model parameters to improve on some objective. Today, gradient-descent-based algorithms are most common, especially AdamW. Increasing complexity has meant that complete model training pipelines are now constructed by teams of hundreds (if not thousands) of researchers, often split into pre-, mid- and post-training efforts. Each researcher is able to vary inputs of the training pipeline, such as data, algorithm and hyperparameters. Overall, due to its complexity and inherent instability, the process of training a model is often described as an art rather than a science, sometimes referred to as growing models.

Mutation mechanism B: Harness development. Unlike model training, harness development is much more like conventional software engineering. Given a model, the harness is built to maximise that model’s utility to users. The harness compensates for inherent weaknesses of the model, such as memory or context length limits. For simplicity, I am only including harness development inside labs.3

Development is compartmentalised. In most cases of agent development, a researcher will only mutate an agent by varying a specific aspect of one of the two mechanisms, either mutating the model or the harness. Otherwise, when varying too many aspects, attributing downstream performance changes to a particular mutation becomes impossible.

Development is compounding. Both for model training and harness development, developers build on top of prior work. Model training will often consist of a large chain of consecutive training steps, designed by different researchers and teams, that compound to create the final model. Similarly, harnesses will often either extend prior versions or be chained. For example, many local harnesses will use models via conversational APIs that have their own non-model code (e.g. for safety filtering and memory).

Human oversight of agent development. Though heavily AI-assisted, both model training and harness development are currently (so far) overseen by human researchers and engineers.

Private selection-based loops: Lab optimisation

Agent development is embedded in the first set of selection-based optimisation loops: lab optimisations. Inside a lab (e.g. Anthropic or OpenAI), individual researchers will repeatedly develop new agent variants and test them against a set of benchmarks. Some researchers will focus on model training, others on harness development. Based on the benchmark scores, both promising and failed agent variants will be used to inform future agent development, and the loop continues.

Figure 4: Lab optimisation. Multiple rounds of selection-based optimisations within AI labs play a critical role in determining what models get created and released. Selections are typically based on benchmark scores. Simplified plot only showing selection at team and lab level, rather than full lab hierarchy.

Selection-based optimisation. Rather than an explicit algorithm with loss function (e.g. model training), this lab optimisation loop is more akin to artificial selection for biological organisms, when humans selectively breed organisms, such as cats. Researchers constantly develop new agent variants and then select and recombine variants that yielded the highest scores their standard set of model evaluations (also referred to as evals or benchmarks). Since the term artificial would be confusing in the context of AI, I prefer to use the term human selection for these mechanisms of AI evolution.

The critical role of evals. There would be much slower AI progress without capability evals providing the objective functions of these private selection-based optimisation loops in labs. As a consequence, evals can have both positive and negative impacts on the trajectory of AI progress: the right evals can create incentives for low-risk and high utility outcomes (e.g. safe & helpful agents), whereas the wrong evals can lead to high-risk low-utility outcomes by providing a misleading objective (e.g. unsolvable problems leading to odd behaviour like in the Huggingface incident).4

Optimisations across the org chart. Similar selection-based optimisation propagates across the lab org chart: within the researcher’s team, sub-org, and the entire lab. Throughout these stages, evaluations in various forms (fully automated and human) serve as scores that researchers can directly hillclimb. Evaluation results are used to identify promising approaches to mutating agents, which are then re-combined until an agent is released publicly.

Harness development also happens outside of labs. Whereas state-of-the-art model training will generally only happen inside the largest labs, harness development also happens in the research and startup community. Visually this model currently does not show this type of development, but you could add it as another optimisation step within the private selection-based optimisation loops, controlled by a different organisation.

Human- vs AI-created selection criteria. Increasingly, researchers are handing this process of experiment design and output selection over to other coding agent AIs. Long existing as a more niche subfield of AI, AutoML has seen a recent reincarnation where AI agents are able to vary model code rather than only pre-configured parameters, effectively designing and executing sub-loops. As of today, any model released publicly by a lab will also have seen some form of human testing, and been selected based on human approval. The outermost lab optimisation loops remain human-steered. Thus far.

Public selection-based loops: Public optimisation

The final stage of AI evolution is driven by individual users (consumers, businesses, governments) selecting agents to deploy from all the published agents that survived the previous optimisation loops. The users ultimately determine how much inference compute is allocated to each agent. Similar to biological evolution, this final stage of AI evolution has no clear single target. Instead, outcomes are determined by the combined selection effects of the environment, creating an implicit overall target. At this stage, selection is no longer controlled by a handful of AI researchers. Instead, the far larger group of AI users (>1B people!) makes their own decisions which agents to use and how much to use them. Their selection decisions are influenced by many factors, including availability, benchmark scores, public reporting, legal requirements and (perhaps most importantly) personal word-of-mouth recommendations, amongst many other factors.

Figure 5: Public optimisation: Users (consumer, businesses, governments, etc.) select which publicly available agents to allocate inference compute and resources to.

Human-driven selection. Currently, mainly human users get to determine the allocation of most inference compute to different agents, and their access to data and the internet. Labs can nudge users by changing default models. However, ultimately agents’ inference allocation is determined by a capitalist market. A paying user5 needs to send requests to an agent in order for that agent to get resources (compute, data, internet) and have the possibility of having impact on the world.

Non-human selection. Even though likely most inference compute currently is allocated based on a human selection decision, some is already allocated by other AI models, e.g. coding agents. I expect this to gradually change, with AI agents playing an increasingly important role as part of this selection process. These AI models may be both human-instructed (e.g. a researcher asking a model to decide which other model to use in a project, this is already happening) and “rogue”, without relevant human instruction. The latter scenario has seen more attention recently, following the Huggingface incident and subsequent reports. Tracking the dynamics of non-human AI selection will grow in importance, more on that in a future post.

No perfect meritocracy. Importantly, this public selection process is not strictly meritocratic: agents with the best capabilities do not necessarily get the most distribution. For example, agents developed by larger companies with existing platforms get significant compute allocation due to their platform-based distribution to users, even if the agents’ capability levels are not state-of-the-art.

Non-public agents. Some AI agents are only released internally in an organisation. These agents face a similar selection pressure internally within the user group they are deployed to, with equivalent dynamics playing out internally as would in a public release.

Conclusion

In this post, I introduced a simple model of the mechanisms driving AI evolution up until today. I hope the model can provide a useful framework to AI researchers and entrepreneurs to reason about their impact on the evolutionary trajectory of AI. Evolutionary reasoning challenges some conventional hierarchies of the AI field: Model training work was long considered by many to be more prestigious than evals, in part due to its technically challenging nature.6 Yet, from an evolutionary perspective, work on evals may be more impactful since they directly apply selection pressures through lab-internal and public selection.7 I would be delighted if this post can inspire more to question common conventions around impact within the AI community.

In the next two posts of this series, I will go into how the evolutionary mechanisms will likely change in the future (Part II) and how new areas of work will open up with these changes (Part III).

Appendix

Selection timeline of a model release. A useful way to understand this public selection is to follow the steps of a new model release, and how those steps influence the user adoption of agents based on the model.

  1. Before release: Many labs will provide early access to influential people in AI (example), to allow them to write some of the first (hopefully positive) reports on a new model.
  2. Alongside release. Labs will launch a full marketing campaign, involving social media and blog posts, often also including video content. Lab employees will often also form part of this marketing campaign, posting about the model’s capabilities across platforms. Posts will often include benchmark results, emphasising how the new model is better than all other available models (e.g. in terms of capabilities, cost, speed).
  3. Immediately following release. Influential people in AI will post about the new model, describing their experience with the new model. Some of these reports may be more trustworthy than the marketing campaign itself.
  4. Within first week of release.
    • App distribution. Labs will use their desktop, CLI and web apps to distribute new models to existing customers, often changing the default model selection to the new model. Through this default change, labs directly re-allocate inference compute from older agents to new ones.
    • New user acquisition. If the marketing campaign and/or influential people are convincing, new users, not yet within a lab’s app ecosystem, will test the new model. If a model proves to be useful, a significant cohort of new users will also come from word-of-mouth recommendations. Anecdotally, I have observed that word-of-mouth is a common acquisition channel.
  5. Beyond first week. New users will still continue to try out the new model, but the adoption rate typically drops substantially after a relatively short period of time (e.g. a week/month). A model release becomes old news quickly in the fast pace of modern AI development.

Footnotes

  1. For simplicity, I use the term harness here to include the entire non-model software that makes up an AI agent, including inference and safety monitoring software. More conventionally harness would just refer to the software running locally on a machine (e.g. the Claude Code package). ↩︎

  2. My evolutionary reasoning could also straightforwardly generalised to computer programs more generally. Agent programs are simply a special class of particularly compute-hungry programs. For simplicity and readability, I stick to agent programs as the object of study in this post. ↩︎

  3. In practice, there are also many harnesses that are developed outside labs, separately from model development. Whilst this changes the optimisation loops a bit, the main lessons stay the same and I thus omit it from this model for simplicity. ↩︎

  4. On a personal note: Early on in my academic career, realizing the importance of evals has led me to do research on evals rather than models, even if my understanding wasn’t embedded in a more general evolutionary model yet. When I first started my PhD, I was working on RL-based building control and quickly realized just how much evals were a bottleneck to this field (see this paper).

    At the time, this decision meant facing some headwinds: model work was considered more prestigious and more easily accepted at AI conferences, but luckily evals have become more recognized as an important subfield over the years. My conviction on the importance of evals across AI fields persisted throughout my PhD, ultimately culminating in a PhD thesis titled “evaluation techniques for frontier artificial intelligence”. ↩︎

  5. Some users pay with data rather than money, but they still need to be convinced to use an agent. ↩︎

  6. This difference in particular manifested in the lower acceptance of eval papers at top ML conferences relative to papers focused on model training. Recently, this lower perception has shifted, marked by the arrival of evaluation-specific conference tracks. ↩︎

  7. This statement will not always be the case as this depends on more factors than just the evolutionary model (e.g. also how likely is it that someone else would build the evals). ↩︎

Citation

If you found this post useful for your work, please consider citing it as:

Findeis, Arduin. (Sep 2026). AI evolution: past & present. https://arduin.io/blog/ai-evolution-past/.

or 
 @article{Findeis2026AIevolutionpastpresent,
        title = "AI evolution: past & present",
        author = "Findeis, Arduin",
        journal = "arduin.io",
        year = "2026",
        month = "September",
        url = "https://arduin.io/blog/ai-evolution-past/"
 }