Please read the introduction of this blog series for context.

In this second part, the different value chain models are mainly illustrated with examples of Western companies, but the Asian or South American models do not differ significantly. I have limited knowledge of African models.

Historical Model (1998-2010)

The Internet gave rise to an entirely new industry: digital information. It quickly became apparent that it could drive enormous value but no clear model emerged until the end of the 1990s. A viable model needed to be built around content produced manually and published online via Web pages.

In the original value chain of the digital information industry - still relevant today - the consumer of information can access content directly via the website's URL if they know it but, in practice, Web indexing providers play a crucial role by helping users find the URLs of pages that might interest them. This service is very valuable for users but early search engines struggled to find an appropriate business model. As they realized that they needed to keep users on their site, they transformed into web portals offering email, news, stock quotes, weather, chat, etc., alongside search. But something was off.

Google solved the problem by focusing purely on search with an algorithm, PageRank, that delivered vastly superior relevance. Google introduced non-intrusive, text-based ads, AdWords, tied directly to search keywords, which proved highly effective and profitable. The model was attractive for advertisers because the conversion rate is typically higher than untargeted advertising. And the model had the significant advantage of allowing the service to be offered free to the end-user.

Google quickly established itself as the leading provider thanks to the quality of its indexing and search service, which still allows it today to accumulate colossal advertising revenues: ad revenues from the search service represent over $200 billion per year, i.e. more than 50% of Alphabet's total revenue (Google's parent company).

In this model, the consumer of information is active, meaning they initiate the search for information on a topic and enter the corresponding keywords into the search engine’s input field. The search algorithm returns the relevant URLs in response, together with ads.

This model is basic by today’s standards but it already contains several vectors of influence through which unconscious biases or propaganda elements (intentional) can be integrated into the content. The main vector lies at the information production stage, through a simple transposition of historical persuasion methods into the digital world. Robert Cialdini established 6 principles of persuasion : reciprocity, scarcity, authority, consistency, liking, consensus. These principles work perfectly well for digital content ; only the distribution strategy changes. To reach an audience, content ranking must be optimized for Google search, so an entire sub-industry emerged in the digital information ecosystem to tackle Search Engine Optimization (SEO).

Independently of SEO practices, web indexing relies on algorithms which, like any algorithm, can introduce biases and thus constitute vectors of influence. Google established itself as the global leader in Web search in the West by offering the best service to end-users. This means that users consider that the algorithm-induced biases are aligned with their expectations, or at the very least acceptable. It should be noted, however, that Google can actively introduce biases to protect its economic interests, by artificially penalizing competitors' links and boosting its own services. These practices have led to numerous fines from competition authorities over the years. They seem to have ceased, but Google now resorts to other, no less controversial ways to protect its economic interests. For example,Google Chrome gives it an additional layer of control leveraging its dominant position in the Web browser market.

A final vector of influence lies at the level of the search engine itself, as certain terms can be censored even before the search is launched, and certain results can be removed before the user sees them. This censorship typically aligns with a regulatory framework for a given market. Europeans can experience this when, at the bottom of Google search pages, a message explains that results have been removed in compliance with data privacy laws or intellectual property protection regulations.

Any moderation constitutes an influence by introducing a bias. The perception of this influence varies across regions and over time, as demonstrated by the different perceptions in Europe and the United States. In the US, the sentiment has completely shifted over the last 12 months, and moderation is now presented as censorship. In reality the line is blurred.

Bias and censorship are two sides of the same coin. The very notion of bias is political since it involves measuring the deviation from a desired state. Yet the desired state is never the actual distribution that information would have if everyone were allowed to express themselves freely and anonymously. It must be stressed that anonymity is fundamentally incompatible with the social contract, if only because property rights, a cornerstone of the social contract, are attached to an identity. Anonymity is perfectly acceptable – and probably desirable – on the internet, but it would be naive to ignore that this implies a minimum of moderation. If this moderation stems from a democratic process, it is difficult to argue that it is a mark of authoritarianism.

Web 2.0 (2010-2022)

During the 2010s, a new model emerged with the Web 2.0. In this model, the content is still produced manually but it is pushed to the user by a recommendation algorithm based on the user's profile. This model is typically examplified with social networks or marketplaces because these businesses could achieve enormous scale through network effects. But the model is also relevant to most apps on your smartphone. Although it will not be covered here, the smartphone has been instrumental to the success of the Web 2.0 model.

This model marked a paradigm shift that was primarily algorithmic. The collection of an increasing amount of user information allowed for increasingly granular profiling, already incorporating AI tools for classification. This profiling removed the dependency on keywords entered by the user during a search and enabled increasingly precise targeting for advertising. Unlike the search engine, the social network can proactively push advertising. The algorithmic revolution therefore resulted in both a qualitative and quantitative increase in advertising.

Again, the clients are not the users but the advertisers. And advertisers love the social media channel. Similar to supermarkets, advertisers just need to pay up to place their product at eye level in the middle of an aisle, but on social media the aisle comes to the consumer, not the other way around. Those who doubted, in 2010, that Facebook would successfully monetize its platform were mistaken: 14 years later, Meta generates $160 billion in advertising revenue per year.

Web 2.0 has unique characteristics in terms of influence vectors but is merely an evolution of its predecessor:

The social network algorithm is at the heart of the model. It is the revenue source for these platforms, but also the origin of many woes for users.

As in the case of Google's indexing algorithm, there can be manual interventions for defensive purposes by the platforms, such as when X penalizes all posts containing external links. But the main objective of a social network's algorithm is not to defend a position. It is instead to establish a feedback loop that pushes users to engage more with the content, which should lead them to produce more content themselves and, above all, more data, which is useful for improving engagement further and, more importantly, to better serve advertisers (qualitatively and quantitatively).

For social media platforms, it's not just about recommending the best content for a particular user (although research papers on the subject can be interesting, like the one from TikTok's parent company). It's primarily about maximizing the "brain time" users devote to the platform, obviously at the expense of any other activity, especially activities requiring prolonged concentration.

Platforms use all sorts of now well-documented methods to encourage users to stay engaged, notably by activating neurotransmitters like dopamine in our nervous systems. The algorithms and the application interfaces are specifically designed to trigger our reward/reinforcement mechanisms at a molecular level. This is what pushes us to constantly refresh the page, and to participate in the engagement economy by rewarding content produced by other users with a "like" or a share. Even if they do not have malicious intentions, content producers in turn adapt their methods to get the maximum out of the engagement economy – at least in terms of dopamine. This is the driver for sensationalist news coverage (X, LinkedIn, Facebook), the aggressive selection of content to share only what flatters the ego (Instagram, Facebook), or ultra-addictive video formats (TikTok, Instagram, YouTube, and even LinkedIn since January 2025).

Users who get hooked are victims of a persuasion technique that goes far beyond Cialdini's consistency principle. There is a methodical conquest of our brain time to subject us to advertising.

This customer, who believes himself free, and lives in innocence, is unaware that he is analyzed without being touched. He is classified, defined as a part of his entire city, his entire province, and his entire country. They know what he eats, what he drinks, what he smokes, and how he pays. They ponder his desires. In Hamburg or Nuremberg, someone may have plotted curves representing the exploitation of his smallest quirks, his slightest needs. He would see himself confounded – he who feels he lives so personally, so intimately – by the number of other personalities - thousands - who prefer the same liqueur, the same fabric as him. For they know more things there about his own country than he knows himself. They know the mechanism of his own existence better than he does, what he needs to live, and what he needs to have a little fun in his life. They know his vanity, and that he dreams of luxury items, and that he finds them too expensive.

They will make what he needs, the apple champagne, the perfumes drawn from everything. The customer does not know how many chemists are thinking of him. They will manufacture exactly what must satisfy, at once, his wallet, his desires, his habits, and they will create for him something of average perfection. It is through servile obedience to his complex desire that they will ensnare him.

Une conquête méthodique (A Methodical Conquest), Paul Valéry, 1897

Servile obedience to the user's complex desire is also the origin of filter bubbles. By flattering our egos, social networks cause the dimensions of our individual universes to collapse and make us docile participants in their information marketplace. Filter bubbles form naturally but constitute levers for networks of accounts working in concert, often operated by the same group, to better influence their audience. For example, bots from Russia, Saudi Arabia, or United Arab Emirates.

This is all the more problematic when the user's complex desire can be shared with advertisers, who can then directly target certain groups in pursuit of objectives exceeding mere commercial stakes. This is how Cambridge Analytica used the information of millions of Facebook users to influence elections in Trinidad and Tobago (2010), and in the United States (Ted Cruz and Donald Trump in 2016). It is also suspected of having influenced the vote in favor of Brexit in 2016.

Large Language Models (LLMs) (2022-2024)

Few would dispute that OpenAI's public release of ChatGPT in November 2022 marked a pivotal moment. It didn't change the overall power structure of information technology – GAFAM largely maintains control in the West – but it fundamentally transformed how bulimic consumers of information are fed. We've entered the era of chatbots built on large language models and, while this initial iteration of the new model evolved rapidly, its significance was profound. This new model represents a radical departure from the past and constitutes the core architecture for the future digital information ecosystem.

In this original LLM-based schema, consumers of information are back in the driver's seat : they actively request content on a specific topic. But instead of keywords, their input (or 'prompt') takes the form of a sentence, a paragraph, or even a multi-page text. More importantly, this time, the content is not directly sourced from human creators but is generated on demand by an artificial intelligence model previously trained on an immense amount of text. Several trillion words for models trained in 2024. All the information from the training corpus is then compressed into a few billion parameters that computer programs can use as scaffolding for a noisy database of all knowledge shared on the Web. Even though using an LLM as a knowledge base is probably not a good idea – for the reasons explained in Part One – it is still the primary use case for the vast majority of users today.

The new information value chain is much more complex than in historical schemas. It is probably too complex to describe exhaustively here. But if we want to cover all the vectors of influence in this new system, we need to explain how these models are trained and lay out the lifecycle of a user's prompt.

• Building a Model

The main ingredients of any large language model are, on the one hand, state-of-the-art infrastructure and, on the other, a phenomenal amount of data.

The infrastructure runs on semiconductors that are marvels of engineering; working at the nanometer scale, capable of performing several trillion operations per second, and networked in the hundreds of thousands in giant data centers. The energy to power these machines, despite their small size, is obviously colossal, and the amount of water needed to prevent overheating is enormous. Everyone could theoretically train a very small model on their personal machine, but that's nothing like training super-powerful models that must then be put into production to serve millions of users. The know-how of engineers who can orchestrate all this, both from a software and hardware perspective, is extremely sought after.

We have already touched on data when addressing common misconceptions about annotation work. However, data stemming from annotation work is not the primary source for training ; it is instead used at a later stage of training. The first step, unsupervised learning, consists of feeding the model an astronomical amount of data mainly sourced from the open internet. Last year, HuggingFace published FineWeb, a dataset representing 15 trillion tokens, and documented the various production steps, from sourcing to processing to make it usable. You can get an idea of what such a dataset looks like by exploring it here.

From this data, the LLM extracts statistical links between words. For example, how they are organized in a sentence or how they are combined. This is how an LLM learns the vocabulary and grammar of any language. It also learns how words are statistically associated with each other, those that often follow one another and those that have no business being together. This is how an LLM builds its knowledge base.

An LLM can, without a doubt, apply a method explained in a textbook but it does not understand concepts; it merely provides probabilities that a word follows another word based on what humans have previously written . This is true – and even more blatant – for other types of generative AI, as in the case of image generation models, which produce robots with noses and pectoral muscles. There is no reason for a robot to have a nose. Let alone pecs, since these muscles are an evolutionary response to a mechanical constraint that robots do not have. Yet humans have often imagined humanoid robots in their image, including a nose and pecs. And the AI model, not understanding the concept of a nose or a pectoral muscle, generates images of robots replicating human biases. Is it necessary to explain that the AI model does not understand the concept of a breast?

Since the data for unsupervised training comes from the Web, there is a similar vector of influence to those in the previous systems: a content author can try to persuade their audience, and the model can "learn" to associate words together in the same way.

At this stage, we have only scratched the surface of training-data-related influence vectors. Base models are hardly usable after the initial training phase. Before putting an LLM in the hands of a user, it must undergo other more-or-less supervised training steps (mid- and post-training). This involves other algorithms and more data, this time created specifically to enable interactions between the model and the user. This is where the click workers come in. If you are sometimes surprised by recurring terms in LLM responses that don't seem natural to you (e.g delve¹), it is often because the reinforcement data was produced by people who were not native speakers, and their language biases were introduced at a later stage of training.

The ChatGPT revolution in 2022 corresponded to the successful scaling of a specific reinforcement technique that OpenAI had bet everything on: Reinforcement Learning with Human Feedback (RLHF). While it is now losing momentum compared to newer techniques², this technique proved remarkably effective for the first generations of consumer-facing LLMs. This reinforcement technique also aligns the model with certain values by teaching it to respond in a certain way to specific requests, or even not to respond at all. RLHF is the origin of the now-familiar phrase "Sorry, as a language model I can't...". Some see this type of intervention as a necessary counterbalance to the influence vectors from unsupervised training data. Others see it as censorship. Without taking sides in this debate, one can simply observe that these reinforcement phases affect the system's entropy (i.e. fewer surprises), which is striking when comparing the base and "instruction-tuned" versions of open-source models.

As Midjourney's CEO pointed out recently, "naive" reinforcement learning has a magnifying effect on biases present in the training data so overly relying on algorithms for post-training is problematic. In fact, humans play a critical role in counterbalancing baises at this stage. But when post-training aligns models with certain values that are not universally shared, the developer is exposed to controversies. This is what happened to Google with its image generation model in early 2024. The model had been reinforced so much to promote diversity that it under-represented white people. And that's an understatement.

The debate is ultimately the same as for search engine or social network moderation, except that, in the case of large language models, the “safety" rules rarely stem from a democratic process. In the United States, some institutions act as if they have the responsibility of making their models simulate values by prohibiting answers to certain questions. While the intention may seem laudable at first glance, it is no different from the Chinese Communist Party's policy of ensuring that publicly accessible models "promote socialist values" (literally, it’s in the law).

There are several techniques to counter these refusal or censorship biases. The simplest is to identify the "direction" of the moderation vector, and then disabling this direction. The moderation direction tends to be orthogonal to all others (see 4.2 of the AllenAI paper²), meaning its removal does not impact the model's other capabilities. This technique, called abliteration, is relatively accessible as it does not require retraining the model. Another related technique involves defining a control vector yourself to guide the model in any direction, which can be applied in the direction opposite to censorship.

In any case, to implement these techniques, one must have access to the model; its parameters and the details of its architecture. Open source or open weight models therefore guarantee some protection against biases intentionally introduced by the developers themselves. Paradoxically, the Chinese are the most inclined to open up models. It is because they benefit the most from the soft power associated with open AI. And so, contrary to a widespread belief, Chinese models allow more leeway around moderation biases. But there is no silver bullet. If all moderation is disabled, it becomes much harder to counter influence vectors present in the unsupervised training data.

• Lifecycle of a Prompt

Abliteration and control vectors introduced, for the first time, an important distinction between an AI tool and an AI machine. The AI tool is the model in its most transparent form : code that anyone can attempt to run, to break down and to rebuild at will, to retrain, or to tweak to change its behavior. The tool allows users to measure the expression of all biases stemming from the training phases, and to discard the model if they consider biases unacceptable.

With an AI machine, you essentially bid farewell to your prompt the moment you submit it, trusting it to a system you don't control. Few realize that prompts do not actually fare well in the machine. But you give up your claim to oversight over what happens to those words once they cross the interface (Web or API). Those words are not yours anymore. And importantly, the model's response, useful or not, generally doesn't grant you ownership either—pay close attention to the terms regarding usage and rights if you plan to use this data for anything else.

So what happens between the moment you hit 'Enter' and the moment a list of bullet points appears on your screen?

First, your prompt goes through an initial moderation filter. Sometimes it is as simple as keyword matching but, most of the time, the filter involves an initial AI model intended for classification. Early Chinese LLMs via API were not very good at concealing this classification, which triggered an alert that stopped the application. If you use a Chinese API, your prompt will be subject to this algorithmic processing, in compliance with the 2023 law on consumer-facing LLMs. But this practice is not exclusive to the Chinese, it is widespread. OpenAI uses classification models on all your prompts and, if the classifier concludes you are not complying with the terms of use, the chatbot stops with an automated response and, in some cases, you can receive a warning email. Anthropic even boasts about using classifiers to determine if the user or the model itself has problematic intent.

The filtering constitutes moderation and thus, a vector of influence. But more critically it flags the user and triggers an automated reporting process. In the above example, it results in an email from OpenAI but could very well take another form and reach a different recipient. These dystopian tools are all the more worrying as we no longer limit ourselves to sending keywords through the interface: we now write entire sentences or even paragraphs to allow a model to interpret our requests correctly; we share documents containing personal information. Seeing AI giants grow closer to governments does not seem like a desirable evolution, but it seems inevitable given the geopolitical stakes of AI.

Not getting bounced at classification stage does not mean that the model's doors are wide open to the prompt. It is common practice to decorate it before letting it in, by attaching additional information so the model processes it in a certain way. This can take the form of system prompts defining rules or just appending text to the prompt to guide the model's response (e.g., reminding it to avoid plagiarism or sexual content). Often, the added sentence includes an instruction not to reveal these additions. None of this is officially documented by the providers.

It's not just about misinforming clients about the service provided, but actively preventing them from learning that the models process altered versions of their requests. Today, no one seems offended by these commercial practices. Instead, Anthropic, the specialist in this matter, wins contracts with the European Parliament and various governments by communicating about its transparency and ethics. In this case, the vector of influence extends beyond the simple relationship between platform and user.

The next step is tokenization. This is where words are turned into tokens according to a conversion map determined algorithmically. While tokenization is far from the sexiest part of a generative AI model, it’s important to understand what happens there because it affects everything that follows.

Taking a step back: our languages encode the reality we perceive using certain vocabularies, grammars, as well as certain styles, tones, and turns of phrase to communicate ideas. Our languages are already an imperfect encoding; it's more about determining "greatest common denominators" to allow all members of a group to share a broadly similar perception of reality. So they can understand each other, communicate and collaborate. The tokenization process is no different in its objectives except that there is no grammar, no turn of phrase, no tone, just a vocabulary made of portions of words, which sometimes get grouped together to represent word strings. The tokenizer's vocabulary must cover all the languages that the AI model will handle. This common vocabulary for the entire world and the promises to make humanity god-like thanks to AI constitute an amusing nod to the myth of the Tower of Babel. That was a cautionary tale though: the ascent went wrong and humanity found itself divided; sent back to their flat world.

There are several techniques to create a tokenizer's vocabulary, but it is not a sector where scientific research seems very advanced. The Byte Pair Encoding (BPE) method³ is the norm nowadays. There are some broadly accepted principles, but their foundations do not feel extremely robust. They could very well be abandoned overnight if someone’s experiments with tokenizers prove conclusive. This kind of shift similar happened with the Chinchilla scaling laws, which had the force of axioms but were abandoned in a few months by the entire industry. A recently published article, demonstrating that a new method can increase performance and reduce inference cost, suggests that a shift could happen very soon.

The creation of the tokenizer inherently introduces biases but, so far, no one seems to have succeeded in leveraging the tokenizer to make it a real vector of influence. Instead, the tokenizer is something everyone endures, and it is blamed for some difficulties models have in performing certain tasks like processing multi-digit numbers.

In the search for influence vectors, the tokenizer can be useful because it can reveal information about the training data. Indeed, if character strings or even word strings are encountered very often by the algorithm defining the vocabulary, then the algorithm can group them into a single token. This optimization leads to far fewer calculations than if shorter tokens had to be processed consecutively. For the optimization to be effective, the vocabulary must be representative of the data that will then be used to train the model, so, if you can find tokens representing long character strings, it might mean that the training data contained many occurrences of these strings.

In 2023, Susan Zhang analyzed unusual tokens in the vocabulary of the tokenizer for the Chinese model Baichuan2. According to her, in the Chinese language, special tokens are easy to spot because few "normal" tokens have more than three characters. And the list of tokens with more than 3 characters in Baichuan2's vocabulary included "Epidemic Prevention," "Coronavirus Disease," "Committee," "Xi Jinping," "Coronavirus," "Nucleic Acid Amplification Tests," "New Coronavirus," "Wear a mask," "Communist Party," "People's Republic of China," "Communist Party of China," "General Secretary Xi Jinping," "Copyright belongs to the original author." In an extended version of the vocabulary published mid-September 2023, she identified tokens meaning "Guided by Xi Jinping Thought on Socialism with Chinese Characteristics for a New Era" or "On our journey to achieve beautiful ideals and big goals".

Once the prompt is converted into tokens, the model can, finally, spring into action. The vast majority of the models' own influence vectors were covered in the section on model training. One element not mentioned so far is the compute budget that the provider can allocate to processing the prompt.

Until a few months ago, this question mainly concerned parameter precision (quantization level, FP8, FP16/bf16…) which allowed cost savings at the expense of response quality. An analogy can be drawn with the difference between a high-definition image and a low-definition image: lower parameter precision requires less storage space and fewer calculations during processing, but the results are less precise. Arbitrages made on model precision were rarely communicated to users. With the advent of reasoning models, the compute budget has taken on a whole new importance because the more the model "thinks," the better the expected result. The provider can therefore, at this stage, influence how users' prompts will be processed.

Furthermore, when discussing control vectors to disable moderation, we indicated that this type of tool allows for guiding a model in a specific direction, which can also be implemented by the provider to censor or promote something ad hoc.

This raises serious questions about the business models of AI providers. Is the current token-based approach built to last? If not, where will they turn for revenue? Could there be incentives to pull the levers of influence available to them behind the veil of interfaces? These specific financial pressures may never fully materialize, because the ground is already shifting again in the digital value chain. The cards were dealt again before everyone truly had a chance to play their hand.

Let's wrap up the prompt lifecycle discussion before turning to the new schema. After exiting the main model, the response enters a post-processing phase. This involves preparing the output for the user and often includes a final screening by a classifier to catch harmful content. Anthropic details such a process, and OpenAI appears to use a similar late-stage check, given that moderation occasionally activates partway through generation.

AI Agents (2024-?)

In this new system, the chatbot doesn't disappear completely; it becomes an interface with a much more complex system. A first important evolution is that, upstream in the value chain, the data used to train the agent is increasingly synthetic, i.e., produced by an AI model. The use of synthetic data can be a decision made by developers when quality data doesn't exist and needs to be produced. But it is also, and primarily, the consequence of the proliferation of synthetic content on the internet.

Having a model produce data that will be used to train another model poses all sorts of problems regarding licensing and intellectual property protection. But above all, it enormously increases users' exposure to biases of all kinds. Especially if the AI agent is given the ability to search the internet to compile content, which is increasingly synthetic.

One question arising in the context of internet content inflation is whether our current indexing technologies can cope with the volume. Or, more precisely, how often would indexing algorithms need to be adapted to continue identifying the signal in the noise, when generative AI produces SEO-optimal information on demand? Google search sometimes seems to struggle, even though we are only at the dawn of the generative AI era. The "Dead Internet" theory, a scenario where finding relevant information is no longer possible because the noise is too great, might be a fantasy like the Y2K bug, but it's a risk that must be considered given the perceptible signs today.

Companies indexing the Web will always find temporary responses via algorithmic fixes, but we must question their place in this new value chain.

AI models cannot be trained on demand to incorporate the latest information. To account for fresh content, relevant information must first be searched for and, once retrieved, passed to the AI model for analysis. This process is called Retrieval-Augmented Generation (RAG). Search will therefore continue to play an important role. And if search becomes challenging due to content inflation, its value-add increases.

The issue is that the current business model of search engines collapses as soon as they stop sending results to individuals but send them instead to AI agents that are not influenced by advertising.

In a pessimistic scenario, the content of the ads would simply be adapted to influence AI agents (see prompt injection below). The most likely outcome, however, is that web-indexing businesses will shift to an API-based model. Search APIs are a little-known corner of the digital information ecosystem but it is already possible to search the internet programmatically. There is no advertising since there's no one to see it. So users have to pay ; it's relatively expensive and daily rate limits are typically low. Google is cheaper than Bing (Microsoft) but still costs $5 per 1000 queries, with a limit of 10,000 queries per day. Brave, Exa, Linkup, or Tavily are examples of alternatives to Google and Microsoft, sometimes up to 50% cheaper, but the indexing is notably worse.

The European Union talks a lot about the stakes of AI but seems blind to their absolute dependence on a very small number of Web indexing companies controlling prices and volumes. Building an independent indexing algorithm should be a priority because AI agents, without Web search, won't get far.

AI agent developers have only two options: creating their own web index or paying a third-party for this service and covering the costs by monetizing search traffic. OpenAI has a partnership with Microsoft (Bing) for Web search. In a recent interview, OpenAI’s CEO Sam Altman spoke about monetization beyond the simple ChatGPT subscription: "A lot of people use Deep Research [OpenAI's agentic system doing web search] for e-commerce, for example, and is there a way that we could come up with some sort of new model, which is we’re never going to take money to change placement or whatever, but if you buy something through Deep Research that you found, we’re going to charge like a 2% affiliate fee or something. That would be cool, I’d have no problem with that. And maybe there’s a tasteful way we can do ads, but I don’t know. I kind of just don’t like ads that much." It is highly likely that OpenAI will eventually build their own Web index because they already have all the building blocks internally to do so; only contractual commitments might prevent them.

Whether or not they stick to the promise of never taking money to modify placement is not very important in practice. If an AI agent processes the list, the display order is not as decisive as for human readers. There are many other factors that can influence what the model considers relevant. Besides the steering of the agent operator (see previous section), the content itself can contain more or less visible instructions that can influence, or even completely derail the model reading them.

This vulnerability, commonly called "prompt injection", concerns not only documents from the open web but potentially any document that the users themselves pass to the model for analysis or summary. There is no known solution to prevent prompt injection today, and everything suggests there never will be. The best prompting experts always manage to get what they want out of LLMs. This incidentally leads to fascinating artistic projects⁴.

It is becoming increasingly common to embed a hidden instruction in a document (by writing in white font on a white background), a Web page (in the HTML code), or even an entire website (in robots.txt or llms.txt files containing instructions for bots). One can hide a quasi-infinite amount of information in Unicode tags which, even if visible, could not be interpreted by humans. Or if they were, they would likely be interpreted as benign, for example as an emoji. This article covers the work of many hackers on Unicode tags.

Prompt injection can aim to influence the end-user but it can be malicious in many other ways. For example, to extract information from an agent that has access to sensitive data. It is very difficult for an LLM to keep a secret, so if it has access to information, it should be considered a security vulnerability. An agent with internet access could be convinced to make a request to download an image from a URL, putting sensitive information in the request body so the server delivering the image can read the information. It could also be convinced to pass sentive information as an argument to the API of a third-party service. Or the agent could hide all this sensitive information in invisible Unicode tags that could be interpreted once the document is published by the user. It is still early days but few realize yet the extent of the risks of AI agents for data security.

On top of prompt injections, there are gaping security flaws left behind by application developers who rely too much on code assistants without reviewing, or sometimes without even understanding the lines of code.

The great danger of the AI agent, which we see expressed particularly with code agents today (see Cursor, Windsurf, or more artisanal systems like MCP), is that we are leaving the information paradigm and entering the action paradigm. We then delegate many decisions to entities that cannot be held responsible.

Many misconceptions about the current effectiveness of AI agents today come down to the unit used to measure their success rates. Pass@n is a common measure: a pass@5 of 80% means the agent succeeded at least once in 80% of the tests it was subjected to, having five attempts for each test. But what does a pass@5 score of 80% imply in real life?

Firstly, it implies that each test result must be verified. However, there are very few domains where verification can be automated. And if it cannot be automated, then the evaluation results might be subjective.

Secondly, it is possible to achieve a pass@5 of 100% with only a 20% success rate. A pass@5 of 80% implies a failure probability per attempt that would be absolutely unacceptable from a human worker. Critically, it implies that every single test result must be verified. Some argue that, because agents’ cost is so low, the number of attempts to pass a test does not matter. This argument is flawed if a [very expensive] human must verify the result after each attempt. To put failure and success probabilities in the context of the most advanced AI agents today, the recently published METR report was based on success rates of 50%.

This report established a “Moore’s law for agentic tasks" based on task duration. It is not entirely clear that task duration is so important. Outside laboratory settings, the essential criteria for properly evaluating an AI agent seem to be cost criteria.

First, the cost of verification: if verification costs nothing, the agent can indeed be asked to retry until it passes a test, even if the expected success rate is less than 1%. Conversely, if verification is extremely costly, the acceptable success rate might be well above 99%.

Then, the cost of failure: when talking about automation by AI agents, we mainly talk about automating communication or, more generally, information processing. An agent is integrated into a longer chain, and the information the agent must produce (if it succeeds in its task) will be used downstream. A failure breaks the communication chain, which can have an extremely high cost. This communication problem is also why multi-agent systems still fail spectacularly today.

Finally, there can be the opportunity cost for humans not performing certain tasks. We will expand on this idea in Part Three. The challenge is not technological but directly relates to our cognitive abilities. The challenge lies in preserving our ability to make informed choices, to exercise critical thinking, and actively participate in civic life, in a world that is increasingly shaped by algorithms and interests beyond our control.

Each stage of the recent evolution of the information value chain has introduced new vectors of influence, whether algorithmic biases, sophisticated persuasion techniques, filter bubbles, opaque moderation, or vulnerabilities of all kinds.

From a societal perspective, this evolution is also accompanied by a transfer of value-add. Traditionally, value-add resided with content producers but, with the emergence of search engines, platforms, and now generative AIs, a growing share of the value is captured by the engineers who design and control these technologies. They are the ones who master the algorithms, infrastructures, and data – all the key elements of the new information economy.

---