Soft Control: Huxley’s Algorithm and the New Surveillance Playbook

Aldous Huxley’s Brave New World appeared in 1932 and imagined a society governed through pleasure, distraction and engineered contentment instead of open coercion. In Brave New World Revisited (1958) he argued that propaganda and persuasion techniques were already making that kind of control more plausible. Neil Postman’s 1985 book Amusing Ourselves to Death later sharpened the contrast: Orwell feared what is forced on us, Huxley feared what we will welcome.

This file uses Huxley as a lens, not as evidence. The lens asks a useful question, which is whether systems built for engagement can shape behaviour at scale without anyone issuing an order. The answer has to come from documented mechanisms, not from the literary comparison.

One of the few direct tests is Facebook’s emotional contagion study, published in the Proceedings of the National Academy of Sciences in June 2014 by Adam Kramer, Jamie Guillory and Jeffrey Hancock. It covered 689,003 users and changed the emotional content of the posts their feeds displayed. When positive posts were reduced, people produced fewer positive and more negative posts of their own, and the reverse when negative posts were reduced.

Two details matter. The measured effects were small, which is why the study is evidence that feeds can nudge behaviour but not that they control it. And the journal later published an editorial expression of concern about the research, which drew wide criticism because users were not asked for informed consent. The study shows both the mechanism and the governance gap around it.

Interface design can steer choices without a single banned word. In September 2022 the US Federal Trade Commission published the staff report Bringing Dark Patterns to Light, drawn from a workshop held in April 2021. It found deceptive design across e-commerce, cookie consent banners, children’s apps and subscription sales, and grouped the most common tactics into four types: disguised advertising, making cancellation hard, burying key terms and junk fees, and tricking people into sharing data.

The European Union’s Digital Services Act goes further by prohibiting online platforms from designing interfaces that deceive or manipulate users or materially distort their ability to make free and informed choices. Regulators have therefore already accepted the narrow form of the soft-control claim: design can be unlawful manipulation. They have not accepted the broad form, that a single system governs society.

The documented effects are real but bounded. Platforms optimise for attention, retention and advertising revenue, and much of what looks like coordinated steering is the side effect of commercial incentives. Users also adapt: they switch platforms, learn how ranking works and form their own networks. A claim that algorithms determine political outcomes with precision is not supported by the experiments and enforcement actions cited here.