The agentic web is arriving, and websites are already redesigning for bots

For a quarter century, the web was built for people with the occasional bot on the side. That ratio is flipping.

In a feature for MIT Technology Review, Grace Huckins traces the idea back to 2001, when Tim Berners-Lee and two colleagues described a Semantic Web navigated by software agents that would, for example, book a parent's physical therapy appointment. It never took off, partly because it required site owners to tag everything by hand.

What changed

LLMs don't need tags. They can read a clinic's site, spot the address and services, and even click through screenshots of pages. And the agents are here: Meta's Muse and OpenAI's Dots are setting download records and getting access to people's email and bank accounts, and Anthropic opened a research preview of its Claude Duet agent in October. Sites are adapting. Time publishes agent-friendly versions of articles and sells ads aimed at agents, and companies such as Etsy and Expedia have built integrations with the big AI firms.

Two ways to build it

One approach teaches agents to use browsers the way humans do. McGill professor Siva Reddy's lab started there, then switched to the other approach after finding that web pages full of moving parts trip agents up: give them text and tool interfaces instead. Connectors that let an assistant work inside Gmail are an early version of that.

The stakes

The feature lays out two futures. One is an open, customizable web that's far easier to use. The other funnels most of the internet through a few dominant platforms' agents, with today's sites and creators struggling to survive. Mozilla's Ajit Varma compares the shift to getting a mobile phone.

For developers, the practical takeaway is that your next important user may not have eyes. Clean APIs, structured data and clear terms for automated access are becoming product decisions, not afterthoughts.

Scoreboard update: AI +1. The web starts redesigning itself for agents.

Stay undead.