Skip to main content

The unseen layer of the agentic web and MCP

One of the biggest large language model vendors is about to be offered as public shares sold on the stock exchange in an initial public offering. Anthropic’s industry emerged from research conducted at Google in 2017 on the generative pre-trained transformer architecture. (GPT is used to create the large language models that now proliferate broadly across the world technology sector.) Simply stated, LLMs generate predictive text, images or code for the future based on things said or indexedd in the past. There is an often quoted idiom that history doesn’t repeat, but it often rhymes with the past. Recently many people have been thinking about the emergence of the LLM industry in light of all the private companies hoping to sell their shares in the near future. I've been thinking and discussing a lot about one of the concepts Anthropic pioneered as a now-standardized web architecture called MCP, model context protocol. (MCP can be thought of like a handshake between agentic systems for summarizing and passing context between LLM models that process collaboratively.) Yesterday at the Android Developer Summit, I attended lectures on a new variant of the MCP approach called Android MCP. This rhymes so much with my past experience and my thoughts of late that I think it worth sharing for other developers.

I come from the open source community, having worked on the open source browser Firefox and working off of a broad set of standards that Mozilla, Yahoo, Microsoft and Google have shared across the last 30 years of the web’s history. I wrote a book about this amazing open innovation culture in The Momentum of Openness, specifically around the open-sourcing of the Netscape browser and the subsequent boom of the web browser and web publishing industry. To take a look forward it may be helpful to take a little look back over the three decades that got us to the generative pre-trained transformer and the LLM boom. Then it will be very easy to see why MCP sticks in my craw a little bit and how I’ve learned to embrace the concept over the past few months.

In 1995 I joined a news-wire called Jiji Tsushin. (時事通信社 commonly referred to as Jiji Press in the US.) Like the American companies Associated Press, Bloomberg and Reuters, our business model was business to business, B2B. We would author and distribute international news content that other newspapers and websites could publish to their own B2C end customers. Daily newspapers like Yomiuri, Asahi, Nikkei and Mainichi would have subscribers in Japan who would pay to get our news secondhand from their preferred marquee brand news outlet. Jiji covered all the international news that no local newspaper had the resources to cover directly. Our revenue was earned across all of the consumer-facing print and digital media publishers downstream. Jiji has thrived through the years on this business model. I was very proud to be a part of it.

Around 1996, during the surge of html-based web content publication boom, Jiji decided we should launch a web-hosted news portal as well. Because of the proliferation of web browsers and web portals, we could expand our distribution beyond print and broadcast and win speed-to-market of our cutting edge journalism. The news-wire businesses compete on getting the news to market faster than anyone else. (I remember my own articles and scoops as great personal adventures and triumphs.) Time to market was value for our direct customers and in turn their customers. In 1996-1997, Jiji's decision to have a B2C portal worked well in broadening Jiji’s ongoing reputation and influence beyond its legacy business of just providing general news through other news outlets.

But thinking of it just philosophically at the time, it seemed that going directly to our customers’ customers with our own web portal undercut their own value of their subscriber service of which Jiji was one component. Wasn’t this a bit too much openness? If the Japanese customers could get their international news directly from our portal, why would they pay their subscriptions to the newspapers? This concept on the preservation of privilege to the revenue stream is sometimes referred to as getting a “clip of the ticket”, or a small portion of a transaction charge already being made in the direct to customer relationship. But it turns out that Jiji’s B2B and B2C revenue streams and service models were profoundly beneficial through this transition. Jiji has stayed independent through the years for its cutting edge journalism and broad market coverage and analysis.

After the web-conversion of Jiji, I decided to get into web design and web services more broadly in the next step of my career and relocated to Japan to launch a search engine. After you have a proliferation of web content, you need a deft means of parsing and distilling it for secondary audiences. Many companies aggregated and distilled content from broad web indexes hosted by AOL, Yahoo, MSN, AltaVista and Excite in those content “boom” years of the World Wide Web that formed the internet of today. Google became an authoritative index powering the back-end web crawling and indexing for those portals. You could see Google's web search as the new analog for the new-wires of today. You may go to a news portal for the content they author. But the web-search they offer there brings in content beyond what that portal itself has to offer, like Jiji was to the newspapers and Japanese newspapers and news portals. 

Alphabet, the company that owns and operates Google, has moved a meta-layer above just indexing web content over 30 years. They iterated on the approaches of preceding technology (often with licenses for the underlying patents when they were proprietary) to improve the reach of their open-source and the broader benefit of web technologies to companies far beyond their own customers and are a power-house of innovation today. Google Translate, one of the services they launched later became an underpinning of how LLM technologies work today. (See Attention is All You Need, for the original source of the generative pre-trained transformer.) This in turn was broadly popularized by OpenAI’s GPT services, which in turn gave rise to Anthropic’s LLM Claude. But the underlying technology resulting in the large language models is built on the corpus of web content that preceded it. LLMs are like the “fairy rings” of yesterday’s internet. (A botanical concept of a ring of trees or other plants that form on the outside of a predecessor like a Redwood tree when burls forming on the trunks of the tree drop and grow in a circle around the predecessor.)

 

Timeline of the Musseum of the Human Web

A few months ago I visited an installation created by the Computer History Museum in collaboration with Parallel.ai. While perusing the Museum of the Human Web San Francisco exhibits, I had a chance to discuss its inspiration with museum curator Lukas Levert and to learn about the web-index that Parallel creates and distributes for its downstream customers. Parallel offers a utility for LLM tools and agentic services to quickly read a web index of all available content without their customers having to build and host their own web index. Do you hear the rhyme with Jiji new-wire B2B and early Google B2B models? Lukas and the Parallel team created this museum pop-up to introduce their thinking about how the human-authored web gave rise to a parallel model of search engines for agentic agents that are emerging as complementary tools for many people these days in their work and creative lives. Agents now author content and experiences on the web based on the multi-modal LLM and generative tools informed by all that happened across the web in the 30 years prior. 

Lukas and I talked at length about the human web being one created for the presentation layer of html that we came to know and love over the last 30 years through web browsers. Seeing that the human web is gradually being re-authored and re-interpreted with machine learning tools for completely new presentation layers and devices. So there needs to be a new kind of web discovery, now tailored for agents without eyes and for transactional web services where the human is the beneficiary, but not the active searcher. The web has been serving communities beyond ears and eyes for a long time. There is a whole working group about building accessibility tools in operating systems, apps and web browsers that is parallel to everything web designers do in html. (Learn more about some of this work here https://www.w3.org/WAI/standards-guidelines/wcag/.) During my time at Mozilla, I became fascinated with accessibility tools in Firefox and still use several of them daily. Underlying the web we see, hear and touch there is an architecture in http that is not graphical nor auditory. If the user or user’s agent doesn’t see with eyes or hear with ears, then it is a waste of resources to build a graphical user interface (GUI) to present to them. So accessibility tools in web browsers strip off the presentation layer and look underneath the html and CSS at the core utility the service or web host is providing, much like the search engines of 30 years ago in how they analyzed web links and content behind the html rendering layer.

To see the analogs I see here, imagine the idea of stripping away the Japanese news site top-pages to see the underlying articles that Jiji, Reuters and Associated Press provides underneath them. Remember that it is typically the top layer that end customers pay for that only trickles down to the news-services licensed by the newspapers and news portals thereafter. If you strip off the paid layer, then marquee web hosts don’t generate their revenue. The customer gets the news they want. But the conduit gets cut out of the transaction. This could seem unfair from the traditional economics underpinning the legacy web. 

But it just so happens that the web itself is undergoing a transition now with more people running their own “aggregation layer” atop the web, through their own agentic personalization tools, such as asking an LLM to summarize a news headline originally published on the New York Times. Parallel makes aggregation and quick web search for agentic systems very easy and efficient. But, agents often don’t have wallets. Its the eye/ear wearers that typically do. The transaction model of web economics is geared to those eyes and ears, as is the whole web design and content aggregation business model. So what happens when the agentic view of the web becomes the predominant means of web interaction?

You may ask why I’m getting so inspired around this 30 year sine wave rhyme with the past. What’s all this got to do with Google and Anthropic? (Beyond the financing and seemingly unstoppable proliferation of LLM web aggregation and parsing tools… ) It’s a new open source standard that Anthropic put forward in 2024 that is now widely adopted by web publishers and services called MCP, model-context-protocol. MCP is becoming as ubiquitous as banner ads were on web pages in the early 1990s. Every website will likely have one in the near future as a presentation-layer to the agentic assembled web. If a site has an MCP server, a non-human web-client or tool can request the underlying content of the web page without wasting time serving the html presentation user interface. You can think of it like going on a blind date with a person who is blind. Wearing a tuxedo or an evening gown isn’t likely going to be relevant. You have to present to the senses that are present. MCP is an approach to respond to any inbound agentic request with just the key content or messages you want the agentic system to present to their respective user or user interface (which may not be visible or auditory in turn). So metaphorically, you have to present what you have with no “window dressing.” 

Lukas and I had a long exchange on this concept later at the We Are Developers World Congress thereafter. As a person who has focused 30 years on building web content and web services for people, it seems demeaning in a way to just “give” the content away without so much as a glance or a handshake. We are perhaps too accustomed to our human and graphical user interfaces (GUI) to be comfortable just revealing all the source content to a robotic web parser. This reminded me of a product initiative and code tool that emerged around 30 years ago as well called robots.txt. This was a means for a web content host to exclude access to its web pages from non-human view and subsequent promotion in their search indexes. Meaning if you didn’t want AltaVista, Inktomi or Google web crawlers to find privileged pages like paywalled content in their indexes, you could instruct them to ignore it. Robots.txt is the inverse of MCP in my view. MCP is and invitation for agentic bots to “please peek under the hood here and show this to your user if relevant to their context." Content that was blocked by a robots.txt file on a domain URL is typically meant only for human consumption. But in today’s web, it is likely that if you don’t reveal your site’s content to an agentic crawler or web crawler, nobody will ever see it unless you entice or direct them to. There is no reason to put something on the web that you intend to stay hidden after all. 

In a discussion with Ian Bicking (an engineer who had built several features of Firefox) recently, we touched on this concept of the agentic crawled web and web accessibility. Paywalls, advertising and web interfaces, even in web accessibility contexts of screen-readers without the html GUI layer, are designed for one-to-one sessions via a browser, app or other web client with a human directly on the other end. Ian joked that we are now focusing on making the web easy to index and experience for bots in ways that sometimes leave the humans behind. News publishers make their content easy to parse and present in the agentic layer resulting in a GPT summary having full access to content via the publisher’s RSS feed or MCP server that is not then also revealed to the human who comes to the publisher directly. Instead a human user may see only a single headline and an invitation to prove humanness or pay before viewing the content their agent referred them to.

Yesterday’s marketing effort to conduct search-engine optimization (SEO) to drive “eyeballs”, as humans were called in some circles, is now being replaced by bypassing the visual layer entirely with MCP optimization. The 10-blue-links of yester-decade search results page is now replaced with an “AI Summary” at the top of many web clients and apps. MCP is the way to say “Tell your agent to read my MCP and then relate that to your human.” This has the implication that the human relation of the former html web is now autonomous, invisible and seemingly impersonal. (Coming from the perspective of someone who started a career in the news content industry 30 years prior.)

To drive the point further in this explication, I’d like to summarize a presentation made by Richard Seroter, the chief evangelist of the Google Cloud Platform, from a recent Android Bay Area Developer Festival. His presentation (summarized on his blog here which can be heard here) focused on a new emerging standard Google is working on called agent-to-UI, abbreviated A2UI.  Adjacent to Google’s work on Agent-to-agent (A2A) protocol and MCP, which govern how agents navigate the web and manage data exchanges, A2UI is similar to the web content accessibility context in the sense that we don’t know exactly what user interface or GUI would be most appropriate for every user or every context a user might be in. (We don’t want to read a newspaper while driving for instance, it’s a mismatch of content to context. An audio representation played through a speaker might be preferable in the context.) So in allowing your presentation layer or GUI to be created ad hoc based on the context and need of the end user is the most appropriate way to make sure your service or content reaches the broadest audience of agentic web representation. 

Stepping back to the news content publisher perspective I started from, you can interpret how the New York Times of around the 2006 proliferation of “Web 2.0” context might have reacted. First they had to have a web portal in 1996. Then a Facebook page and an RSS feed, bypassing that portal front door, in 2006 and now just an MCP server in 2026 with someone/something else authoring the top presentation layer with A2UI and AI summaries. As our future agentic AI readers present more and more of the surface layer of the web to a seemingly far-removed reader, how do we shift our roles as builders of the web? Are we always going to be further and further dis-intermediated from the discussion with our human web users? They seem to no longer be “our readers”, nor are we “their hosts”. Web designers now cater to a smaller portion of the overall human experience that is relating to their content sources in this bot-intermediated process. 

I’ve experimented with many different approaches to web content syndication and aggregation myself. As a web developer and site host, I published my first web page in 1998 after leaving Jiji and started building and working on other people’s websites and apps. I tend to think of content creation and distribution from a visibility and discoverability angle, like the traditional SEO perspective of web crawlers, ads or marketplace promotions that garner engagement in the traditional internet market models. At yesterday’s Android Developer Summit, the story pivoted a bit further still for me. The agentic web presentation layers (MCP, A2A, A2UI) and the parallel.ai machine-level view of the web has seemed very abstract and not yet enticing from the perspective of an author and web publisher that I identify as. During the sessions on Android MCP, and ML Kit discussions I finally saw a means of engaging in driving more discovery and engagement that counters the sense of disintermediation I’d sensed from the agentic layer. Google’s engineers offered a new twist on MCP specifically applied to mobile applications. The concept of MCP in an operating system application publicizes its skills the way that a web-based MCP service might publicize its content or utilities. This means that the specific processes my Android users engage with in their daily lives, might be directly enhanced if I demonstrate certain app functionalities that contribute to their daily activities in complementary ways. Instead of being dis-intermediated in the agentic flow, I may have my app functions specifically called in an agentic discussion or UI flow if my tool or app is the best available to serve the direct user need! The Android MCP doesn't usurp my app's content or function into the agent, it refers-out to a skill I've advertised that I can serve in the user's direct experience. This in turn could result in more engagement directly in the agentic layer of the tools that Gemini and Android already offers on device without my application needing itself to have a direct agentic function built into my app. 

Though I’m not currently promoting my web content into agentic discovery, beyond what I publish here on Google’s hosted RSS reader, I can see my path as an Android developer using Android MCP to contribute more to my users' future engagement and benefit with the tools I support and promote in the Android ecosystem. What had started as a means for agents to engage on the web between automated loops and content indexing now has a direct tie into a way that I engage with my customers in the mobile app ecosystem.


Home





Comments

Popular posts from this blog

The Momentum of Openness - My Journey From Netscape User to Mozillian Contributor

(Update: Because this post is exceedingly long, I have decided to make it available as a printed book: Momentum of Openness  It will remain free to read here.) Insider story behind the cover image: Mozilla's mascot derived from the name of the Mosaic browser and the trademarked name of a large mythical beast from Japanese culture which would rise from the oceans to protect mankind against peril. You may see this mythical creature in Bugzilla, or featured in popular web browsers like Chrome when they are having issues addressing your requests. I like to call it "The Mozilla" because it serves as a protector of all that's good. When I first came to the headquarters of Mozilla, I had to get a picture being bitten by the Mozilla. You'll understand why we feel so affectionately about this symbolic icon as you read the story of my journey to web development below. Foreword Shepard Fairey's Dino Working at Mozilla has been a very educational experience over the past...

“Novel view synthesis” fine in photos or grief bots perhaps, but not for science bots

I've been reading about the opportunities and perils of chatbot technologies recently. This is in part spurred by books written by Karen Hao and Sarah Wynn-Williams about industry players in the sector and in-part inspired by the recent articles on psychological peril for young people engaging with chatbot apps discussed in recent news where bots allegedly prompt humans into self-harm after humans prompt them for advice. Separately, I have also been exploring approaches to capture 3 dimensional holograms called Gaussian Splats. Gaussian Splat synthesis does not use neural network stable diffusion. The two approaches seem metaphorically similar though. One is generative, one subtractive. One helps you see the real world with greater clarity, the other can be used to create fictional images. So I've been thinking about this boundary of truth enhancing and truth abstracting. My views aren't so much about the software approaches themselves, but rather what people can and ten...

Far-seeing Devices for Accessibility

The German word for TV is Fernseher, meaning far-seer. I often think about that concept of the fixture of our living rooms which allows us to teleport to perspectives of other places far away. A mode of communion with others, distraction, learning. We are societally connected across the world like never before. We tend to live our lives situationally in our local communities, then at some point in our evenings we teleport our awareness into the lives of others for the snippet of time that came to be known as prime time . This slot of our societal calendars is reputed to have the broadest attention span of collective conscious focus. It came to have that moniker because of marketers seeking to have some time during the hour of evening news or entertainment that would give their messages the broadest appeal to the space-portal's "share of voice" in this communal time of focus. When terrestrial TV fragmented into multi-platform and multi-screen surface areas along with the p...