As someone involved in the API standards space for a long time, I’ve always been interested in how software talks to software. One of the most exciting things to me about AI has been the possibility of automating key aspects of it, like making it possible to build powerful software agents that talk with the world of APIs around us.
Now, Model Context Protocol (MCP) has emerged as the AI-native standard for continuing on the path started by previous standards like REST, SOAP and XML-RPC. Introduced late last year by Anthropic, MCP is an open standard that simplifies connecting data and tools to LLMs, obviating the time-consuming need to create specific integrations, connectors or prompts for each database, tool or cloud service.
As agentic systems become increasingly complex and developers make more enterprise capabilities available as tools, having clear and agreed-upon rules is critical for the growth and success of autonomous workflows. It enables AI agents to make decisions about which tools to use, in the right order to accomplish a task. It’s a lot like the way REST helped enable efficient data exchange, flexibility, and scalability for APIs; MCP exhibits many similarities to what happened in the API world, and some important differences.
Your chat window as interaction point
In the API world, we thought about APIs as either general-purpose, loosely coupled services that could be reused for a wide variety of use cases, or “experience” APIs, which were designed to provide the specific functionality necessary to support a specific UX pattern. Netflix was often our go-to example for experience APIs — it coined the term — but just about any mobile app you use relies heavily on experience APIs.
Now we have a new point of interaction: your ChatGPT window or your Claude Desktop.
Here’s an example. I want to edit an Excel spreadsheet or Google Sheet. Each has an MCP server that expresses capabilities in a manner that the application understands. An Excel MCP, for example, could literally state something like: “I provide an internal state in the form of a spreadsheet, which is a matrix of rows and columns that are addressed by letters and numbers, respectively. Each has a numerical formula, and I provide a set of tools (or functions) you can use to modify this spreadsheet.”
How does this sort of experience get built? It’s not through the purpose-specific experience APIs of the mobile age, but it’s also not achieved by returning to the cognitive burden of the web services or even the microservices eras. It requires something new: a capabilities-oriented approach that exposes tools and the semantic context for helping LLMs understand how to use those tools to satisfy the user’s request. MCP is designed to step into this gap.
A developer could virtually create this MCP server by copying and pasting this information from an Excel user manual.
MCP enables you to ask an agent a question (“make me a spreadsheet of how much I can spend on groceries over the next year”) and, behind the scenes, the agent has MCP servers available to expose these capabilities. Because the agent can reason — it can break down problems into tasks and execute them iteratively — it can use MCP to bring in additional context via other tools and APIs, and generate a highly relevant, useful response.
Retrieval-augmented generation – and then some
So, the context is your prompt plus the context your agent has retrieved. Sound familiar? That’s because in many ways, MCP is an evolution of retrieval-augmented generation (RAG). But it takes it a big step further by providing a structured way of how APIs, services, tools or capabilities are described to an agent so it knows how to use them.
In simple terms, RAG feeds external information into a model at inference time. MCP provides a well-defined contract for which tools and capabilities are available, and, more importantly, a rich semantic description of how they can be used.
And this makes chat clients, which are sometimes derided as simple and elementary forms of AI — ”just chatbots,” extremely powerful.
Making models smart about services
The idea of making it possible for LLMs to use tools originated in the ReAct paper in late 2022 and was widely discussed in 2023. Meta released its Toolformer research paper in February 2023 and OpenAI followed with its preview release of function calling and JSON schema support in March, paving the footpaths introduced by frameworks like LangChain.
At the same time, work was done to try to modularize and package the use of tools. OpenAI’s ChatGPT plugins were a good example of this. Also introduced in March 2023 and built on the OpenAPI specification, these plugins helped ChatGPT access up-to-date information, run computations or use third-party services. The plug-in model was also adopted by Microsoft, and DataStax was proud to be a design partner for the GitHub Copilot plugin model; we delivered the Astra DB extension for GitHub Copilot at GitHub Universe in 2023.
Unfortunately, LLMs at the time had not yet become reasoning models, and so the utility of these plugins was limited. Agentic use cases were possible with these models, but in practice, they were somewhat clunky.
The release of Anthropic’s Claude Sonnet 3.5 model in June 2024 was when we really saw things get serious in this area, making it possible for companies like Cursor and others to leverage these capabilities in the developer space, and for Anthropic to increasingly promote general-purpose agentic use cases for its models, leading the introduction of MCP as a standard way of packaging and integrating tools into AI clients. This meant that developers could build MCP servers that delivered an out-of-the-box integration of their apps and services for agents to easily use.
You can get MCP wrong
With APIs, we learned that API design matters. Great APIs, like those from Stripe or Twilio, were designed for the developer. With MCP, design matters too. But who are we authoring for?
You’re not authoring for a human, you’re authoring for a model that will pay close attention to every word you write. And it’s not just design, it’s the operationalization of MCP that is also important, and another point of parallelism with the world of APIs. As we used to say at Apigee, there are good APIs and bad APIs. If your backend descriptions are domain-centric — as opposed to business or end-user centric — integration, adoption and developers’ overall ability to use your APIs will be impaired. A similar issue can arise with MCP. An AI might not recognize or use an MCP server’s tools if its description isn’t clear, action-oriented or AI friendly.
A final thing to note, which in many ways is very new to the AI world, is the fact that every action is “on the meter.” In the LLM world, everything turns into tokens, and tokens are dollars, as NVIDIA CEO Jensen Huang reminded us in his NVIDIA GTC keynote this year. So, AI-native apps — and by extension the MCP servers that those apps connect to — need to pay attention to token optimization techniques necessary for cost optimization.
There’s also a question of resource optimization outside of the token/GPU space. An overly enthusiastic agent could melt down your enterprise backend service by making multiple calls to it to retrieve the data it needs. Your MCP server needs to keep this in mind and either reduce the burden through techniques like caching or advise the agent as to the costs associated with leveraging the MCP’s tools in its reasoning processes, and suggest that it act accordingly.
Let software speak
As I wrote earlier, agentic AI has the potential to completely transform business processes. Unleashing the full automation and decision-making power agents can inject into enterprise architectures — and ensuring accurate, relevant results — requires the ability for them to call external functions, APIs, and tools.
While MCP isn’t the only standard being developed right now to facilitate this (though it is the current front-runner), the important thing to understand is that facilitating software’s ability to talk to software is how we’ve progressed in all of the major technology waves we’ve experienced.
