0:02Welcome here for this session. My name is Leo. I'm a solution engineer at Cloudflare. I come from France and I work in Paris with a lot of startups and scale -ups, so I'm helping them to implement our products. And today I'm going to do a presentation about CodeMode, which is a way to implement MCP server, a more efficient way for token users. and also in the way the agent is going to behave. So I'm going to go to the technical and the theory, but I'm going to also show you a real-life example of one of my customers in France who implemented it for production services. And I hope you will be able also to use it for your application.

0:50Maybe before going through the presentation, who is using MCP within their organization, agent? Okay, so almost everybody. So I guess you are facing some issue.

1:03Who knows Codemode already? Or a few people. So that's great because I'm going to go back to the basics. Before explaining the theory behind Codemode, I'm just going to explain a bit how we were using agents at Cloudflare, how we were using MCP. One way to use MCP is, of course, through like... an agent coding platform can be open code it can be cloud code codex all of that and the way you do this every time you want to plug a new MCP for your agent to be able to interact with external system you need to plug a server and then the the right server is gonna have different tools so you can plug as many as you want it can be like our internal portal it can be the cloth API it can be the docks but in your use case it can be whatever system you want your coding agent to interact with.

1:55It can be GitHub, it can be anything.

2:00For non-technical people, you can also build some kind of chat, agentic or application. And this is something we released sometimes like Lawflare where we allowed non-technical people to interact with AI. And there was a way to connect MCP if the, I don't know, salespeople or marketing people wanted to interact with like the Wiki, Jira or whatever system. if they needed to. And we allow them to connect and plug MCP and different tools. But with something, we set up a limit in the number of MCP they were able to connect. And some of you might wonder why, some of you might know why we are putting a limit. It's because when you plug too many MCP in an LLM, each of them have a lot of tools and if you multiply the number of tools you are a bit context because every time you start a new session for this agent you put all the tools all the mcp and his context is kind of exploding so the agent is less relevant less efficient it's consuming also way more token for nothing if i just want to check like my last email or something in the wiki to do some research i don't have to plug the entire mcp catalog for my enterprise so this is why we put the limit.

3:17If not, you get like with context problem.

3:25This is even more relevant if you want to plug. So I showed an example if you want to plug a lot of system, but even if you want to plug only one system but which is really complex, like the Cloudflare API, you can have an issue because the Cloudflare API have more than 2,500 endpoints, which is a lot and if you want to create like a tool for each of this endpoint or each of this use case you end up with a lot a lot of tools um so one thing we did at the beginning we thought it was a good idea to divide um our mcp two different categories so we didn't have a super mcp with uh that your agent could interact with to do everything so we had different mcp for different use case that was great because it was working well each of them have like few tools but the issue is if your agent if you wanted your agent to be able to do everything on Cloudflare you are limited because you need to plug another one you need to unplug one to to select the tool he can access and it was too much so we needed a way to to find a strategy for the agent to be able to perform everything without providing too many tools and flooding the context And so this is not something that was made by Cloudflare.

4:46Actually, if you read some papers from Apple, they already did some research about it before us. But we realized, some of the engineers in our team realized that LLM are way better at writing code than writing tool code because, of course, they were trained on a lot of data, a lot of code base. So it's way more. for an LLM model to write a bit of code API calls than writing tool code which is something less natural for it so going in that direction we think okay maybe instead of giving a lot of tools to my model to my LLM I can just give him like an open API spec and I can ask him please write like API calls to target this endpoint so this is a bit the strategy we wanted to go in And so what would happen if I ask my super, in theory, agent to protect me against DDoS attack instead of going and calling tools?

5:47The agent is going to write code, fetch some API endpoints. So this is just for illustration, but this is a bit of the context. So he's going to say, okay, which API endpoints would I need to create rules to protect me against DDoS? He's going to get the endpoints, and once he gets the endpoints, he's going to be able the request because he knows the correct body to send to perform the action. So that's way different than the way we do today if you want to call tools. And that's what we wanted to implement. So how is it working in practice? So you have still the LLM, which is still interacting with an MCP. Why? Because an MCP is a really cool way to interact. It's universal. You can plug it really easily. And every LLM is compatible.

6:33You still have an MCP, but with only two tools, one which is search, the other one is execute. The search tool allows the model to get the endpoint, get the information he wants to perform the call. Then he writes the code and sends the code to the execute tool. So when the execute tool is called, what's happening is the MCP needs to execute that code, right? The execute tool. And to execute that code, Of course, it's untrusted because it was generated by DLLM, so we don't want our server to execute an arbitrary code that we don't perform any control on it. So we need a little sandbox, and I mean, we have something really ideal for that, which is called a dynamic isolate, which can allow you to trigger little workers.

7:21It's a little isolate that can allow you to execute JavaScript, and that's the way you can do and execute the code mode. So I fetched the endpoint API, my LLM write the code, he said to my server, this is the code I want to execute, the server get the code, execute it in the little sandbox, and then this is how we perform the action. So that's the theory behind code mode.

7:46And if you implement this for the entire API, you can see that it's a way more effective way than just doing a raw open API spec. And we typed it to... to be more token efficient, and that's the kind of number we can manage to get.

8:08And I'm just going to show you a little example. So this is my backup demo. I will try to do a live one, but with LLM, it can fail at any moment. So I created a Jira-like system. So it's to manage tickets for engineering. So it's really simple. You can perform only two tasks. there is two mode one classic MCP maybe it's too small I don't know if I can yeah so one is classic MCP and the other one is implemented with codemod so they have the same feature but the two different way to implement it so if I ask them something really simple which is like listing the five most recent tickets we might see that codemod is a bit overkill because is something really simple, just one tool call, listing tickets, I want to see the five last ones, results, and then my LLM can interpret the response.

9:00In code mode, I have to write code to execute the code, so that's maybe a bit overkill for only one single action, and you can see actually that in token consumption, it's consuming a bit more. So I'm not saying that you should use code mode in every situation. If it's just like simple task, you might not need it. But if we try with a more complex prompt, which is, if it's really long, I'm not gonna, I mean, you're gonna see it here, but like listing tickets, closing some tickets, stuff like this, we're gonna see that. So in MCP, I'm doing a lot of tool call. One by one, waiting for the response, then doing it. After that, it's done.

9:46It's a bit more longer in code mode because I don't remember which model I'm using, but it's taking a bit of time. But we should see in a few seconds that I generated a bit of JavaScript to perform the API call, as I said. The value is returned. Everything is done. And we should see that the token consumption is way more So for this complex task, having a lot of action to perform. lot of tool calls we see that we're 50 % more efficient in token usage so and this is for one single agent one single prompt but you can imagine that within an organization if you have like a lot of agents in parallel a lot of complex tasks it can scale and can save you a lot of token usage so that that's one of the main benefits of code mode uh token efficiency uh for i mean of course the cost of the token but also or the efficiency of the agent, how well he's going to perform to do the task.

10:53So it worked well. I can just skip this.

10:59And just one example. So I have a customer in France, which is like a health platform. Everybody in France who want to book an appointment with a doctor, they might use it. It's a complex workflow, like the one I showed you before, because you need to get the referral record of the other patients, you need to find the directory of all the practitioners within the region, then you have to find the slot, to boot the slot, so there is a complex workflow with different steps, different tool calling. So if you want an agent to be able to perform all of that, you would need to implement different MCPA within different systems. So that's where CodeMod can be useful, because instead of building different MCPA, different tools and then chaining everything together and maybe having a more dumb agent which is a bit lost sometimes with all the tools he have he can just get the api and and call them uh sequentially so finding uh the referral checking every calendar and then booking uh for for the agent so that's a real life use case that was implemented um yeah and maybe I'm just going to show you a quick example of code to what it looks like if you want to implement it yourself.

12:20So basically, we have an SDK that can allow you to deploy MCP on Cloudflare. So you can deploy an MCP the classic way with different tools and stuff like this, but you can also deploy MCP with code mode in Cloudflare. So the way we did it for our API, if you have like a really complex... in a really complex system. You can also do it really easily on Cloudflare and the way it looks is here you can see I'm loading an open API spec so something really basic that you might all know already and the way we use it so everything is bundled in an SDK so that's really easy for you to use. I have a spec here that I can reuse in my API open API MCP server bundler and then everything is set up.

13:06really easy to set up nothing crazy you don't have to manage the infra because it's running on workers so no kubernetes no containers everything is

13:16stateless also with the new mcp spec that's really easy to deploy and yeah so that's a really simple one that's easy because i only have two endpoints but even for more complex use case that's way easier than going from scratch implementing tools calls like this um yeah and if you were have like an MCP running with different tools. You can also implement code mode with a wrapper we have in the SDK, so you don't have to go from scratch. If you already started building something and you face a limit in the number of tools and so on, you can work from that and move to the code mode API. I'm going to also show you, so this is if you want to build your own MCP for an external audience, I would say.

14:02Something also useful that we have at Cloudflare is if you want to expose MCP to your internal user. So I don't know the role of all the people in the room. I guess some of you are building stuff, building APIs, building MCP, but some of you might also be responsible for the internal infra and the MCP you are exposing for your user. So there is something really cool at Cloudflare which is called mcp portal so i don't know if you know it so it's part of the of the glossary offering and it allows you to expose in a really safe way external mcp so not the one you are building to your developers and your users so basically everybody within your organization who needs to access mcp you can expose them via the portal and you can have like logs of all the record the access control you can put policy plugged in your Okta or Entry ID and access whoever is able to access whatever MCP and block some policies and allow some of these people.

15:06And what's cool, so if I have only two MCP in my portal, it's easy. But if I give like 20 MCP, 30 MCP, 40 MCP, I face the same issue I was mentioning before, which is people are getting too many MCP, too many tools. So there is a way. in the portal to enable code mode by default so everything which is going to be accessed by this user using this MCP portal will be optimized with a code mode so it's going to be really more efficient for them so there is two benefits of the portal of course code mode for everything I mentioned before but also a really secure and safe way to expose MCP an external system to your developers and your users And this kind of thing, we're using it extensively at Cloudflare, so everybody in the company that wants to connect to an MCP, they have to go through our internal portal to authenticate and everything is optimized that way.

16:04That's it for the overview.

16:12If you have some questions, I might be available to answer them at the end, but that's it. Thank you.