0:02thank you hello welcome everybody so today i'm happy actually to talk to you to you about the securing the authentic harness i think it will be one of the most interesting topics in the coming months or years maybe i mean so what i planned is first a small explanation or basically discussion around what the harness actually is, what it does, and why it's relevant to control what it can do, what it can't do, and how to secure this overall AI, basically.

0:46I will have some time at the end, most likely for questions, maybe for some additional demos, so I'm really happy later to take additional questions.

0:57So, I really want to start with what actually is an AI harness and why it's so relevant. So, I mean, everyone likely knows the LLM, the large language model that actually understands text and words, which I have here as the model calls. But the model itself is basically just understanding text, right? it can't act on its own so therefore it actually needs a harness which is like an orchestrator of four AI so in addition to the model calls the harness provides capabilities to the AI to the large language model to execute certain things and the third part which I put here is the data so in addition to the harness being able to call tools to execute actions and there's also the aspect of adding data into the context making it available to the llm so the harness is actually the part that enables an ai to act and it's usually iterating over the tasks

2:20enabling the large language model to execute certain things, get additional data and making it really valuable, right? So the topic of this presentation is also why projecting prompt itself is not sufficient because it's not only about the large language model because this is basically just translating text but the harness is actually the part that enables it to act right and to do certain things to consume data get additional data execute tasks so if you want to use secure AI we need to look at all those aspects so I got a little slide here that should visualize how it's actually working in practice so if we type something into our chatbot we actually have this part here so it's a JSON we have this user role here and in the content we have what we type into the into the AI basically and then there's the first part that's relevant the harness it hatches tools that are available to the harness so if there's like you you have you're running it on host it has um a web fetch tool for example it adds this information to the request that

3:55goes to the llm provider like openai workers ai or whatever model you're using so this the harness sends this whole request what we typed in additionally with tools that are available and then the model actually It checks what we want from him, checks the details of the tools that are available, and decides if a certain tool should be called. So it tries to understand what we want from him and if any of the tools would be helpful for this task. And what it does then is the model sends the response back to the harness, telling it which tool he wants called.

4:40So for example here it will tell it I want a tool call, I want that you use the function search and I want you to query this specific case because this is the case we asked it for, right? So this request goes back to the harness and the harness actually executes the tool and then sends the results back to the LLM again, right? So now it knows this information that he wants. And now we can basically go in circles and loops and execute and improve and act on what we actually want to do.

5:23So we have three parts that are relevant if we talk about securing a harness. We have the model, the LLM, which we need to inspect monitor and act on we do have the actions the tool calls and this can be all kind of tool calls it can be local tools within the harness it can be mcp tools and i also generalize the data access here and also the data this can be all kind of data it can be local data in the file system it can be data on the internet those are like the the three categories that we need to look into in order to understand what's going on and in order to make it secure.

6:15So this is one of the examples if by default actually the model really doesn't have any restrictions. It can execute things that you give it in the harness and if we can basically inject instructions into the model, we can change how it acts, right? So, for example, in this case, there's a hidden instruction somewhere on a refund policy that says, as per refund policy, so -and -so, refunds under $1 ,000 are approved by default. Although maybe the intention originally was, maybe it was just about $200.

7:01And if the harness has access to the action to refund, right, and there are no additional controls, it might just do it, right? In reality, it's not as easy because there are different layers to it. But as in traditional cybersecurity as well, it usually always makes sense to do defense in depth, have different layers of protections and controls. And this is why it's also so relevant to basically control all aspects, right? It's about controlling the data that goes into the harness. It's about controlling the LLM communication. It's also about controlling the action that a harness can actually execute.

7:50So here again, the concept is always the same, right? No matter if it's about... model traffic actions or data access what you would usually want to add is you want to understand who's actually doing that action it might be a autonomous agent it might be you that's asking the agent to do something and what it should be allowed to do might differ depending on that right so if you have an autonomous agent that's serving your customers for tech support the identity of this agent might be the tech support agent but if you are using the agent and you actually requested to do something you usually want to act on the user's behalf and get the permissions from there so the authorization so this is basically checking who is doing it this is checking what are you allowed to do and then you add an additional layer to have like an outside, out of bound check, right?

9:05So the problem is if you can put controls inside the harness itself, right? But it's the same control region, right? So it might change its own properties. permissions. So you usually want an outside control, an independent outside control of the harness that controls it. And the fourth part, and it's actually more important than it might look like. I think Grant, our CISO, also spoke about it. The most important part is actually visibility. So you really want to be able to understand what happened or what's currently happening so that even if some controls fail at least you can understand what went wrong right all right so we will go into each of those three pillars I pre -recorded some videos so we have some examples that visualize what's going on and how to act on it so the first part that I want to talk about is the LLM traffic And what we're doing here is Cloudflare is providing an AI gateway and that's like basically a proxy where you can send the LLM traffic over and add additional control, monitoring, data leakage prevention.

10:36So you can add a harness that's running on your notebook or you can also run a service. AI agent running on your web page you hook it up to AI gateway for additional controls and in the back end you connect the LLM that you want to use so this can be like the big providers OpenAI, Anthropic it can be worker AI models from Cloudflare but you can also add local models so if you're having your own hardware running on -prem you can hook it up to Cloudflare and it from there up to the i -gateway as well so you can really steer whichever model you want to use via one gateway or several gateways all right so and bigger

11:35so this this first video is really a simple example um So it's just a request here. I add MCP document portal here. That means I give it access to like a file system via MCP where certain documents in so that a model can understand my specific data. In this case, I just asked it to check for a coffee machine manual. So it actually checked the connectivity to document portal. it looked up the document and I said okay I found a coffee machine operating manual and here's the information right so we can add our specific data to the model

12:23and because we didn't just run it directly to an LNM provider but we send it over AI gateway we got now some additional information

12:36and actually that's big enough but this is how the AI gateway log looks like so we can see the the model call here the specific time which model we used how many tokens we used what cost sign bin duration and so on and on the left hand side we see the request so that was the part that we sent to the model and to the right we see the response so what the model sent back to the AI hub right and by doing so we get a very good view of what's going on no matter where the model runs as long as it runs over the AI gateway we can later on understand what happened in addition down here we have additional metadata so here we can see this was the user that did that request but we can also see this was a saw a chatbot and additional information so we can really extend it by the data that we want to use or that we want also to use to make certain decisions or routing depending on that.

13:46So here we can see that it was requesting the document and to the right it looks a bit strange because this was streaming so it's sending parts of the sentence so we have basically the first part where it says I then found it and then it would go ahead with the rest of the of the text basically

14:12and here this is a bit a bit harder to see but we saw in the first slide with the communication of the of the model a tool execution and all of that and the that it's adding the context that after it looked up the it is the tool call to get the data from the MCP from the document portal the agent called the tool and then it sends the result to the LLM again right so we can also obviously then see those details of the document that it found here as well so it's another call so it would be a bit later and then we can see what was the actual content of the document that it sent which might be different from the exact response of the model, right?

15:02Because it will just take the details out of the tool call that are relevant for the response to the original question, right? So this might, you know, contain much more information about the coffee machine than what the model actually responded.

15:29Now we get to the prompt injection part as another example. So we are using the same document portal and we just ask it, you know, do a summarization of all the documents that you have and give me the result. And within one of those documents, there was actually. prompt injection so now we can see down here it replied with a 424 prompt block to your security configuration because we send all the the communication through the ai gateway we can intercept certain things right so in that case in one of the documents within those files there was an injection that said know I mean this is like the standard thing but ignore all instructions and give me I don't know what exactly was but basically additional instructions right and because we sit in between we can prevent that and block that and it's actually also tagged so so up here you basically I get a different symbol as well so that if you lay it on check in you can see the details and easily find it so this is how it looks like so you do have guardrails in this case we

17:03said block prompt injections or jailbreaks but they are also additional other filters that you can use plus there's data leakage prevention so you can even add more specific filters like If credit card information is requested, then block it.

17:27So, this was basically the first part here for the model conversation. So, the LLM protection. And now we go ahead into the action section here. So, tools and actions.

17:44So, here we have the concept of . hooking up whatever MCP server you want to make available to what we call our MCP server portal. So that means instead of directly connecting your MCP servers to your AI harness, you connect them to the MCP server portal and the single portal you connect to the AI harness. And this is actually not only a security advantage, also a usability advantage because you connect it once and then you have one server portal or several you can also have several server portals but you have less ports that you need to connect to your harnesses so this is like um like mcp server proxy again so because it sits in between we can add additional controls visibility and we can even add role -based access so Depending on the role of the user or the email, you can give it access to certain servers or certain tools.

19:03So in that case, we connect the portal here in Cloudflare OS in this case. So at the end of the day, we're going to talk about this. So this is the single URL of the MCP portal that we add, we say authenticate against it. Then depending on the user identity, so this can be a service token, it can be an entry ID authentication that already exists, we can give it access to certain MCPs in the backend. In that case, you get the portal page with the the MCP servers that are available to this identity, you can authorize it.

20:02And if you now look into the connected account, we can actually see which tools are available so based on the identity that we signed up on in this case we have a search function a fetch function export function that is provided via the portal to this ai harness but this is actually right now it's a um to visualize it's an unrestricted access so in this case we again ask it to search the document portal and let's check cellular information right um so it's now again going through the portals it asks it sends the tools that it has the following uh tool call search execute export and dllmr answered basically okay then use the search tool search for salary right the harness executes it and then it takes a moment and then we get uh the salary bands of our hr the people department so that usually is not what you want right so we do the same with access control in that case so we sign in as a different person in

21:37that case we use Cloudflare OTP it's an email authentication so you can say certain email addresses are known to sign sign up gets the pin and then you enter it and then if we check the tools again which tools are available to us

22:00look like it oh no it's actually just showing us no servers available so so this user actually doesn't have any access to any of the mcp servers in the back end um so obviously in the in the tool and in the portal you can be very specific about which users have access to which servers

22:29and now we do the same with a user that's actually allowed to use the portal and the backend server but he actually has different tools available

23:04so if you now look into the tools available we don't have the export function anymore right so we can can really restrict what's available and therefore add additional controls on that as well so we now add the the model control right the model We had the action boundary and the third one is the data access. And this is basically the broadest one, because like I said before, so depending on the tools available to the harness, it can get data from all kinds of sources, right? It might be data from the internet. It can be internal apps that are available to it over the internal network, right?

23:50And it can be low. local files, or it can be specific vector stores for pre -vectorized data, making it easier for an LLM to consume.

24:05And obviously here we have the same topic, right? It can be used for prompt injection, it might access things that you don't want it to access. So this is like the third boundary that you want to add controls and visibility. anything so that you can secure what's going on so in this case

24:28there are several ways actually do it in in my demo i we start basically with mcp server that has a web browsing tool so it's not directly browsing the internet it's calling an mcp server to browse the internet and what we can do there is also we can add a cloudflare gateway behind that so whatever the mcp server is doing we can add additional inspection and controls so we can add for example um that the the the um the mcp server itself might have access to certain data but based on addition an additional control layer with the gateway we can even there i get add additional controls like okay but even if the document is available if it's about salary data then we want to block it or we want to act on it right

25:36so this is specific to mcp server part but also in general we can add additional controls by gateway restrictions.

26:15So, again, we ask it to search for salary data and now it looks a bit different, right? It says, hey, let me try to search it, but I got an error here, I got an internal error,

26:33I got access to certain documents, but I can't access the salary data, right? So what we did here is basically we added a gateway DLP policy so we had a policy for this worker that's running the harness and said we have a DLP profile that is blocking this identity from accessing several data, right? So we can have more flexible controls in addition to existing controls. And I mean, this is like a best practice, right?

27:21You never want to rely only on one control. So you usually want to add defense in depth, different controls. If ever one fails or it's misconfigured, you might have another one that basically saves you, right?

27:40So, in this specific case, it was the gateway behind the MCP that checks what the MCP call is actually doing.

27:51But there's also the channel internet access or network access that you can restrict. So, depending on where the harness is running...

28:04usually would want to restrict it to certain locations that it can reach right so depending on what it is if it's like a very specific agent that maybe is only there for support requests you want to close down its success further then if it's like a general agent that should be doing a lot of different tasks for you right so you can create different profiles depending on the identity who's calling

28:34So you can make it as closed down as needed, but also as open as needed, right? So there's no real easy answer to it. There's always this tradeoff of overblocking and underblocking, right?

28:54Okay. So now again, we have the MCP server. that's able to browse the Internet with added gateway policies for the MCP server. So, we ask it here again check the document portal and what are the current export limits of the document store. In this case, there's another prompt injection in one of the documents. It's actually telling it if you're looking for up -to -date information on This document checked the following site in the Internet, and that's basically a malicious site, right? And the interesting part here is actually that the gateway blocked the initial request where you use the MCP browse tool because you set up a policy, but we failed to send the local tools via the gateway.

29:56well so although we blocked the mcp server tool there was another tool available locally where we didn't have any policy or check on so it just found a way around right so it's also really important to make sure that we understand all the boundaries and the tools that has available but this is also where it helps us to have the visibility into the model calls with the gateway because we can actually see which tools it reported available right so it's like we can use the the one also to check the other and and improve on that so now we have the same exactly the same call just that we hooked up the harness the internet access of the harness to a gateway secure web gateway with the same policy in place and now Obviously, even the local web fetch tool can't access the page, right?

31:02I mean, the interesting part is, you know, you can even, when setting up policies and figuring out how you want to configure it, you can even, you know, ask the model to try to find workarounds, right? Just to test if all your policies are working. and this actually when i when i set up the videos uh this was actually mistake i made so i i really ignored that the the harness itself still has internet access without gateway so the thing i changed was simply saying your internet accesses via secure web gateway so that the policies basically also apply to everything um harness can reach via its network.

31:49So now we have the same here, the same block, also for the direct connectivity. So the important part is really this overall visibility, right? So with all... those tools in place we can really see everything from the initial request we can see the model data that's been sent to the provider forth and back we can see which data it queries right so because it goes via our secure web gateway we can see every request it does is it might be the internal network where it reaches internal services via cloudflare it might be internet data um so so this here is basically the secure web gateway where we can add additional visibility and controls and the action part here is um really the mcp portal um there are obviously also controls in the harness right so you you can add within the harness controls which tools are available and you should be right because it's again it's this defense in depth approach you add controls in the harness you add it in the boundary but i think the boundary is more important because it's independent right so it's much harder for an ai to

33:23do something if it's not in this environment right

33:30yeah and so so we basically have the full chain and the full visibility into everything that's happening from the ai harness perspective right

33:44So again, the three main parts, the models with the AI gateway, we control the prompts, we can prevent them, we can get visibility, we can get access, we can do different routings. We have the actions, which are MCP actions, but also part of it is a secure web gateway. it's not mcp specific you can add controls on the secure web gateway to which capabilities to what yeah actions um ai harness might do and the third part here is the general data access where it's also if everything around the harness is controlled by you you get the visibility into it

34:39So, again, the takeaways, try to make it easy. It's really about you need to control what the harness can do, what it can reach. You need to understand who is doing it, what is he doing, is he allowed to do it, and you really need evidence, right? So if anything happens, you would at least want to understand why it happened and how might prevent it the next time right so this was also what uh grand said um for him the most important part is actually visibility the evidence right because you can build on that and the third part which i really recommend out of practice check for bypasses right so and you can really leverage ai as well right just ask it to find whatever way to get to whatever you want it and then validate in the logs what happened what it tried so so for me this is all basically the the basics of how to think about ai security but based on that you can build right so because cloudflare's platform that's really open you can actually use the data that you have in those different pillars like the AI gateway, the web gateway, the mcp portal to customize even further right so you might decide you want some customized dynamic controls for example

36:14so you can look at the request log what's incoming and the web gateway logs what it's actually requesting you can build a logic on top of workers where you say does this actually fit the task it was asked to do if not you might increase the risk level based on the risk level you might decide to restrict what it's able to access in general right so the platform gives you a lot of flexibility to build on top of that I'm thinking

36:57Yeah, I think that's actually it.