0:02It's nice to see you folks. My name is Sunil Pai. I used to work on the AI agents team until about a month ago, and then I started working with Ali. But the last two years of my work are now in the slide deck, I guess. It's kind of weird. So the way you make slide decks in 2026 is you open up Claude Code and you say, just find everything I've done in last two years and make it palatable and make it look like I know what I'm talking about which is wonderful uh before I get into it I need a favor from you folks Kyle actually I need you to help me out here I need you to make it look like I'm having an argument with these people and take a photo because that will make this the best talk that exists can you take it so you are really mad at me okay okay can you can you just quickly take a photo yeah thanks okay cool yeah okay I'm gonna send that in work chat Okay, awesome.

0:56Excellent. So my name is Sunil Pai. I've been with Cloudflare for about five years, including a startup and stuff. And I am a React developer. I was a front-end guy. And I was like, I'll hang out with the back-end kids. And I thought building AI agents on durable objects would be like a two-week mission. Just get in, write some code. And it ended up taking so much of my life. And this is a little journey through. it what I learned and how I could tie it back to lessons that I'd already known before so first a quick story also Opus does these animations now for free like like this is just a react app like in my browser just love it the new world means that people are doing deploys multiple times a day there's no idea of a release cycle and everybody's saying okay no do any pushes for the next hour while I do a push.

1:52We have customers who are, I personally know of at least one customer that's doing about 200 deploys a day. And we heard that there is one customer that's doing four digits, which is more like every few minutes and stuff. And the thing about agents and like just request response cycles, your standard e-commerce websites is that they're actually long running processes in the background. They're doing things even without a browser connected. That was the big late 2025, early 2026 thing. So this is an example of a customer who was using the Cloudflare Agents SDK and it was going on for minutes. Yeah, roughly, it was getting close to about two hours and they'd already done like three deploys. And then the Cloudflare Agents SDK decides to break it saying, oh, this process has been going on for too long.

2:45It's been going on for too long, so it's probably hanging. so we're just going to kill it. Which is dumb as hell. That's my code, just so we're clear. I'm responsible for that happening. I had it in the thing where I'm like, oh, the moment it hits, so much time, it dies. And while revisiting, it's weird, because you do want to have some sense of boundaries, and it turned out time was the wrong one. So we'll get back to this use case, but I want you to know, this was a big goof-up that led me down. bunch of stuff. This is a very 2025 -ass quote. Oh, quoting agent that dies when your laptop sleeps. That's a tool. I want to understand, has San Francisco discovered amphetamine on their laptops, or do they still keep the screens open on the buses? I loved the three weeks when Twitter was just hundreds of those photos.

3:35But the idea is that you want to take your agent from something that you're running, like an app, and you want to treat it like an infrastructure that's running in the cloud, not on your machine, basically. And I built, well, we built something called Project Think in, I would say, in April. We shipped it in April where we took what we called an opinionated take on what an agent loop should be running on durable objects. Some folks here might know that I'm the biggest durable object fan in the world, which also means I hate it the most. Like I'm deep familiar. with everything that makes it suck. And that's what true romance is like, right? Like after you've spent 20 years with your wife, I'm coming up on my 16th wedding anniversary myself and we tell each other that if any of us angers each other, we can destroy them overnight just because we know where all the skeletons are hidden.

4:32Okay, so this is what a standard agent loop looks like. It doesn't matter. It's just pseudocode at this point. I didn't even write it. I'm like, okay, it looks fine. And it seems it's nice to model an agent loop as a while loop. And you say, for every step, you pass it whatever your messages array is so far. And if it's done, you return the text. Otherwise, if there are any tool calls that were returned in the response, run that tool, add it all to the messages array, Wonderful mental model. It's kind of weird, though, because we spend all... this time um working on um optimizing request response cycle and now we're just like oh here's 600 messages we're going to make it 600 messages pass it back we're going to get another 602 messages back it's it's vulgar is what it is but the entire world like like commerce capitalism runs on this request response cycle right now which i thought is quite funny uh we can but each there is a there is a rough edge with almost every line in this uh this call is the most expensive call because inference cost stuff because everyone is using the top frontier model at this point and it's not usually awaited it is streamed in your build per token it can stall it can decide it's doing something else in the background might be an async call in the background the tool call is another one which is some of these things are doing payments they're making

6:04destructive calls on databases,

6:09and basically a complete mess where if you don't actually run it just once, then all hell will break loose. Some of them might never run. Just a number of things that can go wrong here.

6:22And even worse is, we spent 30 years building these stateless infrastructures and libraries and an ecosystem around it. But it turns out if you actually hold it in memory and your isolate or your server dies, you're screwed. Because if you want to start it again, it is a very, very expensive process. And even the user experience sucks. I don't know if you remember the first versions of ChatGPT. If you closed the tab or you closed the app, it would die midway. Or sometimes it would wipe out everything that had happened and it would come back in. Just completely awful and very, very expensive. Over the last year, some other things have happened. So, METR, is that what you folks call it here? METR, the research group?

7:10They've discovered this Moore's law for agent execution where every few months, the time basically doubles. And I think that actually, it's not a linear line. It's going away from, I think it's quadratic is what it is. Like, it's been doubling, but I think it's now tripling, quadrupling. And that's the hope inference companies have. you right they'd like you to set up a machine that just runs 24 hours a day and the only thing you're doing is providing context taking stuff out oh that's how they're stealing software engineer jobs okay that's how they're trying to get the jobs away from all of us they're like yeah it's just running on these servers all the time another nice thing that's happened is I remember especially in 2025 there were a number of agent frameworks that just I don't hear about them anymore but they were responsible for bringing about the common stack for what an AI agent looks like.

8:02So you have a harness, you have sandboxes that you can spin up and spin down. You have a durable log where you keep all your messages and not just the inference messages. You might have some UI-specific metadata, et cetera, in it. Then you have APIs for running them as managed groups, like Cloud Managed Agents, I think is the most, I don't know whether it's the most popular one, but it at least framed the phrase. And the other, there are durable execution services and libraries that people use for building in guarantees, consistency guarantees, et cetera, for these things. I'm going really slow on time. And then earlier this year, even the MCP spec went stateless. It's why for the longest time, the best MCP servers ran only on durable objects, which was fun.

8:48It pissed off a lot of our competitors. But now it's stateless. You can run it on Vercel. my point being like we discovered all these problems that were running in a single machine and it turns out that all of these things are solved systems in the world are solved problems in the world of distributed systems the idea of a process being killed mid -turn and what do you do when it comes back well that's just crash recovery that's why you need a write-ahead log that's why you need some form of replication over there a tool call being retried Love, $12 word, idempotency. I used to say $5 word, but inflation is inflation.

9:27The idea of idempotency, it's just a wonderful word. It's what software engineers with about three, four years of experience tell each other to show that they know what they're talking about. Beautiful word. The idea of having two sub-agents working on a single source of truth, two or more, really. This is the split-brain problem, which is why you need a single writer and you need cues and you need some way of synchronizing. changes. Waiting days for a human. That's just it. Like when you do approval, you can't just do an await in your, wherever your code is waiting for a human. They might come back days later and click the approve button, right? So now you need some, again, you need a durable execution engine. There are like durable timers. What happens when that approval is not even valid anymore? You work on, how do you deal with rate limits, runway spend? How do you attach budgets for all?

10:17these sorts of things and the one that i really don't think we've actually solved which is how do you deal with context windows uh either even a million tokens is not enough anymore i've been working on please go downstairs expo floor there's this weird thing called city that remembers which is a big isometric thing and i've added dragons and a big mecha kitty that you can now use and hopefully by the end of today you'll be able to kick the buildings over this is going to be for whoever's videoing this right now It's an actual exhibit downstairs. Point being, I've been doing that in a single open code thread for about six weeks now. I have not looked at what context compaction it's a phrase. At some point, it starts doing stuff, and I'm like, okay, I'm going to have coffee, come back, and it just works. Anyway, so new names, old problems, and it turns out the answers for them are the same too.

11:06The thing I like about agents are that they are one-to-one, right? Opus came up with this extremely basic... ass analogy about it being a restaurant and diners etc but an agent is a personal chef every user gets their own task their own context at some point it'll do generative ui so the website looks different for every user sure whatever as an in from an infrastructure angle though it's interesting because if you try to do it the traditional way with a system that builds you for uh like needs to keep the machine all the time and maintains everything in memory then you end up with this very inefficient architecture where you're just burning electricity on machines. But with stuff like durable, I say durable objects, but something that gives you an actor model style

11:55entity system, it's nice that these things only wake up when you need them to. Durable objects are actors. Quick show of hands. I'm saying things assuming people know what durable objects are. Enough hands went up. Yes, okay, so. objects are actors. Fun fact, they're actually called actors in the Cloudflare code base. Famously, durable objects are the most piece of shit name for technology ever. But they're actually called actors and I shit you not, when they were going live in 2020, in 2020, that was when the beta blog post went out, Kenton did try to call them agents and he was shot down. We have a thing inside Cloudflare where we're like, yeah, Kenton was six months ago. We should have just called it agents. I suspect it would have affected the stock price.

12:41Anyway, so because they're actors, they have a bunch of things that they're associated with. There's an identity, an ID. You have one thread that works on it. It has storage and you can hibernate. Hibernation actually came a couple of years later, but the fact that it can sleep when it's not doing any work and it wakes up on messages. Okay, I'm going to start scooting through these things because I'm actually going quite slowly on time. Your mental model should be there are some things inside a durable object or an agent that stick around even if it shuts down and comes back to life. So on the agent class in the agents SDK, it's this .state, you have SQL, you have a SQL database, all our schedules and alarms are durable and work even if the isolate goes down. We have workspace files, that thing has been renamed to computer and I think we actually shipped it just today.

13:27And then on every web socket you can actually set a little bit of state that persists across hibernation. the things that don't are regular javascript variables class fields set timeout in-flight fetches etc the annoying thing about durable objects and that's just a distributed systems problem is that there's no on evict callback you can't really know when a system is shutting down much like a sigterm signal which there's good bad things about it but it does mean that you need to be prepared for all of these things by the way just a way to clear the big punchline here is that the agents SDK takes care of all of this shit for you. Like it just makes durable objects actually usable. Anyway, so there are five ways that these things break. I will just go through all of them. I mean, I have a slide for each of them. One is how do you keep a durable object alive when you're doing a bunch of IO? There's a dot conveniently.

14:16I wrote a dot keep alive function that you can wrap your functions in and it uses a durable object alarm and it just keeps the actual insight. alive even if it feels like dead air. Durable objects itself are shipping things now that make that the default behavior, but until it did, I had to keep a live around. I have this data structure called a fiber. Again, so it's what someone with 12 years of experience tries to say. Well, I've introduced a new kind of data structure, which I completely stole from React, by the way. And there are tasks where it looks very much like any of the other durable execution engines. And you can use these things to. wrap functions that you think should live through eviction, et cetera.

15:02We did have a recover function, and it was about a third of the entire code base. It was mental, and then we just deleted it. This is Opus now quoting an Opus quote from back then. It's self-referential Opus. But we removed the recovery mode itself, and instead we just simplified the data structure down, and we said no. happens just keep recording it and when it recovers have a strategy to like no head the second one is what we call durable streams uh we hate i hate i absolutely hate having to do um i forget what it's called a recovery a recoverable uh inference streams anthropic used to have a first class api for it they removed it open ai still has it i kind of wouldn't trust it to like stick around um the idea being that if you actually do have if your thing does blow up in the middle of LLM streams how do you recover like from that moment and the way that you do it is a shit ton of code I'm not even going into the point being that there are ways to do it but I did write a blog post and we are shipping in Cloudflare AI Gateway a way to enable this for every provider and every kind of inference the idea is that you will back every single inference stream with a durable object that your agent talks to so if your agent actually shuts down. You haven't lost the stream.

16:22You reconnect, and you get the rest of it back from it. I'm not the kind of guy who says you should read my blog, but you should read my blog.

16:33Again, back to idempotency, especially when you're doing stuff with money or just database destructive actions, you want the idea of adding an idempotency key when you actually do stuff like refunds, etc. I don't need to spend this very standard durable execution stuff. say very standard, I'm saying it means your coding agent knows it, so he'll take care of it. He'll see the API and be like, okay, I'll figure out what the thing for it is. something. This is one of my favorite implementations. Every durable object has only one alarm, but really you want to manage like a number of things with that single alarm. So we implemented the schedule API. We have a .queue API tasks. keep alive, destroy that.

17:20Maintain a SQLite table

17:26and every time you try to say, hey, do this thing after two minutes, do this thing after three hours, do this thing at like 9 a.m., it saves it in that table and the single alarm just goes through this and runs it. I love it. This solves so many problems. I don't work on the Durable Objects team, but they keep promising me that this will be a first-class API. Until then, you can use my implementation. That's fine. Back to the example that we had. It turns out that the mistake that we were doing was that we were trying to think in terms of time, but that means nothing in a world where something is taking 30 seconds, 30 minutes, 30 hours, 30 days, 30 years, who knows, right? Really want to do it, you want to be able to tell the agent that some work is happening and every time some work is happening, you want to reset what your counter is and that is what your budget is.

18:14should be this is what the api for it looks like and you can actually pass there's a hook saying hey should i just keep recovering based on the context which happens every time the thing wakes up in general you can just say return true it's fine um i say this but like at least there are two people inside this room who are pissed off with how much durable objects cost if you don't keep an eye on them and one of them is sitting right there oh there's another one right here there are more than two in this room. Regarding concurrency, this is my absolute favorite feature of DurableObjects, which is as a dumb JavaScript programmer, I do not have to worry about multi-threaded implementations anymore. Everything happens on a single thread, put it in a queue to make writes. Make writes, yeah.

19:05These are actually two notes from Cognition and from Cursor. Did it happen because I spoke to them and I don't know if I'm allowed to share what I shared over there. Like I said, the great thing about DurableObject is your mental model is that it's happening in a single JavaScript thread and you do stuff. A thing that we did argue about a lot internally, especially after we shipped DurableObject facets, is should we have multiple threads in a single DurableObject and have every thread be a facet turned out to be annoying. annoying uh should we have all chat threads in a single duo also turned out to be annoying so instead what we do now is multi-layer where we have a user hub that mean one durable object that maintains all the threads oh sorry as well as and then having a durable object for each one it doesn't matter spin up a million billion trillion of these like if you end up reaching a limit on this your next step should be to tweet at cloudflare saying oh you guys did a big you talk about durable objects being you should be able to spin up a trillion and it didn't work and Dane will get pissed off and then message the rest of us and we have to fix it so that should be our main there are you can do infinite ones of these um the other thing and i'll quickly go through these because i assume there are other talks with strategies like code mode where you can actually write code that calls tools you can take uh 1 .17 million is the how much context tokens the entire cloudflare api takes which means you cannot fit the entire Cloudflare API

20:37in any inference provider right now. But you can bring it down to 1,000 tokens by using things like code mode.

20:44And the beautiful thing about dynamic workers, of course, is you will never leak your API key and suffer. You will never leak your API key because it's just not available for it to thing. Anyway, so five rules for agent architecture that the agent's SDK and your agent will take care of you. One agent, one actor, one durable object. Make sure that the idempotency thing is something that you have to take care of. Base it on a user ID, a time, some kind of transaction value. Make sure that any work you're doing is being written down in a database because that's where it sticks. Make sure you're budgeting for work. Time doesn't mean anything in 2026. And make sure you use things like dynamic workers for actually... Uh...

21:32for what you give access to your agent. I did want to show a little demo here, but I'm out of time, which is what I feel like the future of AI applications should look like, and they're not chat agents. I built a thing where you can embed an agent into existing apps like Ableton, et cetera. In fact, that's my demo. It's a little digital audio workstation where you work alongside AI instead of it taking over for you. It's on my blog.

21:59I don't even care about this. The point being that all the new problems that we discovered in the last 24 months or so are all old problems in very old textbooks if you bother reading books at all nowadays. And you will find the answers for scaling these things to millions, billions reliably in those books. That's my talk. Thank you very much.