0:03this is a runtime for developers and their agents i i i hate this title but you know um they asked me about a month ago say hey come up with a come up with some slides i am so freaking adhd that coming up with slides a month a month in advance is like asking me to do it a year in advance there's no way so these got done yesterday whether or not i actually followed the slides we'll Um, I'll, I'll, I'll do my best. Um, uh, there's only a few in there that, that really, you know, kind of highlight what I'm talking about. I'd rather just kind of just, just talk about what we're, uh, uh, you know, talk about it. We've only got about 20 minutes. Um, there's other talks throughout the day that kind of go into a lot more detail of, you know, uh, kind of the business piece that I'm going to be touching on. So, uh, quick introduction, uh, for myself.

0:50Um, I am James Snell or J.A. Snell everywhere. Uh, I'm a senior principal engineer at Cloudflare. I work on the workers' runtime itself. I have been a Node core developer, part of the TSC there for over 11 years. I was actually part of the launch of the OpenJS Foundation, getting IOJS and Node back together. I've been around for quite a while. All the gray in the beard, I call this my Node gray because dealing with all the open source drama really comes from that. I am a delegate. for TC39 and TC55. These are the committees that actually drives the standards for JavaScript and cross-run time compatibility. And I'm also working on a book.

1:37I have to do the shameless plug. JavaScript in-depth, Manning. It's almost done. I got like two chapters left. Then it's out there. But what are we going to be talking about?

1:50Agents need to be able to write. and run code, right? This is becoming just a truism. This code needs to be secure. It needs to run quickly. It needs to run well. It needs to be scalable. This is just the fact of our ecosystem today. Increasingly, what we're seeing, more and more of the workers that are being deployed, more of the software that's being deployed right now is increasingly coming from... not from people. This is just a lot of the bi-coding applications. These are just, you know, almost the vast majority of workers that are being deployed today are coming from agents. That code needs to be secure. It needs to be scalable.

2:36It needs to be set up in such a way that we don't have to worry about these rogue agents going off and just doing whatever they want, right? We need a secure sandbox to be able to run these. And as we start to look at things, what we've discovered is that the worker's architecture is actually really ideally suited to this, and it comes down to three fundamental primitives. Those primitives are the security model, the isolation model that we have, which I'll just talk about briefly in a bit. It's also the execution model, like how we actually deploy this code, how we actually run it. those primitives are for both stateless and stateful workers, regular workers or durable objects.

3:22And then it also comes down to the APIs, like what is the compatibility model for these? Why is that important? All right. Now, throughout the day, there's going to be much more in-depth talks. This is just a quick conversation here. I encourage you to attend the other sessions, particularly Kenton and Ryan's talks. uh here um what these will start to go in in in depth on are kind of what we're building on this platform how we're actually using these primitives uh kenton uh has been working on this thing called cloudflare os which is really a agent harness built on top of workers um what that means what the the design um architecture goes into this what the trust model needs to deliver in order to make that happen uh and then just a couple of weeks ago you might have the news that Dino has been joining, is joining Klaus there, which is fantastic.

4:17The Dino runtime itself, we're still figuring out what's going to happen with that. There's still lots of conversations happening there, but the team that's been working on Dino has joined the workers' runtime. Ryan is really going to be talking about what that means, what they're going to be focusing on, why that happened, and why we think these primitives, particularly durable are really the way that agents should be running code moving forward. So here I'm just kind of touching on the high level points. They'll be going into a much deeper level of what this actually means. And then we have the other talks that kind of go into this in a lot more detail.

5:00Okay, this is the key thing. Agents do not need a new kind of runtime, all right? They need something that is familiar, consistent, and secure. There are, there's been studies done that these models that are being trained, most of the time they're going to write code that is just familiar. It's what they know, right? And the platform that exists, the one that all these models are being trained on, fundamentally is Node, right? They're being based on the NPM ecosystem. They're being based on all the open source code that is available on GitHub, which overwhelmingly is. the node apis this is where we go into you know if you look at it bond when they came out there we're going to be compatible with node that was their mission right it's going to be just a faster note dino originally came out we're going to be web platform based but where do they end up node compatibility uh workers our initial uh approach was we are going to be a web development model right we based on web streams and web crypto and and these web apis you And now what we're focusing on is node compatibility.

6:09That node compatibility has to be there. Now, our definition of node compatibility is different than Dino and Bond, but at the same time, we have to build these familiar primitives that are there. But let's go back to the three pillars. Three pillars, the security model. And then we have the execution model and then the compatibility. On the security model, workers who... always been based on this idea of isolates and sandboxes. It's been built in from the very beginning. We've not had to go through and re -architect these things to be secure by default. Fundamentally, the architecture itself is secure by default. And we accomplished that. Yes, we have these isolates. In a single worker's process, we can run thousands of workers at the same time.

6:57Every one of those workers is an individual V8 isolate. every V8 has its own heap, its own memory, right? The way we orchestrate these things, those things remain isolated. That's one layer. The second layer and the more important layer is our bindings model. And the bindings model is fully based on just the capabilities driven deployment. Rather than just running an application like a node, which has access to your full operating system, right? The security model in node is if it runs, it's trusted, which is insane, right? Uh, so in order to secure node, uh, for agent development, you have to put it in this sand, you know, in a container or some kind of sandbox and really lock that thing down. Well, in workers, it only has access to the things that it needs, right?

7:43We do have a, a, a file system for like the node faucets and APIs, but it's a virtual file system that's completely in memory and isolated to that one, uh, uh, to that one, um, worker, right? Uh, if you need to give it, uh. network access, right? That has to be configured specifically for that worker. If you need to give it access to private resources, that has to be configured specifically for that thing. And it only has access to exactly what you give it. So it doesn't have this open environments where it can just kind of access whatever it wants. It really does stay constrained. And it's been that way from the beginning, right? I'm not going to say that it's completely 100%, you know, secure. There's always, you know, things that, that, that. come up. But from the very beginning, it's architected around this model of this secure by default. I'm only going to give it the capability that it needs, right?

8:31So that's the first primitive. The second primitive is durable objects and just the stateless workers. So stateless worker is basically you send one request, it does one invocation, and then it moves on, right? It shouldn't really maintain state between those things. Durable objects, though, the thing that we're really looking at for agents, right? Agents need to be able to take some task, take it for some period of time, maintain state, multiple requests, multiple steps, multiple pieces of the workflow. This is the primitive that we think is fundamentally the approach that you should be using agent developments on, right? And this goes to one of the key reasons why the Dino strategy here makes a lot of sense.

9:21are doing, and what Ryan will go into much more detail on, is kind of why that primitive is what we think is the thing that really is the correct approach for moving forward. But also, it needs to be something that you can do beyond just Cloudflare's infrastructure, right? You should be able to run durable objects on your infrastructure. You should be able to deploy it on your You should be able to take that outside of Cloudflare's infrastructure and run that, but still do it in a way that is clear, consistent, and compatible with what we're doing in workers. So our strategy there is really to go all in on durable objects, really to take that and expand that out and really make that a fundamental primitive of how you do development with agents long term.

10:17keeping the security model in place, keeping that isolation, keeping the capability injection there as we go. Now, the third pillar really comes down to the compatibility model. So the part that I work on with workers primarily is the API. Now, one of the fundamental truths of developing with agents is that as developers, we can to pay less and less attention to the actual APIs. We don't necessarily care what the APIs are. You care about what the architecture is. You care about what you're trying to do, but let the agents actually worry about the APIs. What we're finding is that in the majority of cases, and there's been actual studies on this,

11:04the agents, without any kind of context, without any additional context, if you leave them just to their own, ask them to write some code, they're going to base it on what they have seen the most. It's just a truism of the model. And what they've seen the most are node compatibility, all the node APIs, buffer, process next tick. These are the things that they try to use. If a runtime does not support those, right, what's going to end up happening is the agent can't discover that it doesn't support those. It will try, it will fail, it will try again. All of that means more tokens use, more cost. The more that we just build this, capability, this compatibility into the runtime, the cheaper it actually becomes for these models, just to write code that just works. It doesn't have to keep trying again, right? It doesn't have to go off and discover a bunch of contexts. Now, we do need to teach the models new things.

11:51And some of the things we've talked about, building skills, we're like, how do you build on workers? How do you do these kinds of things? But the primitives, and in this case, node APIs, just need to be there. And in August of just a couple months ago, we've enabled node compatibility by default for all new workers. It's just there. Not all the APIs work. Some of them are just non-functional stubs. Some of them will actually throw if you try to use them. But the point is they are there. And if something does try to use them, there's at least some feedback that says this is how they're used.

12:37But one goal, one difference in the way that we approach web node compatibility that is different than what Dino and Bunn have tried to do is we have no intention of ever being 100% node compatible. There are a huge number of APIs there. It just never makes sense in this model. I'm talking about things like child process and a lot of the VM APIs. These things just don't make any sense.

13:07to that end, there is a subset of Node APIs that do make sense, that are consistent, that are core. We have been involved with ECMA, basically a committee called Winter TC, or TC55. We helped found that when it was originally Winter CG, a community group under W3C. We helped in the transition of getting that to ECMA. One of the things... we have published recently there is a minimum common web API. And it's an actual standard now through ECMA. What that defines is the minimum set of web APIs. So web streams, readable stream, web crypto. I mean, these are the things that all runtimes should have.

13:52That's out there. Great. It's a great minimum common denominator. The next. spec that we're working on now and i should have the first draft of this by end of next week is the minimum common node api these are the minimum apis that every runtime should support and implement in a consistent way with actual implementations not just stubs that should exist on every runtime that claims node support or needs to be able to run javascript in a portable way right uh and the goal of this is really to have this be something that the models can learn from this is what they can rely on this is what developers can rely on this is the minimum thing that a set of apis that if you're building a framework if you're building any application that is needs to run not just on node but on any of these run times these are the things that have to be there and and and and uh really have to work in a consistent way so So that is something that we're hoping to progress through to TC55 rather quickly.

15:03I'm actually hoping to see if we can advance it by the end of the year. So the first draft should be ready by next week. So with these three pillars, like, again, we have the security model, which hasn't changed. You know, it's been there from the beginning. We have the execution model. Again, it hasn't changed. We've had it for years. It's nothing new that we've come up with. And we have the compatibility model. which again hasn't changed it's been we've been working on node compatibility for years but all of these primitives together the you know they they really form the basis of what the minimum what we need for agents to really go out there and and thrive and and and really unleash what they're what we what we need them to do without the risks right and the risks being you know rogue agents just going off and doing whatever they wants, you know, burning a bunch of tokens, you know, running up costs.

15:54The thing that has been missing is the deployment on your own infrastructure. You know, they, you know, yes, we think the Cloudflare infrastructure is where you should be running these things, but these primitives really should be something that are generalizable to everyone. So the work that we're doing on Worker D, integrating cell D, integrating, you know, the work that Dino did to really make this something that you can deploy in your own environment. The driving of the standards, right? This is the minimum API surface that's out there. All of these things that we're going to be working on moving forward are really going to be geared towards making this development pattern that today only exists for Cloudflare, making this the way that you write software for agents moving forward on any infrastructure, right?

16:50So it doesn't matter if you're using Node, we're going to have consistent APIs. We're working on a new server API, for instance, that is based on workers model that will work on Node as well. So code will just be portable back and forth. That's something that we're also doing in TC55. We're working on new streams APIs that also make this a bit more portable. So we're working on these primitives really designed to take this way of writing software and make it the... general way that you do it moving forward. All right. On any plus. So that, like I said, this is just a quick, um, uh, uh, introduction. The other talks here, um, uh, you know, definitely encourage you go, go sit on these. If you, if you can't, um, um, catch one, catch the recordings afterwards. Um, I'm going to be out in the speaker's lounge.

17:37Um, you know, I can answer questions, go into, into detail, um, or catch me anytime. Don't miss the sessions, and yeah, that's it.