Serverless runtimes for agents, and where the loop lives
Pull-quote: “A serverless function is a request that ends. An agent loop is a conversation that pauses. Every hard part of putting one inside the other comes from that mismatch.”
Serverless is the wrong axis for the agent runtime decision. The question that decides your architecture is where the loop lives between two model calls, and what holds state while nothing is executing. Answer that and the plan tier and the bill follow. Answer “is serverless cheaper” first and you will pick a runtime that cannot hold a forty-step task.
This post is for an engineer choosing a host for an agent that already works locally. Every platform limit quoted here comes from vendor documentation and is cited inline, because these numbers change and a stale number is worse than none.
What you will be able to do
- State the execution ceiling for Lambda, Lambda MicroVMs, Azure Functions, and AgentCore Runtime, and name the one that constrains your workload.
- Explain why an HTTP-triggered Azure Function times out at 230 seconds even when its configured timeout is 30 minutes.
- Decide whether cold start is a real cost for your agent or an inherited worry from web-service thinking.
- Choose between externalised state, durable execution, and a long-lived process, and say what each costs.
- Write a checkpointed agent step and prove it does not re-execute after an interruption.
- Sort agent workloads by run length against whether a human is waiting.
Why does an agent loop break the serverless request model?
Because a serverless function is a request that ends, and an agent loop is a conversation that pauses. A function invocation begins with an input, computes, returns, and its memory is gone. An agent loop makes a model call, waits, executes a tool, waits again, and needs everything it learned in step three still present at step eleven. What survives the gap between two model calls is the whole question.
That gap is longer than most teams expect. A single model call on a reasoning-heavy task can take tens of seconds, and an agent that makes twelve of them has spent most of its wall-clock time waiting on a network response. Billing favours serverless here: AgentCore Runtime bills CPU against actual active processing and typically does not charge during I/O waits (AgentCore Runtime documentation), while a reserved container pays for the waiting. Correctness cuts the other way, because every pause is a place a scale-in event or a deploy can kill the process.
What are the actual execution limits, and which one bites first?
The ceiling that stops most agent workloads is not the one on the pricing page. Here are the documented limits, with the source for each.
| Runtime | Execution ceiling | Source |
|---|---|---|
| AWS Lambda, standard function | 900 seconds, not adjustable | Lambda quotas |
| AWS Lambda durable functions | One year per durable execution | Lambda durable functions |
| AWS Lambda MicroVMs | 28,800 seconds, not adjustable | Lambda quotas |
| Bedrock AgentCore Runtime | 8 hours per session | AgentCore Runtime |
| Azure Functions, Flex Consumption | 30 minutes default, no maximum | Azure Functions hosting |
| Azure Functions, Consumption, legacy | 5 minutes default, 10 maximum | Azure Functions hosting |
Three limits absent from that table decide more deployments than the ones in it.
The 230 second wall. Regardless of the function app timeout, 230 seconds is the maximum time an HTTP-triggered Azure Function can take to respond, because of the default idle timeout of the Azure Load Balancer in front of it. A function configured for 30 minutes still has its response dropped at 230 seconds, and Microsoft’s guidance is the Durable Functions asynchronous pattern. This is the most common surprise in Azure agent deployments: the timeout looks generous and the failure looks like a network fault.
The scale-in grace period. On Flex Consumption and Premium plans, a function execution gets a 60 minute grace period during scale-in and a 10 minute grace period during platform updates. A long run is not protected by having no timeout, it is protected for 60 minutes and then the platform reclaims the instance.
Worker startup. Azure Functions allows a maximum of 60 seconds for the language worker process to start, and that value is not configurable. An agent that imports a heavy dependency tree at module scope hits this first, and the error will not mention your agent.
Does cold start matter for an agent loop?
Only when a person is waiting for the first token, and then it matters enormously. In every other case it is a rounding error against the model latency you were going to pay anyway.
Split it by who is watching. In a human-facing turn a cold start adds seconds in front of a model call that already takes seconds, and the interaction feels broken rather than slow. The fix is capacity: Flex Consumption and Premium plans support always-ready instances, and Azure Container Apps avoids the problem when the minimum replica count is at least one. In a background loop nobody perceives the cold start, and a nightly agent that runs for eleven minutes does not care that the first 800 milliseconds started a runtime.
A third case changes the trade, because snapshot-based resume gives warm state without a warm bill. Lambda MicroVMs boot from a pre-initialised Firecracker snapshot, and a suspended MicroVM resumes with memory and disk intact in sub-second to single-digit seconds, which is why they suit interactive coding sandboxes (Lambda MicroVMs guide). Suspend and resume run from an idle policy with autoResumeEnabled, maxIdleDurationSeconds, and suspendedDurationSeconds. A user who steps away for lunch costs nothing and returns to the same shell.
Where does agent state live between steps?
There are three answers, and the one you pick determines everything else about the runtime.
Externalise every step. After each iteration, write the message history, the plan, and the tool results to a store, and read them back on the next invocation. Any stateless runtime works. Two costs follow. Latency accumulates, a round trip on every step multiplied by the step count. And the store becomes a correctness surface, because a partial write between the tool executing and the result persisting produces a run that repeats a side effect on resume.
Keep the process alive. State stays in memory for the length of the session, the reason container-hosted agents feel easy. AgentCore Runtime is the serverless version: a dedicated microVM per session with isolated CPU, memory, and filesystem, up to 8 hours. You pay for it in the ceiling, and in a crash at minute fifty losing fifty minutes.
Checkpoint and replay. The runtime records each completed operation and, after an interruption, re-runs your code from the top while substituting stored results for work already done. This is durable execution, the only one of the three that survives an arbitrary process death without recovery code you wrote.
What is durable execution, and why does it change the answer?
Durable execution is a runtime mechanism that saves a checkpoint after each completed operation, then recovers from a failure by replaying your function from the beginning and skipping operations already in the log. AWS Lambda durable functions implement it for executions lasting up to one year (Lambda durable functions).
Your function receives a DurableContext and calls five operations through it: step, wait, wait-for-condition, callback, and invoke (durable execution SDK). The mapping to an agent is direct. One iteration becomes one step, with retries and checkpointing supplied. A human approval becomes a callback that consumes no compute while it waits. A retry in four hours becomes a wait that costs nothing.
async def handler(event, context):
plan = await context.step("plan", lambda: model_plan(event["goal"]))
for i, task in enumerate(plan.tasks):
result = await context.step(f"task-{i}", lambda: run_tool(task))
if result.needs_human:
await context.callback(f"approve-{i}") # no compute while waiting
return await context.step("summarise", lambda: model_summary(plan))Two constraints come with it, both load-bearing. Your code must be deterministic, and the SDK enforces this by validating that operation names and types match the checkpoint log, so a step name built from a timestamp fails on resume. The log also has a size: Lambda allows 3,000 durable operations per durable execution and 100 MB of persisted storage (Lambda quotas). An agent that checkpoints every token exhausts both.
Azure has the same mechanism under another name. The Durable Task extension for Microsoft Agent Framework gives agents persistent sessions and automatic checkpointing, on Azure Functions or your own compute. Microsoft states that sessions survive crashes, restarts, and scaling events, that approvals and timed waits last hours to weeks without consuming compute, and that a Flex Consumption plan scales to thousands of concurrent sessions or to zero (Durable Task extension for Microsoft Agent Framework).
Prove the mechanism is real rather than configured. Run a two-step agent, kill the process between step one and step two, and invoke it again. Step one must not run twice, its result should come back from the log. Two calls in your external system mean the operation is non-deterministic or outside a step.
Which agent shapes fit serverless, and which do not?
Two axes decide it: how long a session runs, and whether a human is waiting on the next token.
Short and unwatched is the easy quadrant. Event-triggered enrichment, a classification agent behind a queue, a nightly summary: a plain function on a scale-to-zero plan is correct and cheap, and cold start is invisible.
Long and unwatched is the durable execution quadrant. Multi-day approvals, batch review of a thousand documents, an agent waiting on a supplier. Do not solve it with a bigger timeout. The failure you are designing against is the platform reclaiming your instance at minute sixty-one.
Short and watched is the always-ready quadrant. A chat turn that calls three tools and answers. Keep it under the 230 second HTTP wall, provision always-ready instances, and stream tokens so time to first token is what the user measures.
Long and watched is where plain serverless functions stop being the right tool. An interactive coding session, a browsing agent a human is steering, anything with bidirectional streaming. You want a live process with a session identity: AgentCore Runtime, a suspendable MicroVM, or a container with a minimum replica count. Here the real question is session affinity.
Where this goes wrong
The HTTP response died at 230 seconds and the logs looked fine. The symptom is a client timeout while the function log shows the run completing two minutes later. The cause is the load balancer idle timeout in front of every HTTP-triggered function, independent of your configured timeout. Return an immediate acknowledgement with a run identifier and let the client poll.
Conversation state outgrew the durable backend. The symptom is an agent that works for twenty turns and then fails to persist. On the Durable Task Scheduler, session state including full conversation history is subject to the backend’s state-size limits, and the maximum entity state size is 1 MB. Compaction is not automatic: Microsoft’s documented approach is to start a new session and summarise the prior context, the discipline in treating context as a budget.
Replay ran the side effect twice. The symptom is a duplicate payment, ticket, or email after a resume. The cause is work performed outside a durable operation, or a step whose name is not stable across replays. Every side effect belongs inside a step, and step names must derive from the plan rather than a clock or a random value.
Streaming was assumed and the backend is request-response. The symptom is a UI that shows nothing for forty seconds and then the whole answer at once. Durable agents built on durable entities are request-response underneath, and streaming happens through response callbacks while the entity returns the complete response afterwards. A durable scheduler also adds latency against in-memory execution, so measure it as in engineering LLM cost and latency.
Common questions
Can an AI agent run on AWS Lambda?
Yes, if the agent finishes inside 900 seconds, Lambda’s hard function timeout, which cannot be raised. Longer agents need one of the alternatives AWS provides: durable functions for checkpointed executions up to a year, Lambda MicroVMs for stateful sessions up to 8 hours, or AgentCore Runtime for hosted sessions up to 8 hours.
Is serverless cheaper than a container for an agent?
Often, for a reason unrelated to the usual serverless argument. Agents spend most of their wall-clock time waiting for model responses, and a runtime that bills active processing rather than reserved capacity does not charge for that waiting, while a container does. The saving disappears if you buy always-ready instances for a workload nobody watches.
What is the maximum timeout for an Azure Function?
The Flex Consumption, Premium, Dedicated, and Container Apps plans default to 30 minutes and enforce no maximum, while the legacy Consumption plan defaults to 5 minutes with a 10 minute maximum. The number that matters more is 230 seconds, the ceiling on how long an HTTP-triggered function can take to respond.
What does durable execution actually give an agent?
Recovery without recovery code. The runtime checkpoints each completed operation and, after a crash or a planned pause, replays your function while substituting stored results for finished work, so a run interrupted at step nine resumes at step ten. It also makes waiting free, so an approval can take days without holding compute.
Do I need durable execution if my agent finishes in two minutes?
No. A two-minute agent that can be safely retried from the beginning does not need checkpointing, and adding it costs latency and a determinism constraint for no gain. The threshold is the first step with a side effect you cannot repeat.
Next steps
Pick your quadrant before you pick a plan. Write down your ninety-fifth percentile session length and whether a human is waiting, then check the limit for the runtime you were about to choose against that number rather than your average.
Then read how much of the reasoning loop you hand to AWS, because hosting and loop ownership are usually decided in the same meeting, and answering them separately produces a runtime that cannot host the loop you chose. If the word “agent” is doing too much work in your planning, what actually makes something an agent supplies the ladder behind the run-length estimate. The component view is in the agent harness, not the model, decides if your agent works.
The checkpointed overnight-batch shape is how we operate scheduled agent work behind FreightCortex. Field notes sit in the Signals index.