# Building a Function-as-a-Service Platform: From MicroVMs to Millisecond Metering

## Blog Details

- **Author**: Naveen R.
- **Date**: September 8, 2026
- **Tags**: serverless, microvm, aws-lambda, cloud-architecture, faas
- **Read Time**: 20 mins

## Introduction

You upload a zip file. Some milliseconds later a function runs, and you are charged for exactly the milliseconds it ran. Nothing was provisioned, nothing was reserved, and if a thousand requests arrive at once, a thousand copies of your code run in parallel. That is the promise, and the promise is doing an enormous amount of hiding.

Underneath, someone had to solve a genuinely hard problem: run untrusted code from tens of thousands of mutually hostile tenants on shared hardware, start a fresh execution environment in tens of milliseconds, pack hundreds of them onto a single host to make the economics work, and account for usage precisely enough to bill it. Those four requirements fight each other. Strong isolation is normally slow and heavy. Fast startup normally means reusing a warm process, which leaks state. High density normally means weakening the boundary between tenants.

This post is about how that fight is actually won. We are going to build a Function-as-a-Service platform in the class of AWS Lambda and Google Cloud Functions, from the virtual machine monitor up: the isolation primitive and why microVMs beat both containers and conventional VMs, the guest boot path, the split between control plane and data plane, placement across a worker fleet, the synchronous and asynchronous invoke paths, snapshot restore to make cold start mostly disappear, burst concurrency and bin-packing, networking and code delivery, the multi-tenant threat model, millisecond metering, and the failure modes you only learn about after you run the thing.

Two notes on what follows. Performance numbers are drawn from published benchmarks, the Firecracker NSDI 2020 paper, and AWS and Google public documentation; real figures vary with region, instance type, runtime, and workload shape. Code is illustrative and compressed to the load-bearing lines, not production implementation.

Here is the whole system before we take it apart.

![High level architecture of a microVM based serverless platform, showing callers reaching an invoke frontend through a load balancer, the frontend consulting a concurrency limiter and an assignment service, the assignment service resolving function configuration from the control plane registry and claiming capacity from the worker manager, a worker host booting or resuming a microVM that pulls its code bundle and snapshot from a code store, and telemetry flowing out to a metering stream, with a separate asynchronous path through a queue and poller.](https://d5osvdbc8um23.cloudfront.net/static-asset/blog_images/system-design-serverless-microvm/01-high-level-architecture.png)

## What a Serverless Compute Platform Must Actually Guarantee

Before choosing any technology, write down what the platform promises. Every design decision later is a consequence of one of these five guarantees.

**Isolation against a hostile tenant.** This is not "tenants should not interfere with each other by accident." The assumption is that some fraction of the code running on your fleet is actively trying to escape, read another tenant's memory, or reach your control plane credentials. The boundary has to hold against someone who is deliberately attacking it, and the platform has to stay safe when a boundary eventually does not hold.

**Startup latency low enough to be inside a request.** A user is waiting on an HTTP response. If creating a fresh execution environment costs a second, the product does not work for interactive traffic, and you are pushed toward keeping environments warm, which reintroduces state leakage. The target for a lightweight runtime is a cold path in the low hundreds of milliseconds and a warm path where platform overhead is a small single-digit number of milliseconds.

**Density high enough that the unit economics close.** You bill per gigabyte-second at a rate around $0.0000166667. That is roughly six cents per gigabyte-hour, against an on-demand host that costs real money per hour. The only way the margin exists is packing: hundreds of concurrent execution environments per physical host, with per-environment overhead measured in single-digit megabytes rather than the hundred-plus megabytes a conventional VM costs. A microVM at roughly 5 MB of overhead is what makes a 256 GB host able to hold hundreds of 512 MB functions instead of a few dozen.

**Elasticity from zero to thousands of concurrent executions.** Lambda publishes an initial burst of 500 to 3,000 concurrent executions depending on region, then a sustained growth rate of an additional 500 per minute. Behind that number is a placement system that must make thousands of scheduling decisions per second, and a fleet that can materialize execution environments at that rate.

**Metering precise enough to bill and cheap enough to ignore.** Billing granularity is 1 ms. Every invocation therefore produces a usage record, and at platform scale that is millions of records per second. The metering path cannot involve a synchronous database write, and it cannot lose records, because the output is an invoice.

Notice what is absent from that list: nothing says the execution environment must be a container, must be reused, or must even be Linux. Those are implementation choices, and the interesting ones.

## The Isolation Decision: Processes, Containers, gVisor and MicroVMs

This is the decision the entire platform is built on, so it deserves the most scrutiny.

| Mechanism | Startup | Memory overhead | Isolation boundary | CPU performance | Safe for hostile multi-tenancy |
|-----------|---------|-----------------|--------------------|-----------------|-------------------------------|
| OS process | under 1 ms | about 1 MB | Same kernel, same host | Native | No |
| Container | 50 to 100 ms | tens of MB | Same kernel, namespaced | 98 to 99% of native | Not on its own |
| gVisor sandbox | 100 to 200 ms | tens of MB | Userspace kernel reimplementation | 85 to 95% of native, syscall-sensitive | Yes |
| MicroVM | around 125 ms | about 5 MB | Hardware virtualization (KVM) | 96 to 98% of native | Yes |

**Processes are eliminated instantly.** Two tenants in two processes on one kernel share the entire Linux system call surface. Any local privilege escalation bug in the kernel is a full cross-tenant compromise. This is fine for running your own code and unacceptable for running the internet's code.

**Containers are the interesting near miss.** A container is a process with namespaces, cgroups, capabilities dropped, and a seccomp filter. It starts fast and it is cheap. But the isolation boundary is still the kernel system call interface, which is enormous: hundreds of syscalls, each with a large parameter space, plus `/proc`, `/sys`, ioctls, and filesystem interactions. The historical record is unambiguous: runc's CVE-2019-5736 let a container overwrite the host runtime binary and obtain root on the host. That class of bug keeps appearing because the attack surface is fundamentally large. Containers are excellent for packaging and for your own trusted workloads. As the sole boundary between mutually hostile tenants, they ask you to bet on the absence of bugs in a million-line interface.

**gVisor shrinks the surface by reimplementing it.** Instead of letting guest syscalls reach the host kernel, gVisor interposes a userspace kernel called the Sentry, written in Go, which implements a large subset of the Linux syscall interface itself, and a separate Gofer process that brokers filesystem access. A guest syscall is trapped and serviced in userspace; only a small, audited set of operations reach the host. The cost is real: syscall-heavy and I/O-heavy workloads pay noticeably, because what was a fast host syscall is now a userspace round trip. Google runs Cloud Run and second-generation Cloud Functions on this model, which is direct evidence it works at scale.

**MicroVMs move the boundary into hardware.** With KVM, the guest gets its own kernel, and the boundary is the CPU's virtualization support plus the virtual machine monitor's device emulation. The guest can make any syscall it likes; those go to the guest kernel, not yours. Compromising the guest kernel gets an attacker nothing except a kernel they already controlled.

Historically the objection to VMs was weight. A QEMU-based VM boots firmware, enumerates a rich emulated machine, and costs on the order of 100 MB or more of host memory. Firecracker's insight was that a serverless VM does not need any of that. It is a purpose-built VMM in Rust, roughly 50,000 lines against QEMU's million-plus, and it emulates almost nothing: virtio-block, virtio-net, virtio-vsock, a serial console, and a partial i8042 keyboard controller that exists only so the guest can trigger a reset. No PCI enumeration, no BIOS or UEFI, no graphics, no USB, no ACPI. That is the entire machine.

The result is a boundary with two properties containers cannot offer together: it is hardware-enforced, and the software portion of it is small enough to actually audit. Firecracker also confines itself, running behind a seccomp allowlist of a few dozen syscalls, so even a compromised VMM has almost no host interface to attack.

So: **microVMs for the data plane**, where untrusted tenant code executes, and ordinary **containers for the control plane**, where only your own trusted services run. Different threat models deserve different answers, and paying microVM costs for your own placement service buys nothing.

## Inside a MicroVM: The Minimal VMM, the Jailer and the Guest Boot Path

Now build the execution environment. On each worker host, a worker agent owns a set of slots; each slot is one microVM.

![Anatomy of a microVM slot on a worker host, showing the worker agent spawning a jailer per slot, the jailer applying chroot, cgroups and seccomp before executing the unprivileged VMM, the agent configuring the VMM over a unix domain socket, the VMM calling into /dev/kvm and booting a minimal guest kernel that runs a tiny init which execs the language runtime, and the VMM exposing a read only rootfs over virtio-block and a tap device over virtio-net.](https://d5osvdbc8um23.cloudfront.net/static-asset/blog_images/system-design-serverless-microvm/03-microvm-anatomy.png)

### The jailer runs first

Before the VMM exists, a small setuid helper builds the box the VMM will live in. This ordering matters: the VMM never runs with privileges it could lose later, because it never had them.

```bash
# Jailer: construct the sandbox, then exec the VMM inside it (illustrative)
JAIL=/srv/jailer/fc/$VM_ID/root
mkdir -p "$JAIL/dev"
mknod "$JAIL/dev/kvm" c 10 232 && chown fc:fc "$JAIL/dev/kvm"

# hard resource ceilings before any guest code exists
cgcreate -g cpu,memory,pids:/fc-$VM_ID
cgset -r memory.max=$((MEM_MB + 8))M   -g memory  fc-$VM_ID
cgset -r cpu.max="$CPU_QUOTA 100000"   -g cpu     fc-$VM_ID
cgset -r pids.max=64                   -g pids    fc-$VM_ID

# unshare namespaces, chroot, drop to an unprivileged uid, then exec
exec cgexec -g cpu,memory,pids:fc-$VM_ID \
  unshare --pid --net --ipc --mount --fork \
  chroot "$JAIL" \
  setpriv --reuid=fc --regid=fc --clear-groups --no-new-privs \
  /firecracker --api-sock /run/fc.sock --seccomp-level 2
```

Read what that buys. The chroot is nearly empty, so a VMM compromise lands the attacker in a directory with no binaries and no host filesystem. The uid is unprivileged and `no-new-privs` prevents regaining anything through setuid binaries. `pids.max` means a fork bomb inside the VMM process tree hits a wall. `memory.max` is set to the guest's memory plus a small allowance, so a VMM memory bug cannot starve the host or its neighbours. The network namespace is fresh and contains exactly one interface. And seccomp level 2 applies the allowlist, so the syscalls available for a host attack number in the dozens rather than the hundreds.

### Configuring and starting the machine

The VMM exposes a REST API over a unix domain socket rather than taking a giant command line. The agent describes the machine, then starts it.

```python
# Worker agent: define the machine over the VMM API socket (illustrative)
def start_slot(sock, cfg, slot):
    put(sock, "/machine-config", {
        "vcpu_count": cfg.vcpus,
        "mem_size_mib": cfg.memory_mb,
        "smt": False,              # no hyperthread sharing across tenants
    })
    put(sock, "/boot-source", {
        "kernel_image_path": "/vmlinux",
        # no init discovery, no udev, no modules: boot straight to our init
        "boot_args": "console=none reboot=k panic=1 pci=off i8042.noaux=1 "
                     "quiet init=/sbin/slot-init",
    })
    put(sock, "/drives/rootfs", {
        "drive_id": "rootfs", "path_on_host": "/rootfs.img",
        "is_root_device": True, "is_read_only": True,
    })
    put(sock, "/network-interfaces/eth0", {
        "iface_id": "eth0", "host_dev_name": slot.tap, "guest_mac": slot.mac,
    })
    put(sock, "/actions", {"action_type": "InstanceStart"})
```

Three details in the boot arguments carry most of the startup win. `pci=off` skips bus enumeration, which is one of the slowest parts of a normal Linux boot. `console=none` removes serial output, which is surprisingly expensive when the host is busy. `init=/sbin/slot-init` replaces the entire userspace bring-up (no systemd, no udev, no service dependency graph) with one static binary whose only job is to set up the runtime and start polling for work. `smt: False` is a security decision rather than a performance one: it keeps a tenant from sharing a physical core's sibling thread with another tenant, which is the precondition for a whole family of microarchitectural side channels.

### What actually happens in those 125 milliseconds

The guest boot path is short because almost everything a normal boot does has been deleted.

1. **No firmware.** There is no BIOS or UEFI stage. The VMM loads an uncompressed `vmlinux` and jumps to the 64-bit entry point with a boot parameter structure it wrote directly into guest memory. A conventional VM spends tens to hundreds of milliseconds here; a microVM spends zero.
2. **Kernel init, roughly 20 to 40 ms.** A stripped kernel with no module loading, no ACPI, no USB, no graphics, and virtio drivers built in. There is nothing to probe because there are only two real devices.
3. **Init, about 1 ms.** A static binary, not an init system. It mounts `/proc`, brings up `eth0` from static configuration handed in through the metadata channel, mounts the tmpfs for scratch space, and execs the runtime.
4. **Runtime initialization, the part that actually varies.** This is where the boot budget goes and where the language choice dominates. A Python or Node process that imports a handler with modest dependencies is fast. A JVM with a Spring context, or a Python function importing a large ML library, can spend hundreds of milliseconds to seconds. The platform's boot path is a fixed cost of about 125 ms to a running userspace; everything above that belongs to the tenant's code.

That last point is why the cold start problem cannot be solved purely by making the VMM faster. Once the platform's own contribution is down to roughly 125 ms, the remaining latency is the tenant's runtime doing work, and the only way to remove work that has already been done is to not do it again. That is the argument for snapshots, and we will get there.

### The device model, and why so little of it

```rust
// The entire machine: two data-path devices (illustrative)
let mut vm = Vm::new(kvm.create_vm()?)?;
vm.set_user_memory_region(GuestMemoryMmap::from_ranges(
    &[(GuestAddress(0), cfg.mem_size_mib << 20)])?)?;

// read-only rootfs, shared across every slot running this function version
vm.add_device(VirtioBlock::new(rootfs_fd, /* read_only */ true)?)?;
// one tap per slot, created and owned by the host
vm.add_device(VirtioNet::new(tap_fd, cfg.guest_mac)?)?;

for i in 0..cfg.vcpu_count {
    let vcpu = vm.create_vcpu(i)?;
    thread::spawn(move || vcpu.run());   // one host thread per guest vCPU
}
```

Every emulated device is attack surface, because device emulation is host code parsing guest-controlled input. The historical VM escapes have overwhelmingly been device emulation bugs. Two virtio devices plus vsock and a serial line is a surface small enough to review line by line, and that is the actual security argument for a minimal VMM, more than the memory savings.

## Control Plane: Function Registry, Placement and the Worker Fleet

The control plane knows about functions and hosts. It deliberately never touches an invocation payload, for two reasons: payloads are the tenant's data and should traverse as few systems as possible, and a control plane outage must not stop invocations that are already placed.

![Control plane and placement, showing a developer calling the control plane API which validates and versions the function, writes configuration to the function registry, stores the code bundle by digest in the code store, and enqueues a bake job; a package builder producing a rootfs and a pre warmed snapshot into the snapshot store; and a placement service reading version configuration and a capacity index to score and assign a slot on the worker fleet, with a worker manager collecting heartbeats and feeding the capacity index.](https://d5osvdbc8um23.cloudfront.net/static-asset/blog_images/system-design-serverless-microvm/02-control-plane-placement.png)

### The registry, and why versions are immutable

```json
{
  "function_id": "fn-7Kq2",
  "version": 14,
  "state": "ACTIVE",
  "runtime": "python3.12",
  "handler": "app.handler",
  "memory_mb": 512,
  "timeout_sec": 30,
  "code": {
    "digest": "sha256:9f2c...",
    "uri": "s3://fn-code/fn-7Kq2/14.zip",
    "rootfs_digest": "sha256:1ab4...",
    "snapshot_id": "snap-fn-7Kq2-14"
  },
  "env": { "DB_HOST": "..." },
  "role_arn": "arn:aws:iam::...:role/fn-7Kq2-exec",
  "vpc": { "subnets": ["subnet-a"], "security_groups": ["sg-1"] },
  "reserved_concurrency": null,
  "provisioned_concurrency": 0
}
```

Publishing a version is a copy-on-write operation: it never mutates an existing version, it creates a new one. This is not bookkeeping fastidiousness, it is what makes the rest of the system tractable. A slot is warm *for a specific version*, so warm-slot reuse is only ever safe if versions are immutable. A snapshot is a snapshot of a version. A rollback is repointing an alias, not rebuilding anything. And two invocations of "the same function" that arrive during a deployment get consistent behaviour, because each is bound to a concrete version at the moment of assignment.

### Baking, done once, off the invoke path

Publishing kicks off asynchronous work that the invoke path must never do: fetch the bundle, verify its digest, build a read-only rootfs image containing the runtime and the handler code, boot one microVM from it, let the runtime reach steady state, and snapshot it. All of that is expensive and all of it is amortized across every future invocation of the version. The invoke path's job is reduced to "find a host that has these artifacts, or can get them quickly."

### Fleet state, and the honest problem with it

The worker manager holds the fleet's capacity picture, assembled from heartbeats:

```go
type WorkerState struct {
    WorkerID     string
    TotalMemMB   int
    CommittedMB  int               // sum of slot memory, not RSS
    ActiveSlots  int
    WarmVersions map[string]int    // version -> idle slot count
    Generation   uint64            // kernel/agent build, for staged rollout
    LastBeat     time.Time
    Draining     bool
}
```

Two properties of this table are worth stating plainly because they shape the placement algorithm.

It is **stale by design**. Heartbeats arrive on an interval, and thousands of placement decisions happen between two heartbeats from the same host. The placement service is therefore always working from a slightly wrong view, and the worker is the authority: a worker that receives a placement it cannot honour rejects it, and the placer retries elsewhere. Any design that assumes the capacity index is accurate will over-commit hosts under burst.

It tracks **committed memory, not used memory**. A 512 MB function is charged 512 MB of the host's budget whether it touches 40 MB or 500 MB. Overcommitting against actual usage would raise density and is exactly the sort of optimization that produces a correlated outage when many tenants get busy simultaneously.

### Placement is a multi-objective problem

The naive goal is bin-packing: fill hosts tightly, buy fewer hosts. Real placement optimizes several things that conflict.

```python
# Score candidate hosts; lower is better (illustrative)
def place(version, fleet, tenant):
    need = version.memory_mb + VMM_OVERHEAD_MB
    best, best_score = None, math.inf
    for w in fleet.healthy():
        if w.draining or w.free_mb() < need:
            continue
        score = 0.0
        # 1. locality: this host already has the artifacts, so no fetch
        if version.id in w.warm_versions:      score -= 100
        elif version.rootfs_digest in w.cache: score -= 40
        # 2. blast radius: avoid stacking one tenant on one host
        score += 25 * w.tenant_share(tenant)
        # 3. noisy-neighbour spread for large slots
        if version.memory_mb >= 3008:          score += 10 * w.large_slots
        # 4. packing, as a tiebreaker only
        score += 0.1 * (w.free_mb() / w.total_mb)
        if score < best_score:
            best, best_score = w, score
    return best   # None: fleet is full, scale out and throttle meanwhile
```

The ordering of those terms is the whole design. Locality dominates because a host that already holds the rootfs and snapshot can produce a slot in milliseconds while a cold host must fetch hundreds of megabytes first. Blast radius is second and it is a security term, not a performance one: concentrating one tenant on one host means a single host compromise takes all of that tenant's traffic, and concentrating many tenants on one host means one host failure hits many tenants. Packing density is last, a tiebreaker. A platform that optimizes packing first gets a cheap fleet with terrible tail latency and a bad blast radius.

Placement must also be sharded. At thousands of decisions per second a single placer is both a bottleneck and a single point of failure, so shard by function id, accept that shards have independent and slightly inconsistent views of capacity, and let worker rejection reconcile the difference.

## Data Plane: The Synchronous and Asynchronous Invoke Paths

The data plane is what a request actually traverses. Its design goal is a short path with few dependencies, because everything on it is in the latency budget and everything on it can fail the request.

![Invoke path and scaling, showing a caller reaching the invoke frontend, a concurrency controller that either admits the request or returns a throttle response, an assignment service consulting an idle slot index that either hits a warm slot for reuse or misses and restores a new slot from snapshot, both paths converging on handler execution and a response, plus a separate asynchronous queue path with retries and a dead letter queue, and a scaling loop growing the slot pool.](https://d5osvdbc8um23.cloudfront.net/static-asset/blog_images/system-design-serverless-microvm/05-invoke-path-scaling.png)

### Synchronous invocation, step by step

1. **Frontend.** Terminates TLS, authenticates the caller, authorizes the action against the function's policy, and validates the payload (Lambda caps a synchronous request at 6 MB). Resolves the alias to a concrete version.
2. **Admission control.** Checks the account and function concurrency counters. This has to happen before placement, because the cheapest way to survive an overload is to reject work before spending resources on it.
3. **Assignment.** Looks for an idle slot already warm for this version. A hit is the common case in steady state and costs about a millisecond. A miss means restoring a slot from snapshot, or on a cold host, fetching artifacts first.
4. **Execution.** The payload is handed to the slot over its local channel. **One concurrent request per slot**, always. This is the single most important invariant in the data plane, and it is worth being explicit about why. It means a handler never has to be thread-safe, a slow request cannot contend with a fast one inside the same environment, and the memory limit is per request rather than shared. It also means concurrency and slot count are the same number, which is what makes the scaling model comprehensible to users.
5. **Response.** Streamed back through the frontend. The slot is returned to the idle pool for its version.

Steady-state warm overhead here is a small number of milliseconds of platform work on top of the handler's own duration.

### Where the state lives

The idle slot index is the hottest structure in the platform: read on every invocation, mutated twice per invocation. It cannot be a strongly consistent global store at these rates. In practice it is partitioned by version and cached close to the assignment layer, with the worker as the authority. Assignment optimistically hands out a slot; the worker confirms or rejects; a rejection costs a retry rather than a correctness failure. Designing for optimistic assignment plus worker authority is what lets the fast path stay fast.

### Asynchronous invocation is a different product

An asynchronous invoke returns as soon as the event is durably enqueued (Lambda caps the event at 256 KB), and the platform takes responsibility for eventually running it. That is a much stronger promise than the synchronous path makes, and it requires machinery the synchronous path does not have:

- **Durability.** The event is persisted before the caller gets its acknowledgement.
- **Retries with backoff.** Lambda retries an asynchronous invocation twice by default, with delay between attempts.
- **A dead letter destination.** After retries are exhausted the event goes to a queue or topic rather than being dropped, because dropping it silently would be the worst possible behaviour for a system that promised to run it.
- **Poller fleet sizing.** The pollers draining the queue must scale with backlog, but they must also respect the function's concurrency limit, or a large backlog would translate directly into a thundering herd against the tenant's own downstream dependencies.
- **At-least-once semantics, stated honestly.** A retry after a partial success means handlers must be idempotent. This is a real constraint pushed onto the tenant, and pretending otherwise causes duplicate-processing bugs in production.

The important architectural note: the poller re-enters the platform through the same synchronous path. There is one execution path, wrapped in different delivery semantics. Two independent execution paths would double the surface where the two can disagree.

## Killing Cold Start: Snapshots, Copy-on-Write Restore and Pre-warm Pools

We established that the platform's own boot path is roughly 125 ms and that the remaining cold start is tenant runtime initialization. Snapshots attack both at once, by doing the work once and then repeatedly restoring the result.

![Snapshot and restore pipeline for eliminating cold start, showing a newly published version cold booting the VMM and guest kernel then initializing the runtime, a snapshot taker pausing the VM and serializing guest memory to a memory file and vCPU and device registers to a state file, both landing in a per version snapshot store, and a restore path that memory maps the snapshot with lazy page in guided by a working set file to resume a slot in about ten milliseconds into a warm pool from which invocations are assigned.](https://d5osvdbc8um23.cloudfront.net/static-asset/blog_images/system-design-serverless-microvm/04-snapshot-restore-cold-start.png)

### Taking the snapshot

```rust
// Snapshot a VM that has reached steady state (illustrative)
fn snapshot(vm: &mut Vm, id: &str) -> Result<SnapshotRef> {
    vm.pause_vcpus()?;                         // quiesce: no vCPU mutating memory
    vm.flush_virtio_queues()?;                 // no in-flight device descriptors

    // guest RAM, page aligned so it can be mmap'ed directly on restore
    vm.dump_guest_memory(&format!("/snap/{id}.mem"))?;
    // vCPU registers, MSRs, interrupt controller, virtio device state
    vm.save_state(&format!("/snap/{id}.state"))?;

    vm.resume_vcpus()?;
    Ok(SnapshotRef::new(id))
}
```

The order is not negotiable. Pause the vCPUs first or the memory file is a torn read of a moving target. Flush the device queues or you restore a VM that believes an I/O completion is coming that will never arrive. Getting either wrong produces a snapshot that restores successfully and then misbehaves subtly under load, which is a genuinely unpleasant class of bug.

The output is two files: guest memory, which for a 512 MB VM is up to 512 MB, and a small state blob. Taking it costs roughly the boot plus init time it captured, and it happens once per version rather than once per invocation.

### Restoring lazily is the whole trick

```rust
// Restore: map, do not copy (illustrative)
fn restore(id: &str, tap: RawFd) -> Result<Vm> {
    let mem = File::open(format!("/snap/{id}.mem"))?;
    // MAP_PRIVATE: pages arrive on fault, writes are private to this slot
    let guest = unsafe { mmap_private_file(&mem)? };

    let mut vm = Vm::new_with_memory(guest)?;
    vm.load_state(&format!("/snap/{id}.state"))?;   // registers, MSRs, devices
    vm.rebind_net(tap)?;                            // fresh tap, fresh MAC
    vm.resume_vcpus()?;                             // running, mid-poll-loop
    Ok(vm)
}
```

Nothing copies 512 MB. The memory file is mapped `MAP_PRIVATE`, so the guest's pages are faulted in from the file as they are touched, and any page the guest writes becomes a private copy. Two consequences follow, and they are the reason this technique works at all.

First, restore latency is decoupled from VM size, because it is proportional to the working set actually touched rather than the memory configured. Bringing a VM back to a running state is on the order of 5 to 10 ms of VMM work; the rest arrives on demand.

Second, one snapshot file backs many slots. Every slot running version 14 maps the same memory file, and pages neither slot has written are shared physical memory in the host page cache. Density improves precisely because most of a freshly restored guest's memory is never written.

The cost is a long tail of page faults during the first requests a restored slot serves. The fix is to record which pages the guest touches during a typical invocation and prefetch that working set at restore time, converting thousands of individual faults into a few sequential reads.

### Restoring is not resuming, and the difference is a security bug waiting to happen

A restored VM believes it is the same machine that was snapshotted, and it is right about that in a way that is dangerous. Everything derived from "unique per machine" state is now duplicated across every slot restored from that snapshot. Concretely:

- **Random number generator state is identical.** Every slot restored from one snapshot has the same entropy pool. If a handler generated a session token, an IV, or a nonce with that pool, thousands of slots produce the *same* value. This is not theoretical; it is the primary correctness hazard of VM snapshotting, and it is why the restore path must reseed the guest's entropy source before any tenant code runs.
- **Clocks are frozen at snapshot time.** The guest's notion of wall clock and monotonic time is stale by however long the snapshot sat on disk. Time must be corrected on resume or handlers see impossible timestamps and expired-looking credentials.
- **Network identity is stale.** The MAC, IP, and any open connection state belong to a machine that no longer exists. Networking is rebound to a fresh tap device, and the guest must be told to re-derive its configuration. Anything the runtime cached about connections, DNS, or credentials before the snapshot has to be invalidated.

A platform that snapshots without addressing all three ships a subtle, high-severity vulnerability. Lambda's SnapStart exposes this to users directly through runtime hooks, so a handler can re-randomize and refresh what it must after restore.

### Pre-warm pools and what they cost

```go
// Maintain a target number of restored, idle slots per hot version (illustrative)
func (p *Pool) reconcile() {
    target := p.predictor.Target(p.versionID)   // from recent arrival rate
    for p.idle.Len() < target && p.host.CanFit(p.version) {
        go func() {
            slot, err := restore(p.version.SnapshotID, p.host.AllocTap())
            if err != nil { p.metrics.RestoreFail.Inc(); return }
            p.idle.Push(slot)      // reseed entropy + fix clock happen in restore
        }()
    }
    for _, s := range p.idle.OlderThan(p.maxIdle) {
        s.Teardown()               // reclaim memory from cooled-off versions
    }
}
```

Pre-warming trades money for latency, and the trade is not subtle: a slot held ready is host memory committed to zero requests. For the platform's own opportunistic pools it is a bet on predicted arrival rates. When a customer wants a guarantee rather than a bet, the honest product is to let them pay for it explicitly, which is what provisioned concurrency is: reserved slots, billed for existing rather than for running.

The mechanisms compose into a latency ladder that is worth stating explicitly, because every optimization in this section is just moving invocations up it:

| Path | Cost | When |
|------|------|------|
| Warm slot reuse | about 1 ms | Steady-state traffic |
| Pre-warmed restored slot | about 1 ms | Predicted or paid-for capacity |
| Snapshot restore on a host with artifacts | roughly 10 to 50 ms | Scaling up a known version |
| Snapshot restore, artifacts fetched first | hundreds of ms | New version, cold host |
| Full cold boot with runtime init | 125 ms plus tenant init, sometimes seconds | No usable snapshot |

## Scaling, Burst Concurrency and Worker Bin-Packing

### Why burst is rate-limited at all

Lambda's published behaviour is an initial burst of 500 to 3,000 concurrent executions per region, then growth of 500 more per minute. Users experience this as an arbitrary limit. It is not arbitrary; it is the composition of several real constraints:

- **Per-host launch rate.** A host can bring up microVMs at roughly 150 per second, and that is competing with the requests it is already serving.
- **Artifact fetch bandwidth.** Scaling a version onto hosts that lack its rootfs and snapshot turns into hundreds of megabytes per host of fetch traffic. Unbounded scale-out is a self-inflicted distributed denial of service against your own code store.
- **Placement throughput.** Thousands of decisions per second against a fleet-wide, deliberately stale capacity view.
- **Fleet headroom is finite.** Instantaneous unbounded growth for one tenant is capacity taken from every other tenant. The rate limit is a fairness mechanism as much as a mechanical one.

Which is also why the throttle response is a feature. Returning a fast, cheap rejection at the frontend is dramatically better for everyone, including the throttled tenant, than accepting work the fleet cannot place and failing it slowly after consuming resources.

### The concurrency counter is the hard part

```go
// Admission: a lease, not a bare counter (illustrative)
func (c *Controller) Acquire(fn string, tenant string) (Lease, error) {
    if !c.tenantLimiter.Allow(tenant) {          // account-wide ceiling first
        return Lease{}, ErrAccountThrottled
    }
    s := c.shardFor(fn)
    s.mu.Lock()
    defer s.mu.Unlock()
    if s.active >= s.limit {
        return Lease{}, ErrFunctionThrottled     // 429, fast and cheap
    }
    s.active++
    // Leases expire. A crashed worker must not leak concurrency forever.
    return Lease{fn: fn, shard: s, expires: time.Now().Add(fnTimeout + slack)}, nil
}

func (l Lease) Release() {
    l.shard.mu.Lock(); l.shard.active--; l.shard.mu.Unlock()
}
```

Two things here are learned the hard way. The counter must be **leased with an expiry**, not incremented and decremented, because the process holding a decrement can die. A platform that leaks concurrency on every worker failure slowly throttles healthy functions to zero, and the symptom (throttling with no traffic) is baffling until you find it. And the counter must be **sharded**, because a single global counter at millions of invocations per second is a contention point; the price of sharding is slight imprecision at the limit boundary, which is a fine trade.

### Bin-packing against multiple resources

```python
# A slot fits only if it fits in every dimension (illustrative)
def can_fit(host, version):
    mem  = version.memory_mb + VMM_OVERHEAD_MB
    # CPU scales with configured memory: 1769 MB is one vCPU's worth
    vcpu = version.memory_mb / 1769.0
    return (host.free_mb        >= mem
        and host.free_vcpu      >= vcpu
        and host.slots          <  host.max_slots      # taps, fds, page tables
        and host.free_iops      >= version.expected_iops
        and host.tenant_share(version.tenant) < MAX_TENANT_SHARE_PER_HOST)
```

Memory is the dimension everyone models and rarely the one that binds first. Coupling vCPU allocation to configured memory (Lambda's ratio is one vCPU per 1,769 MB) is what makes the model tractable for users: one knob, and CPU scales with it. But hosts also run out of file descriptors, tap devices, page table capacity, and IOPS long before their memory is exhausted, and the per-tenant share cap can reject a placement on a host with plenty of everything. In practice a large host that could hold 500 slots by memory arithmetic sustains meaningfully fewer once CPU contention and I/O interference are accounted for, and the correct response is to measure achieved density rather than trust the arithmetic.

## Networking, Code Delivery and the Guest Filesystem

![Networking and storage for a slot, showing code bundles fetched by digest from the code store into a deduplicated host cache, mounted as a shared read only base rootfs with a per slot copy on write overlay and a separate scratch drive attached to the microVM, and the microVM's virtio-net traffic passing through a per slot tap device into a recycled network namespace, then through host NAT with a per slot bandwidth cap to the internet, alongside a VPC attachment plumbing a shared elastic network interface into the namespace, and credentials handed to the runtime at start.](https://d5osvdbc8um23.cloudfront.net/static-asset/blog_images/system-design-serverless-microvm/06-networking-and-storage.png)

### One tap and one namespace per slot

```bash
# Per-slot network: a namespace containing exactly one interface (illustrative)
ip netns add "slot-$ID"
ip tuntap add dev "tap-$ID" mode tap user fc
ip link set "tap-$ID" netns "slot-$ID"
ip -n "slot-$ID" addr add 169.254.0.1/30 dev "tap-$ID"
ip -n "slot-$ID" link set "tap-$ID" up

# rate limit the slot so one tenant cannot saturate the host NIC
tc -n "slot-$ID" qdisc add dev "tap-$ID" root tbf \
   rate 100mbit burst 32kbit latency 50ms

# every slot uses the same guest-side addresses; SNAT gives it a real identity
iptables -t nat -A POSTROUTING -s 169.254.0.2 -j SNAT --to-source "$HOST_IP"
```

The design choice worth calling out is that every slot's guest sees an identical private address, and the host translates. That makes the guest side of the network configuration a constant, which in turn is what allows a single snapshot to be restored into thousands of slots without any per-slot guest network configuration. It also means slots cannot address one another: there is no shared bridge, only a namespace per slot with one interface and a NAT gateway. Lateral movement between tenants has no path at layer 2.

Rate limiting is not optional. Without it, one tenant doing bulk data transfer degrades every other slot on the host, and "my function got slower and I changed nothing" is an unfixable support ticket.

### Reaching a customer VPC without paying for it on the cold path

Attaching a function to a customer's private network was historically the most expensive thing you could do to your cold start. The naive implementation creates an elastic network interface in the customer's subnet when an execution environment starts, and interface attachment takes on the order of ten seconds. That was the well-known VPC cold start penalty, and it was severe enough to make VPC-attached functions unusable for interactive traffic.

The fix is to stop doing per-slot attachment: create a small number of shared interfaces per subnet and security group combination, up front, and plumb slots into them through a network mapping layer. Attachment cost moves off the invoke path entirely and is amortized across every function sharing that subnet and security group. AWS shipped this in 2019 as Hyperplane-backed VPC networking. The lesson generalizes: when a per-request cost is dominated by resource creation, the answer is usually to create the resource once and multiplex, not to make creation faster.

### Code delivery is a caching problem

The invoke path cannot afford to download and unpack a code bundle. So the code store is content-addressed and the host cache is deduplicated by digest:

1. **Fetch by digest,** not by name. Content addressing makes the cache trivially correct, since a digest that is present is the right bytes, and no invalidation logic is needed.
2. **Deduplicate below the bundle level.** Most bundles are mostly runtime and shared dependencies. Chunking bundles and caching chunks means a new version of a function typically fetches only its changed chunks. AWS published this approach for container images on Lambda, using chunk-level deduplication with convergent encryption so that chunks can be shared across tenants without one tenant learning what another has deployed.
3. **Tier the cache.** Host-local first, then a zone-local cache, then the origin. Scaling a popular version to a thousand hosts should hit the origin once.
4. **Verify before use.** The digest is checked before the rootfs is built, because a corrupted or substituted bundle is arbitrary code execution as another tenant.

### The guest filesystem, deliberately boring

```
/                      read-only squashfs, shared by every slot on this version
├── bin, lib           minimal userspace, static where possible
├── opt/runtime        the language runtime
├── var/task           the handler code
├── tmp                writable tmpfs or scratch drive, 512 MB to 10 GB
└── proc, sys, dev     minimal, mounted by init
```

Root is read-only and shared, which is what makes hundreds of slots on a host cheap: they map the same pages. Writable state is confined to `/tmp` and lives and dies with the slot. There is no persistence, which is a deliberate constraint rather than a missing feature. If a slot could persist state, then warm reuse would leak data between invocations of different callers, and the stateless execution model that makes the whole platform safe to reuse would be gone.

One subtlety: `/tmp` survives across invocations *within* the same slot, because the slot is reused. Tenants discover this and cache things there, which is fine, but it means `/tmp` contents from a previous request in the same slot are visible to the next. That is the same tenant and the same version, so it is not a cross-tenant leak, but it is a real behaviour that has to be documented rather than accidental.

## Multi-Tenant Security and the Blast Radius Model

The threat model is explicit: assume tenant code is hostile, assume it is trying to escape, and assume that eventually something will work. The design goal is not "no compromise" but "compromise is contained, detected, and cheap to remediate."

### Layers, and what each one is actually for

1. **Hardware virtualization.** The guest kernel is the guest's problem. The boundary is CPU-enforced.
2. **A minimal, audited VMM.** Roughly 50,000 lines of Rust with two data-path devices. Device emulation is the historical source of VM escapes, so there is very little of it.
3. **VMM self-confinement.** The jailer's chroot, namespaces, cgroups, unprivileged uid, and a seccomp allowlist of a few dozen syscalls. This layer exists specifically to make a VMM compromise, layer 2's failure mode, not equal a host compromise.
4. **No sibling-thread sharing.** SMT is disabled for guest vCPUs so two tenants never share a physical core's sibling thread, which is the precondition for a large family of microarchitectural side channels.
5. **Slot lifecycle.** A slot serves one tenant and one version for its whole life, and is destroyed rather than reassigned. Reuse within one tenant and version is safe; reuse across tenants would put the entire isolation argument on the correctness of a scrubbing routine.
6. **Network isolation by construction.** A namespace per slot with one interface and no shared bridge. There is no layer 2 path between tenants to attack.
7. **Least-privilege credentials.** Each slot receives short-lived credentials for the function's own role, injected in-guest, with no route to the host's identity or the control plane.

Layers 2 and 3 are the pair that matters most and the one most often collapsed in descriptions of this architecture. The minimal VMM reduces the *probability* of escape. The jailer reduces the *consequence*. Neither substitutes for the other.

### Blast radius, quantified honestly

If a tenant escapes its guest and compromises the VMM, the jailer means the attacker holds an unprivileged process in an empty chroot with a few dozen syscalls. If they then defeat the jailer and the host kernel, they have one worker host: every slot currently on it, which is hundreds of executions, and the credentials of the workloads running there.

That is the number the architecture is designed around, and it drives concrete decisions:

- **Spread tenants across hosts** so one host is not one tenant's entire footprint. The per-tenant share cap in the placement scorer is this control.
- **Give the host no interesting credentials.** A worker's own identity should be able to fetch artifacts and report telemetry, and nothing else. If compromising a worker yields control plane access, the blast radius is the fleet rather than the host.
- **Make hosts disposable.** Detection triggers drain, terminate, and reimage. No investigation happens on a host that is still serving traffic.
- **Cycle hosts routinely,** so an undetected persistent foothold has a bounded lifetime by default rather than only when someone notices.
- **Detect at the layer that should never see activity.** Tenant code doing strange things is normal and not a signal. A *VMM process* making an unexpected syscall is not normal, because its syscall set is tiny and fixed. Monitoring the layer with the narrowest expected behaviour is where the highest-signal alerts live.

## Metering, Billing and Observability Without a Long-Lived Process

![Metering and observability for ephemeral execution, showing a microVM invocation emitting lifecycle events to the worker agent which records start and end timestamps and peak memory, one usage record per invocation flowing into a metering stream and a billing aggregator that rolls up per account and writes an idempotent usage ledger, guest stdout and stderr tailed out of band by a log shipper into a searchable log store keyed by request id, spans emitted to a trace collector, and aggregated metrics feeding alerting.](https://d5osvdbc8um23.cloudfront.net/static-asset/blog_images/system-design-serverless-microvm/07-metering-and-observability.png)

### Metering has to be correct, because it is an invoice

Billing granularity is 1 ms, at roughly $0.0000166667 per gigabyte-second, plus a per-request charge. So every invocation produces a record, and at platform scale that is millions of records per second whose accuracy is a financial and legal matter.

```go
type UsageRecord struct {
    RequestID   string    // idempotency key, all the way to the ledger
    AccountID   string
    Version     string
    StartUnixNs int64
    DurationNs  int64     // measured by the host, not reported by the guest
    InitNs      int64     // cold start init, reported separately, not billed as duration
    MemoryMB    int       // configured, which is what is billed
    PeakRSSMB   int       // observed, for the customer's own right-sizing
    Outcome     string    // ok | error | timeout | oom
}

// Billed duration: configured memory times wall time, at millisecond resolution.
func (r UsageRecord) GBSeconds() float64 {
    billedMs := math.Ceil(float64(r.DurationNs) / 1e6)     // round up to 1 ms
    return (float64(r.MemoryMB) / 1024.0) * (billedMs / 1000.0)
}

func (r UsageRecord) Cost() float64 {
    return r.GBSeconds()*0.0000166667 + 0.0000002   // compute + per-request
}
```

Several decisions in that small struct are load-bearing:

- **The host measures duration, not the guest.** A guest-reported number is tenant-controlled input, and tenant-controlled input must never determine a bill.
- **Configured memory is billed, observed memory is reported.** The tenant reserved 512 MB and the platform committed 512 MB of a host to them, so that is the billable quantity. `PeakRSSMB` is given back as advice, not as a charge.
- **Initialization time is tracked separately.** Conflating cold start init with execution duration makes cold starts look like the tenant's slow code, and makes the platform's own latency invisible in its own metrics.
- **The request id is the idempotency key end to end.** The path from worker to ledger involves batching, streams, and retries, so records will be delivered more than once. Deduplicating at the ledger on request id is the only way the invoice is right.

The pipeline is: batch on the host, publish to a durable stream, aggregate asynchronously, write an idempotent ledger. No synchronous database write on the invoke path, ever. And the host buffer must be bounded with an explicit policy for what happens when the stream is unavailable, because "buffer forever" is how a telemetry outage becomes a worker outage.

### Observability when there is nothing to attach to

Every habit built up around long-lived servers is unavailable here. You cannot ssh in; the machine is gone. You cannot attach a profiler; the process lived for 40 ms. You cannot scrape a metrics endpoint on an interval; there is nothing there between requests. Everything has to be emitted during the invocation, by a system that outlives it.

**Logs.** The guest writes to stdout and stderr; the agent tails them out of band and ships them, tagged with the request id. The interesting problem is the ending: a slot can be destroyed the instant a handler returns, so the flush has to be part of teardown rather than best effort, or the logs explaining a failure are exactly the ones that get lost. Backpressure needs an explicit policy too: when the log pipeline is behind, dropping with a counted, visible drop metric is better than blocking invocations, and far better than dropping silently.

**Metrics.** Emitted by the agent, not the guest, so they exist even when the handler crashed on its first instruction. The set that actually matters: invocation count, error count, duration distribution, **init duration as its own series**, throttle count, concurrent executions, and OOM kills. Init duration deserves its own series because it is the platform's own performance, and averaging it into duration hides both cold start regressions and their fixes.

**Traces.** The agent injects trace context into the environment before the handler runs and emits a span for the invocation, including the init phase for cold starts. Without a platform-emitted span, a cold start appears in a distributed trace as unexplained latency in the caller.

**Crashes.** On segfault, panic, OOM, or timeout, capture what can be captured (exit reason, cgroup memory events, a bounded core dump) before teardown, and attach it to the request id. A timeout in particular should record where the handler was when the platform killed it, because "your function timed out" with no further information is the least useful message the platform can send.

The unifying principle: the request id is the join key for the entire system. Logs, metrics, traces, usage records, and crash artifacts all carry it, and it is the only thing that reassembles an execution that no longer exists.

## Failure Modes and What Breaks in Production

The architecture above is the part you design. This is the part you learn.

**A worker host dies.** Every slot on it is gone, which is hundreds of in-flight executions. Synchronous invocations fail and the caller sees an error; asynchronous ones are retried by the poller, which is precisely the durability difference the two paths promised. The subtle failure is not the lost requests, it is the leaked concurrency: those slots held concurrency leases, and if leases do not expire, the affected functions are permanently throttled by a fleet that thinks they are still running. This is why admission control uses leases with expiry.

**A snapshot is subtly wrong.** Not corrupt, which is caught by digest verification, but semantically wrong: taken before the runtime truly reached steady state, or with a device queue mid-flight. Every slot restored from it misbehaves in the same way, so the failure is version-scoped, correlated, and hits 100% of that version's cold starts while warm slots stay healthy. That signature (a new version failing only on scale-up) points at the snapshot rather than the code. Remediation is to invalidate the snapshot, fall back to cold boot, and rebake. The deeper lesson is that snapshot validation belongs in the bake pipeline: restore it once and run a synthetic invocation before any real traffic can reach it.

**Thundering herd.** Traffic jumps by an order of magnitude in seconds. Burst concurrency absorbs the first wave from warm and pre-warmed slots, then the growth rate limit takes over, then fleet scale-out, which is 60 to 90 seconds away because that is how long a new host takes to be useful. In that window, throttling is the correct behaviour, and it must be fast and cheap. The failure mode to actively avoid is a herd against your own code store: a thousand hosts simultaneously fetching the same 300 MB artifact. Tiered caching and jittered fetch turn a self-inflicted outage into a slow ramp.

**A noisy neighbour.** One tenant saturates disk or network on a shared host and everyone else's tail latency degrades. This is the hardest class to debug because the symptom appears in a function whose own code and traffic did not change. Per-slot cgroup and traffic-control limits are the prevention; per-host contention metrics are what make it diagnosable at all, since nothing in the affected tenant's own telemetry will show a cause.

**Resource exhaustion inside a slot.** A handler leaks memory and hits its cgroup limit. The cgroup does its job: the slot is OOM-killed, the invocation fails with a distinct error, and the neighbours are unaffected. The requirement is that the tenant can tell an OOM from a crash from a timeout, because the fixes are completely different and a generic "execution failed" leaves them guessing.

**Partition from the control plane.** Workers cannot reach the registry or the metering stream. Because the data plane does not depend on the control plane for already-placed slots, existing warm traffic keeps serving, which is the payoff for that separation. What stops is placement of new versions, and telemetry buffers start filling. The right behaviour is: keep serving, buffer with a bound, degrade the newest functionality first, and never let a control plane problem take down execution. Getting this wrong (a synchronous registry lookup on the invoke path) converts a control plane blip into a full platform outage, and it is the most common way an otherwise sound design fails.

**Poison pill in the async queue.** One event deterministically crashes the handler, gets retried, crashes again, and if retries are unbounded or the dead letter destination is missing, it blocks or endlessly recycles the queue. Bounded retries plus a dead letter destination are not optional features; they are what keeps one bad event from consuming a function's entire concurrency budget indefinitely.

## Conclusion

The recurring shape of this design is that every guarantee the platform makes is paid for somewhere specific, and the engineering is in choosing where.

Hostile multi-tenancy is paid for with hardware virtualization, and the bill for that (weight and boot time) is negotiated down by deleting almost the entire virtual machine: no firmware, no bus enumeration, two devices, an init that is one static binary. That gets the platform's own contribution to cold start to roughly 125 ms with about 5 MB of overhead, which is what makes both the security model and the density model affordable at once.

Startup latency is paid for with memory and storage. Snapshots move boot and runtime initialization off the request path, and copy-on-write mapping makes restore proportional to the working set rather than the configured memory, so one snapshot file backs thousands of slots. The catch is that a restored VM is a duplicate of a machine, entropy pool and clock included, and a platform that does not reseed and re-time on restore has shipped a security bug rather than an optimization.

Elasticity is paid for with admission control. Burst limits, growth rate limits, sharded leased concurrency counters, and fast cheap throttling are all the same idea: rejecting work you cannot place is better for every tenant, including the one being rejected, than accepting it and failing slowly.

Density is paid for with careful placement, and placement is where the priorities are most easily inverted. Locality first, because artifact fetch dominates cold placement. Blast radius second, because it is a security control. Packing efficiency last, as a tiebreaker. Optimize packing first and you get a cheap fleet with bad tail latency and a bad worst case.

And the whole thing is only operable because of two decisions that look like bookkeeping and are not: immutable versions, which make warm reuse and snapshots safe to reason about, and a request id threaded through logs, metrics, traces, usage records, and crash artifacts, which is the only way to reconstruct an execution that ceased to exist before anyone asked about it.

None of these mechanisms are exotic on their own. Hardware virtualization, copy-on-write memory, content-addressed caching, leased counters, and bin-packing are all old ideas. What makes a serverless platform hard is that the guarantees interact, so the isolation choice constrains the cold start strategy, the cold start strategy creates a security obligation, and the density target reshapes placement. Getting one of them right in isolation is straightforward. The engineering is in the seams.