Designing an AI Image and Video Generation Platform like Midjourney or Sora

When I design an image and video generation platform like Midjourney, Sora, Runway or Firefly, I start from one uncomfortable fact: every request is a batch job pretending to be an interactive feature. A user types a prompt and expects four images in a few seconds, or a short clip in a minute or two. Behind that prompt, a diffusion-style model runs dozens of denoising steps on a GPU, and for video it may hold a whole multi-GPU node for minutes. The user watches a progress bar the entire time and will notice if it freezes, restarts or silently disappears.

That makes this a very different system from a chat model. An LLM inference platform streams tokens and lives or dies on time to first token, continuous batching and the KV cache. A media platform has no tokens to stream. Its unit of work is a job that costs seconds to minutes of GPU time, produces large binary artifacts, and must be checked for safety before anyone sees it.

In this post I walk through how I would build this platform from scratch: scoping, requirements and capacity estimates, a high-level architecture, and then four deep-dives:

• The GPU job queue: priority tiers, fair share and preemption
• Long-running jobs: checkpoints, progress streaming and retries
• Asset storage and delivery: variants, upscales and the CDN
• Safety filtering and provenance: moderation, C2PA and watermarking

Scoping the Problem and Clarifying Assumptions

I am building the serving platform: the system that turns a prompt (and optionally a reference image) into finished images or clips, stores them and delivers them. I treat the models as black boxes with known cost profiles. I do not describe how Midjourney or Sora work internally, because those details are not public; I describe techniques that apply to any diffusion-style or similar iterative generator.

What I'm including:

• Text-to-image jobs that return a grid of candidates (I assume 4 per job)
• Derived jobs on an existing image: upscale, vary, inpaint, extend
• Text-to-video and image-to-video jobs producing short clips (5 to 10 seconds)
• Paid fast tiers, a slower "relax" tier and a free tier
• A personal gallery, shareable links and a public explore feed
• Safety checks on prompts, input images and outputs, plus provenance metadata

What I'm NOT building here:

• Model training, fine-tuning and dataset pipelines
• A video editor or timeline
• Text guardrails in depth. Prompt moderation reuses the ideas of an LLM guardrail layer, so I focus on image and video specific checks.
• Billing internals. I assume a credit ledger exists and show where it is charged.

Key assumptions:

• Image models run on a single H100-class GPU per job and batch the 4 candidates of one job together
• Video models need a full 8-GPU node per job
• Jobs are iterative: the model runs a fixed number of steps (say 20 to 50), and its state between steps can be saved and resumed
• Users tolerate seconds of wait for images and minutes for video, as long as they see progress

How this differs from the classic system: the closest classic system is a video transcoding pipeline: upload, queue, process on workers, store renditions, deliver over a CDN. The differences are cost, interactivity and risk. A transcode is cheap, nobody watches it happen, and it cannot invent harmful content. It also differs from LLM serving: there is no KV cache to manage and no token stream, batching happens once at job start rather than every step, and scheduling looks more like a cluster batch scheduler with a latency target.

Functional and Non-Functional Requirements

Functional Requirements

1. Submit: accept a prompt, parameters (model, aspect ratio, resolution, steps, seed, duration) and optional reference images; return a job id immediately.
2. Progress: show queue position, step progress and low-resolution previews while the job runs.
3. Deliver: store outputs, show them in the gallery and serve them quickly worldwide.
4. Derive: upscale, vary, inpaint or extend an output as a new job linked to its parent.
5. Prioritise: run fast-tier jobs ahead of relax and free jobs, fairly between users in a tier.
6. Moderate: block disallowed prompts and outputs before the user sees them, with an audit trail.
7. Provenance: label every output as AI-generated in a way that survives common handling.

Non-Functional Requirements

Back-of-the-Envelope Capacity Estimation

These are planning assumptions, not measurements. The method matters more than the numbers: estimate jobs, convert them to GPU-seconds, then size the fleet.

Traffic:

• 3 million daily active users, 5 image jobs each per day: 15 million image jobs per day, about 175 per second on average and 520 at peak (3x)
• I assume 200,000 video jobs per day, about 1.3% of all jobs: about 2.3 per second

GPU cost per job (assumptions I would replace with benchmarks):

• Image: a batch of 4 candidates at 1024x1024 takes about 6 GPU-seconds
• Video: a 5-second 720p clip takes about 2 minutes on an 8-GPU node, so about 960 GPU-seconds

image: 175 jobs/s x 6 GPU-s   ≈ 1,050 GPUs busy on average
video: 2.3 jobs/s x 960 GPU-s ≈ 2,200 GPUs busy on average
peak (3x) at 75% target utilisation:
   image ≈ 1,050 x 3 / 0.75 ≈ 4,200 GPUs
   video ≈ 2,200 x 3 / 0.75 ≈ 8,800 GPUs (if none of it could wait)

The lesson is in the ratio: video is about 1% of jobs and about two thirds of GPU time. That is why relax tiers, spot capacity and off-peak scheduling matter far more for video. If half of video work can be deferred to the trough, the peak fleet shrinks dramatically.

Storage: 15M jobs x 4 candidates = 60 million images per day. At about 1.5 MB per original PNG plus about 200 KB of variants, that is about 100 TB per day, on the order of 35 PB per year if everything stayed hot. Video adds about 2 TB per day (200K clips at about 10 MB with renditions). Most candidates are glanced at once and never opened again, so lifecycle tiering is a large part of the budget.

Delivery: if each user views about 200 thumbnails (20 KB) and 20 display images (300 KB) a day, that is about 10 MB per user, 30 TB per day of egress, roughly 3 Gbps on average before video playback. Almost all of it should come from a CDN.

High-Level Architecture and Core Components

Component Walkthrough

API Gateway. Authenticates users and API keys, validates parameters, checks credits against the estimated cost and enforces an idempotency key on submit, so a double click never creates two paid jobs. It returns a job id in milliseconds; nothing on the GPU path is synchronous with the client request.

Prompt moderation. A fast text classifier and policy check on the prompt, plus image checks on any reference image. It runs before queueing, because blocking a prompt costs milliseconds while blocking an output costs GPU time.

Job service. Owns the job record: owner, parameters, estimated GPU-seconds, tier, state, parent job and credit reservation. It uses a relational database, because jobs, credits and lineage are transactional and relational.

GPU job scheduler. Holds per-tier queues, applies fair share and decides which job goes to which pool. Workers pull jobs by taking a lease. This is the first deep-dive.

GPU workers. Each hosts one model on one GPU (images) or a full node (video). It loads weights once, then loops: lease a job, run the steps, emit progress, checkpoint if the job is long, upload the output.

Progress hub. Fans out step events to watching clients over Server-Sent Events, with a polling fallback. The second deep-dive.

Asset storage and CDN. Originals and variants in object storage, metadata in the database, delivery through a CDN with signed URLs. The third deep-dive.

Output safety and provenance. Classifiers on outputs, then signing and watermarking. Only after this step is an asset visible. The fourth deep-dive.

Request Flow

1. The client sends POST /v1/jobs with a prompt, aspect_ratio: "16:9", tier: "fast" and an Idempotency-Key header.
2. The gateway authenticates, runs prompt moderation and asks the job service to create the job, which estimates 6 GPU-seconds, reserves credits and writes the job as QUEUED.
3. The scheduler places the job in the fast-tier queue. The client opens GET /v1/jobs/{id}/events and sees its queue position.
4. An image worker that already holds the model leases the job, runs 30 steps on the batch of 4, and emits progress and an occasional blurry preview.
5. The worker uploads raw outputs to a private staging area and marks the job MODERATING. Output checks pass, provenance is attached, and the asset service publishes the images and their variants.
6. The job becomes SUCCEEDED, credits are settled, and the event stream delivers the final CDN URLs.

Scaling It Out

• Pools by model and shape. Image and video models have completely different hardware shapes, so they get separate pools. Within a pool I keep workers pinned to a model, because swapping weights takes far longer than running a job.
• Global scheduling. A 2-minute video job does not care whether its GPU is 80 ms away. Gateways are regional for fast submits, but the scheduler places work in any region with capacity. Only the output upload crosses regions. That lets me buy GPUs wherever they are available and cheapest.
• Spot capacity for relax work. Relax and free jobs run on preemptible instances when available. Because long jobs checkpoint, losing a spot node costs seconds of rework, not the whole job.
• Autoscale on queue age. GPUs are always busy when there is any queue, so utilisation says nothing. I scale each pool on the age of the oldest job per tier against its target, and pre-warm ahead of the daily peak because loading weights onto a new node takes minutes.

Deep-Dive: The GPU Job Queue: Priority Tiers, Fair Share, and Preemption

The Unit Is GPU-Seconds, Not Jobs

A 512x512 image at 20 steps and a 10-second 1080p video are both "one job", and their costs differ by a factor of thousands. If I count jobs, one video-heavy user quietly consumes the capacity of hundreds of image users. So every job gets a GPU-seconds estimate at submit, from a simple model fitted on real runs:

def estimate_gpu_seconds(job) -> float:
    profile = MODEL_PROFILES[job.model]            # fitted from benchmark runs
    pixels = job.width * job.height * job.frames   # frames = 1 for images
    work = pixels * job.steps * job.num_outputs
    est = profile.fixed_overhead_s + work * profile.seconds_per_unit
    return est * profile.gpus_per_job              # 8 for video, 1 for images

This number drives credits, fair share and placement. I log actual GPU-seconds next to each estimate and refit the profiles regularly, because a new model version or kernel shifts them.

Priority Tiers

• Fast: paid users spending fast credits. Target: start within seconds.
• Standard: paid users whose fast credits ran out, and API customers on a standard SLA.
• Relax: unlimited-but-slow plans and the free tier. No latency promise; eligible for spot and preemption.
• Internal batch: evaluations and safety red-team sweeps that fill the trough.

Tiers decide who goes first when GPUs are scarce. Credits decide whether a user may submit at all. I keep those concepts separate.

Fair Share Within and Across Tiers

Strict priority starves relax users completely during a long peak, turning "slow" into "never". Across tiers I use weighted fair queuing (for example 70% fast, 25% standard, 5% relax when all are busy, with idle capacity available to anyone). Within a tier I use deficit round-robin by user, charged in GPU-seconds: each user earns a quantum per round and spends it on their next job's estimate. A user who submits 40 jobs at once does not push everyone else back by 40 jobs.

I also cap concurrent jobs per user per tier (say 3 image jobs and 1 video job). It is blunt, but it stops a script from filling a pool and keeps the waiting set small.

Placement and Batching

This is where the design differs most from LLM serving. An LLM engine batches continuously, adding and removing sequences every step. A diffusion job runs a fixed number of steps over a fixed-size latent, so I batch at job start:

• Image jobs: the 4 candidates are one batch on one GPU. When a pool is lightly loaded, I may merge small jobs with the same model, resolution and step count, but only if it does not delay the head of the queue.
• Video jobs: need all 8 GPUs of a node at once, which is gang scheduling. If image jobs take scattered GPUs, a video job may wait forever for a full node. I keep video on dedicated nodes, and if I ever share nodes, I reserve and drain a whole node for a waiting video job instead of letting image work refill it.
• Model affinity: the scheduler strongly prefers workers that already hold the right weights, and moves a worker to another model only when that model's queue age stays above target for a while. A model swap is the GPU equivalent of a cold start.

Preemption

Preemption is useful but expensive, so I limit it:

• Never preempt image jobs. They finish in seconds; waiting is cheaper than wasting the work.
• Preempt long jobs only at a checkpoint. The job saves its state, releases the node and returns to the front of its queue with its progress intact.
• Only lower tiers are preemptible. Fast-tier video can preempt relax-tier video; nothing preempts fast-tier work. Spot interruptions are preemptions I do not control, so only checkpointing jobs go to spot.
• Cap preemptions per job (say 2), so a relax user does not watch the same clip restart all afternoon.

Failure Modes

• Bad estimates. If a new model is underestimated, fair share breaks quietly. I alert when the actual-to-estimate ratio drifts per model.
• Hot model. A trending model can flood one pool. I autoscale it, and if capacity runs out, lower default resolution or steps for the relax tier rather than letting its queue grow without bound.
• Scheduler restart. Jobs are durable rows, so a restarted scheduler rebuilds its queues from QUEUED jobs while leased workers keep running.

First vs later: first, a durable job table, per-tier queues with per-user round-robin, a concurrency cap and pull-based workers with leases. Later, GPU-second fair share, cross-region placement, spot pools and checkpoint-aware preemption, once video volume justifies them.

Deep-Dive: Long-Running Jobs: Checkpoints, Progress Streaming, and Retries

A Job Is a Leased State Machine

A minutes-long job will meet node failures, spot interruptions, deploys and out-of-memory errors. I model it as an explicit state machine: QUEUED, LEASED, RUNNING, CHECKPOINTED, MODERATING, then SUCCEEDED, BLOCKED or FAILED. Every transition is a conditional update that checks the current state and lease owner, so two workers can never both believe they own a job. A worker that lost its lease and wakes up later finds its update rejected and discards its result.

A worker takes a lease with a short expiry (say 30 seconds) and heartbeats every 10 seconds. If the node dies, the lease expires and a lease supervisor requeues the job. This is the same task-ownership pattern a distributed scheduler uses, and I keep it identical on purpose because it is well understood and easy to test.

-- Lease the next job for this worker's model (one transaction)
UPDATE jobs
SET state = 'LEASED', lease_owner = :worker_id,
    lease_expires_at = now() + interval '30 seconds', attempt = attempt + 1
WHERE id = (
  SELECT id FROM jobs
  WHERE state = 'QUEUED' AND model = :model
  ORDER BY dispatch_rank           -- precomputed by the fair-share scheduler
  FOR UPDATE SKIP LOCKED
  LIMIT 1
)
RETURNING id, params, checkpoint_uri;

At high volume the scheduler assigns jobs from memory and the database becomes the durable record rather than the hot path, but the lease semantics stay the same.

Checkpoints

An iterative generator has a natural checkpoint: the latent state after step k, plus sampler state, random seed and conditioning inputs. Image jobs do not checkpoint at all; rerunning a few seconds of work is cheaper than writing state. Video jobs checkpoint:

• Every N steps or every 20 to 30 seconds of wall time, whichever comes first
• On a preemption signal or a spot interruption notice
• At segment boundaries, for models that generate long clips in chunks

Checkpoints go to object storage under the job id and attempt. Writing one every half minute is a small cost next to minutes of GPU time. A resume only happens on the identical model version and sampler config recorded in the job; if the version has rolled forward, I restart from scratch rather than mix states from two models. GPU kernels are not always bit-exact across hardware, so a resumed clip may differ slightly from an uninterrupted one, which is acceptable because the user never saw the other version.

Progress Streaming

The worker publishes small events to a topic keyed by job id: queue position, then step 12 of 30, then occasionally a low-resolution preview decoded from the current latent. The progress hub relays them over Server-Sent Events, which fits a one-way stream, passes through proxies and gives me resume through event ids.

id: 41
event: progress
data: {"job_id": "j_8f2c", "state": "RUNNING", "step": 12, "steps": 30, "eta_s": 7}

id: 42
event: preview
data: {"job_id": "j_8f2c", "url": "https://cdn.example.com/p/j_8f2c/12.webp"}

id: 43
event: done
data: {"job_id": "j_8f2c", "assets": ["a_91", "a_92", "a_93", "a_94"]}

The rules I hold to:

• Progress is best-effort; the job record is the truth. A reconnecting client sends Last-Event-ID and the hub replays from a short buffer, or the client polls GET /v1/jobs/{id}. Losing an event must never lose a job.
• The GPU loop never waits on the network. Publishing is fire-and-forget into a local buffer.
• Previews are small, rate-limited and checked. They are unmoderated output, so a fast classifier screens them and they are blurred by default (fourth deep-dive).
• ETA from history. I compute it from this worker's measured step time and the queue's recent drain rate. A bar that jumps backward erodes trust faster than an honest longer estimate.

Retries

Retries are capped (say 3 attempts) with backoff and jitter. I also track failures per worker: a GPU that keeps failing jobs is drained and quarantined automatically, because one faulty card can fail thousands of jobs in an hour.

The user pays for outcomes, not attempts. At submit I reserve credits, on SUCCEEDED I settle them, and on FAILED or BLOCKED I release them per policy. Settlement is keyed by job id, so a duplicated final step charges once.

First vs later: first, leases, heartbeats, the retry table and SSE with a poll fallback; image jobs need nothing more. Checkpoints, preemption and spot capacity arrive with video, when jobs become long enough for restarts to hurt.

Deep-Dive: Asset Storage and Delivery: Variants, Upscales, and the CDN

Separate Bytes from Metadata

Each output becomes an asset with an owner, the job that produced it, its parent (for upscales and variations), visibility and safety state. Metadata lives in a relational store; bytes live in object storage under a content-addressed key, so identical bytes are stored once and URLs can be cached forever.

CREATE TABLE assets (
  id            TEXT PRIMARY KEY,
  owner_id      TEXT NOT NULL,
  job_id        TEXT NOT NULL,
  parent_id     TEXT REFERENCES assets(id), -- set for upscale / vary / extend
  derive_op     TEXT,                       -- 'upscale_2x', 'vary_subtle', ...
  kind          TEXT NOT NULL,              -- 'image' | 'video'
  content_hash  TEXT NOT NULL,              -- sha256 of the original bytes
  visibility    TEXT NOT NULL DEFAULT 'private', -- private | unlisted | public
  safety_state  TEXT NOT NULL,              -- approved | restricted | removed
  storage_class TEXT NOT NULL DEFAULT 'hot',
  created_at    TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX ON assets (owner_id, created_at DESC);
CREATE INDEX ON assets (parent_id);

The parent_id chain is the asset's lineage: grid, then variation, then upscale. Users browse it in the UI, and safety depends on it: when an original is removed, its derivatives must be found and removed too.

Variants

A gallery never needs the original PNG, so I produce variants:

• Eagerly, after moderation: a thumbnail for grids, a WebP or AVIF display copy, and for video a poster frame plus an adaptive streaming ladder (HLS or DASH at a few bitrates).
• Lazily, on demand: any other size or format, generated by a resizer at the CDN origin and then cached.

Eager variants keep the first gallery view fast; lazy ones avoid paying for sizes nobody requests. Variant building is CPU work on cheap autoscaled workers off a queue. The video ladder is ordinary transcoding, and I would reuse an existing pipeline rather than build one.

Upscales and Variations Are New Jobs

An upscale looks like an image operation, but it is a GPU job with its own estimate, tier and moderation. It goes through the same job path with parent_id set, so it gets fair share, retries and progress for free. Its output is checked again, because an upscaler can add detail that was not in the original; a safe parent does not guarantee a safe child.

Delivery Through the CDN

• Immutable URLs. Every variant URL contains the content hash, such as /i/9f3a.../thumb_256.avif, served with Cache-Control: public, max-age=31536000, immutable. A changed image gets a new URL, so updates never need a purge.
• Signed URLs for private assets. The gateway issues short-lived signed URLs (or signed cookies for a gallery page) that the CDN verifies at the edge. Bytes are still cached at the edge; the signature only gates access.
• Public explore feed. Popular public images get huge view counts and are ideal CDN content. An origin shield in front of the resizer collapses misses from many edges into one origin request.
• Purge only for removals. The one case that needs invalidation is a takedown: a safety removal, a legal request or a user deletion. I delete the objects, mark the asset removed and purge its URLs. Because URLs are content-addressed and every variant and derivative is in the database, I know exactly which URLs to purge.

Lifecycle and Cost

At about 100 TB per day, storage grows faster than anything else. My lifecycle policy: everything hot for 30 days, when most re-visits happen; then candidates nobody opened, favourited, upscaled or shared move to an infrequent-access tier, and their originals later to archive with a restore delay on download. Variants stay hot longer because they are small and power gallery browsing. Staging data (raw unmoderated outputs, previews, checkpoints) is deleted within hours to days, since it has no lasting value and carries safety risk.

Failure modes: metadata and bytes drifting apart. I write the object first, then the row, and a reconciler deletes orphaned objects after a grace period. A viral image on cold edges can also hammer the resizer, which the shield and eager display variants absorb.

First vs later: first, originals plus two eager variants, immutable and signed URLs on a CDN. Later, lazy resizing, AVIF negotiation, the video ladder and lifecycle tiering as storage shows up on the bill.

Deep-Dive: Safety Filtering and Provenance: Prompt and Output Moderation, C2PA, Watermarking

Where Media Safety Differs from Text Safety

I reuse text guardrail techniques for the prompt, but images and video add problems text does not have:

• The output can be harmful even when the prompt is not. Innocent prompts can produce explicit images, and attackers phrase around prompt filters. Output checks are mandatory.
• Input images carry risk. Image-to-image and image-to-video jobs can start from a real photo of a real person.
• Some content is illegal to produce or hold. CSAM is not a policy setting: it must be blocked, preserved as required and reported (in the US, providers report to NCMEC). I would take legal guidance per jurisdiction rather than design this from first principles.
• Likeness and IP. Realistic images of real people who never consented, and recognisable copyrighted characters or logos, are trust and legal risks.

Layered Checks

I check at three points, from cheap to expensive:

1. Before generation. Prompt classifiers and blocklists; on reference images, NSFW and minor-detection classifiers, hash matching against known abuse material, and face detection to decide whether likeness rules apply. This stops most clear violations before any GPU time is spent.
2. During generation. For video, a fast classifier on decoded preview frames every few steps. If a clip is heading somewhere clearly disallowed, I abort early and save the remaining GPU time. It is a cost feature as well as a safety one.
3. After generation, before visibility. Every image, and video frames sampled at a fixed rate plus scene changes, pass NSFW and violence classifiers; CSAM perceptual-hash matching against industry hash lists plus classifiers for novel material; face matching against a protected-persons list (public figures and people who opted out); and embedding similarity against protected characters and marks.

# Output policy for one surface (illustrative)
surface: public_explore
checks:
  csam:          { action: block_preserve_report, fail_mode: closed }
  nsfw:          { block_above: 0.80, review_above: 0.50, fail_mode: closed }
  violence:      { block_above: 0.90, review_above: 0.60 }
  likeness:      { realistic_style: block_on_match, stylised: review_on_match }
  ip_similarity: { review_above: 0.85 }
video:
  sample_fps: 2
  also_sample: scene_changes
on_review: hold_private

Thresholds differ by surface: a verified adult's private gallery can allow more than the public feed. That is why policy is configuration keyed by surface and account, not code.

Fail Closed, with Humans in the Loop

Output checks fail closed: if the classifier service is down, assets wait in MODERATING. A few minutes of delay during an incident is far better than publishing unchecked media. Borderline results go to a human review queue with the asset held private and blurred by default, which needs its own tooling and reviewer wellbeing protections. Review decisions become labelled data for offline threshold tuning. Every decision is written to an audit log with the policy version that made it, so I can explain any block and handle appeals.

Provenance: C2PA Content Credentials

C2PA (the Coalition for Content Provenance and Authenticity) defines Content Credentials: a cryptographically signed manifest attached to a media file recording how it was made, including that an AI model generated it, by which service and when, and which edits followed. I attach one to every original and every download, signed with keys from a managed signing service. Upscales and variations carry a manifest that references their parent, mirroring the lineage in the asset table.

Embedded metadata is easy to strip: a screenshot, a re-encode or a social network's upload pipeline removes it. So I also keep the manifest server-side, indexed by content hash, to answer "did we make this" for exact copies.

Invisible Watermarking

For copies that have lost their metadata, I add an invisible watermark in the pixels or across frames: a signal designed to survive resizing, compression and mild cropping, readable by a detector with the right key. Google's SynthID is one public example of the approach. I treat any watermark as a robust hint, not proof, because no scheme survives every edit or a determined removal attempt.

A verification endpoint checks all three for any image, so trust and safety teams and partner platforms can verify provenance without asking me.

Failure Modes

• Over-blocking. Aggressive thresholds block harmless art and frustrate paying users. I track block rate and appeal-overturn rate per category and model version, and treat a jump in either as a regression.
• Adversarial iteration. Users probe around filters. Rate limits on repeated blocked prompts and escalating account-level enforcement work better than an ever-longer blocklist.
• New model, new failure modes. Each release gets a red-team sweep on the internal batch tier and launches with stricter thresholds that are relaxed with data.

First vs later: first, prompt moderation, output NSFW and CSAM checks that fail closed, a review queue and an audit log; that is the minimum to launch. Then C2PA, likeness and IP checks, in-flight video checks, watermarking and the verification endpoint.

Conclusion: Bringing It All Together

An image and video generation platform looks like a creative tool on the outside and like a cluster batch scheduler with a live audience on the inside. The model is the most visible part, but the product's quality comes from everything around it: who gets the next GPU, what happens when a node dies at minute two of a clip, how an image reaches a phone on another continent in under 100 ms, and whether anything harmful gets through.

The core contrast with LLM serving is the shape of the work. An LLM platform schedules tokens every step and manages KV cache memory. A media platform schedules whole jobs measured in GPU-seconds, batches them once at start, and must make minutes-long work durable and visible.

If I had to summarise the design in four decisions:

1. Price, queue and share in GPU-seconds. Separate tiers from credits, use fair share within tiers, gang-schedule video and preempt only at checkpoints.
2. Make every job a leased, checkpointed state machine. Progress is best-effort over SSE, the job record is the truth, and users pay for outcomes, not attempts.
3. Write once, cache forever. Content-addressed immutable URLs, eager small variants, lazy everything else, signed URLs for private assets, and lifecycle tiering from day one.
4. Check before anyone sees it, and label what you make. Layered checks that fail closed, then C2PA, a server-side record and a watermark, because each provenance layer covers the others' gaps.

Related reading: Designing an LLM Inference Platform, Designing a Distributed Job Scheduler, Designing a Content Delivery Network, and Designing an LLM Guardrails and Safety Layer.