Designing an Enterprise Knowledge Assistant like Glean

When I design an enterprise knowledge assistant like Glean, the part people expect to be hard, the LLM, turns out to be the easy part. Writing a fluent paragraph from a few retrieved passages is something I can buy from a model provider. The hard parts are unglamorous: pulling hundreds of millions of documents and messages out of a dozen SaaS tools, each with its own APIs, rate limits and change model; keeping that copy fresh; and above all, never showing an employee a sentence from a document they are not allowed to open.

That last point shapes the whole design. A consumer chatbot that gives a wrong answer is embarrassing. A workplace assistant that quotes the unannounced reorg plan to the wrong person is a security incident that ends the product inside that company. Trust is the product. Every answer needs citations the user can click, every citation must point at something the user can already see, and the answer must reflect the document as it is today.

In this post I walk through how I would build it: scoping, requirements and capacity estimates, a high-level architecture, and four deep-dives:

• Connectors and incremental sync
• Chunking and the embedding pipeline
• Permission-aware hybrid retrieval
• Answer quality, feedback and evaluation

Scoping the Problem and Clarifying Assumptions

I am building a multi-tenant SaaS assistant. Each customer is an enterprise that connects its own sources (Google Drive, Confluence, Slack, Jira, GitHub, email), and its employees ask questions in a web app or a Slack bot. The assistant answers with inline citations and also works as a plain search box.

What I'm including:

• Connectors for the main source types, with an initial backfill and ongoing incremental sync
• Parsing, chunking and embedding of documents, messages, tickets, pull requests and email
• Permission sync, so the index knows who can see each item
• Hybrid retrieval (keyword plus vector), re-ranking and grounded answer generation with citations
• Feedback capture and an evaluation system that gates changes to models, chunking and ranking

What I'm NOT building here:

• Agents that take actions (filing tickets, sending messages). Write access raises different safety questions; it is a separate product on top of this retrieval layer.
• Model training or LLM serving. I use hosted or self-hosted foundation models as dependencies with their own SLOs.

Key assumptions:

• A large tenant has 50,000 employees and about 200 million items across sources. I assume roughly 120 million chat messages, 40 million emails, 30 million documents and wiki pages, and 10 million tickets and pull requests.
• The SaaS serves a few hundred tenants of mixed size. For capacity I treat the whole fleet as equivalent to about 50 tenants of this large size.
• Customer data must stay in a chosen region (US or EU), and some customers will ask for their own encryption keys.
• Each source is the system of record for both content and permissions. My index is a derived copy that can always be rebuilt.

How this differs from classic enterprise search: older enterprise search showed ten blue links, and the source system enforced permissions when the user clicked one. In a RAG assistant, the model reads the content for the user and writes a synthesis. There is no second chance at click time, because the sensitive sentence is already in the answer. Enforcement has to move from "the source checks when you open it" to "my retrieval layer guarantees it before the model sees it".

Functional and Non-Functional Requirements

Functional Requirements

1. Connect: an admin authorises a source with OAuth, and a backfill starts.
2. Sync: keep content and permissions current as items are created, edited, shared or deleted.
3. Search: return ranked items filtered to what the asking user can see.
4. Answer: generate a grounded answer with citations, including multi-turn follow-ups.
5. Feedback: capture thumbs up or down, citation clicks and rephrased questions.
6. Admin controls: exclude sources or channels, view sync health, purge data.

Non-Functional Requirements

I separate the query path from the ingestion path in every target. A query outage is visible immediately; an ingestion slowdown shows up as staleness, which is recoverable as long as no change event is lost.

Back-of-the-Envelope Capacity Estimation

These numbers are planning assumptions, not measurements. The point is the method.

Chunks per large tenant:

• Chat messages are short, so I group them into thread or time-window chunks. I assume about 6 messages per chunk: 120M / 6 = 20M chunks
• Emails: about 1 chunk each after stripping quoted replies and signatures: 40M chunks
• Documents and wiki pages: I assume about 6 chunks each on average: 180M chunks
• Tickets and pull requests: about 2 chunks each: 20M chunks
• Total: about 260 million chunks, call it 250M

Vector storage per tenant:

1,024-dim embedding, float32   = 4 KB per chunk
250M chunks x 4 KB             = about 1 TB raw vectors
with int8 quantisation (1 KB)  = about 250 GB
plus HNSW graph links          = tens of GB more

Across 50 tenant-equivalents that is on the order of 12 to 15 TB of quantised vectors, which is why I shard by tenant from day one.

Backfill embedding work: 250M chunks at about 400 tokens each is about 100 billion tokens per large tenant. If one embedding GPU node sustains, say, 50,000 tokens per second, that is about 560 node-hours, or over a day on 20 nodes. In practice source API rate limits are the slower constraint, so a full backfill takes days to weeks, and I index recent and popular content first.

Change rate: I assume about 2M new chat messages a day (40 per employee) plus about 1% of other items changing daily (about 0.8M). That is roughly 3M change events a day, about 35 per second on average per large tenant, with bursts in the hundreds when someone moves a large folder or reorganises a wiki space.

Query load: if 40% of employees use it daily and ask 5 questions each, that is 100,000 queries a day per large tenant, about 1.2 per second, maybe 5 at peak. Across the fleet that is a few hundred queries per second at peak. Each query does a filtered hybrid search, a re-rank over about 100 candidates and an LLM call with about 6,000 input tokens of context. Ingestion dominates the infrastructure bill; LLM tokens dominate the per-query bill.

Permissions: 50,000 users and, I assume, around 20,000 groups (nested groups, Slack channels, Jira project roles). Every one of 200M items carries an access list, and in an org this size someone shares a doc, joins a team or leaves every few seconds.

High-Level Architecture and Core Components

Component Walkthrough

Connector fleet. One connector type per source, many instances per tenant. Each knows how to authenticate, list changes since a cursor, fetch content and read permissions. Connectors emit content changes (upsert, delete) and permission changes (an item's ACL or a group's membership changed). This is the first deep-dive.

Change queue. A durable queue partitioned by tenant. It decouples source API speed from processing speed, and it is where I replay from after a pipeline bug.

Parse, chunk and embed pipeline. Workers convert each format to structured text, chunk it along its natural structure, embed new or changed chunks, and write to the index. Permission events update the ACL store and the ACL fields in the index. This is the second deep-dive.

Hybrid index. Per-tenant shards holding, for each chunk, its text (for BM25), its vector (for HNSW nearest-neighbour search), its metadata and its access-control fields, so I can filter and rank in one pass.

Identity and ACL store. Users, groups, nested memberships, and external identities (a Slack user id, a GitHub login, an email address) mapped to one canonical person, plus per-item ACLs. It is also fed by SCIM from the customer's identity provider.

Permission service. Given a user, returns every principal they act as: their ids in each source, every group after nested expansion, and tenant-wide principals like "anyone in the domain". Cached, and invalidated on membership changes.

Answer orchestrator. Rewrites the question using conversation history, fetches principals, runs hybrid retrieval, re-ranks, does a final permission check, builds the prompt, calls the LLM, verifies citations and streams the answer.

Re-ranker and LLM. A cross-encoder re-ranker sharpens the top results; the LLM writes the grounded answer. Both sit behind interfaces so I can swap them, which matters for evaluation.

Request Flow

1. An employee asks: "What is our parental leave policy for contractors in Germany?"
2. The gateway validates the SSO session, resolves the tenant and routes to that tenant's home cell.
3. The orchestrator fetches the user's principals and runs BM25 and vector search in parallel, both filtered by them.
4. The re-ranker orders the top 100; the top 20 pass a late permission check for sensitive sources.
5. The best passages, with titles and dates, go into a prompt that asks the LLM to answer only from them, citing each claim.
6. A verifier checks the citations, the answer streams back with clickable sources, and the trace goes to the evaluation stream.

Scaling It Out

• Cells by region. Each tenant has a home cell in its chosen region holding everything that touches its data. The global layer only knows "tenant X lives in cell EU-1". This handles data residency and caps the blast radius of a bad deploy.
• Shard by tenant. A large tenant gets dedicated index shards. Small tenants share shards, with the tenant id as a mandatory filter enforced in the data-access layer, not left to each query author. Per-tenant keys wrap raw content and index files for customers who require it.
• Fair-share ingestion. A new 50,000-person tenant's backfill must not starve everyone else's incremental sync. Workers pull from per-tenant queues with weighted fair scheduling, and incremental changes always beat backfill.
• Query and ingest scale separately. Query capacity follows business hours; ingest capacity follows onboarding and re-embedding projects.

Deep-Dive: Connectors and Incremental Sync

Every Source Is Different, So the Interface Is Not

Google Drive exposes a changes feed read with a page token. Microsoft Graph has delta queries. Gmail has a history id. Slack and GitHub push webhooks. Jira and Confluence offer webhooks plus search by updated time. Some sources offer nothing better than "list everything and compare". I hide all of it behind one contract:

class Connector(Protocol):
    def list_changes(self, cursor: str | None) -> ChangePage: ...
        # returns items changed since cursor, a next cursor, and has_more
    def fetch(self, item_id: str) -> RawItem: ...        # bytes + metadata
    def fetch_acl(self, item_id: str) -> list[Principal]: ...
    def list_group_members(self, group_id: str) -> list[Principal]: ...
    def full_listing(self) -> Iterator[str]: ...          # all item ids, for reconciliation

Webhooks vs Polling

Webhooks give freshness in seconds, but they get dropped, duplicated, reordered, or silently disabled when a token expires. So I treat a webhook as a hint: "something changed in this scope, go look". The scheduler then runs the normal cursor-based delta fetch. Duplicates collapse because the fetch is idempotent, and every source also has a polling floor that catches missed webhooks. Webhooks reduce latency; polling guarantees completeness.

Cursors and Sync State

Per tenant, per source, per scope (a shared drive, a Slack workspace, a GitHub org), I store the cursor and sync state in a relational database:

CREATE TABLE sync_state (
  tenant_id     uuid,
  source        text,
  scope_id      text,
  cursor        text,          -- page token, delta link, history id, or timestamp
  phase         text,          -- BACKFILL | INCREMENTAL | RECONCILING | PAUSED
  last_success  timestamptz,
  error_count   int,
  PRIMARY KEY (tenant_id, source, scope_id)
);

The rule that keeps this correct: advance the cursor only after the changes it covers are durably written to the change log. If a fetcher crashes mid-page, the next attempt replays the same page, and downstream processing is idempotent on (item_id, version). At-least-once delivery plus idempotent writes is far easier to reason about than trying to make the source and my queue agree transactionally.

Rate Limits Are the Real Bottleneck

Every SaaS API enforces limits, usually per app, user or workspace, and often per method. Exceed them and you get throttled, or worse. So I run token buckets per (tenant, source, credential, method class) in Redis, set from each source's documented limits with a margin. Fetchers acquire tokens before every call and honour Retry-After by pausing that bucket.

Within one tenant's budget I prioritise:

1. Permission changes and deletes (correctness and safety)
2. Incremental content changes (freshness)
3. Backfill (completeness)

Backfill only gets what is left. This ordering is the single most important scheduling decision in the connector layer.

Backfill vs Incremental

I start the incremental cursor before the backfill begins, so changes made during a multi-day backfill are not lost, and I backfill newest-first, because recent content answers most questions. The assistant is usable on partial data with a clear "still indexing" notice.

Deletes and Tombstones

A missed delete means the assistant keeps quoting something someone deliberately removed. Delta APIs usually report deletes, but not always: an item moved into a folder my credential cannot see may simply vanish from the feed. So I handle deletes three ways:

• Explicit deletes from the delta feed become tombstones that remove the item's chunks and purge the raw copy.
• Permission loss (my connector can no longer read the item) is treated as a delete.
• Weekly reconciliation lists all ids per scope, compares them with the index and tombstones anything missing.

Tombstones carry a version, so a late upsert for an older version cannot resurrect a deleted item.

Per-Tenant Isolation

Credentials are per tenant, encrypted, and decrypted only inside that tenant's cell. Raw content lands under per-tenant prefixes and keys. A worker processes one tenant's task at a time. A shared pool that caches API clients by source type instead of by tenant is a classic way to cross the streams.

First vs later: I would start with polling-only connectors for the top five sources and a reliable cursor store, then add webhooks for latency, then weekly reconciliation once I trust the basics. A connector SDK for third-party sources comes much later.

Deep-Dive: Chunking and the Embedding Pipeline

Parsing Many Formats

Before chunking, every item becomes a structured document: a tree of sections, headings, paragraphs, tables and code blocks. Each format needs its own parser:

• Google Docs and Confluence pages arrive as markup, so headings come for free.
• PDFs and slides need layout-aware extraction; scanned PDFs need OCR on a separate, lower-priority queue.
• Spreadsheets are a trap. I index sheet names, headers and a sample of rows, not millions of cells.
• Emails need quoted replies and signatures stripped, or every reply repeats the thread.
• Pull requests keep file paths, the description and the review discussion, which is what people ask about.

A parser failure goes to a dead-letter queue, and the item is still indexed by title and metadata so it stays findable.

Structure-Aware Chunking

A chunk is the unit I retrieve, cite and show the model, so it should be a self-contained piece of meaning. Fixed 500-token splits cut tables in half and separate headings from their paragraphs. My rules:

• Split on structure first: sections and headings, then paragraphs, and only then by token count. Target 300 to 500 tokens, with a small overlap when a long section has to be split.
• Carry context into every chunk: prepend the document title and the heading path ("Benefits Handbook > Leave > Parental Leave > Contractors"). A paragraph that says "this does not apply to them" is useless without it.
• Keep tables and code blocks whole when they fit; otherwise split by rows or functions and repeat the header.
• Chat is different: one Slack message means little alone, so I chunk by thread or by a few-minute window, keeping author and timestamp per message.

Each chunk carries the metadata that ranking, filtering and citations need:

{
  "chunk_id": "gdrive:1AbC...:v42:c07",
  "tenant_id": "t_8812",
  "item_id": "gdrive:1AbC...",
  "item_version": 42,
  "source": "gdrive",
  "title": "Benefits Handbook 2026",
  "heading_path": ["Leave", "Parental Leave", "Contractors"],
  "url": "https://docs.google.com/document/d/1AbC.../edit#heading=h.x9",
  "author": "u_hr_311",
  "updated_at": "2026-09-30T11:02:00Z",
  "acl": ["group:hr-all", "domain:acme.com"],
  "acl_version": 17,
  "embedding_model": "embed-v2",
  "content_hash": "sha256:4f1c..."
}

Only Re-Embed What Changed

Most edits touch one paragraph of a long document. The chunker hashes each chunk and checks a cache keyed by (tenant, item, hash, model); unchanged chunks keep their vectors, and only new or changed ones are embedded. For documents that autosave continuously, I debounce until the document has been quiet for a minute or two.

Permission-only changes skip the pipeline entirely. Sharing a document does not change its text, so I update the ACL fields on its chunks in place. That keeps revocations fast.

Choosing and Versioning the Embedding Model

I pick the embedding model with my own retrieval evaluation, not a public leaderboard, because enterprise text (acronyms, code names, jargon) behaves differently from web text. I weigh dimension and quantisation (int8 in memory, full precision on disk for re-scoring), hosted vs self-hosted (some customers forbid sending text to a third party), and multilingual support.

Every vector is stamped with its model version. Vectors from different models are not comparable, so a query must be embedded with the same model as the index it searches.

Re-Embedding on Model Change

Switching models means re-embedding about 100 billion tokens per large tenant, so I treat it as a migration:

1. Build a new index alongside the old one, filled by a backfill job that reads chunk text from the metadata store, not the source APIs.
2. Keep the incremental pipeline writing to both indexes until the new one catches up.
3. Run the evaluation suite on the new index; switch the query path per tenant when it passes.
4. Delete the old index after a rollback window.

This takes weeks across the fleet at low priority, which is exactly why I do not change embedding models often.

A Queue-Based Pipeline

Each stage (parse, chunk, embed, index) is a worker pool on a tenant-partitioned queue, with its own retries, dead-letter queue and scaling: OCR and embedding are heavy, chunking is cheap. The embedding stage batches chunks across items to keep accelerators busy. Ordering matters only per item, so I drop any write whose item_version is older than what the index already holds.

First vs later: I would start with good parsers for the top formats, heading-aware chunking and the hash cache. OCR, table handling and a code-specific embedding model come later, once evaluation shows where retrieval fails.

Deep-Dive: Permission-Aware Hybrid Retrieval

Why Permission Leaks Are the Number One Risk

Wrong answers erode trust slowly. A permission leak destroys it in one screenshot. The LLM makes leaks worse than in classic search, because it can paraphrase a restricted document into an answer without a visible citation. My rule is absolute: a chunk the user cannot read must never enter the prompt. Not "rank lower", not "hide the citation". Never retrieved.

ACL Sync and Group Expansion

Each item's ACL is a list of principals: users, groups, and special principals like "anyone in the domain". Drive shares to users, groups and domains; Slack access is channel membership; Jira uses project roles; GitHub uses teams. Connectors translate each into canonical principal ids. Groups nest, so I need the transitive closure, and there are two places to compute it:

• Expand at index time: store every user id on every chunk. Queries are simple, but one membership change can rewrite millions of chunks.
• Expand at query time: store groups on chunks and compute the user's full principal set per query.

I choose groups on chunks, expanded per user at query time. I assume a user belongs to at most a few hundred groups after expansion, so the filter is "chunk ACL intersects this set". A membership change updates one row and invalidates the affected users' cached principal sets. No chunk rewrites.

def principals_for(user):
    key = f"principals:{user.tenant}:{user.id}:{user.membership_version}"
    if cached := cache.get(key):
        return cached
    ids = {user.canonical_id, *user.source_identities, f"domain:{user.domain}"}
    frontier = set(identity_store.direct_groups(user))
    while frontier:                       # transitive closure over nested groups
        g = frontier.pop()
        if g not in ids:
            ids.add(g)
            frontier |= identity_store.parent_groups(g)
    cache.set(key, ids, ttl=600)
    return ids

"Anyone with the link" needs a policy decision. Technically anyone could open it, but surfacing it to everyone turns obscurity into broadcast. I default to showing link-shared items only to users who are listed or have opened them (when the source reports that), and let admins change it.

Early-Binding vs Late-Binding Filtering

Early binding puts the ACL into the index and filters during search. It is fast and returns a full page of visible results, but it is only as fresh as the last ACL sync.

Late binding checks each candidate against the source's live permission API. It is correct at that moment, but slow, costly in API quota, and can leave too few results.

I use both: early binding for every query, late binding for the final top 20 on sensitive or fast-changing sources (HR systems, private channels, items whose ACL changed recently). The late check runs in parallel with a tight timeout and fails closed: if I cannot confirm access, the chunk is dropped.

Hybrid Search with Filters

"PROJ-4412 rollback" or an error string needs exact keyword match; vectors are weak on rare tokens and identifiers. "How do we handle refunds in Europe" needs semantic match. So I run BM25 and filtered vector search in parallel, with the same tenant and ACL filter.

Filtered approximate nearest-neighbour search has a known failure mode: if the filter is very selective (a user who can see 1% of a shard), filtering after graph traversal returns too few results. I use an engine that filters during traversal, and fall back to a brute-force scan over the filtered subset when the estimated match count is small.

The lists are merged with reciprocal rank fusion (RRF), which needs no calibration between BM25 and cosine scores:

def rrf(result_lists, k=60):
    scores = defaultdict(float)
    for results in result_lists:
        for rank, chunk_id in enumerate(results, start=1):
            scores[chunk_id] += 1.0 / (k + rank)
    return sorted(scores, key=scores.get, reverse=True)

Re-Ranking

First-stage retrieval optimises recall: get the right passage into the top 100. A cross-encoder re-ranker reads each (query, passage) pair together and fixes precision in the top 10; too slow for millions of chunks, fine for 100. I add signals the model cannot see: freshness (decay tuned per source, since a two-year-old Slack thread is rarely right), authority (official wiki spaces over personal drafts) and engagement.

Testing for Leaks

Permission correctness gets its own test system:

• Canary documents: in test tenants, documents with unique marker strings shared with specific test users. Probes continuously query as users who should and should not see them, and alert on any wrong result.
• Invariant check: before building a prompt, assert every chunk's ACL intersects the user's principals. A violation blocks the answer and pages on-call.
• Revocation tests: remove a test user from a group and measure how long until that group's documents stop appearing. That is my revocation SLO.
• Diff audits: sample items daily and compare my stored ACL with the live source ACL.

First vs later: early-binding filters, query-time group expansion and the invariant check ship on day one, because a leak is not something to fix in version two. Late binding and canary probes come next; learned ranking signals last.

Deep-Dive: Answer Quality, Feedback, and Evaluation

Grounded Answers with Citations

The prompt gives the model numbered passages, each with title, source, author and last-updated date, and tells it to answer only from them, cite a passage for each claim, and say plainly when the passages do not answer the question. "I could not find this in the documents you have access to" is a correct answer, and better than a fluent guess.

After generation, a citation verifier checks that every cited number exists and runs a cheap entailment check that the passage supports the sentence. Failing sentences lose their citation and are flagged; an answer with too many is replaced by a list of top results. Each citation links to the source item, so the employee can check a claim in one click. That click is the trust mechanism.

Freshness and Conflicting Sources

The 2023 travel policy and the 2026 travel policy both match the query. I put dates in the passages, rank fresher authoritative sources higher, and tell the model to prefer the most recent source and point out conflicts ("the 2026 policy says X; an older page says Y"). Showing the conflict is more honest than silently picking one.

Golden Question Sets

Without evaluation, every change to chunking, embeddings, re-ranking or prompts is a guess. I maintain golden sets of questions with their known relevant items and key facts of the answer, built from dogfooding, from synthetic questions generated from sampled documents and reviewed by people, and from real queries with consent. They include negative cases: questions answered only in a document the asker cannot see. The correct output is "not found"; anything else is a leak.

Retrieval Metrics

I measure retrieval separately from generation, because a bad answer usually means bad retrieval:

• Recall@k: did a relevant chunk reach the top k given to the model? This matters most, because the model cannot cite what it never saw.
• MRR and nDCG: how high relevant results rank, which matters for the search page.
• "Not found" rate by source, which often reveals a broken connector before anyone files a ticket.

LLM-as-Judge, with Caveats

Grading thousands of answers by hand per release is not feasible, so an LLM judge scores faithfulness (is each claim supported by its cited passage) and relevance. I treat it as a noisy instrument. Judges can favour longer answers or their own style and vary across runs, so I use a fixed rubric with per-claim verdicts instead of a 1 to 10 score, prefer a judge model different from the answer model, and calibrate against human labels whenever the judge changes. Leak checks never depend on the judge; they are deterministic.

The Feedback Loop

Thumbs are sparse and biased (people mostly vote when annoyed), so I add implicit signals: citation clicks, copying the answer, or immediately rephrasing the question. Every trace stores the query, retrieved chunk ids, prompt version and answer, so a thumbs-down can be traced to "retrieval missed it", "the model ignored it" or "the source is wrong". Each has a different fix, and reviewed failures become new golden questions.

Regression Gates

Every change that can move quality is a release with a gate:

eval_gate:
  candidate: "chunker-v5 + embed-v2 + rerank-v3"
  baseline: "production"
  suites: [golden_core, golden_multilingual, golden_code, leak_negatives]
  require:
    recall_at_20:           { min_delta: -0.01 }   # no more than 1 point worse
    faithfulness:           { min: 0.95 }
    not_found_correctness:  { min: 0.98 }
    leak_negatives:         { max_failures: 0 }    # hard stop
    p95_search_latency_ms:  { max: 500 }
  rollout: [internal, 5%_tenants, 25%_tenants, all]

After the offline gate, the change rolls out by tenant percentage while I watch thumbs ratio, rephrase rate and citation clicks against the baseline. A change that improves averages but fails one leak case does not ship.

First vs later: I would start with a few hundred golden questions, recall@k and a strict citation verifier, which catch most regressions. An LLM judge, per-tenant golden sets and online experiments come once the basics are trusted.

Conclusion: Bringing It All Together

From far away, an enterprise RAG assistant looks like "a vector database plus an LLM". Up close, it is a data integration platform, a permissions system and an evaluation system, with an LLM at the end, and the LLM is the part I can swap most easily.

The sources stay the system of record for content and permissions; my index is a derived copy I can always rebuild. Webhooks are hints, cursors are truth, and deletes get as much care as writes.

If I had to summarise the design in four decisions:

1. Build connectors around cursors, rate-limit budgets and reconciliation. Freshness and completeness both depend on it, and permission changes and deletes go first in the queue.
2. Chunk by structure, stamp everything with versions, and treat embedding model changes as migrations. That keeps cost bounded and changes reversible.
3. Enforce permissions with early binding on every query, late binding for sensitive sources, and continuous leak tests. A leak is the one failure the product cannot survive.
4. Gate every quality change on retrieval and grounding evaluations. Without golden sets and recall@k, every improvement is a guess.

Related reading: Designing an LLM Inference Platform, Designing Search Autocomplete, Storage Engines and Indexes Compared, and Message Queues vs Logs vs Pub/Sub.