Design Pastebin — System Design Interview Guide

Functional requirements

• Create a paste: text body, optional title, optional language for syntax highlighting
• Get a short URL for sharing
• View a paste: render text with optional syntax highlighting
• Optional: paste expiration (10 min, 1 hour, 1 day, 1 week, never)
• Optional: private/unlisted pastes accessible only via the URL
• Optional: edit history / revisions

Non-functional requirements

• Scale: ~50M pastes created per month, ~500M page views per month
• Read-heavy: ~10:1 reads to writes (one paste typically viewed by several people, plus crawlers)
• Read latency: <200ms p99 for viewing a paste
• Write latency: <1 sec p99 for creating a paste, including URL generation
• Availability: 99.95%; reads must remain available even during write-path degradation
• Storage: a typical paste is <10 KB; cap individual pastes at ~10 MB

Capacity estimation

Public scale assumptions: ~50M new pastes/month = ~20/sec average, ~100/sec peak. Reads at 10:1 = ~500M views/month = ~200/sec average, ~2K/sec peak. Modest by web standards.

Storage: average paste ~5 KB (most pastes are 1-2 KB log snippets or code; long tail of larger pastes). 50M pastes/month × 5 KB = 250 GB/month = 3 TB/year. After 10 years: ~30 TB. Easily fits in commodity sharded storage.

Metadata per paste ~500 bytes (paste_id, short_code, title, language, owner, created_at, expires_at, visibility, view_count). At 600M pastes × 500 bytes = 300 GB metadata. Indexes ~2x of that.

Short-code namespace: 7-character base-62 codes yield ~3.5 trillion possibilities. Plenty of headroom. 6-character codes (~56 billion) are enough for ~100 years of growth.

Read patterns: a small fraction of pastes attract most views. The top 1% of pastes account for 80%+ of views (Pareto distribution). The long tail (most pastes ever created) attracts very few views — many are viewed once and never again.

View_count writes are voluminous: 200M+ view increments/month. Synchronous writes per view would amplify the write QPS by 10x; production systems batch view-counter writes (in-memory accumulator flushed every minute).

High-level design

Three core services: paste service, short-URL service, and view-tracking. Plus an object-storage tier for the actual paste content.

Paste service handles create and read. On create: validate input, generate a unique short_code (see deep dive on collision strategies), write paste content to object storage, write metadata (short_code, owner, title, language, expires_at, content_storage_url) to a sharded relational store keyed by short_code, return the short URL.

On read: look up metadata by short_code in the relational store, check expiration and visibility, fetch content from object storage, return rendered HTML (with syntax highlighting if a language was specified). Reads are cached aggressively at the edge.

Short-URL generation: see deep dive. Options range from random + collision check, to a monotonic counter encoded in base-62, to a hash of the content.

View-tracking: each view emits a view event (paste_id, timestamp, ip-hash) to a streaming queue. A worker consumes the stream and aggregates view counts per paste, flushing to the metadata row periodically. The view counter on the metadata row is eventually consistent — exact view count isn't load-bearing for the product.

Expiration: scheduled cleanup of expired pastes. A background sweeper periodically scans for expired rows and deletes content from object storage plus metadata from the relational store. The scan is bucketed by expiration timestamp (an index on expires_at) so each sweep is bounded. Lazy expiration also works: on read, check expiration and treat expired pastes as not-found, deferring physical cleanup.

Edge cache: the read path is heavily cached. Cache key is short_code; TTL is short for newly-created pastes (so updates are visible) and long for old pastes (which rarely change). On cache miss, fetch from origin; populate cache. With proper cache hit rates (95%+), the relational store sees almost no read load on popular pastes.

Deep dive — the hard problem

Two deep dives: short-code generation strategies, and storage cost optimization for the long tail.

Short-code generation — three viable strategies, each with tradeoffs.

Strategy A: random + collision check. Generate a random 7-char base-62 code; check the metadata store; if collision, retry. Simple, but every create requires a read. At 100 writes/sec peak this is acceptable. Collision probability rises as namespace fills: at 1% of 62^7 used (~35 billion pastes), collision probability is ~1%; at 50%, it's 50%. With 600M pastes at year 10, namespace utilization is <0.02% — collisions are essentially zero. Production-acceptable for pastebin-scale.

Strategy B: monotonic counter + base-62 encode. Maintain a global counter; on create, increment and encode the value in base-62. Zero collisions, fastest write path. Tradeoff: counter is a single point of contention. Solution: distribute counter ranges — each application server pre-allocates a block of 1000 IDs from a coordination service; local increments within the block need no coordination. Counter values are sequential, which can be a privacy concern (allows an attacker to enumerate recent pastes). Mitigation: shuffle the counter through a reversible permutation (e.g., Feistel cipher) so consecutive IDs produce non-consecutive codes.

Strategy C: hash of content. SHA-1 of the content, truncate to 42 bits, base-62 encode. Deterministic — same content gets the same short code (saves storage on duplicate pastes). Tradeoff: hash collisions become real with truncation; need a collision-resolution scheme (salt with creator's user_id, or rehash with a counter). Most production pastebins pick (A) or (B); (C) is opportunistic for popular content.

Storage cost optimization: the long tail of rarely-viewed pastes is expensive. A paste created 5 years ago that's never been viewed since still consumes storage. Two production tactics.

Tiered storage by age and access: pastes go through hot → warm → cold tiers. Hot tier (recent, frequently-accessed) is on fast SSDs or in-memory cache. Warm tier (older but occasionally-accessed) is on standard object storage with normal pricing. Cold tier (old, rarely-accessed) goes to archival storage (Glacier-style — much cheaper per-byte but slower to access, ~minutes-to-hours retrieval latency).

Tier transitions: a background job migrates pastes between tiers based on access patterns. Pastes not accessed in 30 days move to warm; not accessed in 1 year move to cold. On a cold-tier access, the user sees a 'paste is being retrieved' message and waits a few minutes. For a free-tier pastebin this is acceptable; for a paid tier you'd keep everything in warm.

Compression: most paste content is highly compressible (code and logs have repetitive patterns). Compress at write time (gzip or zstd) — typically 3-5x reduction. Stored in object storage in compressed form; decompressed at read time. Minimal CPU cost.

Third tradeoff: anti-abuse. Pastebins are commonly used for malicious purposes (credentials, malware payloads, illegal content). Production systems run content classifiers on paste creation, flag suspicious pastes for review, and accept abuse reports from the public. Mention this as an operational concern — interviewers reward the candidate who raises it.

Fourth: rendering and syntax highlighting. Done at read time, server-side, with output cached. If a paste specifies a language, the read path runs the content through a syntax-highlighter and returns rendered HTML. Cached output means the highlighter runs once per paste per version. Mention this as a separate cache layer.

Common mistakes

• Storing paste content in the metadata relational store instead of object storage — kills both stores at scale and limits paste size
• Forgetting view-count batching — synchronous view increments on every read amplify write QPS 10x
• Designing a single hot tier without tiered storage — long-tail pastes dominate storage cost
• Skipping URL collision strategy entirely — production must explain how short_codes are generated
• Ignoring anti-abuse — pastebins are a known abuse target and interviewers expect a defense layer

Likely follow-up questions

• How would you support paste editing with full revision history?
• What changes if pastes could be very large (100 MB+)?
• How would you implement password-protected pastes where only users with the password can view?
• How would you support team workspaces where a group of users share a private paste namespace?
• How would you detect and remove pastes containing leaked credentials (API keys, passwords)?