Designing Dropbox / Google Drive

The Problem That Hides Distributed Systems Hardness

You edit a document on your laptop. A few seconds later, the same edit appears on your phone. You go offline, make some changes, come back online, and everything quietly synchronizes in the background. This experience feels like magic.

Behind it is one of the most demanding distributed systems problems in production: keeping arbitrary files synchronized across multiple devices and users, in real time, while handling petabytes of storage, conflict resolution, sharing permissions, and unreliable networks.

Dropbox, Google Drive, OneDrive, iCloud Drive, Box: they all solve the same core problem. The architecture has converged on a similar pattern. This article walks through it.

Step 1: Requirements

Functional Requirements

Non-Functional Requirements

Reliability: never lose data. Files survive any single (or multi) machine failure. Backups, replication.
Latency: small file changes propagate within seconds. Large file uploads progress visibly.
Scalable: hundreds of millions of users, petabytes total. Each user might have hundreds of GB.
Bandwidth efficient: minimize what gets transferred. Network bandwidth is expensive both for the service and the user.
Cost effective: storage cost dominates the unit economics. Aggressive deduplication.

Step 2: Capacity Estimation

Two takeaways. First, raw storage is in exabytes. Dedup matters enormously: a 4x reduction saves billions in infrastructure cost. Second, the operation rate is high but not extreme; what matters more is the bandwidth and storage cost per operation.

Step 3: The Big Insight — Block-Level Deduplication

Naive design: store every file as a single binary blob. When the file changes, re-upload the whole thing. Simple, wrong.

Why it is wrong:

A 100 MB file with one byte changed becomes a 100 MB upload. Wasteful.
Two users with the same MP3 store it twice. Wasteful.
Resumable uploads after a network failure require knowing which parts of the file made it. Hard with whole-blob storage.

The right approach: split every file into blocks (chunks). Hash each block. Store blocks by their content hash. The "file" is just a list of block hashes plus metadata.

Block Size

Common block size: 4 MB. Big enough that overhead per block is small. Small enough that incremental updates only retransmit a few blocks.

A 100 MB file becomes 25 blocks. Change one byte in the middle, and only 1-2 blocks (the one containing the change) need to upload. The other 23-24 blocks are already stored.

Content-Addressed Storage

Each block is identified by the SHA-256 hash of its content. Blocks are stored in a giant key-value store: hash => block content. This is called content-addressed storage.

Two important properties:

Identical content stored once. If two users upload the exact same MP3, both files reference the same blocks. The MP3 is stored once globally.
Tamper-evident. If anyone modifies a block, its hash changes, so all references break.

The Metadata Layer

The file is metadata: name, parent folder, creation time, modification time, owner, and a list of block hashes plus their order. Metadata is small (a few hundred bytes per file). Stored separately from blocks, in a regular database.

The split is critical: blocks live in cheap object storage (S3-style); metadata lives in fast databases.

Step 4: Full Architecture

The Three APIs

Block API handles raw block transfer. Upload a block (POST with content; server hashes and stores). Download a block (GET by hash). Pre-flight check (does this hash already exist? if so, no need to upload).

Metadata API handles the logical file system. Create a file with this name, this parent, and these block hashes. Rename. Move. Delete. List folder contents. Get version history.

Notification API tells clients about changes. Long-poll or WebSocket. When the user's filesystem changes (from another device, or because someone shared a file), the client gets notified and pulls down the changes.

The split between Block and Metadata APIs is critical. Block uploads are huge but rare. Metadata operations are tiny but frequent. Different scaling concerns; different infrastructure.

Step 5: The Sync Protocol

Each client maintains a local view of the user's files. The protocol keeps it in sync with the server.

Upload Flow (Local Change)

1. Client detects a file changed locally (file system watcher).
2. Client splits the changed file into blocks. Hashes each.
3. Client checks each block hash against the server: "do you already have this?" Server returns a list of blocks it doesn't have.
4. Client uploads only the missing blocks (typically a small fraction of the file).
5. Client calls Metadata API: "this file now consists of these block hashes." Server records the new file version.
6. Server emits a change event for the user's other devices.

Download Flow (Remote Change)

1. Other device receives notification: "files changed."
2. Client calls Metadata API: "what changed since version N?"
3. Server returns a delta: list of files changed, with their new block hashes.
4. Client compares: which blocks are new (not in local cache)?
5. Client downloads only the missing blocks.
6. Client reassembles the file locally.

The key efficiency: only changed blocks transfer. A 100 MB file with a 1 MB change costs 1 MB of bandwidth, not 100.

The Notification Channel

How does Device B know to sync when Device A makes a change? Long-polling or WebSockets to the Notification API. When Device A's metadata commit completes, the server fans out a notification to all other connected devices for that user.

If a device is offline, it misses the notification. On reconnect, the client asks "what changed since I was last connected?" and catches up.

Step 6: Conflict Resolution

Two devices, both offline, both edit the same file. They reconnect. Now what?

Last-Write-Wins

Simplest. The server takes whichever update arrives last. Other versions silently disappear.

Bad UX. Users hate when their work disappears without warning. Avoid this for any product where users care about their content.

Both-Versions (Conflicted Copy)

The strategy used by Dropbox. When the server detects a conflict, it accepts the second update as a separate file with a name like:

filename (Device B's conflicted copy 2026-04-12).docx

The user sees both versions and resolves manually. Inelegant but safe: nobody loses work. Dropbox specifically chose this because users prefer "weird filename" over "missing data."

Operational Transform / CRDT

For real-time collaborative editing (Google Docs, Notion), neither last-write-wins nor both-versions works. You need to merge concurrent edits at a finer grain.

Operational Transform (OT) and Conflict-free Replicated Data Types (CRDTs) are mathematical techniques that allow two clients to make independent edits and converge on the same final document. Both are complex to implement correctly. Most file sync services don't use them; they're for structured editing apps that need true real-time co-editing.

What Dropbox Actually Does

Both-versions for general files. For Dropbox Paper (their docs product), full collaborative editing with OT/CRDT under the hood.

The right answer depends on the type of content. A general file sync service should default to both-versions. Specialized collaboration products should layer real-time editing on top.

Step 7: Storage Choices

Block Store: object storage (S3 or equivalent). Keyed by block hash. Effectively unlimited capacity. Cheap. Each block stored with multiple replicas for durability (typical 3x replication, or erasure coding for larger blocks).

Metadata DB: sharded SQL database (often Postgres or MySQL). Sharded by user_id. Each user's filesystem tree fits on one shard. Schema includes file metadata (name, type, parent_folder_id, block_hashes, created_at, modified_at, version, deleted_at) and folder structure.

Versioning: immutable references. A new version of a file is a new metadata row pointing at potentially different blocks. Old versions remain accessible until garbage collected. Old blocks remain in the block store as long as any version references them.

Block Store cleanup: a background job tracks reference counts. When no metadata row references a block, it can be deleted. Often delayed for safety (don't immediately delete; wait 30 days in case of metadata corruption).

Hot vs Cold: recent blocks are accessed often; old archived blocks rarely. Move cold blocks to cheaper storage tiers (S3 Glacier-style). Trade-off: retrieving cold data has latency.

Step 8: Sharing and Permissions

Users want to share files and folders with other users. The permissions model:

Permission Levels

Owner: full control. Can delete, change permissions, etc.
Edit: can modify and add files, but not delete the share.
Comment: can view and leave comments.
View: read-only.

Implementation

A permissions table: (resource_id, user_id, level, granted_by, granted_at). Resource is a file or folder. When a user accesses something, the system checks: does this user have permission?

For folders: permission is inherited by children. Granting "edit" on /Projects gives edit on everything inside. The permission check walks up the tree until it finds a permission entry.

Public Links

"Anyone with the link can view." Generate a long random token. Map the token to a (resource_id, level). Anyone with the URL gets that access without authenticating.

Public links can be revoked (delete the token), expired (TTL), or password-protected (require additional secret).

Cross-User Sharing and Storage

If User A shares /Projects with User B, User B sees the folder in their tree. The folder is shared, not duplicated. Both users see the same files. Edits by either propagate to both.

This is metadata-only: User B's filesystem includes a "shared with me" pointer to A's folder. The actual blocks are stored once.

Step 9: Bandwidth Optimizations

Block-level dedup is the biggest. But several other tricks reduce data transfer:

Compression

Blocks are compressed before storage and transfer. Modern algorithms (zstd) achieve significant ratios for text. Less for already-compressed data (images, video).

Delta Sync Within a Block

Sometimes a block is mostly the same as its predecessor with small changes. Send just the diff (rsync-style rolling checksums). Saves bandwidth at the cost of CPU.

LAN Sync

If two devices on the same network have the same files, sync between them locally instead of through the cloud. The user's office or home network has higher bandwidth than their cloud connection.

Dropbox's LAN sync feature implements this: clients on the same subnet discover each other and exchange blocks directly, only using the cloud as a coordinator.

Resumable Uploads

Block-level dedup makes resume free. If 80 of 100 blocks succeeded before the network dropped, only the remaining 20 need re-upload after reconnection.

Smart Sync (Selective Sync)

Users with limited disk space don't want every file synced locally. The client shows files but only downloads them on access. The cloud is the source of truth; the local copy is a working set.

Implemented via OS-level file system extensions (placeholders that look like real files but trigger downloads on access).

Step 10: Real-Time Notifications

The Notification API tells clients to sync. Implementation choices:

Long Polling: client opens an HTTP request that the server holds. When something changes, server responds. Client immediately makes another long-poll. Simple, broadly supported by firewalls.
WebSockets: persistent bidirectional connection. Lower overhead. Modern default.
Push notifications (mobile): when the app is closed, OS-level push (APNs, FCM) wakes it up.

The notification carries minimal info: "your filesystem version is now N." The client then queries the Metadata API to learn what specifically changed.

Notification Fan-Out

When a metadata commit happens, the server figures out which devices need to know. For a single-user file change, that's their other devices. For a shared file, all collaborators.

Fan-out is asynchronous. Stored in a queue, consumed by notification workers that route to the right user's connection.

Step 11: Edge Cases and Operational Concerns

Empty Files and Tiny Files

A 0-byte file has no blocks. Metadata-only. A 100-byte file fits in less than one 4MB block; it still uses one block (with most space empty). Some implementations inline very small files into metadata to avoid block-store overhead.

Very Large Files

A 100 GB video file is 25,000 blocks. The metadata entry has a list of 25,000 hashes. Manageable but not trivial. Some systems use Merkle-tree-style nested metadata for huge files.

Encryption

Server-side encryption at rest is standard. Client-side encryption (zero-knowledge, where the server can't read your files) is a niche product feature. Trade-off: you cannot do server-side dedup if blocks are encrypted client-side.

Convergent encryption (encrypt block with key derived from its content hash) preserves dedup at the cost of a known information leak: the server can tell if two users have the same file.

Trash and Restore

Deleted files go to trash. Visible to the user for some retention period (30 days). Then permanently deleted. Trash is just a metadata flag (deleted_at set) with a background job to actually purge.

Quota Enforcement

Users have storage limits. The metadata service tracks per-user usage. Block uploads check the quota before accepting. Important: usage is computed from logical (per-user) size, not physical (deduped) size. A user storing the same MP3 as 100,000 others still uses their quota for it.

Network Reliability

Mobile devices have flaky connections. The client retries failed uploads with exponential backoff. Block-level dedup means resumes are cheap.

File System Differences

macOS, Windows, Linux, and various file systems have different rules: case sensitivity, character restrictions, max path length, max filename length. The sync engine normalizes paths and handles cross-platform incompatibilities (e.g., a Windows-illegal character in a Linux-created filename).

Sync Loops and Storms

If two devices are misconfigured, they might trigger each other to update repeatedly. Defensive design: rate limit per-file updates, detect oscillation, alert ops.

The Cold Storage Problem

Old files are rarely accessed but still cost storage. Move them to cheaper tiers. But too aggressive tiering causes user frustration when they finally want a file (slow retrieval). Balance via observed access patterns.

Step 12: Recap of Key Decisions

Block-level deduplication. The single most important design choice. Saves storage and bandwidth.
Content-addressed block store. Hash IS the key. Identical content stored once globally.
Separate Block API and Metadata API. Different scaling characteristics. Different infrastructure.
Both-versions conflict resolution. Safe default. Users prefer "weird filename" over data loss.
Long-poll or WebSocket notifications. Real-time enough for sync UX.
Metadata in SQL, blocks in object storage. Right tool for each job.
Background reference counting for block GC. Old blocks deleted when no reference remains.
LAN sync for high-bandwidth local networks. Cloud as coordinator, peer-to-peer for transfer.

The One Thing to Remember

The design hinge of file sync is the block-level abstraction. Once you treat files as content-addressed blocks, deduplication is automatic, incremental sync is natural, and storage costs collapse. The metadata layer rides on top, tracking which blocks make up each file. Most of the other features (sharing, versioning, conflict resolution) are policies on top of this foundation. The hard problems are not in the architecture; they are in the operational details: cross-platform file system quirks, the long tail of file types and sizes, network unreliability, and the inevitable user expectation that nothing should ever go missing. Get the foundation right (blocks + metadata) and the rest is configuration.