CDN Deep Dive

The Problem That Physics Forces On You

You browse a website. You expect it to load fast. The page should appear within a second, the images should fade in immediately, the videos should start without buffering. Most of the modern web meets this expectation. None of it would be possible without CDNs.

Here is the constraint nobody can engineer around: the speed of light. New York to Tokyo, even on a perfect fiber link, is about 70 milliseconds one way. Round-trip is 140ms minimum. Add TCP handshake, TLS handshake, server processing, and you easily pass 500ms before a single byte of useful content arrives. Multiply that by the dozens of resources on a typical page (HTML, CSS, JS, images, fonts) and the user is staring at a blank screen for many seconds.

The fix is the only one possible: move content closer to the user. If a Tokyo user can fetch from a Tokyo data center instead of Virginia, the round-trip drops from 140ms to 5ms. Twenty-eight times faster. Across many requests on a page, this is the difference between fast and slow. Between users staying and bouncing. Between a viable product and a dying one.

That is what a CDN does. It is global infrastructure dedicated to bringing content close to users. This article walks through how it actually works, beneath the marketing.

Step 1: What a CDN Actually Is

A Content Delivery Network is a network of servers (called edge nodes, POPs for Points of Presence, or just "edges") distributed worldwide. Every major CDN has hundreds to thousands of these locations across continents. When a user requests content, the CDN serves it from the edge node nearest to them rather than from your origin server.

Three things make this work:

Caching: the edge stores copies of content. The first request fetches from origin; subsequent requests hit the cache.
Routing: some mechanism (usually anycast) directs each user's request to their nearest edge.
Programmability: the edge can do more than serve cached files. Modern CDNs run code at the edge, transform responses, enforce security policies.

Every CDN provider (Cloudflare, Akamai, Fastly, AWS CloudFront, Google Cloud CDN, Azure CDN) implements these three principles. The differences are in the details: number of POPs, caching capabilities, programmability features, pricing model, and security add-ons.

Step 2: Anycast Routing — How Users Find the Nearest Edge

The trick that gets users to the nearest POP without any DNS magic. The CDN announces the same IP address from all its POPs simultaneously. Internet routing protocols (BGP) deliver each user's traffic to the topologically nearest one.

How Anycast Works

Normally, an IP address is hosted at one location. 1.2.3.4 belongs to one server. Routing brings packets there.

Anycast breaks this. Multiple servers in different locations all claim the same IP. BGP (Border Gateway Protocol, the routing system the internet runs on) sees multiple paths to 1.2.3.4 and picks the shortest for each ISP. As a result, a user in Tokyo and a user in London both connect to "1.2.3.4" but reach different physical servers.

This is invisible to the application. The user does not know which edge they hit. They just connect to a name like cdn.example.com which resolves to the anycast IP, and packets get routed.

Why Anycast Beats DNS-Based Geo Routing

An alternative is DNS-based routing: the DNS server returns different IPs for different users based on their location. This works but has problems:

DNS caches lie about user location. If a user's resolver is in a different city, the user gets routed to the resolver's location, not their own.
DNS is slow to update. Failover from a failed POP to a healthy one takes minutes (TTL must expire).
Anycast is automatic. If a POP fails, BGP withdraws the route and packets flow to the next-nearest POP within seconds.

Most modern CDNs use anycast for the steady-state routing and DNS for some edge cases (regional content restrictions, custom hostname-to-edge mapping).

Step 3: Cache Hierarchies

Modern CDNs are not a single layer. They are tiered, with multiple cache levels between the user and the origin.

What Each Tier Does

Edge POP: closest to the user. Fast network, smaller cache (often just SSDs, sometimes just memory). Hundreds globally. Most cache hits happen here.

Regional cache: larger, fewer locations. Catches misses that the edge couldn't serve. Has more storage, can hold less-popular content.

Origin shield: a single cache layer in front of your origin. The whole point: ensure your origin sees only ONE cache miss per object, no matter how many edges around the world request it.

Origin: your actual servers. The source of truth. Should receive minimal traffic if the CDN is doing its job.

Why Tiering Works

The math: a popular object hits the edge cache 99% of the time. Of the 1% misses, the regional cache serves 90%. Of those misses, the origin shield serves 90%. By the time a request reaches origin, you are talking 0.001% of incoming traffic. Origin can be modest hardware.

Without tiering, every cache miss at every edge would hit origin. With hundreds of edges, even a 1% miss rate on each would multiply to swamp origin. Tiering is what makes the math work.

Step 4: What Gets Cached and What Doesn't

Not all content can be cached. The CDN respects HTTP cache headers from the origin response.

Cache-Control Directives

Cache-Control: public, max-age=3600 — cache this for 1 hour anywhere (browsers, CDNs).
Cache-Control: private, max-age=600 — browsers can cache for 10 minutes, CDNs cannot (per-user data).
Cache-Control: no-store — do not cache anywhere.
Cache-Control: no-cache — cache it, but always revalidate before serving.
Cache-Control: s-maxage=86400 — CDN-specific TTL (separate from browser).

What Each Class of Content Looks Like

Static assets (CSS, JS, images, fonts): long max-age (months or years). These rarely change; when they do, you change the URL (cache busting via versioned filenames).

HTML pages: short or no cache. They change frequently and might be personalized.

API responses: typically not cached, since they often vary per user. Some public read-only APIs (geo lookups, currency rates) are cached aggressively.

Video streaming segments: long max-age. Each segment is immutable once produced.

User-specific content (profile, dashboard, cart): never cached at the CDN. Always fetched from origin per-user.

The Vary Header

Sometimes you want different cache entries based on request headers. Vary: Accept-Language tells the CDN to keep separate cache entries per language. Vary: Accept-Encoding for gzip vs brotli vs uncompressed.

Be careful: every Vary axis multiplies the cache. Vary: User-Agent can fragment a single object into millions of cache entries (every browser version is different). Cache hit rate plummets. Use Vary sparingly and intentionally.

Step 5: Cache Keys

A cache key is what the CDN uses to identify a unique cached object. By default it is the full URL: scheme, host, path, query string. Two requests with the same URL hit the same cache entry.

Customizing Cache Keys

Sometimes URL alone is not the right key. Examples:

Ignore irrelevant query params. Tracking parameters (utm_source, gclid) don't change content. Configure the CDN to strip them from the cache key. Otherwise ?utm_source=email creates a separate cache entry from ?utm_source=twitter, halving cache hit rate.

Include cookie value. If your site shows different content based on a "country" cookie, include that cookie in the cache key. Beware: each unique cookie value means a separate cache entry.

Geographic variants. If you want different content per country, the CDN can include geo data in the cache key.

Cache key configuration is the single biggest factor in CDN performance. Misconfigure it and you might be caching at 5% hit rate while paying premium pricing.

Step 6: Purging and Invalidation

You changed your homepage HTML. The old version is cached at hundreds of POPs worldwide. You need to evict it.

Purge Mechanisms

URL purge: remove a specific URL from all POPs. Modern CDNs do this within seconds.
Tag-based purge: tag entries when caching. Later, purge all entries with a tag. Useful for "everything related to product 42" if all related responses are tagged with product-42.
Wildcard purge: purge all URLs matching a pattern (example.com/products/*). Slower, more disruptive.
Full purge: wipe everything. Last-resort hammer; subsequent requests all hit origin until cache repopulates.

Versioned URLs (The Best Approach)

Instead of purging, change the URL. Append a version hash or timestamp:

/static/style.v123abc.css

When CSS changes, the URL becomes /static/style.v124def.css. The HTML that references it also updates. Old cached copies of v123 sit unused until evicted naturally. New URL = no cache hit = guaranteed fresh content.

This is best practice for static assets. Build tools generate versioned filenames automatically. Many production CDNs serve only versioned static URLs and never need to purge.

Stale-While-Revalidate

An HTTP cache directive that says "if the entry is expired, serve it anyway and revalidate in the background." Users always get a fast (potentially stale) response. The cache catches up asynchronously.

Great for content that's mostly fresh enough. Used heavily by CDNs. Avoids the "cache stampede" where many users hit origin simultaneously when an entry expires.

Step 7: Edge Compute

Modern CDNs go beyond static caching. They run code at the edge, close to the user. The category names vary: Cloudflare Workers, AWS Lambda@Edge, Fastly Compute@Edge, Akamai EdgeWorkers, Vercel Edge Functions.

What Edge Compute Does

Modify requests. Strip query parameters, normalize headers, add authentication tokens.
Modify responses. A/B test by serving variant HTML, inject scripts, rewrite URLs.
Compute responses. Generate JSON responses to API requests entirely at the edge, with no origin involved. Static content with dynamic personalization.
Authentication and authorization. Reject unauthorized requests at the edge before they touch origin.
Routing. Send some requests to one origin and others to another. A/B test backends. Geo-route to regional origins.

Why Run Code at the Edge?

Latency. A request that can be answered at the edge (5ms away) doesn't need to round-trip to origin (often 100ms+ away). For some workloads (auth checks, A/B test routing, simple personalization), this is a 20x speedup.

Origin offload. Code that runs at the edge doesn't run at origin. As your traffic grows, origin scales linearly with cache misses, not total traffic.

Limitations

Edge compute environments are constrained. Limited memory per request, short execution time, no persistent storage at the edge (sometimes a small KV store is offered). Not for everything; suited for fast, stateless transformations.

Step 8: Security Functions

Modern CDNs are not just performance infrastructure; they are also security infrastructure.

DDoS Mitigation

An attacker tries to flood your site with traffic. Without protection, your origin goes down. CDNs absorb DDoS at the edge: the malicious traffic hits the edge, gets identified, and is dropped before it reaches origin.

The CDN's network is engineered for terabits-per-second of capacity. A single small site behind a CDN gets the same DDoS protection as the largest sites: amounts of attack traffic that would melt origin are routine for the CDN.

Web Application Firewall (WAF)

Filter malicious requests at the edge. SQL injection attempts, cross-site scripting, common exploit patterns — the WAF blocks them before they reach your application. Rule sets cover OWASP Top 10 and continue evolving with new threats.

Good WAFs let you customize: block known bad IPs, allow specific patterns, create custom rules for your app. Most provide a "managed mode" with sensible defaults out of the box.

Bot Management

Distinguish humans from bots. Allow good bots (Googlebot for SEO). Challenge or block bad bots (scrapers, credential stuffers, ticket bots). Techniques include JavaScript challenges, behavior analysis, and known-bot databases.

TLS Termination

HTTPS connections terminate at the edge. The CDN handles certificate management, TLS handshakes, and cipher selection. Your origin can serve plain HTTP to the CDN over a private connection (or HTTPS with origin certificates).

This means you don't manage TLS certificates manually. The CDN provider does it. Let's Encrypt automation, certificate rotation, modern cipher suites all handled.

Rate Limiting

Cap how many requests a single IP can make. Prevents abuse without a separate rate limiter. Configure per-endpoint limits. CDN edges enforce them at line rate.

Step 9: Cost Economics

CDN cost is mostly bandwidth out of the edge. You pay per byte delivered to users. Pricing varies widely by provider and volume:

Bandwidth costs typically range from a fraction of a cent per GB (high volume) to several cents per GB (low volume).
Some providers (Cloudflare) include unlimited bandwidth in their flat plans.
Some (AWS, Akamai) bill per GB transferred.
Geographic regions vary: Asia-Pacific egress is more expensive than US/EU.

Cache Hit Rate Drives Cost

Origins serve cache misses. Origin bandwidth has its own cost (often higher than CDN egress). The higher your CDN cache hit rate, the less origin bandwidth you pay for.

A 95% cache hit rate means 95% of your bandwidth comes from CDN edges (cheaper) and 5% from origin (more expensive). A 50% cache hit rate means you pay full price both ways for half your traffic.

Cache misconfiguration (wrong cache keys, short TTLs, unnecessary Vary headers) destroys hit rates. Optimization: long TTLs where possible, versioned URLs, custom cache keys to ignore tracking parameters.

Origin Shield Saves Origin Cost

Even on a cache miss, the origin shield (an extra layer between regional caches and origin) ensures only ONE actual origin fetch happens worldwide per object. Without it, hundreds of regional caches might each independently fetch the same object on miss.

Bandwidth Negotiations

At very high volumes, CDN bandwidth costs are negotiable. Custom pricing can be a fraction of list price. Multi-CDN setups (using two providers and routing between them) give negotiating leverage.

Step 10: Major CDN Providers

Cloudflare

The developer favorite. Generous free tier (unlimited bandwidth on free plan, with caveats). Strong edge compute (Workers). Competitive pricing at higher tiers. Aggressive on security: free DDoS protection, WAF, bot management. Built-in DNS and SSL. Good for most modern web apps.

Akamai

The granddaddy. Largest network globally (more POPs than anyone). Used by enterprises with high-stakes traffic: banks, governments, e-commerce giants. Sophisticated control, deep customization. Expensive. Customer-facing experience is more "enterprise sales" than "self-service signup."

Fastly

Programmable, fast purges (sub-second). Strong real-time logging. Used by tech-forward companies (Stripe, GitHub historically). Compute@Edge built on WebAssembly. Caters to engineering-heavy teams.

AWS CloudFront

Tightly integrated with AWS. If your origin is on AWS, CloudFront fits naturally. Lambda@Edge for compute. Pricing is per-GB-egress, no flat plans. Convenient for AWS-native shops.

Google Cloud CDN

Tied to Google Cloud. Uses Google's global network (which is genuinely huge). Less customizable than dedicated CDNs. Convenient if you're already on GCP.

Bunny.net, KeyCDN

Smaller, cost-focused providers. Bandwidth at a fraction of major-provider prices. Fewer features but suitable for static content delivery at scale.

Picking a Provider

For most modern web apps: Cloudflare. Generous free tier, strong feature set, easy onboarding.
For enterprise with strict compliance and customization: Akamai.
For developer-driven teams wanting programmability: Fastly or Cloudflare Workers.
For AWS shops: CloudFront integrates well.
For pure cost optimization on static assets: Bunny.net or similar.

Step 11: Multi-CDN Strategies

Some teams use two CDN providers simultaneously. Reasons:

Resilience: if one CDN has an outage (yes, they do), the other takes over.
Performance: different CDNs perform differently in different regions. Mix to optimize globally.
Cost negotiation: ability to shift traffic gives leverage in pricing.
Feature mix: use one for static assets (cheap), another for dynamic acceleration (more expensive but faster).

Implementation: a load balancer (DNS-based or TCP-based) routes traffic between CDNs. Sophisticated setups use real-time performance data to route each user to the better-performing CDN at that moment.

Multi-CDN is operationally complex. Most teams start with one and add a second only when motivated by an outage or cost.

Step 12: Edge Cases and Operational Concerns

The Cache Stampede

A popular cache entry expires. A million concurrent users miss the cache simultaneously. Without protection, all million requests hit origin at once. Origin overwhelmed.

Solutions: stale-while-revalidate (serve old version, refresh in background); request coalescing at the edge (one origin fetch per object regardless of concurrent requests); origin shield.

Cache Poisoning

Malicious user manages to get a "bad" response cached for a popular URL. Now everyone gets the bad response until the cache expires.

Mitigation: validate inputs strictly; never cache responses based on untrusted user input; use cache keys that exclude user-controlled headers; quick purge mechanisms when poisoning is detected.

Origin Failover

Origin goes down. CDN can serve cached content. But cache misses fail. Some CDNs offer "serve stale on origin error": even after expiration, serve the cached version when origin is unreachable.

Multi-region origin: configure the CDN with primary and backup origins. If primary fails, traffic flows to backup.

SSL/TLS Certificate Management

The CDN needs a certificate for your domain. Options:

CDN-issued (free, automatic; SAN certificates that include the CDN's customers).
Custom certificate you upload.
Origin certificate for CDN-to-origin connections.

Most teams use CDN-issued for the user-facing connection and a separate origin cert for CDN-to-origin.

Logging at Scale

Edge POPs log every request. Aggregating logs from hundreds of POPs is non-trivial. Modern CDNs ship logs to your S3, BigQuery, or analytics platform in near real time. Volume can be enormous: a busy site easily produces terabytes of access logs per day.

Monitoring

Cache hit rate, origin response time, error rates, geographic latency. Most CDNs provide a dashboard. Beyond that, you set alerts for hit-rate drops (could indicate misconfiguration), origin error spikes, geographic anomalies.

Geo-Restrictions and Compliance

Some content is blocked in certain countries (legal restrictions, regional rights). The CDN enforces this at the edge: requests from blocked geos return 403.

GDPR, data residency: some data must stay within certain regions. The CDN provider must comply. Most major CDNs offer regional configurations.

Mobile Networks and Lossy Connections

Mobile users have higher latency and packet loss. The CDN's edge being close helps, but TCP/HTTP optimization at the edge helps too. Some CDNs implement custom protocols (HTTP/3, QUIC) to handle packet loss better.

Step 13: Recap of Key Decisions

Anycast routing. Single IP, many physical locations. Users automatically reach the nearest.
Tiered caches. Edge → regional → origin shield → origin. Most requests served before reaching origin.
HTTP cache headers control everything. Cache-Control, max-age, Vary. Misconfigure them and performance dies.
Versioned URLs over purging. Change the URL when content changes; old cache entries decay naturally.
Edge compute for low-latency dynamic responses. Auth, A/B routing, personalization at the edge.
Built-in security. DDoS, WAF, bot management, TLS termination all included.
Cache hit rate drives cost. 95% hit rate = 5% origin bandwidth.
Origin shield protects origin. One miss per object globally, regardless of edge count.

The One Thing to Remember

A CDN is a cache that happens to be everywhere in the world, plus the security infrastructure that comes with being in the path of all your traffic. The wins (latency, bandwidth offload, DDoS protection, TLS automation) are huge for any service serving users globally. The complexity (cache headers, purging strategies, dynamic vs static, edge compute) requires real care to get right, but the payoff is significant. For most modern web applications, putting a CDN in front isn't optional; it is table stakes. The question isn't whether to use one but which one and how to configure it. Cache hit rate is the metric that defines whether you're getting your money's worth. Versioned URLs and good Cache-Control headers are the foundations. Everything else is layered on top.