Caching

Why It Matters

Databases are slow compared to memory.

If every request hits the database, your system will not scale.

Example:

• Database response time: 200 ms
• Cached response time: 20 ms

At 10,000 requests per second, this difference is massive.

Without caching:

• high latency
• database overload
• poor user experience

What This Concept Actually Means

Caching stores frequently accessed data in a fast storage layer like memory.

Instead of:

• API -> Database (200 ms)

We use:

• API -> Cache (20 ms)

Think like this:

• Cache = shortcut to data
• Database = source of truth

How It Works

Let’s take a product catalog API.

Without Cache

• Every request hits database
• DB handles 2,000 queries per second
• Traffic: 8,000 requests per second

Result:

• DB overload
• latency increases to 500 ms

With Cache

• Cache hit rate: 80 percent
• 8,000 requests -> 6,400 served from cache
• Only 1,600 hit database

Result:

• DB stays within limit
• latency drops to 50 ms to 100 ms

Flow:

Diagram
100%
flowchart LR
  User --> API
  API --> Cache
  Cache --> API
  API --> Database
Zoom or open fullscreen for a closer look
visualized by

Key Techniques / Variations

1. Cache Aside (Lazy Loading)

Flow:

• Check cache
• If miss, fetch from DB
• Store in cache

Use case:

• product listings

2. Write Through Cache

Flow:

• Write to cache and DB together

Use case:

• systems needing consistency

Trade-off:

• higher write latency

3. Write Back Cache

Flow:

• Write to cache first
• DB updated later

Use case:

• high write systems

Risk:

• data loss if cache fails

4. TTL (Time To Live)

Cache expires after some time.

Example:

• product data cached for 5 minutes

Trade-offs and Design Decisions

When to use caching

• read-heavy systems
• repeated queries

Pros

• reduces latency (200 ms to 20 ms)
• reduces database load
• improves scalability

Cons

• stale data risk
• cache invalidation complexity
• extra infrastructure

Real-world example

• Database capacity: 3,000 queries per second
• Traffic: 12,000 requests per second
• Cache hit rate: 75 percent

Calculation:

• DB load = 25 percent of 12,000 = 3,000

System works without overload.

Architecture / Flow Diagram

Diagram
100%
flowchart LR
  Client --> API
  API --> Cache
  Cache --> API
  API --> Database
Zoom or open fullscreen for a closer look
visualized by

Failure Modes To Watch

• cache stampede (many requests miss cache at once)
• stale data issues
• cache eviction problems
• memory limits

Design Checklist

Ask yourself:

• What data is frequently accessed?
• What is acceptable staleness? (e.g., 1 minute)
• What is cache size limit?
• What is eviction strategy? (LRU, LFU)
• What is expected cache hit rate?

Summary

• Caching improves performance and scalability
• It reduces database load significantly
• Use cache for read-heavy workloads
• Always plan for invalidation and expiry
• Cache is critical for high-performance systems