Derived Projections

Concepts Covered

• Source of truth
• Read models
• Denormalized views
• Projection drift
• Rebuildability
• Event-driven projection workers
• Consistency tradeoffs
• Read-path optimization

Definition

A derived projection is a computed view of data optimized for a specific read pattern.

It is derived because it is not the original source of truth. It is a convenient representation built from source data, committed events, or both.

Examples:

source truth: post_likes rows
projection: post_like_counts

source truth: messages
projection: user_conversation_inbox

source truth: click events
projection: analytics dashboard totals

The Pain That Forces Projections

The most normalized source model is often not the best read model.

In a like system, the source truth may be:

post_likes(user_id, post_id, created_at)

This table is excellent for answering:

Did user_7 like post_42?

But it is expensive for answering this on every feed render:

How many users liked post_42?

Counting all matching rows every time can become too slow. The product needs a fast answer, so the system creates a read-optimized projection:

post_like_counts(post_id, like_count)

The Naive Version

A naive system computes every answer directly from source truth:

SELECT count(*)
FROM post_likes
WHERE post_id = 'post_42';

That is correct, but correctness alone is not enough. If a feed shows 50 posts and each post performs a count over a large table, the read path becomes expensive.

The opposite naive mistake is treating the projection as the only truth:

post_like_counts is the source of truth

That is dangerous. If a counter update is duplicated, missed, or corrupted, the system may have no reliable way to know the real answer.

Mental Model

Source truth answers:

What really happened?

Derived projections answer:

What shape does the product need to read quickly?

The projection is allowed to be optimized, denormalized, cached, or eventually consistent because it can be rebuilt or repaired from the source.

How Projections Work

Common flow:

1. Source write commits.
2. Event is emitted or source table is scanned.
3. Projection worker processes the change.
4. Read model is updated.
5. Product reads from the projection.

Example like flow:

LikeCreated event
  -> counter projection worker
  -> increment post_like_counts
  -> feed reads count quickly

Example chat flow:

MessageCreated event
  -> inbox projection worker
  -> update user_conversation_inbox
  -> app opens conversation list quickly

Why Projections Are Usually Eventually Consistent

Projection workers process after the source write. That means the projection may lag.

This is often acceptable:

• A like count can update a moment later.
• An unread count can briefly lag.
• An analytics dashboard can be behind by minutes.

But the acceptable lag must be a product decision, not an accident.

What Projections Guarantee

Derived projections can provide:

• faster reads
• product-specific query shapes
• lower repeated computation
• decoupled read models
• rebuildable views

They do not automatically guarantee:

• immediate freshness
• no drift
• no duplicate event effects
• simple rebuilds
• correctness without source truth

Operational Reality

Watch:

• projection lag
• event consumer lag
• projection update failures
• drift between source and projection
• rebuild duration
• reconciliation corrections
• read fallback latency
• projection schema migration failures

Failure modes:

• Projection misses events.
• Projection processes duplicate events.
• Source schema changes but projection logic does not.
• Projection becomes stale without alerting.
• Nobody knows how to rebuild it.
• Product decisions depend on stale state that should have been strongly consistent.

Related Topics

• Instagram Likes System
• WhatsApp-Style Messaging System
• Eventual Consistency
• Projection Drift
• CQRS Read Model
• Reconciliation Job