API Composition Patterns for Complex Data Requirements

The Problem with Multiple Data Sources

Modern applications rarely pull all their data from a single database table. A product detail page might need inventory counts from a warehouse service, pricing from a billing service, reviews from a content service, and shipping estimates from a logistics service. The question is not whether you need to compose data from multiple sources — it is how you do it without creating a slow, fragile mess.

The naive approach is to let the client make four separate API calls and stitch the results together in the frontend. This works for simple cases but falls apart quickly. The client becomes tightly coupled to the internal service topology. Latency compounds across sequential calls. Error handling gets distributed across the UI layer. And if one service is slow, the entire page feels slow because the client is waiting on the weakest link.

API composition patterns solve this by moving the aggregation responsibility to the backend, where it can be optimized, cached, and made resilient to partial failures.

Gateway Composition

The most common composition pattern uses an API gateway as the aggregation layer. The client makes a single request. The gateway fans out to the relevant services, collects the responses, merges them, and returns a unified response.

This is conceptually simple but has real implementation decisions:

Parallel vs. Sequential calls. If the downstream calls are independent, the gateway should make them concurrently. If service B needs data from service A's response, the calls must be sequential. Most real compositions are a mix — some calls can be parallelized, others have dependencies. Modeling this as a directed acyclic graph of calls, rather than a flat list, gives the gateway the information it needs to maximize parallelism.

Partial failure handling. When one of four downstream services times out, what does the gateway return? The options range from failing the entire request (simple but harsh) to returning partial data with a degradation indicator (complex but user-friendly). The right choice depends on whether the missing data is critical or supplementary.

Response shaping. The gateway is also the right place to trim fields, rename properties, and restructure data for the client's needs. This keeps downstream services from having to maintain client-specific response formats. It also naturally leads into the backend for frontend pattern when different clients need different compositions.

Materialized Views and Precomputed Aggregates

Gateway composition works well when the data is relatively small and the downstream services respond quickly. But when composition involves joining large datasets or performing calculations across services, doing it at request time gets expensive.

The alternative is precomputing the composed view. When the underlying data changes, an event triggers a process that updates a read-optimized materialized view. The API serves directly from this view without making any downstream calls at read time.

This is the read side of CQRS applied to cross-service data. It trades write-time complexity for read-time simplicity. The materialized view is eventually consistent — there is a delay between when the source data changes and when the view reflects it — but for many use cases (product catalogs, dashboards, reporting) eventual consistency measured in seconds is perfectly acceptable.

The implementation typically involves an event consumer that listens for changes from relevant services and updates a denormalized read store. The read store can be a document database optimized for the specific query patterns the API needs to serve.

Choosing the Right Composition Strategy

The decision between runtime composition and precomputed views comes down to three factors:

Freshness requirements. If the data must be current to the millisecond — account balances, inventory for high-demand items — runtime composition is necessary. If staleness measured in seconds or minutes is acceptable, precomputed views are faster and more resilient.

Query complexity. Simple key-based lookups across a few services are well-suited to gateway composition. Complex aggregations, joins, or calculations that span multiple services are better handled by precomputed views that do the heavy lifting asynchronously.

Downstream service reliability. If the services you are composing from are highly available and fast, runtime composition is straightforward. If they are unreliable or slow, precomputed views insulate your API from their instability.

In practice, most systems use both patterns. Critical, high-traffic reads get precomputed views. Lower-traffic or freshness-critical reads use gateway composition. The event-driven architecture that powers the precomputed views often provides benefits beyond just API composition — it becomes the backbone for analytics, notifications, and audit trails.

Composition is where distributed systems either feel seamless to the end user or expose their internal complexity through slow, inconsistent experiences. Getting it right is one of the highest-leverage architectural decisions in a service-oriented system.

If you are designing APIs that need to compose data from multiple services and want to get the architecture right the first time, let's talk.