Enterprise Integration Patterns
Common patterns for connecting enterprise applications and when to use each approach. A practical guide to messaging, APIs, and event-driven architectures.
Fundamental Integration Approaches
Point-to-Point Integration
The simplest approach: System A connects directly to System B. If you need to connect A to C, you build another direct connection. With just a few systems, this works fine. With many systems, it becomes a tangled web of connections where changing one system can break connections to many others.
When it works: Few systems, stable interfaces, limited integration scope.
When it fails: When the number of systems grows. With N systems, you potentially need N × (N−1)/2 connections - exponential complexity.
Hub-and-Spoke (ESB)
A central integration hub manages all connections. Each system connects only to the hub, which routes and transforms messages between systems. An Enterprise Service Bus (ESB) is the common implementation of this pattern.
When it works: Many systems to integrate, need for centralised management, complex transformation requirements.
When it fails: Can become a performance bottleneck. Single point of failure. Can become expensive to license and maintain.
Event-Driven Architecture
Systems publish events when things happen ("order placed," "customer updated") without knowing who's listening. Other systems subscribe to events they care about. This decouples producers from consumers; neither needs to know about the other.
When it works: Systems that need loose coupling, real-time requirements, scenarios where multiple systems need the same data.
When it fails: When you need synchronous request/response patterns. When eventual consistency isn't acceptable.
Core Messaging Patterns
Message Queue
Messages are placed in a queue by producers and consumed by a single consumer. The queue guarantees delivery and ordering, even if the consumer is temporarily unavailable. Messages wait until it's ready.
Use case: Order processing where each order must be handled exactly once by a single processor.
Publish-Subscribe
Publishers send messages to a topic. Multiple subscribers can listen to the topic and each receives a copy of every message. Unlike queues, messages go to all interested parties.
Use case: Customer profile updates that need to reach your marketing platform, support system, and analytics tool simultaneously.
Request-Reply
A requester sends a message and waits for a response. This provides synchronous-like behaviour over asynchronous infrastructure; useful when you need immediate feedback.
Use case: Real-time inventory checks during order placement.
Message Router
A component that inspects messages and routes them to different destinations based on content or rules. Useful when different message types need different handling.
Use case: Routing support tickets to different queues based on product category or customer tier.
Message Transformation Patterns
Message Translator
Converts messages from one format to another. Your ERP speaks XML; your e-commerce platform speaks JSON. The translator sits between them, converting on the fly.
Content Enricher
Adds information to messages that wasn't in the original. An order message might contain only a customer ID; the enricher looks up full customer details from your CRM before forwarding.
Content Filter
Removes information that the receiver doesn't need or shouldn't see. Your internal order data might include profit margins that external systems shouldn't receive.
Aggregator
Combines multiple related messages into a single message. Individual line items from different sources might aggregate into a complete order before processing.
Splitter
Breaks a complex message into multiple simpler messages. A batch order might split into individual line items, each processed separately then rejoined.
Reliability Patterns
Guaranteed Delivery
Ensures messages aren't lost even if systems fail. Typically implemented by persisting messages to disk before acknowledging receipt. If processing fails, the message can be redelivered.
Dead Letter Queue
Messages that can't be processed (invalid format, no handler, repeated failures) go to a special queue for manual inspection rather than blocking normal processing or being silently discarded.
Idempotent Receiver
Receivers that can handle the same message multiple times without side effects. If network issues cause a message to be delivered twice, the receiver produces the same result as processing once. Essential for reliable at-least-once delivery.
Making Operations Idempotent
- Use unique message IDs and track which have been processed
- Design operations as "set state to X" rather than "increment by Y"
- Include all data needed in the message rather than relying on external state
- Use database transactions with upsert operations
Transactional Outbox
When a service needs to update its database and publish a message, there's a risk of inconsistency: the database updates but the message fails to send. The outbox pattern writes messages to a database table in the same transaction as the business data, then a separate process reliably publishes from the outbox.
Choosing the Right Pattern
| Requirement | Recommended Pattern |
|---|---|
| One consumer per message | Message Queue |
| Many consumers per message | Publish-Subscribe |
| Immediate response needed | Request-Reply or synchronous API |
| Handle failures gracefully | Guaranteed Delivery + Dead Letter Queue |
| Complex routing logic | Message Router or ESB |
| Different data formats | Message Translator |
| Loose coupling between systems | Event-Driven with Pub-Sub |
Modern Implementation Technologies
Message Brokers
RabbitMQ: Flexible, supports multiple messaging patterns, good for traditional enterprise integration.
Apache Kafka: Designed for high-throughput event streaming, excellent for event-driven architectures at scale.
AWS SQS/SNS: Managed cloud services: SQS for queues, SNS for pub-sub. Simple, scalable, no infrastructure to manage.
Azure Service Bus: Microsoft's enterprise messaging with advanced features like sessions and transactions.
Integration Platforms
MuleSoft: Full integration platform with visual design, API management, and extensive connectors.
Dell Boomi: Cloud-native iPaaS with low-code integration building.
Azure Logic Apps / AWS Step Functions: Cloud workflow orchestration for building integration flows.
Summary
Enterprise integration patterns provide proven solutions to common integration challenges. Rather than inventing solutions from scratch, understanding these patterns lets you choose appropriate approaches and communicate clearly with your team.
The key is matching patterns to requirements. Simple integrations don't need complex patterns. Complex requirements need strong patterns, but also more expertise to implement correctly. Start simple, and introduce patterns as complexity demands.
Key takeaways
- Point-to-point works for two systems. Beyond that, you need a pattern.
- Pub-sub decouples senders from receivers. Essential for scalable architecture.
- Idempotent operations and dead letter queues are reliability essentials, not nice-to-haves.
- Choose message brokers based on your throughput, ordering, and durability requirements.