Phase 2: Core Components

Module 05: MessageBus – Event Routing LayerHow components communicate across the platform

In Module 02, we saw that the platform routes communication through the MessageBus. This module explains how that messaging layer works and why it matters for a production-grade trading system.

1. Why Do We Need a Message Bus?

❌

Without a Message Bus

Components invoke one another via direct dependencies. Adding a new component requires modifying all interconnected peers. 10 distinct components spawn 45 connections.
Result: A tightly coupled and brittle architecture.

✅

With a Message Bus

All components interact solely with the message bus protocol. Introducing a new component merely requires subscribing to topics of interest.
Result: Clean decoupling, effortless horizontal scalability.

flowchart LR
                        subgraph before["❌ Direct Connection — Mesh Topology"]
                        direction TB
                        A1["DataEngine"] <--> B1["ExecutionEngine"]
                        A1 <--> C1["RiskEngine"]
                        B1 <--> C1
                        end

                        subgraph after["✅ Message Bus — Star Topology"]
                        direction LR
                        A2["Component A"] --- MB["📡 MessageBus"] --- B2["Component B"]
                        C2["Component C"] --- MB --- D2["Component D"]
                        end
Figure 1: Component connections without a message bus (left) vs. with a message bus (right)

2. Three Communication Patterns

The MessageBus structurally implements three distinct communication patterns, designed for specific quantitative trading scenarios:

2.1 Publish/Subscribe (Pub/Sub)

📢 One-to-Many Broadcast

Publishers do not need to know which subscribers exist, and subscribers do not need to know who produced the message. They communicate asynchronously via loosely-coupled Topics.

sequenceDiagram
                        participant DE as 📊 DataEngine<br />(Publisher)
                        participant MB as 📡 MessageBus
                        participant S1 as 🎯 Strategy A<br />(Subscriber)
                        participant S2 as 🎯 Strategy B<br />(Subscriber)
                        participant Port as 💰 Portfolio<br />(Subscriber)

                        Note over S1,Port: Initialization: Subscribe to topics
                        S1->>MB: subscribe("data.quotes.BTCUSDT.BINANCE")
                        S2->>MB: subscribe("data.quotes.BTCUSDT.BINANCE")
                        Port->>MB: subscribe("data.quotes.*")

                        Note over DE: Ingests new market data tick
                        DE->>MB: publish("data.quotes.BTCUSDT.BINANCE", quote)
                        MB->>S1: Route quote tick
                        MB->>S2: Route quote tick
                        MB->>Port: Route quote tick
Figure 2: Pub/Sub Pattern — DataEngine broadcasts a quote; multiple components receive it concurrently.

2.2 Request/Reply (Req/Rep)

🔄 Synchronous Query Paradigm

A component issues a request and explicitly awaits the targeted response. Ideal for scenarios requiring historical data ingestion or state queries.

sequenceDiagram
                        participant S as 🎯 Your Strategy
                        participant MB as 📡 MessageBus
                        participant DE as 📊 DataEngine

                        S->>MB: request(DataType.BAR, instrument_id)
                        MB->>DE: Forward RPC Request
                        DE->>DE: Query historical bars
                        DE->>MB: response(bars_data)
                        MB->>S: Return payload
Figure 3: Req/Rep Pattern — Strategy querying historical data.

2.3 Point-to-Point (Direct Messaging)

📬 Targeted Delivery

Sends a payload directly to a specific, explicitly named component. Typically leveraged for execution command routing.

sequenceDiagram
                        participant RE as 🛡️ RiskEngine
                        participant MB as 📡 MessageBus
                        participant EE as ⚙️ ExecutionEngine

                        Note over RE: Risk checks passed
                        RE->>MB: send("ExecutionEngine", submit_order_cmd)
                        MB->>EE: Direct imperative delivery
Figure 4: Point-to-Point Pattern — RiskEngine routes a validated order command directly to the ExecutionEngine.

3. Topic Routing Semantics

The MessageBus leverages topic strings to route messages, conceptually mirroring the channel paradigm in Redis PubSub or MQTT:

Topic SchemaSemantic DefinitionExample
data.quotes.{symbol}.{venue}Top-of-Book (BBO) quote for a specific instrumentdata.quotes.BTCUSDT.BINANCE
data.trades.{symbol}.{venue}Execution trade ticks for a specific instrumentdata.trades.ETHUSDT.OKX
data.bars.{bar_type}Aggregated bar emissiondata.bars.BTCUSDT.BINANCE-1-MINUTE-LAST
events.order.{venue}Order state transition events for a specific venueevents.order.BINANCE
events.position.*Aggregated position events (wildcard)Intercepts all position-related events
💡 Wildcard Subscriptions

Using the * operator allows matching of any segment. For instance, data.quotes.* intercepts every BBO quote broadcast across the entire system. This is highly useful for aggregate monitoring nodes or system-wide risk algorithms.

4. The Message Pipeline

Understanding how an event flows through the system:

flowchart LR
                        subgraph flow["Message Pipeline Workflow"]
                        direction LR
                        A["1️⃣ Generation"] --> B["2️⃣ Publish"] --> C["3️⃣ Route Match"] --> D["4️⃣ Synchronous
                        Dispatch"] --> E["5️⃣ Completion"]
                        end

                        F["⏱️ Key Characteristics:<br />Synchronous execution<br />Guaranteed ordering"]
                        F -.-> D

                        style flow fill:#0f1419,stroke:#4fc3f7,stroke-width:1px
Figure 5: The synchronous dispatch workflow of a message.
⚠️ Implications of Synchronous Dispatch

This implies that when the DataEngine fires a quote event, the pipeline blocks and awaits all subscribers (your Strategy, Portfolio, etc.) to resolve execution before advancing. The advantage is guaranteed event ordering in chronological sequence. The disadvantage is that if your strategy processing introduces excessive latency, it will degrade the throughput of the entire core system.

5. Redis External Bus Topologies

In live trading environments, the internal MessageBus can structurally bind to an external Redis Cluster, enabling:

💾

Message Persistence

All transient messages are written to Redis Streams. System crashes incur zero data loss; the engine recovers state from Redis after reboot.

🌐

Inter-Process Communication (IPC)

Multiple discrete TradingNodes can share telemetry data over Redis. An essential requirement for scalable distributed deployment architectures.

📊

External Monitoring

External analytical tools (e.g., custom dashboards, monitoring nodes) can passively subscribe to Redis trading events for real-time macroeconomic monitoring.

flowchart LR
                        subgraph node1["🖥️ TradingNode 1"]
                        s1["Strategy A"]
                        mb1["MessageBus"]
                        s1 --> mb1
                        end

                        subgraph node2["🖥️ TradingNode 2"]
                        s2["Strategy B"]
                        mb2["MessageBus"]
                        s2 --> mb2
                        end

                        subgraph redis["🔴 Redis Cluster"]
                        streams["Redis Streams<br />Message Persistence"]
                        end

                        mb1 <-->|"Pub/Sub"| streams
                        mb2 <-->|"Pub/Sub"| streams
Figure 6: Multiple nodes sharing messages via Redis.

6. Rate Limiting (Throttler Engine)

External Exchange APIs stringently enforce connection rate limits. The MessageBus natively embeds a Throttler to dynamically preempt API ban limits:

🚦 Core Throttler Mechanics

Assuming Binance enforces a hard limit of 10 requests per second:

  • The throttler is configured with limit=10, interval=1s.
  • The first 10 order permutations pass through immediately.
  • The 11th order permutation enters a buffer queue, delaying dispatch until the next rolling second begins.
  • This structurally ensures maximum throughput (TPS) utilization without risking API rate limit bans from the exchange.

7. Interactive Exercise

🧠 Exercise: Selecting the Communication Pattern

Which communication pattern should be used in the following scenarios? (Pub/Sub, Request/Reply, Point-to-Point)

  1. The DataEngine ingests a new BTC tick and needs to distribute it to all active strategies → ____
  2. A strategy intends to query historical data for the past 100 bars → ____
  3. The RiskEngine passes validation and needs to route the synthesized order immediately to the ExecutionEngine → ____
  4. An order execution fires, demanding state updates in both the Portfolio and the strategies → ____
  1. Pub/Sub — One canonical publisher, multiple decoupled consumers.
  2. Request/Reply — Synchronous query requiring specific data and waiting for the payload response.
  3. Point-to-Point — Specific command dispatched directly to a targeted component.
  4. Pub/Sub — Atomic execution transition demanding concurrent notifications to multiple components.

8. Module Summary

📋 Key Takeaways
  1. The Message Bus is the exclusive communication conduit between all components, enabling complete decoupling.
  2. Three Patterns: Pub/Sub (Broadcasting), Request/Reply (Synchronous Queries), and Point-to-Point (Targeted Commands).
  3. Topic Routing: Utilizing granular topic strings and wildcard matchers to route to subscribers.
  4. Synchronous Dispatch: Operations conclude synchronously on a single thread to preserve deterministic event ordering.
  5. Redis Extension: Supports message persistence, Inter-Process Communication (IPC), and scalable distributed configurations.
  6. Throttler: Automatically limits API call frequencies to prevent exchange limit breaches.
← PreviousModule 04: Domain Model Next →Module 05B: Data Management