Qlaster: shared-memory streaming when your Solana services share a host

Almost every way you stream Solana data today assumes the consumer is somewhere else. Yellowstone gRPC serializes account and transaction updates and ships them over the network; that's the right call when your indexer is in another datacenter. But a surprising amount of Solana infrastructure — RPC sidecars, real-time indexers, MEV searchers, simulation services — runs on the same physical host as the validator feeding it. For those, Qlaster (by Brew Labs) asks the obvious question: if the producer and consumer share memory, why pay for a socket at all?

Qlaster is a Rust system for shared-memory data streaming between colocated Solana services. A single sender fans account/transaction updates out to multiple local consumers over per-consumer rings in /dev/shm, coordinated by a Unix-domain control socket. It's Linux-only and unapologetically low-level — and that's exactly the point.

The core idea: rings in /dev/shm, not bytes on a socket

The hot path never touches a socket. Each consumer gets its own SPSC ring (single-producer, single-consumer) backed by a memory-mapped file in /dev/shm. The sender writes an update once into the ring memory; the consumer reads it in place. No per-consumer serialization, no kernel copy across a socket buffer, no syscall per message on the read side. That's the "zero-copy hot path" — the update lands in mmap'd memory and is consumed where it sits.

The wins that fall out of that decision:

• One write, many readers. Per-consumer rings mean the sender isn't re-encoding the same update N times for N consumers.
• Sender-side filtering. Each consumer subscribes to what it cares about (account pubkeys, owners, transaction opt-in) and the sender filters before writing — so a consumer's ring only ever sees relevant traffic. Large filter sets fall back to a bloom pre-reject to keep the check cheap.
• Backpressure that doesn't stall everyone. If a slow consumer fills its ring, it gets dropped rather than blocking the dispatcher and starving every other consumer. In a fan-out system that's the correct failure mode — one bad consumer can't take down the feed.
• Opportunistic LZ4. Big account payloads (≥ 500 KiB) get compressed — but only when an entropy sample predicts a real win, so you don't burn CPU compressing incompressible data.

How a consumer connects: the control socket and SCM_RIGHTS

Shared memory needs a way to bootstrap — a consumer has to learn which ring is its and get a way to be woken when data arrives. Qlaster uses a Unix-domain socket (UDS) purely as a control channel: handshake, subscription, and wakeup setup. The clever part is what travels over it. Using SCM_RIGHTS — the Unix-socket mechanism for passing file descriptors between processes — the sender hands the consumer:

• the path to its /dev/shm ring, and
• an eventfd the sender signals when it writes, so the consumer can block-and-wake instead of busy-polling.

That eventfd is what makes the design practical: a consumer sleeps until the sender pokes its eventfd, then drains the ring. No spinning a core to watch shared memory, but also none of the latency of a full network round-trip. The sender provisions one ring + one eventfd per consumer at connect time.

What it looks like to use

The API is two halves — stand up a sender fed by a Tokio broadcast channel, and connect consumers that drain frames into a lock-free queue you poll:

use qlaster::sender::{SenderConfig, ShmTransportConfig, setup_sender};
use qlaster::consumer::setup_shm_consumer;
use qlaster::metrics::QlasterSenderMetrics;
use std::sync::Arc;
use tokio::sync::broadcast;

// Sender: your validator/Geyser source publishes into a broadcast channel.
let (updates_tx, _) = broadcast::channel(128);
let cfg = SenderConfig { shm: ShmTransportConfig::defaults("/tmp/qlaster.sock") };
let sender = setup_sender(cfg, updates_tx.clone(), None,
    Arc::new(QlasterSenderMetrics::new())).await?;
tokio::spawn(sender.run());

// Consumer: connect, subscribe to the accounts you care about, drain.
let mut consumer = setup_shm_consumer("/tmp/qlaster.sock", 9000).await?;
consumer.subscribe(vec![pubkey], vec![]).await?;
while let Some(update) = consumer.updates.pop() {
    // read in place — no copy off a socket
}

Under the hood, sender/ binds the UDS and provisions the rings/eventfds; consumer/ receives the ring path + eventfd over SCM_RIGHTS and drains frames into crossbeam_queue::ArrayQueues the caller polls; shm/ holds the ring buffer, eventfd, and UDS framing primitives. It leans on the standard Rust systems toolkit — tokio for the async control plane, crossbeam for lock-free handoff.

Where this fits — and where it doesn't

Qlaster isn't a Yellowstone competitor; it's a different layer. A natural topology is Geyser/Yellowstone for the network boundary, Qlaster behind it for same-host fan-out: one process ingests the validator's Geyser stream and re-publishes to a dozen local consumers over shared memory, each filtering for its own slice. It's in the same "get closer to the metal" spirit as Firedancer on the validator side and DoubleZero on the network side — except here the optimization is the absence of a network.

The honest read

Qlaster is a sharp tool with a narrow blade. The constraints are real and non-negotiable: Linux-only (it depends on eventfd(2), memfd_create, and SCM_RIGHTS), and sender and consumer must share a host and the same /dev/shm. If your consumers are anywhere else on the network, this isn't for you — use gRPC and move on. It's also early (a handful of commits, a small public repo) and low-level enough that you're signing up to understand rings, eventfds, and fd-passing to operate it well. But for the specific, increasingly common case — high-throughput Solana services colocated with their data source — eliminating the serialize-and-copy tax is exactly the kind of unglamorous win that shows up in p99 latency and CPU budget. It's a clean piece of systems engineering aimed at a real bottleneck.

References

• brewlabshq/qlaster-public — the repo (by dhruvsol, Brew Labs)
• Yellowstone gRPC — network-boundary Geyser streaming
• Firedancer — the validator-client landscape
• DoubleZero — a dedicated network layer for Solana

The fastest packet is the one you never send. When your producer and consumer already live on the same host, Qlaster turns "stream over the network" back into "point at the same memory" — which is what it always should have been.