Crate grafos_mq 

Lease-based message queue over fabric resources.

grafos-mq provides partitioned topic-based messaging where each partition is a ring buffer in leased fabric memory. Producers send messages with round-robin or key-hash partitioning. Consumers poll from assigned partitions with offset tracking and seek support.

Architecture

Producer -----> TopicManager -----> Partition 0 (ring buffer in MemLease)
           |                   |--> Partition 1
           |                   |--> Partition N
           |
Consumer <----> OffsetStore
           |
ConsumerGroup --> partition assignment (decentralized, lease-based liveness)
           |
DlqRouter -----> DLQ Topic (separate topic for dead letters)

Quick start

use grafos_mq::topic::{TopicManager, TopicConfig};
use grafos_mq::producer::Producer;
use grafos_mq::consumer::{Consumer, SeekPolicy};
use grafos_mq::offset::MemOffsetStore;

let mut mgr = TopicManager::new();
mgr.create("events", TopicConfig::default())?;

// Produce
let mut prod = Producer::new("events");
prod.send(&mut mgr, b"hello")?;
prod.send(&mut mgr, b"world")?;

// Consume
let store = MemOffsetStore::new();
let mut consumer = Consumer::new("events", "my-group");
consumer.assign(&[0, 1, 2, 3]);
consumer.seek(&mgr, &store, SeekPolicy::Earliest)?;
let msgs = consumer.poll(&mgr, 100)?;
assert_eq!(msgs.len(), 2);

Feature flags

Feature	Default	Effect
std	Yes	Enables std in grafos-std
durable	No	Enables durable partition spill to block storage
observe	No	Enables observability hooks
exactly-once	No	Enables epoch fencing for exactly-once semantics

Modules

consumer

Consumer: poll messages from assigned partitions, commit offsets, seek.

dlq

Dead-letter queue: nack tracking and routing to a DLQ topic after max retries.

group

Consumer group: decentralized partition assignment with lease-based liveness.

locator

TopicLocator for cross-application topic discovery via grafos-locator.

message

Message envelope for MQ records.

offset

Offset storage: trait and in-memory implementation.

partition

Partition: ring buffer in leased memory with slot-based indexing.

producer

Producer: send messages to topic partitions.

topic

Topic management: create, open, delete topics with configurable partitions.

uri

MQ URI type: fabric-mq://pool/topic[/partition].