Skip to main content

Messaging

Draft specification

Maturity: Draft — not yet validated by a reference implementation. Subject to change during Runtime development.

See Protocol Specifications for reading order and the full specification index.

Abstract​

This specification defines the Faberio Messaging service — the internal asynchronous communication mechanism used by the Runtime to implement replication between nodes within an information space.

Messaging is a runtime-internal protocol. Applications do not send or receive messages directly. The Replication service uses messaging to synchronize state, distribute change notifications, and coordinate updates across participating nodes.

1. Scope​

This specification defines:

  • message format and envelope
  • delivery modes (unicast, topic-based)
  • topic subscription management
  • delivery semantics
  • information space scoping
  • security requirements for messaging
  • runtime-internal usage constraints

Applications MUST NOT access the Messaging service directly. See section 3.6.

This specification does not define:

2. Motivation​

Replication requires asynchronous communication between nodes. When application state changes on one node, the Runtime must notify remote nodes and coordinate synchronization without involving application code in the transport layer.

Messaging provides this internal communication channel. Together with RPC — used for synchronous replication operations — messaging enables the Runtime to implement replication transparently.

Applications interact with replication through the database and file storage APIs. See Distributed Database and Distributed File Storage for replication models. They neither publish nor subscribe to messages, and they do not initiate RPC calls for node-to-node communication.

Messaging operates over transport connections established by the Node Connectivity service and is scoped to information spaces defined by the Node Discovery service.

3. Concepts​

3.1 Message​

A message is a signed, self-contained unit of asynchronous communication. Every message has a UUIDv7 identifier and belongs to exactly one information space.

3.2 Topic​

A topic is a named channel within an information space. Nodes subscribe to topics to receive messages. Topics enable publish-subscribe communication without direct coupling between publishers and subscribers.

3.3 Publisher​

A publisher is a Runtime replication component that sends messages to a target node or topic. Applications do not act as publishers.

3.4 Subscriber​

A subscriber is a Runtime replication component that receives messages from a topic. Applications do not act as subscribers.

3.5 Delivery Mode​

Faberio messaging supports two delivery modes:

ModeDescription
UnicastMessage delivered to a specific target node
TopicMessage published to a topic; delivered to all replication subscribers

3.6 Application Access​

Applications MUST NOT use the Messaging service directly.

Applications do not publish messages, subscribe to topics, or receive message deliveries. All messaging is performed by the Runtime as part of replication implementation.

When an application modifies replicated state — through the database API, file storage API, or other platform services — the Runtime automatically generates and delivers the necessary messages to synchronize state across nodes. Application code remains unaware of the underlying messaging layer.

The same constraint applies to RPC: applications MUST NOT invoke RPC directly. RPC is used internally by the Replication service for synchronous coordination between nodes. See RPC specification.

4. Messaging Architecture​

4.1 Implementation Model​

Messaging is a transport-layer control mechanism for replication — not a replicated database table. The specification deliberately separates the data plane from the signaling plane.

Data Plane vs Control Plane​

LayerMechanismNatureVisibility
Data planeReplicated database and file storagePersistent application stateApplications read/write via platform APIs
Control planeMessaging and RPCEphemeral replication signalsRuntime-internal only

Applications store data in replicated tables. When a record changes, the Runtime detects the change and sends a messaging notification to remote nodes. Remote nodes then synchronize the actual record from the data plane — the message itself does not carry the full application state as a durable store.

Messaging MUST NOT be implemented as a replicated application table. Using a replicated messages table would create a circular dependency: replication would need to replicate messages in order to deliver messages that trigger replication.

Runtime Internal Storage​

The Runtime MAY maintain local, non-replicated structures to support messaging:

StructurePurposeReplicated
processed_message_idsDeduplication of at-least-once deliveryNo
Replication changelogOptional internal sync log used by the replication enginePer replication design

These structures belong to the Runtime implementation. They are not exposed to applications and are not part of the distributed application data model.

Implementation Flow​

  1. Application writes to replicated state via the database or file storage API.
  2. Replication engine detects the change in the data plane.
  3. Messaging sends an ephemeral notification to remote nodes over the transport connection.
  4. Remote replication engine receives the message and checks processed_message_ids for deduplication.
  5. Replication engine synchronizes the actual record in the local data plane.
  6. Application on Node B reads the updated data through the same platform API — unaware of messaging.

5. Message Format​

5.1 Envelope​

Every message MUST be wrapped in a signed envelope with the following fields:

FieldTypeDescription
idUUIDv7Unique message identifier
information_space_idUUIDv7Information space the message belongs to
source_node_idstringCryptographic Node ID of the sender
delivery_modestringunicast or topic
targetstringTarget Node ID (unicast) or topic name (topic)
message_typestringPlatform-defined message type
payloadbytesReplication payload
timestampintegerUnix timestamp in milliseconds
correlation_idUUIDv7Optional reference to a related message or object
signaturebytesEd25519 signature over the envelope payload

5.2 Message Types​

The message_type field identifies the replication operation. Platform-defined types include:

TypeDescription
platform.replication.syncFull state synchronization request
platform.replication.changedReplicated state change notification
platform.replication.conflictConflict detection notification

Applications MUST NOT define custom message types. Application-level events are expressed through replicated state, not through direct messaging.

Replication message types are asynchronous signals only. Synchronous state transfer and conflict resolution use RPC methods — platform.replication.fetch, platform.replication.snapshot, platform.replication.compare, and platform.replication.resolve. See RPC — Relationship to Messaging.

5.3 Payload​

The payload field contains the replication data. Payload format is defined by the platform replication module. Payloads that reference objects MUST use UUIDv7 identifiers as defined in the Object Identifier specification.

5.4 Signing​

The envelope MUST be signed by the sending node's private key. The signature covers all fields except signature itself, serialized in canonical format.

Receiving nodes MUST verify the signature before processing the message within the Replication engine.

6. Delivery Modes​

6.1 Unicast​

In unicast mode, a message is addressed to a specific target node identified by target (Node ID).

Unicast is used when the sender knows the specific node that should receive the message.

6.2 Topic​

In topic mode, a message is published to a named topic. All nodes that have subscribed to the topic within the information space receive the message.

Topics are scoped to an information space. A subscription in space A does not receive messages from space B, even if the topic name is identical.

7. Topic Management​

7.1 Topic Naming​

Topic names MUST be UTF-8 strings defined by the platform. Recommended format:

platform.replication.<entity>

Examples:

  • platform.replication.database
  • platform.replication.file
  • platform.replication.state

Applications MUST NOT create or subscribe to custom topics.

7.2 Subscription​

The Runtime replication engine subscribes to platform topics. Subscriptions are internal to the Runtime and are not exposed to applications.

FieldDescription
topicPlatform topic name
information_space_idUUIDv7 of the information space

Applications MUST NOT register topic subscriptions.

7.3 Subscription Propagation​

Topic subscriptions are NOT globally replicated. Instead, when a node publishes to a topic, the Runtime delivers the message to all known peers in the information space. Each receiving node checks local subscriptions and processes messages in the Replication engine.

For efficient topic routing, implementations MAY maintain a topic interest registry shared between peers via peer exchange.

7.4 Unsubscription​

The Runtime manages subscription lifecycle internally. Applications are not involved in subscription or unsubscription.

8. Delivery Semantics​

8.1 At-Least-Once Delivery​

Messaging provides at-least-once delivery semantics by default. A message MAY be delivered more than once in case of network retries or reconnection.

Applications MUST handle duplicate state updates. Idempotency SHOULD be achieved using the message id (UUIDv7) — the Replication engine MAY track processed message IDs to deduplicate.

8.2 Ordering​

Messaging does NOT guarantee global ordering across nodes. Messages from a single publisher to a single target are delivered in send order when sent over the same transport connection.

Topic messages MAY arrive out of order relative to other publishers.

8.3 Offline Nodes​

Messages sent to offline nodes are NOT queued by default. If the target node is unreachable, the message is dropped.

Applications requiring guaranteed delivery to offline nodes SHOULD use replication services to persist state rather than relying on message queuing.

When a node reconnects after an extended offline period, it MUST request a full state copy via platform.replication.sync instead of relying on missed change notifications. See Distributed Database and Distributed File Storage.

Implementations MAY provide optional message buffering for recently disconnected peers, but this is not required by the specification.

9. Information Space Scoping​

All messages MUST include information_space_id. Messages MUST NOT cross information space boundaries.

The Runtime MUST verify that the information_space_id in the message matches the connection's information space.

10. Runtime Authorization​

Messaging is performed by the Runtime on behalf of replicated applications. Before sending messages, the Runtime MUST verify that the originating application is authorized for the target information space as defined in the Node Discovery specification.

Applications without authorization for an information space MUST NOT trigger replication — and therefore no messages are generated for that space.

11. Security Requirements​

Implementations MUST:

  • sign all messages with the sending node's private key
  • verify signatures on all received messages before delivery
  • transmit messages only over encrypted transport connections
  • enforce information space scoping on all messages
  • verify application authorization before generating replication messages

Implementations MUST NOT:

  • expose messaging APIs to applications
  • deliver messages directly to application code
  • allow applications to publish, subscribe, or invoke RPC
  • deliver unsigned messages
  • forward messages across information space boundaries

12. Examples​

12.1 Unicast Replication Sync​

{
  "id": "0195a3f0-9000-7897-abcd-000000000010",
  "information_space_id": "0195a3f0-8000-7890-abcd-000000000001",
  "source_node_id": "a3f8c2e17b4d9a0e5f1c8b2d6e4a7f90",
  "delivery_mode": "unicast",
  "target": "b4e9d3f28c5e0b1f6g2d9e8c7f6b5a4e1",
  "message_type": "platform.replication.sync",
  "payload": "{ \"object_id\": \"0195a3f0-9100-7898-bcde-000000000011\" }",
  "timestamp": 1718000000000,
  "correlation_id": "0195a3f0-9050-7897-cdef-000000000005",
  "signature": "7d3a...4f2e"
}

12.2 Topic Replication Notification​

{
  "id": "0195a3f0-9200-7899-cdef-000000000012",
  "information_space_id": "0195a3f0-8000-7890-abcd-000000000001",
  "source_node_id": "a3f8c2e17b4d9a0e5f1c8b2d6e4a7f90",
  "delivery_mode": "topic",
  "target": "platform.replication.database",
  "message_type": "platform.replication.changed",
  "payload": "{ \"object_id\": \"0195a3f0-9150-7898-abcd-000000000013\" }",
  "timestamp": 1718000001000,
  "signature": "9e4b...1a8c"
}

12.3 Replication Trigger​

1. Application updates a database record on Node A (via database API)
2. Runtime detects state change
3. Runtime publishes platform.replication.changed to topic: platform.replication.database
4. Replication engines on Node B and Node C receive the message
5. Receiving runtimes synchronize state — application code is not involved

12.4 Internal Subscription Flow​

1. Runtime replication engine subscribes to topic: platform.replication.database
2. Remote node publishes platform.replication.changed
3. Runtime on receiving node processes message internally
4. Replicated state updated — application sees consistent data via database API

12.5 Full State Sync After Extended Offline​

Example for the full protocol target (database and file storage). The reference MVP implements node-level DB and FS replication in M2; the Community Hub demo exercises both in M3 — see Implementation Roadmap.

1. Node C was offline for 48 hours
2. Node C reconnects — missed change notifications are not available
3. Node C sends platform.replication.sync to Node A
4. Node A responds with full state via platform.replication.snapshot (RPC)
5. Node C applies complete database and file storage state locally
6. Incremental synchronization resumes for subsequent changes

13. References​