Message Definitions Registry Service - Version 1.0-rc2 đź”—

Abstract đź”—

This specification defines a message and event catalog extension to the xRegistry document format and API specification. The message definitions registry service (or “message catalog”) allows for declaring the metadata (headers, properties, attributes) of messages and/or events and their relationships to schemas, and for grouping those declarations such that they can be associated with endpoints. With the help of this metadata, users can provide constraints for messaging channels, declaring precisely which messages or events are permitted to be sent or can be expected to be received, and how these messages can be distinguished.

Table of Contents đź”—

Overview đź”—

The goal of this message metadata model, in conjunction with the endpoint model, is to provide metadata structure to asynchronous topics, streams and queues in a way that is similar to how table and column definitions provide structure to databases. Database schemas structure data that you have. Endpoint definitions with their referenced, or embedded message definitions, with referenced, or embedded, schema definitions structure data that will be transmitted.

Continuing that analogy, the endpoint scope corresponds to the database, the message groups are schema scopes, and the message definitions correspond to tables. The schema associated with a message definition determines the column layout of the table. As event streams often end up landing in databases for long-term archival and analysis, this structural alignment is very helpful.

Managing the description of the payloads of those messages and events is not in scope, but delegated to the schema registry extension for xRegistry. Schemas are linked from messages and event declarations through a URI Reference.

Message Definitions đź”—

A Message Definition (“message”) describes constraints for the metadata of a message or event. It declares, for instance, the concrete values or permissible value patterns for the type, source, and subject attributes of a CloudEvent.

A message definition has two key purposes:

1 - A message definition is a template. When an application needs to raise an event, the message definition declares precisely, which attributes, headers or properties need to be set on the event envelope or on the protocol message, and which constant values are to be set, or which type of pattern constraints apply. Such templates can be evaluated by tools and code generators, which can then create an interface or event factory for the application programmer that minimizes opportunities to introduce mapping errors.

2 - A message definition is a filter condition. When an application needs to demultiplex an incoming stream, messages, or events that come through the same channel, it can use a collection of message definitions as a set of candidate messages that are expected and can be handled. The metadata of the incoming events or messages is tested against the declarations of the candidate definitions, attribute by attribute, yielding none, one, or more matches for the incoming message. If there is no match, the incoming message is unexpected and will likely be sidelined. If there is one match, the message can be passed on to the application. If there are multiple matches, the message might be evaluated further by matching the body content against the associated data schema.

For message and event producers, the template aspect is the primary concern. Following the declared rules, including validating the payload against the associated schema, ensures that all consumers receive messages with the expected content.

For message and event consumers, the filter aspect is the primary concern. Consumer applications and frameworks can rely on the declared message metadata to demultiplex and dispatch messages and events to handlers and they can sideline non-conformant messages, for instance into dead-letter queues.

Message Metadata đź”—

Any message definition covers up to three aspects:

1 - Envelope: An “envelope” is a transport independent metadata convention that lets a producer convey the context of an event or message to consumers or intermediaries (like publish/subscribe routers) without them having to understand the particular payload format or having to read the payload. The only predefined envelope model in this specification is CNCF CloudEvents 1.0. Once the “envelope” selector is set, constraints for the attributes of the CloudEvents envelope can be defined in the “envelopeoptions”.

2 - Protocol: The “protocol” selector picks a specific application protocol to which the given message is bound. If a protocol is chosen, constraints for the protocol-specific message model can be defined in “protocoloptions”. For instance, if you choose the “MQTT/5.0” protocol, you can constrain the topic path to which this message can be sent and/or the quality-of-service QoS level that is to be used. This specification covers the full metadata set of the application protocols AMQP/1.0, MQTT/3.1.1, MQTT/5.0, Kafka, NATS, and HTTP.

3 - Payload: The “dataschema*” and “datacontenttype” attributes declare the content type of the payload and a schema that can be used to construct, validate, decode and/or encode the payload. The “dataschemaformat” declaration selects the kind of schema (e.g. XML Schema, Protobuf, Avro or others) that is embedded at or referenced by the chosen “dataschema*” attribute.

A message definition MAY contain any combination of envelope, protocol, and payload declarations. A payload-only declaration can be useful if messages are sent through varying protocols without a fixed envelope model and are distinguished by content-type (including parameters) or even only through whether payload schema definitions match an incoming message.

While attributes (or properties or headers; depending on protocol nomenclature) can be defined in the envelopeoptions and protocoloptions, the general pattern used in this model is that the attribute can have a name, a description, a type and a value. Setting a value makes that value constant for all instances of that message, which is useful for discriminators like AMQP’s subject property or CloudEvents’ type attribute.

The “uritemplate” type permits the values to have embedded placeholders (Level 1 URI templates), which turns the values into templates for publishers where the application inserts context information into designated places. For consumers, the templates act as pattern matching filters and to extract context values into named variables.

Reusing Message Definitions đź”—

In complex event-driven enterprise applications it might be desirable to reuse common definitions across different applications. There might also be a need to route the same set of CloudEvents through a specific protocol route, like MQTT, where it would be useful to provide protocol-specific hints in addition to the CloudEvents declaration like requiring a specific topic path pattern for the event type.

The basemessage attribute, an xid-typed reference to another message definition, enables such reuse scenarios. Any message definition MAY refer to another message definition as its base, which effects a copy of the base message’s definitions into the message that defines it. The mechanism is transitive, which means that basemessage relationships MUST be resolved recursively as long as they do not result in circular references.

All aspects of the collected base message are “shadowed” by the definitions of the message that references it. If the base message defines an “envelope” but no “protocol”, the new definition can add the aforementioned MQTT aspects with a new “protocol” selector and corresponding options.

The xref attribute from xRegistry Core allows aliasing of message definitions across message groups, which will be further discussed in the next section.

Message Groups đź”—

All message definitions MUST be contained in a group. A message group can be formed for many reasons. At a minimum, the group will be an access control boundary in many implementations of this specification even though none is mandated.

In many cases, message groups will be a logical boundary around a set of related events that are commonly routed through the channel. A (primitive) air travel luggage handling system might have “checkedin”, “loaded”, “unloaded”, and “delivered” events related to items it moves. If those are reported to external parties through the same channels, it likely makes sense to group them.

A benefit of message groups is that they are very similar to “interfaces” in common programming languages. When a message group is associated with a channel, like a queue, the messages contained in the group form the permitted and expected message set. The association becomes a contract.

This association is formalized in the Endpoint Registry, a related registry that allows associating one or more message definition groups with an endpoint, thus effectively defining a contract for the endpoint. An "endpoint" as defined in that specification is also a message definition group in itself, with the message definitions following the rules of this specification.

When message definitions are to be recombined into different groups, the original message definitions can be referenced instead of copied. For instance, a pubsub topic might take 4 messages as input but a filtering subscription might only yield 1 of those messages as output. In this case, one would create a distinct message group for the subscription and “xref” the message definition from the input group (sample to follow in the merged doc).

Similarly, one could create a new message group to be associated with an MQTT endpoint if the messages annotate CloudEvent definitions with MQTT protocol options using the “basemessage” mechanism. (sample also to follow)

Notations and Terminology đź”—

Notational Conventions đź”—

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

For clarity, OPTIONAL attributes (specification-defined and extensions) are OPTIONAL for clients to use, but the servers' responsibility will vary. Server-unknown extension attributes MUST be silently stored in the backing datastore. Specification-defined, and server-known extension, attributes MUST generate an error if corresponding feature is not supported or enabled. However, as with all attributes, if accepting the attribute would result in a bad state (such as exceeding a size limit, or results in a security issue), then the server MAY choose to reject the request.

In the pseudo JSON format snippets ? means the preceding attribute is OPTIONAL, * means the preceding attribute MAY appear zero or more times, and + means the preceding attribute MUST appear at least once. The presence of the # character means the remaining portion of the line is a comment. Whitespace characters in the JSON snippets are used for readability and are not normative.

Terminology đź”—

This specification defines the following terms:

Message and Event đź”—

A message is a transport wrapper around a message body (interchangeably referred to as payload) that is decorated with metadata. The metadata describes the message body without an intermediary having to inspect it and carries further information useful for identification, routing, and dispatching.

In this specification, message is an umbrella term that refers to all kinds of messages as well as to events as a special form of messages.

The definition of message from the CloudEvents specification applies.

Envelopes and Protocols đź”—

An envelope is a transport protocol-independent message metadata convention. The CNCF CloudEvents specification is an example of a message envelope and is the only envelope explicitly defined in this specification.

A similar transport protocol-independent message metadata convention is, for example, the W3C SOAP 1.2 envelope for which support could be added by an extension.

This specification uses protocol as a selector into the protocol-specific message metadata that is defined under the protocol.ifvalues section of the model. When a known protocol is explicitly specified for a message definition, the protocoloptions section MAY contain constraints for the protocol-specific metadata.

Message Definition Registry Model đź”—

The formal xRegistry extension model of the Message Definitions Registry resides in the model.json file.

For easy reference, the JSON serialization of a Message Registry adheres to this form:

{
  "specversion": "<STRING>",
  "registryid": "<STRING>",
  "self": "<URL>",
  "shortself": "<URL>", ?
  "xid": "<XID>",
  "epoch": <UINTEGER>,
  "name": "<STRING>", ?
  "description": "<STRING>", ?
  "documentation": "<URL>", ?
  "labels": {
    "<STRING>": "<STRING>" *
  }, ?
  "createdat": "<TIMESTAMP>",
  "modifiedat": "<TIMESTAMP>",

  "capabilities": { ... }, ?
  "model": { ... }, ?
  "modelsource": { ... }, ?

  "messagegroupsurl": "<URL>",
  "messagegroupscount": <UINTEGER>,
  "messagegroups": {
    "<KEY>": {                                  # messagegroupid
      "messagegroupid": "<STRING>",             # xRegistry core attributes
      "self": "<URL>",
      "shortself": "<URL>", ?
      "xid": "<XID>",
      "epoch": <UINTEGER>,
      "name": "<STRING>", ?
      "description": "<STRING>", ?
      "documentation": "<URL>", ?
      "labels": { "<STRING>": "<STRING>" * }, ?
      "createdat": "<TIMESTAMP>",
      "modifiedat": "<TIMESTAMP>",
      "deprecated": { ... }, ?

      # MessageGroup extension attributes
      "envelope": "<STRING>", ?                 # e.g. CloudEvents/1.0
      "protocol": "<STRING>", ?                 # e.g. HTTP/1.1

      "messagesurl": "<URL>",
      "messagescount": <UINTEGER>,
      "messages" : {
        "<KEY>": {                              # messageid
          "messageid": "<STRING>",              # xRegistry core attributes
          "versionid": "<STRING>",
          "self": "<URL>",
          "shortself": "<URL>", ?
          "xid": "<XID>",

          # Start of default Version's attributes
          "epoch": <UINTEGER>,
          "name": "<STRING>", ?
          "description": "<STRING>", ?
          "documentation": "<URL>", ?
          "labels": { "<STRING>": "<STRING>" * }, ?
          "createdat": "<TIMESTAMP>",
          "modifiedat": "<TIMESTAMP>",
          "ancestor": "<STRING>",

          # Start of Message extension attributes
          "basemessage": "<URL>", ?            # Message being extended

          "envelope": "<STRING>", ?            # e.g. CloudEvents/1.0
          "envelopemetadata": {
            "<STRING>": <JSON-VALUE> *

            # CloudEvents/1.0 "envelope" the "envelopemetadata" is of the form:
            "<STRING>": {
              "type": "<TYPE>",                # Default=string
              "value": <ANY>, ?
              "required": <BOOLEAN>            # Default=false
            } *
          }, ?
          "envelopeoptions": {
            "<STRING>": <JSON-VALUE> *
          }, ?

          "protocol": "<STRING>", ?            # e.g. HTTP/1.1
          "protocoloptions": { ... }, ?

          "dataschemaformat": "<STRING>", ?
          "dataschema": <ANY>, ?
          "dataschemauri": "<URI>", ?
          "dataschemaxid": "<XID>", ?
          "datacontenttype": "<STRING>", ?
          # End of Message extensions and default Version's attributes

          "metaurl": "<URL>",
          "meta": { ... }, ?
          "versionsurl": "<URL>",
          "versionscount": <UINTEGER>,
          "versions": { ... } ?
      } ?
    } *
  } ?
}

Message Definition Groups đź”—

The Group plural name (<GROUPS>) is messagegroups, and the Group singular name (<GROUP>) is messagegroup.

The following attributes are defined for the messagegroup object in addition to the xRegistry-defined core attributes:

envelope (Message Group) đź”—

protocol (Message Group) đź”—

Message Definitions đź”—

The Resource plural name (<RESOURCES>) is messages, and the Resource singular name (<RESOURCE>) is message.

Different from schemas, message definitions do not contain a version history. If the metadata of two messages differs, they are considered different messages.

When CloudEvents is used for a particular message, it is RECOMMENDED that the message's messageid attribute be the same as the CloudEvents type attribute. Doing so makes for easier management of the meta-model by correlating the look-up value (id) of messages with their related events.

The following extensions are defined for the message Resource in addition to the core xRegistry Resource attributes:

basemessage đź”—

envelope đź”—

Same as the envelope attribute of the messagegroup object.

Since messages MAY be cross-referenced ("borrowed") across message group boundaries, this attribute is also REQUIRED and MUST be the same as the envelope attribute of the messagegroup object into which the message is embedded or referenced.

Illustrating example:

"messagegroupsurl": "...",
"messagegroupscount": 2,
"messagegroups": {
  "com.example.abc": {
    "messagegroupid": "com.example.abc",
    "envelope": "CloudEvents/1.0",

    "messagesurl": "...",
    "messagescount": 2,
    "messages": {
      "com.example.abc.event1": {
        "messageid": "com.example.abc.event1",
        "envelope": "CloudEvents/1.0",
         # ... details ...
        }
      },
      "com.example.abc.event2": {
        "messageid": "com.example.abc.event1",
        "envelope": "CloudEvents/1.0",
        # ... details ...
      }
  },
  "com.example.def": {
    "messagegroupid": "com.example.def",
    "envelope": "CloudEvents/1.0",

    "messagesurl": "...",
    "messagescount": 1,
    "messages": {
      "com.example.abc.event1": {
        "uri": "#/messagegroups/com.example.abc/messages/com.example.abc.event1",
        # ... details ...
      }
    }
  }
}

envelopemetadata đź”—

envelopeoptions đź”—

protocol đź”—

protocoloptions đź”—

dataschemaformat đź”—

dataschema đź”—

dataschemauri đź”—

dataschemaxid đź”—

datacontenttype đź”—

Metadata Envelopes and Message Protocols đź”—

This section defines the metadata envelopes and message protocols that are directly supported by this specification.

Metadata envelopes lean on a protocol-neutral metadata definition like CloudEvents. Message protocols lean on a message model definition of a specific protocol like AMQP, MQTT or Kafka.

A message can use either a metadata envelope, a message protocol, or both.

If a message only uses a metadata envelope, any implicit protocol bindings defined by the envelope apply. For instance, a message definition that uses the "CloudEvents/1.0" envelope but no explicit protocol implicitly applies to all protocols for which CloudEvents bindings exist, and using the respective protocol binding rules.

If a message uses both a metadata envelope and a message protocol, the message binding rules takes precedence over the metadata envelope rules. For instance, if a message definition uses the "CloudEvents/1.0" envelope and an "AMQP/1.0" protocol, then the implicit protocol bindings of the "CloudEvents/1.0" envelope are overridden by the "AMQP/1.0" protocol rules.

If a message uses only a message protocol, only the metadata constraints defined by the message protocol rules apply.

Property Definitions đź”—

The following attributes are used to define the properties associated with the headers, properties or attributes defined for message:

description đź”—
required đź”—
specurl đź”—
type đź”—
value đź”—

If the type property has the value uritemplate, value MAY contain placeholders. As defined in RFC6570 (Level 1), the placeholders MUST be enclosed in curly braces ({ and }) and MUST be a valid symbol. Placeholders that are used multiple times in the same message definition MUST to represent identical values.

When validating a message property against this value, the placeholders act as wildcards. For example, the value {foo}/bar would match the value abc/bar or xyz/bar.

When creating a message based on a metaschema with such a value, the placeholders MUST be replaced with valid values. For example, the value {foo}/bar would be replaced with abc/bar or xyz/bar when creating a message.

If the type property has the value timestamp and the value property is set to a value of 0000-01-01T00:00:00Z, the value MUST be replaced with the current timestamp when creating a message.

Metadata Envelopes đź”—

This specification only defines one metadata envelope: "CloudEvents/1.0".

CloudEvents/1.0 đź”—

For the "CloudEvents/1.0" envelope, the envelopemetadata object contains a property attributes, which is an object whose properties correspond to the CloudEvents context attributes.

As with the CloudEvents specification, the attributes form a flat list and extension attributes are allowed. Attribute names are restricted to lower-case alphanumerical characters without separators.

The base attributes are defined as follows:

Attribute Type
specversion string
id string
type string
source uritemplate
subject string
time timestamp
dataschema uritemplate
datacontenttype string

The following rules apply to the attribute declarations:

The values of all string and uritemplate-typed attributes MAY contain placeholders using the RFC6570 Level 1 URI Template syntax. When the same placeholder is used in multiple properties, the value of the placeholder is assumed to be identical.

The following shows the format of a CloudEvents "envelopemetadata" section for a message (see the model file for the complete definition):

"envelope": "CloudEvents/1.0",
"envelopemetadata" {
  # "CloudEvents/1.0" envelope metadata
  "specversion": {
    "value": "1.0",
    "type": "string"
  },
  "id": {
    "value": "<STRING>", ?
    "type": "string"
  },
  "type": {
    "value": "<STRING>", ?
    "type": "string"
  },
  "source": {
    "value": "<STRING>", ?
    "type": "string"
  },
  "subject": {
    "value": "<STRING>", ?
    "type": "string"
  },
  "time": {
    "value": "<TIMESTAMP>", ?
    "type": "timestamp" ?
  },
  "dataschema": {
    "value": "<URITEMPLATE>", ?
    "type": "uritemplate" ?
  },
  "*": {
    "value": <ANY>, ?
    "type": "<TYPE>"
  }
}

The following example declares a CloudEvent with a JSON payload. The attribute id is REQUIRED in the declared event per the CloudEvents specification in spite of such a declaration being absent here, the type of the type attribute is string and the attribute is required even though the declarations are absent. The time attribute is made required contrary to the CloudEvents base specification. The implied CloudEvents datacontenttype attribute value is application/json and the implied CloudEvents dataschema attribute value is https://example.com/schemas/com.example.myevent.json:

{
  "envelope": "CloudEvents/1.0",
  "envelopemetadata": {
    "type": {
      "value": "com.example.myevent"
    },
    "source": {
      "value": "https://{tenant}/{module}/myevent",
      "type": "uritemplate"
    },
    "subject": {
      "type": "urireference"
    },
    "time": {
      "required": true
    },
  },
  "dataschemaformat": "JsonSchema/draft-07",
  "dataschemauri": "https://example.com/schemas/com.example.myevent.json"
}

For clarity of the definition, you MAY always declare all implied attribute properties explicitly, but they MUST conform with the rules above.

Message Protocols đź”—

"HTTP/1.1", "HTTP/2", "HTTP/3" protocols đź”—

The "HTTP" protocol is used to define messages that are sent over an HTTP connection. The protocol is based on the HTTP Message Format and is common across all versions of HTTP.

The protocoloptions object MAY contain several properties as defined below:

Property Type Description
headers Array The HTTP headers. See below
query Map The HTTP query parameters
path uritemplate The HTTP path
method string The HTTP method
status string The HTTP status code

HTTP allows for multiple headers with the same name. The headers property is therefore an array of objects with name and value properties. The name property is a string that MUST be a valid HTTP header name.

The query property is a map of string keys to string values.

The path property is a URI template.

The method property is a string that MUST be a valid HTTP method.

The status property is a string that MUST be a valid HTTP response code. The status and method properties are mutually exclusive and MUST NOT be present at the same time.

The values of all string and uritemplate-typed properties and headers and query elements MAY contain placeholders using the RFC6570 Level 1 URI Template syntax. When the same placeholder is used in multiple properties, the value of the placeholder is assumed to be identical.

The following example defines a message that is sent over HTTP/1.1:

{
  "protocol": "HTTP",
  "protocoloptions": {
    "headers": [
      {
        "name": "Content-Type",
        "value": "application/json"
      }
    ],
    "query": [
      {
        "name": "foo",
        "value": "bar"
      }
    ],
    "path": "/foo/{bar}",
    "method": "POST"
  },
  "dataschemaformat": "JsonSchema/draft-07",
  "dataschemauri": "https://example.com/schemas/com.example.myevent.json",
}
"AMQP/1.0" protocol đź”—

The "AMQP/1.0" protocol is used to define messages that are sent over an AMQP connection. It is based on the default AMQP 1.0 Message Format.

The protocoloptions object MAY contain several properties, each of which corresponds to a section of the AMQP 1.0 Message, as defined below:

Property Type Description
properties Map The AMQP 1.0 Message Properties section
application-properties Map The AMQP 1.0 Application Properties section.
message-annotations Map The AMQP 1.0 Message Annotations section
delivery-annotations Map The AMQP 1.0 Delivery Annotations section
header Map The AMQP 1.0 Message Header section
footer Map The AMQP 1.0 Message Footer section

As in AMQP, all sections and properties are OPTIONAL.

The values of all string, symbol, uri, and uritemplate-typed properties MAY contain placeholders using the RFC6570 Level 1 URI Template syntax. When the same placeholder is used in multiple properties, the value of the placeholder is assumed to be identical.

Example for an AMQP 1.0 message type that declares a fixed subject (analogous to CloudEvents' type), a custom property, and a content-type of application/json without declaring a schema reference in the message definition:

{
  "protocol": "AMQP/1.0",
  "protocoloptions": {
    "properties": {
      "message-id": {
        "required": true
      },
      "to": {
        "value": "https://{host}/{queue}"
      },
      "subject": {
        "value": "MyMessageType",
        "required": true
      },
      "content-type": {
        "value": "application/json"
      },
      "content-encoding": {
        "value": "gzip"
      }
    },
    "application-properties": {
      "my-application-property": {
        "value": "my-application-property-value"
      }
    }
  }
}
properties (AMQP 1.0) đź”—

The properties property is an object that contains the fixed properties of the AMQP 1.0 Message Properties section. The following properties are defined, with type constraints:

Property Type Description
message-id (see note below) uniquely identifies a message within the message system
user-id binary identity of the user responsible for producing the message
to uritemplate address of the node to send the message to
subject string message subject
reply-to uritemplate address of the node to which the receiver of this message ought to send replies
correlation-id string client-specific id that can be used to mark or identify messages between clients
content-type symbol MIME content type for the message
content-encoding symbol MIME content encoding for the message
absolute-expiry-time timestamp time when this message is considered expired
group-id string group this message belongs to
group-sequence integer position of this message within its group
reply-to-group-id uritemplate group-id to which the receiver of this message ought to send replies to

The message-id permits the types ulong, uuid, binary, string, and uritemplate. A value constraint for message-id property SHOULD NOT be defined in the message definition except for the case where the message-id is a uritemplate.

application-properties (AMQP 1.0) đź”—

The application-properties property is a map that contains the custom properties of the AMQP 1.0 Application Properties section.

The names of the properties MUST be of type symbol and MUST be unique within the scope of the map. The values of the properties MAY be of any permitted type.

message-annotations (AMQP 1.0) đź”—

The message-annotations property is a map that contains the custom properties of the AMQP 1.0 Message Annotations section.

The names of the properties MUST be of type symbol and MUST be unique within the scope of the map. The values of the properties MAY be of any permitted type.

delivery-annotations (AMQP 1.0) đź”—

The delivery-annotations property is a map that contains the custom properties of the AMQP 1.0 Delivery Annotations section.

The names of the properties MUST be of type symbol and MUST be unique within the scope of the map. The values of the properties MAY be of any permitted type.

header (AMQP 1.0) đź”—

The header property is an object that contains the properties of the AMQP 1.0 Message Header section. The following properties are defined, with type constraints:

Property Type Description
durable boolean specify durability requirements
priority integer relative message priority
ttl integer message time-to-live in milliseconds
first-acquirer boolean indicates whether the message has not been acquired previously
delivery-count integer number of prior unsuccessful delivery attempts

The footer property is a map that contains the custom properties of the AMQP 1.0 Message Footer section.

The names of the properties MUST be of type symbol and MUST be unique within the scope of the map. The values of the properties MAY be of any permitted type.

"MQTT/3.1.1" and "MQTT/5.0" protocols đź”—

The "MQTT/3.1.1" and "MQTT/5.0" protocols are used to define messages that are sent over MQTT 3.1.1 or MQTT 5.0 connections. The format describes the MQTT PUBLISH packet content.

The protocoloptions object contains the elements of the MQTT PUBLISH packet directly, with the user-properties element corresponding to the application properties collection of other protocols.

The following properties are defined. The MQTT 3.1.1 and MQTT 5.0 columns indicate whether the property is supported for the respective MQTT version.

Property Type MQTT 3.1.1 MQTT 5.0 Description
qos integer yes yes Quality of Service level
retain boolean yes yes Retain flag
topic_name string yes yes Topic name
payload_format integer no yes Payload format indicator
message_expiry_interval integer no yes Message expiry interval
response_topic uritemplate no yes Response topic
correlation_data binary no yes Correlation data
content_type symbol no yes MIME content type of the payload
user_properties Array no yes User properties

Like HTTP, the MQTT allows for multiple user properties with the same name, so the user_properties property is an array of objects, each of which contains a single property name and value.

The values of all string, symbol, and uritemplate-typed properties and user properties MAY contain placeholders using the RFC6570 Level 1 URI Template syntax. When the same placeholder is used in multiple properties, the value of the placeholder is assumed to be identical.

The following example shows a message with the "MQTT/5.0" protocol, asking for QoS 1 delivery, with a topic name of "mytopic", and a user property of "my-application-property" with a value of "my-application-property-value":

{
  "protocol": "MQTT/5.0",
  "protocoloptions": {
    "qos": 1,
    "retain":  false,
    "topic_name": "mytopic",
    "user_properties": [
      {
        "name": "My Application Property",
        "value": "Value 1"
      }
    ]
  }
}
"KAFKA" protocol đź”—

The "KAFKA" protocol is used to define messages that are sent using the Apache Kafka RPC protocol.

The protocoloptions object contains the common elements of the Kafka producer and consumer records, with the headers element corresponding to the application properties collection of other protocols.

The following properties are defined:

Property Type Description
topic string The topic the record will be appended to
partition integer The partition to which the record is to be sent or has been received from
key string The key that is associated with the record, UTF-8 encoded
key_base64 binary The key that is associated with the record as a base64 encoded string
headers Map A map of headers to set on the record

The key and key_base64 properties are mutually exclusive and MUST NOT be present at the same time.

The partition property is included because there are cases where applications use partitions explicitly for addressing and routing messages within the scope of a topic.

The values of all string, symbol, uritemplate-typed properties and headers MAY contain placeholders using the RFC6570 Level 1 URI Template syntax. When the same placeholder is used in multiple properties, the value of the placeholder is assumed to be identical.

Example:

{
  "protocol": "Kafka",
  "protocoloptions": {
    "topic": "mytopic",
    "key": "thisdevice"
  }
}
"NATS" protocol đź”—

The "NATS" protocol is used to define messages that are sent using the NATS protocol.

The protocoloptions object contains the available elements of the NATS message for the HPUB operation.

The following properties are defined:

Property Type Description
subject uritemplate The subject the message will be published to
reply-to uritemplate The subject the receiver ought to reply to
headers Array A list of headers to set on the message

The values of all string, symbol, uritemplate-typed properties and headers MAY contain placeholders using the RFC6570 Level 1 URI Template syntax. When the same placeholder is used in multiple properties, the value of the placeholder is assumed to be identical.

Example:

{
  "protocol": "NATS",
  "protocoloptions": {
    "subject": "mytopic",
    "reply-to": "replytopic"
  }
}