Dynamic Structured Data

Dynamic Structured Data (DSD) is a system for describing, serializing and deserializing messages in loosely coupled systems. Its (de)serialization facilities may be used in the same way as JSON, XML or even Protocol Buffers. Its message definition facilities may be used with its own text and binary serialization schemes, or they may be used with JSON or XML. DSD has been used successfully with "large" systems including modern 64-bit servers as well as "tiny" systems like 8-bit "Internet of Things" sensors and "mid-range" 32-bit embedded or mobile devices.

Features which differentiate DSD from JSON, XML or ProtoBufs include:

Though not differentiating features, the designers of DSD found these features important:

What's in DSD?

The Dynamic Structured Data specification includes:

Putting it All Together

The "DSD Workflow" is expected to be something like this:

  1. An application developer uses the Message Definition Facility (MDF) or Application Semantic Description Facility (ASDF) to describe the composition and message exchange pattern of application messages.
  2. MDF/ASDF message descriptions are (optionally) used by a software tool to create a parser to parse application messages. Even if automated tools are not used to create custom parser instances, MDF and ASDF are useful mechanisms for succinctly communicating message exchange patterns and the composition of protocol messages between humans.
  3. A transport for the message protocol is selected. Once the transport is selected, it should be straight-forward to create or integrate a DSD message parser into your application based on the Message Transport Specification (MTS).
  4. Depending on your platform and preferences, you may wish to select a restrictive policy for messages your application receives that do not conform to message definitions. A restrictive policy would be to reject non-conforming messages. This will reduce the load on the application layer; your app will simply never see non-conforming messages. A non-restrictive policy may be better for loosely coupled systems where version skew may introduce non-conformant messages. The choice is entirely up to the application developer; DSD models sufficient information to allow restrictive policies to be enforced, but these policies can be relaxed based on developer preference.
  5. Once the transport is selected and the non-conformant message policy is selected, an application should be able to start receiving messages. We assume the receiver knows what serializtion format is used either by looking at the mime type of a HTTP(S) request or by developer fiat. When a message is received and the application knows which serialization scheme to use, it can begin to parse the message, converting it into a suitable internal format from the line format defined by the transfer syntax.
  6. Once the message is parsed and converted into an internal format, it is passed to the application which will then take appropriate action, possibly sending a response.

Example DSD Messages

Examples often illustrate concepts succinctly in a way descriptive text cannot. The author(s) of this document felt it important to include a few simple examples of DSD messages. A more extensive example DSD/Text message may be found at example.dsd.

Example 1 : A DSD/Text serialized message

   # Simple message directing a robot to go to a particular location and
   # submit a report the URL given after it gets there.
     "fast" = True
     "command" = "move"
     "destination" = [ -12.8 101.4 ]
     "report" = "https://example.com/c/c422a2ba-46f6-9a0a-2531e95e345b"
     "timeout" = 60 # normally this is 90 seconds

In this example, we see a few things:

Example 2 : A DSD/Text message from a tiny microcontroller

   [ 1 32 17 ]
   [ 1 32 19 ]
   [ 1 31 19 ]
   [ 4096 "fault : restarting" ]
   [ 1 31 19 ]

This is a very simple message you might expect to come from a temperature / humidity sensor. It might be a sequence of temp / humidity measurements 60 seconds apart interspersed with alert messages. There are two important features to observe from this example:

Example 3 : A DSD/Binary message with the same data as Example 2

   0x10 0x00
   0x07 0x09
        0x03 0x01 0x01
        0x03 0x01 0x20
        0x03 0x01 0x11
   0x07 0x09
        0x03 0x01 0x01
        0x03 0x01 0x20
        0x03 0x01 0x13
   0x07 0x09
        0x03 0x01 0x01
        0x03 0x01 0x1F
        0x03 0x01 0x13
   0x07 0x15
        0x03 0x02 0x00 0x10
        0x05 0x0F 0x66 0x61
        0x75 0x6C 0x74 0x20 
        0x3A 0x20 0x72 0x65
        0x73 0x74 0x69 0x6E
   0x07 0x09
        0x03 0x01 0x01
        0x03 0x01 0x20
        0x03 0x01 0x13

From this we can see:

Example 4 : A DSD message serialized as JSON replicating the contents of Example 1

     "_type": "m",
     "fast": true,
     "command": "move",
     "destination": [ -12.8, 101.4 ],
     "report": "https://example.com/c/c422a2ba-fec1-9a0a-2531e95e345b",
     "timeout": 60

From this example, we can see:

Example 5 : A DSD message serialized as a well-formed XML message

   <!-- Simple message directing a robot to go to a particular          -->
   <!-- location and submit a report the URL given after it gets there. -->
       <integer>60</integer> <!-- normally this is 90 seconds -->

Some important points to note:

Example 6 : A DSD Message Definition Specification for the message in Example 1

   # This is an example message definition
   MESSAGE example_message_name
       BOOLEAN fast
       STRING command
       ARRAY destination [ INTEGER ... ]
       STRING report
       INTEGER timeout


  1. VWRAP Abstract Type System XML Serialization DTD