CBOR & Qt

A Look at Handling CBOR Data in Qt 5.12

By Leon Lin

Concise Binary Object Representation (CBOR) format is a compact form of binary encoding for data serialization. It is similar to JSON in that most data is represented by name-value pairs, and its design emphasizes compactness for better processing and transmission speeds. The format was created by the Internet Engineering Task Force’s (IETF) Constrained RESTful Environments (CoRE) working group, and is intended to be used with the CoAP protocol to support the Internet of Things.

Qt has added support for handling CBOR data in 5.12. This includes classes to encapsulate CBOR data, as well as classes to handle encoding and decoding CBOR data. By using these classes, you can save the trouble of having to write your own CBOR data encoders and decoders.

CBOR Data Types

CBOR supports three major groups of types:

  • Basic types (integers, doubles, floats, booleans, etc.)

  • Strings and byte arrays

  • Containers (arrays and maps)

Additional types can be defined by using tags. A number of additional supported types have been established with tags, including dates, UUIDs, and URLs.

There are also the simple types of null and undefined. Null refers to an optional value (like std::optional) that is not provided. Undefined values refer to values that are not provided because of an error or other problem. Boolean true and false are also simple types in CBOR.

The exact way in which this data should be encoded in CBOR format is described in RFC 7049 at https://tools.ietf.org/html/rfc7049.

Qt Support for CBOR

The main supporting classes are:

  • QCborValue - encapsulates a CBOR value
  • QCborStreamReader - reads CBOR data from a QByteArray or QIODevice
  • QCborStreamWriter - writes CBOR data to a QByteArray or QIODevice

QCborValue

QCborValue is a class that can encapsulate all types of data supported by CBOR. You can construct a QCborValue from a data stream or from many types of data, including integers, doubles, strings, and even more complex formats such as date/time values and URLs. QCborValues may also be constructed from JSON values without loss of information.

Once you have a QCborValue, you can do a number of useful things with them, such as conversions to CBOR or JSON format for transmission or comparisons with other QCborValues.

QCborStreamReader

This class allows the user to read CBOR data, either from a QByteArray or a QIODevice. Its API is similar to StAX and the class can be used in a similar fashion as QXmlStreamReader.

QCborStreamWriter

This class allows the user to write out CBOR data, either to a QByteArray or a QIODevice. This is done with append() functions that take all sorts of data types as input. You can also create arrays and maps by first calling startArray() and startMap(), then following with append() calls to encode the contents of the map. When finished, call endArray() and endMap().

Here's an example, a short application that demonstrates the use of both QCborStreamReader and QCborStreamWriter.

The application allows the user to create a list of inventory items, with each item having attributes such as a name, price per unit, and UUID. The user can generate a CBOR data for the items to be saved in a file. The same file then can be later read by the application to reconstruct the list.

CBOR example

Internally, an inventory item is represented by the class InventoryItem. The list of inventory items is represented by a QAbstractListModel with a QList of InventoryItems as its data store. In the UI user can add new items to the list by pressing the Add button and then modifying the attributes of the new item that appears and clicking the Apply button. The user may also delete existing items by selecting one and pressing the Delete button.

Writing Out CBOR Data

CBOR comes into play when the user selects Export CBOR File from the File menu. The user is prompted by a file dialog to supply a name and path. The application then writes CBOR data for the inventory list to the specified file.

To write the CBOR data, the application has a class called InventoryCborWriter. When saving CBOR data, the function writeCborFile is used. This function opens a file and then creates a QCborStreamWriter with that file:

  QFile file(filePath);
  bool ret = file.open(QIODevice::WriteOnly);
  if (!ret)
      return false;
  QCborStreamWriter writer(&file);

From there, using the QCborStreamWriter to write out CBOR data is extremely easy. The inventory list is represented by an array of inventory items, each of which is represented by a map of key-value pairs. To write out the array, QCborStreamWriter::startArray() is used to signal the start of the array:

writer.startArray();

The application then loops over all the inventory items, writing out CBOR data for each. To specify the start of a map, QCborStreamWriter::startMap() is called. Then, to actually write out the CBOR data, various overloaded versions of QCborStreamWriter::append() are used:

// Each inventory item is a map
for (InventoryItem item: items)
{
	writer.startMap();

	// Name
	writer.append("Name");
	writer.append(item.getName());

	// UUID -- append a tag before the byte array containing
	// the UUID data
	writer.append("ID");
	writer.append(QCborKnownTags::Uuid);
	writer.append(item.getID().toByteArray());

	// Price
	writer.append("Price");
	writer.append(item.getPrice());

	// Quantity
	writer.append("Quantity");
	writer.append(item.getQuantity());

	// Used -- boolean is a CBOR simple type
	writer.append("Used");
	if (item.getUsed())
		writer.append(QCborSimpleType::True);
	else
		writer.append(QCborSimpleType::False);
	writer.endMap();
}

As we are writing out a map, we write out key-value pairs. The method append() is first called with a QString representing the key, then another append() is called that takes a type appropriate for the value being written out. So we use append() calls that take a QString, a double, and an integer to write out CBOR data for the name, price, and quantity of the inventory item.

To write out a UUID, which is a kind of tagged data in CBOR, we call append() with the key name (“ID”) and then append() with a known tag, QCborKnownTags::Uuid. QCborKnownTags is an enum that represents all tags supported by the CBOR RFC, including tags for date/time, URLs, and regular expressions. This is followed by another append writing out the actual data associated with the tag -- in this case, a byte array from our inventory item’s QUuid attribute. Finally, we can write out booleans using the version of append that takes a CBOR simple type, in this case QCborSimple::True or False.

To complete each map, we call QCborStreamWriter::endMap(). Once all inventory items have been written out, we call QCborStreamWriter::endArray().

Reading CBOR Data

Reading an existing CBOR data file to construct an inventory item list is more complex, but made easier by Qt. We have a different class, InventoryCborReader, to handle this task. This class has several variables to help with the decoding of the CBOR data:

m_currentKey -  a QString that specifies the current map key being processed
m_currentItem - an InventoryItem that is the current one being read
QList<InventoryItem> - the list of InventoryItems that will be returned when we finish reading the CBOR file.

The process begins when the user selects Import CBOR File from the File menu. The user selects the CBOR file from a standard file dialog. The path of the selected file is then passed to InventoryCBorReader::readCborFile().

From there, some initialization is handled, including opening a QFile from the supplied file path. If that is successful, the QFile is passed into a QCborStreamReader.

bool InventoryCborReader::readCborFile(QString filePath)

{
  m_itemList.clear();
  m_currentKey = QString();
  QFile file(filePath);
  bool ret = file.open(QIODevice::ReadOnly);
  if (!ret)
      return false;

  QCborStreamReader reader(&file);
  ret = readCborData(&reader);
  file.close();
  return ret;
}

readCborData(), which takes the QCborStreamReader as an argument, is the heart of the code that decodes the CBOR data file. It has a while loop that keeps going as long as the reader finds more elements at the current level and doesn’t report an error:

while (!reader->lastError() && reader->hasNext())
{
	QCborStreamReader::Type type = reader->type();

	...

readCborData() gets the type of the next element in the stream and processes the element accordingly. If it’s an array, which we expect as our CBOR data is set up to have a single array, we call QCborStreamReader::enterContainer() to signal that the following elements are in a container. We then call readCborData() again to handle the elements in the array; when there are no more (read->hasNext() in the while loop returns false) this call to readCborData() will return and then we will leave the container with QCborStreamReader::leaveContainer(). (This is all assuming there are no errors; if one is caught decoding will stop and readCborData() will return false.)

We also call enterContainer() when entering a map. A new InventoryItem is created to hold the data from the map’s contents. As with arrays, readCborData() is called to read all the elements in the map, and on return, leaveContainer() is called.

Most elements are handled simply. Once their type is identified, we get the element’s value using the appropriate function in QCborStreamReader (toUnsignedInteger(), toInteger(), toDouble()) and pass the value to an appropriate handler function defined in InventoryCborReader so that the current InventoryItem can be updated. Each value is assigned based on the current tag. In this example, since each type we handle is associated with only a single tag, that’s not really necessary but the code allows for extensibility in the event a type is associated with more than one tag. Once we’re done handling the element, we call QCborStreamReader::next() to advance to the next element.

Some types of CBOR data need special handling. In particular, strings and byte arrays can’t simply be read once. For strings, QCborStreamReader::readString() needs to be called repeatedly until no more string chunks can be read:

while (r.status == QCborStreamReader::Ok)

  {

     str += r.data;

     r = reader->readString();

  }

We handle byte arrays in a similar fashion with QCborStreamReader::readByteArray():

while (r.status == QCborStreamReader::Ok)

  {

     uuidArray.append(r.data);

     r = reader->readByteArray();

  }

readString() and readByteArray(), when finished with reading a whole string or byte array, automatically advances the reader to the next element, so it is not necessary to call next() when it is finished.

Strings in our example are mainly used as keys for map values. m_currentKey keeps track of the current key and is assigned when a string is read if it is empty. When m_currentKey is not empty then when a new value is read the key is used to determine what in InventoryItem is assigned the new value. For example, when reading an unsigned integer:

void InventoryCborReader::assignUnsignedIntegerValue(quint64 value)
{
   if (m_currentKey == "Quantity")
	   m_currentItem.setQuantity(value);
   m_currentKey = QString();
}

When decoding of the CBOR data is finished, m_itemsList is populated with the list of inventory items. After confirming that the decoding was without error, InventoryListModel gets this list through InventoryCborReader::getItemList() and updates its internal model.

This example only scratches the surface of what can be done with CBOR and Qt. For more information on CBOR, check out the helpful links below. And you can read more on Qt-related topics here.

References

  1. IETF’s RFC for CBOR: https://tools.ietf.org/html/rfc7049

  2. Wikipedia article on CBOR: https://en.wikipedia.org/wiki/CBOR

  3. Qt Documentation for CBOR classes:  http://doc.qt.io/qt-5/qtcborcommon.html, http://doc.qt.io/qt-5/qcborvalue.html, http://doc.qt.io/qt-5/qcborstreamwriter.html, http://doc.qt.io/qt-5/qcborstreamreader.html