Internet-Draft SenML CORECONF October 2024
Gudi, et al. Expires 10 April 2025 [Page]
Workgroup:
t2t Research Group
Internet-Draft:
draft-gudi-t2trg-senml-as-coreconf-00
Published:
Intended Status:
Standards Track
Expires:
Authors:
M. Gudi
Institut MINES TELECOM; IMT Atlantique
L. Toutain
Institut MINES TELECOM; IMT Atlantique
Alejandro Fernadez
Institut MINES TELECOM; IMT Atlantique
Jean-Marie BONNIN
Institut MINES TELECOM; IMT Atlantique

SenML is CORECONF (almost)

Abstract

SenML is one of the data formats used by the Internet-of-Things (IoT) devices to send simple sensor readings and device parameters over the network. However, a lack of a YANG model for SenML means it cannot be used by the applications which already use YANG for data modeling and validation. Furthermore, some of the encoding formats and tools available for YANG models, cannot be used by the devices sending data in SenML format. This document provides one of the ways to model SenML data in YANG. Additionally, SenML data is encoded into CORECONF format using this YANG model to concisely represent the data.

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."

This Internet-Draft will expire on 10 April 2025.

Table of Contents

1. Introduction

In its simplest form, an IoT device consists of at least one sensor, and ability to send measurements of this sensor over the network. Occasionally, the device parameters can also be sent over the network to monitor and manipulate its behavior. Such devices are constrained on energy, network (in its availability as well as bandwidth), and data processing capabilities as they embed low power processors. Consequently, SenML is an appropriate choice for representing this nature of data from these devices.

As much as SenML is useful in building simple IoT applications, a well-defined data model for the same would allow developers and engineers alike to build more complex data systems if the data can be modeled in YANG. Subsequently, the YANG model of SenML can leverage SID based CORECONF representation of its data to further reduce network footprint and improve its interoperability with other network devices.

2. SenML Format

SenML or Sensor Measurement Lists is an data format used by the constrained devices to send sensor information over the networks [RFC8428]. These measurements are often structured as key-value pairs where the keys (also known as fields) describe the associated sensor data. Each field has a well defined label, whether it is mandatory to be included and the permitted values it can carry. SenML also reduces sending redundant information over the network by introducing concepts such as base-name, base-unit, base-version, base-time and base-value.

The format specifies how the application payload can be serialized in three popular formats before sending it over the network- JSON (JavaScript Object Notation) [RFC8259], CBOR (Concise Binary Object Representation)[RFC8949] and XML (Extensible Markup Language).

Although a SenML record has a well-defined Concise Data Definition Language (CDDL) for JSON and CBOR representations in section 11 of [RFC8428], the lack of an accompanying data model (specifically in YANG), means it is harder to use it for applications with strict requirement for data organization and validation. Additionally, SenML CDDL cannot be used directly to extend to other data formats such as CoAP Management Interface (CORECONF)[I-D.ietf-core-comi].

3. YANG Model Language

YANG or Yet Another Next Generation is data modeling language used to describe the organization and constraints on the configuration and state data of the network devices. This data is typically exchanged using NETCONF or RESTCONF protocols [RFC7950]. As YANG models are considered de-facto interchange formats for a particular protocol (or application) which allows network device manufacturers to build inter-operable devices. YANG has rich data types, language features, and supports several extensions for constructing user-defined data types, recursive data models and allowing inheritance to reuse existing data models [RFC6095] among others.

Logically, to model SenML format into a YANG model, the measurements can be designed as YANG lists and each SenML record is a grouping containing leaves of SenML fields and values [RFC9254]. Additional constraints and rules can be added to the model ensuring conformance with SenML specification. Visually it can be represented as follows:

Table 1
SenML Element YANG Equivalent
Measurement List
Field Leaf
Label Leaf Name
Value Type Leaf Type
Record Grouping/Container

Section 11 of [RFC8428] specifies how SenML format is described for encoding into JSON, XML and CBOR. However, IoT devices which use YANG data models don't have any easy way to incorporate and validate their SenML measurements. Additionally, in absence of a YANG model, low powered devices which send measurements in SenML, cannot use CORECONF representation model without resorting to some sort of intermediate data processing. Hence, if SenML has a YANG model, these low powered devices can describe their data entirely in it. Their data can further be transformed in CORECONF, which complies with SenML-CBOR encoding rules before transmitting it over the network.

To enable accurate, fast and efficient transmission of data conforming to YANG models, Schema Identifiers (known as SIDs) can be generated and assigned for each YANG element [I-D.ietf-core-sid] . These SIDs can be used to transform data in CORECONF format as demonstrated in [I-D.toutain-t2t-sid-extension]. An example of this is also described in Transforming SenML to CORECONF (Section 6) below.

4. YANG Model for SenML: Considerations

As described in [RFC8428], SenML measurements consists of field and a value, and each field is identified by its label (which are different for JSON and CBOR). Each value has a well-defined data-type for encoding in JSON, in CBOR or in XML as outlined in section 12.2 RFC 8428. Hence, to describe a generic SenML model in YANG, it is necessary to represent each field as a YANG leaf with the most appropriate YANG type associated with the type from the "XML Type" Column.

A user data type is created in the YANG model to make XML Integer like type-definition. XML Double type follows IEEE 754-2008 definition, which results in approximately 15 significant digits. However for the YANG model, XML Double is replaced by a Number type, which is a decimal64 type with 5 precision digits and has a range of [-92233720368547.75808, 92233720368547.75807]. This is chosen arbitrarily to balance precision and range in YANG but can be changed by the user later.

5. YANG Model for SenML: Implementation

<CODE BEGINS> file "senml.yang"

module senml {
  yang-version 1.1;
  namespace "urn:ietf:params:xml:ns:yang:senml";

  prefix sen;

  organization
    "IMT Atlantique";

  typedef Integer {
    type union{
      type int8;
      type int16;
      type int32;
      type int64;
    }
  }

  typedef Number {
    type decimal64{
      fraction-digits "5";
    }
  }

  grouping senml-grouping {
    leaf bn {
      type string;
    }

    leaf bt {
      type Number;
    }

    leaf bu {
      type string;
      must "(. = 'm' or . = 'kg' or . = 'g' or . = 's' or . = 'A' or . = 'K'
      or . = 'cd' or . = 'mol' or . = 'Hz' or . = 'rad' or . = 'sr' or . = 'N'
      or . = 'Pa' or . = 'J' or . = 'W' or . = 'C' or . = 'V' or . = 'F' or . = 'Ohm'
      or . = 'S' or . = 'Wb' or . = 'T' or . = 'H' or . = 'Cel' or . = 'lm' or . = 'lx'
      or . = 'Bq' or . = 'Gy' or . = 'Sv' or . = 'kat' or . = 'm2' or . = 'm3'
      or . = 'l' or . = 'm/s' or . = 'm/s2' or . = 'm3/s' or . = 'l/s' or . = 'cd/m2'
      or . = 'W/m2' or . = 'bit' or . = 'bit/s' or . = 'lat' or . = 'lon' or . = 'pH'
      or . = 'dB' or . = 'dBW' or . = 'Bspl' or . = 'count' or . = '/' or . = '%'
      or . = '%RH' or . = '%EL' or . = 'EL' or . = '1/s' or . = '1/min' or . = 'beat/min'
      or . = 'beats' or . = 'S/m' )" {
        description
          "The unit must be from the approved unit list";
        error-message
          "Invalid unit. Unit must be one from SenML Units Registry";
      }
    }

    leaf bv {
      type Number;
    }

    leaf bs {
      type Number;
    }

    leaf bver {
      type Integer;
    }

    leaf n {
      type string;
    }

    leaf u {
      type string;
    }

    choice valueleaf {
      description
        "Exactly one Value field MUST appear unless there is a
         Sum field, in which case it is allowed to have no Value field";
      leaf v {
        type Number;
      }

      leaf vs {
        type string;
      }

      leaf vb {
        type boolean;
      }

      leaf vd {
        type string;
      }
    }

    leaf s {
      type Number;
    }

    leaf t {
      type Number;
    }

    leaf ut {
      type Number;
    }
  }

  list e {
    key t;
    uses senml-grouping;
    min-elements 0;
    max-elements unbounded;
  }
}

<CODE ENDS>

The pyang tool can be used to visualize the YANG tree:

$ pyang -f tree senml.yang
module: senml
  +--rw e* [t]
     +--rw bn?         string
     +--rw bt?         decimal64
     +--rw bu?         string
     +--rw bv?         decimal64
     +--rw bs?         decimal64
     +--rw bver?       Integer
     +--rw n?          string
     +--rw u?          string
     +--rw (valueleaf)?
     |  +--:(v)
     |  |  +--rw v?    decimal64
     |  +--:(vs)
     |  |  +--rw vs?   string
     |  +--:(vb)
     |  |  +--rw vb?   boolean
     |  +--:(vd)
     |     +--rw vd?   string
     +--rw s?          decimal64
     +--rw t           decimal64
     +--rw ut?         decimal64

An example instance of JSON data instance for the above YANG model is presented below:

file "senml_example.json"
{
    "senml:e": [
        {
            "bn": "urn:dev:ow:10e2073a0108006:",
            "bt": 1220076.10,
            "n": "temperature",
            "u": "Cel",
            "v": 23.1,
            "t" : 1
        },
        {
            "n": "humidity",
            "u": "%RH",
            "v": 67.3,
            "t": 2
        }
    ]
}

It is possible to validate this YANG model against a SenML data in json encoding as shown below using yangson library:

<CODE BEGINS> file "validate_senml_yang.py"
import json
import yangson

from yangson import DataModel as DM
from yangson.enumerations import ContentType

# This function is required to convert all numerical values to strings due to yangson's strict typing
def convert_numbers_to_strings(d):
    for key, value in d.items():
        if isinstance(value, (int, float)):
            d[key] = str(float(value))
        elif isinstance(value, dict):
            convert_numbers_to_strings(value)  # Recursive call for nested dictionaries
        elif isinstance(value, list):
            for item in value:
                convert_numbers_to_strings(item)
    return d

# Create a data model from model definition in JSON format and path to YANG modules
dm = DM.from_file("./yang-library-senml.json", ["./"])

# Load the SenML data in JSON format
jsonData = json.load(open("senml_example.json"))

jsonData = convert_numbers_to_strings(jsonData)
rootInstanceNode = dm.from_raw(jsonData)

# Validate the root instance node
try:
    rootInstanceNode.validate(ctype=ContentType.all)
    print("Validation successful")
except:
    print("Validation failed")
<CODE ENDS>

6. Transforming SenML to CORECONF

Next, for the above YANG model, we can generate the SIDs using pyang tool starting from integer 60000, as follows:

$ pyang --sid-extension --sid-generate-file=60000:100 --sid-list senml.yang

SID        Assigned to
---------  --------------------------------------------------
60000      module senml
60001      data /senml:e
60002      data /senml:e/bn
60003      data /senml:e/bs
60004      data /senml:e/bt
60005      data /senml:e/bu
60006      data /senml:e/bv
60007      data /senml:e/bver
60008      data /senml:e/n
60009      data /senml:e/s
60010      data /senml:e/t
60011      data /senml:e/u
60012      data /senml:e/ut
60013      data /senml:e/v
60014      data /senml:e/vb
60015      data /senml:e/vd
60016      data /senml:e/vs

File senml@unknown.sid created
Number of SIDs available : 100
Number of SIDs used : 17

However, we can modify the SIDs manually so that the resultant CORECONF will have deltas conforming to SenML-CBOR Labels as defined in section 12.2 of [RFC8428]:

SID        Assigned to
---------  --------------------------------------------------
60010      module senml
60000      data /senml:e
59998      data /senml:e/bn
59994      data /senml:e/bs
59997      data /senml:e/bt
59996      data /senml:e/bu
59995      data /senml:e/bv
59999      data /senml:e/bver
60000      data /senml:e/n
60005      data /senml:e/s
60006      data /senml:e/t
60001      data /senml:e/u
60007      data /senml:e/ut
60002      data /senml:e/v
60004      data /senml:e/vb
60008      data /senml:e/vd
60003      data /senml:e/vs

SID allocation rule states that for every unique YANG identifier, there is a corresponding unique SID number[I-D.ietf-core-sid]. This rule is broken here to ensure the delta for "n" is computed to be 0 to match SenML-CBOR label definition. Hence, top-level key "e" and "n" have the same SID. However, "e" is not part of existing SenML label set as defined in the section 4.3 of [RFC8428].

The resulting CORECONF diagnostic format of the data instance is shown below:

{60000: [{-3: 1.001,
          -2: 'urn:dev:ow:10e2073a0108006:',
          0: 'temperature',
          1: 'Cel',
          2: 23.1,
          6: 1},
         {0: 'humidity',
          1: '%RH',
          2: 67.3,
          6: 2}]
}

Corresponding CBOR encoded hexadecimal is shown below:

CBOR Hex:

a119ea6082a621781b75726e3a6465763a6f773a3130653230373361303130383030363a22fb3ff004189374bc6a006b74656d7065726174757265016343656c02fb403719999999999a0601a4006868756d6964697479016325524802fb4050d333333333330602

If the module name contained in the first 4 bytes ("a119ea60") is removed, then the resultant CBOR conforms to SenML CBOR specification and can be easily parsed by SenML parsers.

Thus, a SenML CBOR representation can be almost CORECONF representation just by adding 4 bytes of the encoded module name.

Finally, a simple comparison of the JSON with the corresponding CORECONF-CBOR encoded form is shown below:

Table 2
4 byte module-name JSON Size CORECONF Size Compression %
Present 136 Bytes 104 Bytes 23.53
Absent 125 Bytes 100 Bytes 19.99

7. Further work

Some areas of future work have been identified to improve SenML YANG modelling-

  1. Base units can be either modelled as enumerated type or identityref type, so they can be assigned a SID value. This should ideally further reduce the CORECONF-CBOR encoded message.

  2. Constraints on having leaves v, vs, vb, vd optional if s is available as described in Section 4.2 of [RFC8428]

  3. Add secondary units described in [RFC8798] in the base units constraints.

  4. Uniqueness for SIDs can be restored by modifying SenML-CBOR label value for "n" to be any unused integer except 0.

8. Normative References

[I-D.ietf-core-comi]
Veillette, M., Van der Stok, P., Pelov, A., Bierman, A., and C. Bormann, "CoAP Management Interface (CORECONF)", Work in Progress, Internet-Draft, draft-ietf-core-comi-18, , <https://datatracker.ietf.org/doc/html/draft-ietf-core-comi-18>.
[I-D.ietf-core-sid]
Veillette, M., Pelov, A., Petrov, I., Bormann, C., and M. Richardson, "YANG Schema Item iDentifier (YANG SID)", Work in Progress, Internet-Draft, draft-ietf-core-sid-24, , <https://datatracker.ietf.org/doc/html/draft-ietf-core-sid-24>.
[I-D.toutain-t2t-sid-extension]
Toutain, L., G, M., and J. A. FERNANDEZ, "SID Extension to efficiently manipulate YANG Data Models", Work in Progress, Internet-Draft, draft-toutain-t2t-sid-extension-00, , <https://datatracker.ietf.org/doc/html/draft-toutain-t2t-sid-extension-00>.
[RFC6095]
Linowski, B., Ersue, M., and S. Kuryla, "Extending YANG with Language Abstractions", RFC 6095, DOI 10.17487/RFC6095, , <https://www.rfc-editor.org/rfc/rfc6095>.
[RFC7950]
Bjorklund, M., Ed., "The YANG 1.1 Data Modeling Language", RFC 7950, DOI 10.17487/RFC7950, , <https://www.rfc-editor.org/rfc/rfc7950>.
[RFC8259]
Bray, T., Ed., "The JavaScript Object Notation (JSON) Data Interchange Format", STD 90, RFC 8259, DOI 10.17487/RFC8259, , <https://www.rfc-editor.org/rfc/rfc8259>.
[RFC8428]
Jennings, C., Shelby, Z., Arkko, J., Keranen, A., and C. Bormann, "Sensor Measurement Lists (SenML)", RFC 8428, DOI 10.17487/RFC8428, , <https://www.rfc-editor.org/rfc/rfc8428>.
[RFC8798]
Bormann, C., "Additional Units for Sensor Measurement Lists (SenML)", RFC 8798, DOI 10.17487/RFC8798, , <https://www.rfc-editor.org/rfc/rfc8798>.
[RFC8949]
Bormann, C. and P. Hoffman, "Concise Binary Object Representation (CBOR)", STD 94, RFC 8949, DOI 10.17487/RFC8949, , <https://www.rfc-editor.org/rfc/rfc8949>.
[RFC9254]
Veillette, M., Ed., Petrov, I., Ed., Pelov, A., Bormann, C., and M. Richardson, "Encoding of Data Modeled with YANG in the Concise Binary Object Representation (CBOR)", RFC 9254, DOI 10.17487/RFC9254, , <https://www.rfc-editor.org/rfc/rfc9254>.

Authors' Addresses

Manoj Gudi
Institut MINES TELECOM; IMT Atlantique
2 rue de la Chataigneraie
CS 17607
35576 Cesson-Sevigne Cedex
France
Laurent Toutain
Institut MINES TELECOM; IMT Atlantique
2 rue de la Chataigneraie
CS 17607
35576 Cesson-Sevigne Cedex
France
Alejandro Fernadez
Institut MINES TELECOM; IMT Atlantique
2 rue de la Chataigneraie
CS 17607
35576 Cesson-Sevigne Cedex
France
Jean-Marie BONNIN
Institut MINES TELECOM; IMT Atlantique
2 rue de la Chataigneraie
CS 17607
35576 Cesson-Sevigne Cedex
France