First of all, this document is far from finished ...
This document describes in detail what version 1.0.0 of Orion-LD implements of the NGSI-LD API specification, v1.5.1, released in November 2021. The origins of Orion-LD is explained as well, particularly how Orion-LD relates to Orion. To top it off, future implementation plans are uncovered, making this document the number one document on the Orion-LD context broker (together with the ETSI NGSI-LD v1.5.1 API spec, of course :)).
To fully understand this document, a reader is assumed to have a working knowledge of NGSI-LD, ideally having read the ETSI NGSI-LD API specification.
- Origins of Orion-LD and its relation to Orion
- Installation and Deployment
- Support of OMA NGSI and NGSIv2 requests
- NGSI-LD Features
- NGSI-LD Requests
- Extra Requests
- Performance Measurements
- Future
The implementation of Orion, the OMA NGSI context broker of the FIWARE European project was initiated by Telefonica I+D back in July 2012. The first year or so, it was just me (Ken Zangelin, now at the FIWARE Foundation) on the project, but then Fermin Galan (also Telefonica I+D) joined the project. Fermin is now working for Telefonica Spain and he's in charge of Orion.
Orion implements the OMA NGSI APIs NGSI9 and NGSI10, and the newer NGSIv2 that was designed by Telefonica I+D.
The work on Orion-LD started in September 2018, by the FIWARE Foundation and as we wanted a broker able to handle both "the old and the new", the Orion repo was forked and that's where the work on adding NGSI-LD to Orion started.
The Orion-LD repository is still a fork of the Orion repository and the idea is to some day merge with Orion. That though is a huge job and we don't have an ETA for this.
To accomplish interworking NGSI-LD/NGSI, Orion's database model is used for NGSi-LD elements. The very same database collections, reusing lots of the old NGSI/NGSIv2 code. Orion's database model was created for OMA NGSI, and when NGSIv2 was implemented, for the same backwards compatibility reason, the database model was not modified. It makes a lot of sense, from a backwards compaibility point of view, but ... the database model differs horrendously from the API data model (except for OMA NGSI, for which the database model was designed), and this makes the broker a lot harder to implement and quite a bit slower as well, as the incoming NGSI-LD payloads must be converted to the OMA NGSI model before pushed to the database.
To deploy Orion-LD 1.0.0 as a docker image ... (Stefan!)
For you brave that prefer to build your own Orion-LD from source code, please follow the installation guide for your preferred distro:
NOTE: To build the v1.0.0 release, instead of using the "develop" branch, use "release/1.0.0"
Since the day Orion repo was forked (September 2018), there have been zero merges Orion -> Orion-LD, so, the OMA NGSI/NGSIv2 part of Orion-LD is only capable of doing exactly what Orion was capable of back in September 2018. A lot has happened for NGSIv2 in the Orion repo since September 2018, thus, using Orion-LD for NGSIv2 is strongly discouraged.
This chapter describes all the features of NGSI-LD and what is implemented or not. Instead of treating each and every NNGSI-LD request separately, they are seen as a whole, as concepts.
Two important concepts that are not yet implemented in Orion-LD are:
- DatasetID
- Forwarding
Forwarding hasn't even been defined in the spec, we're still discussing it in ETSI ISG CIM. Can't really be implemented until the API is defined.
DatasetID is a concept that existed in OMA NGSI, as "MetadataID". We implemented that for Orion, NGSIv1, as it was part of the OMA NGSI specification. Nobody ever used the feature, afaik, and it is quite complex. When Telefonica I+D defined NGSIv2, to replace OMA NGSI, the whole concept was left out. For Orion-LD, implementing NGSI-LD, where the datasetId concept is back ... well, let's just say it's not a top priority. It will be implemented, we're just not putting it very high on the backlog of Orion-LD.
Anyway, a Proof-Of-Concept has been implemented for both forwarding and DatasetID. In general, these two concepts aren't supported by Orion-LD as of right now (v1.0.0). Details of the PoC's are found below.
All requests regarding Entities/Attributes are fully implemented (remember, except for DatasetID and Forwarding). Entity creation/updates/deletions are correctly stored in both the "current state" and the "temporal evolution DB" (if troe (Temporal Representation of Entities) is turned on), the appropriate notifications are sent, etc.
NGSI-LD's new and improved Query language is implemented to about 99%. One simplification has been made: AND and OR aren't allowed to mix in the same level - parenthesis needed.
Example: a) q=A==12&B==10|C==8 is NOT ALLOWED b) q=(A==12&B==10)|C==8 is ALLOWED
It's a minor limitation, you can still do ALL OF IT, you just need to add a few parenthesis. It actually has a quite nice advantage as well - no confusion about operator precedence. Just look at the two examples above. Had expression "a" been allowed, would it be identical to expression "b" ?
All "BATCH" operations (/entityOperations/*) are fully implemented. As always, with the exception of datasetId and forwarding.
Batch operations carry an array of entities in its payload. If the very same entity is present more than once in this array, then we have a problem.
Imagine that one of the entity-instances set the attribute A to 15 and the other entity-instance sets A to 46. The chronological order is everything here. Should A be set to 15 or to 46?
To deal with this problem, a decision was made to treat the entity array as "lower array-index => older". This means that if in index 0 and 1 of the array, an entity E1 is created, then index 0 is treated as the creation and index 1 is provoking an error.
With updates, a similar thing happens. First the update of index 0 is treated, then the update of indes 1 may overwrite the previous update.
For the temporal evolution of entities, each and every instance of an entity in the entity array are pushed to the TRoE database, "in order".
As modifiedAt and created at are built-in properties and set by the broker, all instances will have the same timestamps. This is why the use of "observedAt" is so important - only way to really know the timing of these requests.
Creation and updates of entities provoke notifications to be sent for all matching subnscriptions.
- TimeInterval not supported
- Subscription Query Language - still using the old one from NGSIv2
Forwarding in NGSI-LD is still under discussion in the ETSI ISG CIM group where we define the NGSI-LD API. Forwarding will be included in the next release of the NGSI-LD API specification (v1.6.1, planned for summer 2022).
Before this is included in the specification, it can't really be implemented, so ... hopefully ready at the end of 2022.
That said, we did implement a PoC for forwarding, for a limited set of requests:
- GET /entities/{entityId}
- PATCH /entities/{entityId}/attrs/{attrName}
So, those two requests actually support forwarding, just, not exactly as it will be defined in the NGSI-LD API specification.
The NGSI-LD API specification will introduce a number of new fields for Context Source Registration, enhancing the registrations to support forwarding. Especially important is a field (currently) called "peerType", accepting three values:
- exclusive
- inclusive
- auxiliary
An exclusive registration means that the registree assumes full ownership of what it registers. For example, if an entity with ID "E1" is registered with attributes "temperature" and "pressure", then those two attributes of E1 are exclusive to the registree. If another registree try to register the same attributes (any of the two) for E1, then this second registration is discarded by the broker. This peer type (or registration mode) is typically used for registering IoT Agents in a broker.
An inclusive registration is meant for federation, won't go into any details here. I did go into details about exclusive registrations because, the PoC implements exclusive registrations only. There's no field for it, Orion-LD just assumes that all registrations are exclusive. Remember, Orion-LD doesn't implement federation just yet.
Now, with this PoC, you can register a different attribute for a number of IoT agents (or brokers, or whatever), on the same entity and when retrieving that entity from the broker where the registrations were made, the broker will send forwarded requests to all the IoT agents (or whatever) and then merge all the attributes into the entity and respond with that. Part of the entity can also reside in the broker itself. But, as the registrations are exclusive, no attribute can be local+registered, not registered twice. Is an attempt to do so is made, an error will occur.
Some HTTP headers are inherited from the initiating request:
- X-Auth-Token
- Authorize
- NGSILD-Tenant - not yet implemented for forwarded requests
For federation to "be a thing" in NGSI-LD, it needs to be specified in the NGSI-LD API specification, which is work in progress as of December 2021 That said, NEC's Scorpio NGSI-LD context broker implements a proprietary federation, supporting "inclusive" registrations and only on GET requests, plus, distribution of subscriptions, for correct notifications.
Orion-LD has its Proof Of Concept for "federating" with exclusive IoT-Agents - also a preprietary implementation.
Both Orion-LD's and Scorpio's approach to federation will be modified as soon as Forwarding is fully defined (and published) in the NGSI-LD API specification
- Details on each and every request
- Including URL parameters
Bla bla bla
This request is not implemented in Orion-LD. Orion-LD populates the TRoE (Temporal Representation of Entities). but t he query of the TRoE database is implemented by a different enabler - mintaka. Please refer to the documentation of mintaka
This request is also implemented by mintaka.
NOT IMPLEMENTED
Like GET /ngsi-ld/v1/temporal/entities, implemented by mintaka.
NOT IMPLEMENTED
Fully Implemented
Fully Implemented
Fully Implemented
Fully Implemented
Fully Implemented
NOT IMPLEMENTED
NOT IMPLEMENTED
NOT IMPLEMENTED
A total of FIVE requests have been implemented for testing and debugging purposes. These requests are not part of the NGSI-LD API and thus has a different URL prefix. Instead of "/ngsi-ld/v1", "/ngsi-ld/ex/v1" is used:
- GET /ngsi-ld/ex/v1/ping
- GET /ngsi-ld/ex/v1/version
- GET /ngsi-ld/ex/v1/tenants
- GET /ngsi-ld/ex/v1/dbIndexes
- POST /ngsi-ld/ex/v1/notify
Stefan!