NGSI entity is missing attributes with manual device provisioning through Rest API

[walterwootz]

IoT Agent Node Lib version the issue has been seen with

3.4.4

Bound or port used (API interaction)

Northbound (Provision API and NGSI Interactions)

NGSI version

NGSIv2

Are you running a container?

No, I am running it natively

Image type

normal

Expected behaviour you didn't see

When calling POST http://localhost:4041/iot/devices a complete NGSI entity should be created, including active and lazy attributes.

Unexpected behaviour you saw

The created entity in Orion includes only lazy attributes and active ones are missing.

Steps to reproduce the problem

# 1. Provision service group

curl --location 'http://localhost:4041/iot/services' \
--header 'fiware-service: opcua_car' \
--header 'fiware-servicepath: /demo' \
--header 'Content-Type: application/json' \
--data '{
    "services": [
        {
            "apikey": "iot",
            "cbroker": "http://localhost:1026",
            "entity_type": "Device",
            "resource": "/iot/opcua"
        }
    ]
}'



# 2. Provision device

curl --location 'http://localhost:4041/iot/devices' \
--header 'fiware-service: opcua_car' \
--header 'fiware-servicepath: /demo' \
--header 'Content-Type: application/json' \
--data '{"devices": [{
  "device_id": "age09_Car",
  "entity_name": "age09_Car",
  "entity_type": "Device",
  "apikey": "iot",
  "service": "opcua_car",
  "subservice": "/demo",
  "attributes": [
    {
      "name": "EngineBrake",
      "type": "Number"
    },
    {
      "name": "Acceleration",
      "type": "Number"
    },
    {
      "name": "EngineStopped",
      "type": "Boolean"
    },
    {
      "name": "Engine_Temperature",
      "type": "Number"
    },
    {
      "name": "Engine_Oxigen",
      "type": "Number"
    }
  ],
  "lazy": [
    {
      "name": "Speed",
      "type": "Number"
    }
  ],
  "commands": [
    {
      "name": "Error",
      "type": "command"
    },
    {
      "name": "Stop",
      "type": "command"
    },
    {
      "name": "Accelerate",
      "type": "command"
    }
  ],
  "contexts": [
    {
      "id": "age09_Car",
      "type": "Device",
      "mappings": [
        {
          "ocb_id": "Events",
          "opcua_id": "ns=3;s=Events",
          "object_id": "ns=3;s=Events",
          "inputArguments": []
        },
        {
          "ocb_id": "EngineBrake",
          "opcua_id": "ns=3;s=EngineBrake",
          "object_id": "ns=3;s=EngineBrake",
          "inputArguments": []
        },
        {
          "ocb_id": "Acceleration",
          "opcua_id": "ns=3;s=Acceleration",
          "object_id": "ns=3;s=Acceleration",
          "inputArguments": []
        },
        {
          "ocb_id": "EngineStopped",
          "opcua_id": "ns=3;s=EngineStopped",
          "object_id": "ns=3;s=EngineStopped",
          "inputArguments": []
        },
        {
          "ocb_id": "Engine_Temperature",
          "opcua_id": "ns=3;s=Temperature",
          "object_id": "ns=3;s=Temperature",
          "inputArguments": []
        },
        {
          "ocb_id": "Engine_Oxigen",
          "opcua_id": "ns=3;s=Oxigen",
          "object_id": "ns=3;s=Oxigen",
          "inputArguments": []
        }
      ]
    }
  ],
  "contextSubscriptions": [
    {
      "id": "age09_Car",
      "type": "Device",
      "mappings": [
        {
          "ocb_id": "Error",
          "opcua_id": "ns=3;s=Error",
          "object_id": "ns=3;i=1000",
          "inputArguments": [
            {
              "dataType": 12,
              "type": "Error Type"
            }
          ]
        },
        {
          "ocb_id": "Speed",
          "opcua_id": "ns=3;s=Speed",
          "object_id": "ns=3;i=1000",
          "inputArguments": []
        },
        {
          "ocb_id": "Stop",
          "opcua_id": "ns=3;s=Stop",
          "object_id": "ns=3;i=1000",
          "inputArguments": []
        },
        {
          "ocb_id": "Accelerate",
          "opcua_id": "ns=3;s=Accelerate",
          "object_id": "ns=3;i=1000",
          "inputArguments": [
            {
              "dataType": 6,
              "type": "Intensity"
            }
          ]
        }
      ]
    }
  ],
  "endpoint": "opc.tcp://localhost:5001/UA/CarServer"
}
]}
'



# 3. Get entity from Orion
curl --location 'http://localhost:1026/v2/entities' \
--header 'Accept: application/json' \
--header 'Fiware-Service: opcua_car' \
--header 'Fiware-ServicePath: /demo'

Orion response:

[
    {
        "id": "age09_Car",
        "type": "Device",
        "Speed": {
            "type": "Number",
            "value": 0,
            "metadata": {}
        }
    }
]

Active attributes are missing.

Configs

{
    relaxTemplateValidation: true,
    logLevel: 'DEBUG',
    timestamp: true,
    contextBroker: {
        host: 'localhost',
        port: '1026',
        ngsiVersion: 'v2',
        jsonLdContext: 'https://uri.etsi.org/ngsi-ld/v1/ngsi-ld-core-context.jsonld',
        service: 'opcua_car',
        subservice: '/demo'
    },
    server: {
        port: 4041
    },
    deviceRegistry: {
        type: 'mongodb'
    },
    mongodb: {
        host: 'localhost',
        port: '27017',
        db: 'iotagent_opcua'
    },
    types: {
        Device: {
            active: [
                {
                    name: 'Events',
                    type: 'Text'
                },
                {
                    name: 'EngineBrake',
                    type: 'Number'
                },
                {
                    name: 'Acceleration',
                    type: 'Number'
                },
                {
                    name: 'EngineStopped',
                    type: 'Boolean'
                },
                {
                    name: 'Engine_Temperature',
                    type: 'Number'
                },
                {
                    name: 'Engine_Oxigen',
                    type: 'Number'
                }
            ],
            lazy: [
                {
                    name: 'Speed',
                    type: 'Number'
                }
            ],
            commands: [
                {
                    name: 'Error',
                    type: 'command'
                },
                {
                    name: 'Stop',
                    type: 'command'
                },
                {
                    name: 'Accelerate',
                    type: 'command'
                }
            ]
        }
    },
    contexts: [
        {
            id: 'age01_Car',
            type: 'Device',
            mappings: [
                {
                    ocb_id: 'Events',
                    opcua_id: 'ns=3;s=Events',
                    object_id: 'ns=3;s=Events',
                    inputArguments: []
                },
                {
                    ocb_id: 'EngineBrake',
                    opcua_id: 'ns=3;s=EngineBrake',
                    object_id: 'ns=3;s=EngineBrake',
                    inputArguments: []
                },
                {
                    ocb_id: 'Acceleration',
                    opcua_id: 'ns=3;s=Acceleration',
                    object_id: 'ns=3;s=Acceleration',
                    inputArguments: []
                },
                {
                    ocb_id: 'EngineStopped',
                    opcua_id: 'ns=3;s=EngineStopped',
                    object_id: 'ns=3;s=EngineStopped',
                    inputArguments: []
                },
                {
                    ocb_id: 'Engine_Temperature',
                    opcua_id: 'ns=3;s=Temperature',
                    object_id: 'ns=3;s=Temperature',
                    inputArguments: []
                },
                {
                    ocb_id: 'Engine_Oxigen',
                    opcua_id: 'ns=3;s=Oxigen',
                    object_id: 'ns=3;s=Oxigen',
                    inputArguments: []
                }
            ]
        }
    ],
    contextSubscriptions: [
        {
            id: 'age01_Car',
            type: 'Device',
            mappings: [
                {
                    ocb_id: 'Error',
                    opcua_id: 'ns=3;s=Error',
                    object_id: 'ns=3;i=1000',
                    inputArguments: [
                        {
                            dataType: 12,
                            type: 'Error Type'
                        }
                    ]
                },
                {
                    ocb_id: 'Speed',
                    opcua_id: 'ns=3;s=Speed',
                    object_id: 'ns=3;i=1000',
                    inputArguments: []
                },
                {
                    ocb_id: 'Stop',
                    opcua_id: 'ns=3;s=Stop',
                    object_id: 'ns=3;i=1000',
                    inputArguments: []
                },
                {
                    ocb_id: 'Accelerate',
                    opcua_id: 'ns=3;s=Accelerate',
                    object_id: 'ns=3;i=1000',
                    inputArguments: [
                        {
                            dataType: 6,
                            type: 'Intensity'
                        }
                    ]
                }
            ]
        }
    ],
    events: [
        {
            ocb_id: 'Events',
            opcua_id: 'ns=3;s=Events',
            object_id: 'ns=3;s=Events',
            fields: [
                {
                    name: 'EventId',
                    type: 'ByteString'
                },
                {
                    name: 'EventType',
                    type: 'NodeId'
                },
                {
                    name: 'SourceNode',
                    type: 'NodeId'
                },
                {
                    name: 'SourceName',
                    type: 'String'
                },
                {
                    name: 'Time',
                    type: 'DateTime'
                },
                {
                    name: 'ReceiveTime',
                    type: 'DateTime'
                },
                {
                    name: 'Message',
                    type: 'LocalizedText'
                },
                {
                    name: 'Severity',
                    type: 'UInt16'
                }
            ]
        }
    ],
    service: 'opcua_car',
    subservice: '/demo',
    providerUrl: 'http://host.docker.internal:4041',
    deviceRegistrationDuration: 'P20Y',
    defaultType: 'Device',
    defaultResource: '/iot/opcua',
    explicitAttrs: false,
    extendedForbiddenCharacters: []
}

Log output

No specific error logs

[AlvaroVega]

Since https://github.com/telefonicaid/iotagent-node-lib/releases/tag/3.4.0 initial entity is not created, so the behavior your describe is right.

[walterwootz]

So what do you suggest for iot agent users? Is it better to let them create the entities manually in orion, or can we automate the creation in some way?

[fgalan]

According to documentation in PR #1551 (soon to be merged)

This means that all entities into the Context Broker are created when data arrives from a device, no matter if the device is explicitly provisioned (via device provisioning API) or autoprovisioned

Does that suffice your use case? In negative case, please explain why don't.

[walterwootz]

Yes, the entity is created once a measure arrives from the device, but the created entity is missing active attributes. Only lazy ones are added.

[fgalan]

The attributes are added to the entity as they come in measures.

For instance:

• First measure for a given device with data for attribute A and B is received -> the entity is created with attributes A and B
• Second measure for the same device with data for attribute B and C is received -> in the enetity attribute A keeps same value, B is updated and C is added

If you need all the possible active attributes in the entity from the very beginning, please consider to pre-provision the entity at CB using the CB NGSIv2 API.

[walterwootz]

Ok thanks @fgalan we understand now.
But there is still a problem regarding our IoTAgent OPC UA: when trying to perform provisioning through API we have tried to add a custom attribute ("mappings", see below) in the body with info about the southbound connection, but it does not get propagated to the deviceProvisioningHandler function.

POST http://localhost:4041/iot/devices

{
    "devices": [
        {
            "device_id": "age01_Car",
            "entity_name": "age01_Car",
            "entity_type": "Device",
            "apikey": "iot",
            "endpoint": "opc.tcp://localhost:5001/UA/CarServer",
            "mappings": [
                {
                    "ocb_id": "Engine_Temperature",
                    "opcua_id": "ns=3;s=Temperature",
                    "object_id": "ns=3;s=Temperature",
                    "inputArguments": []
                }
            ],
            "attributes": [
                {
                    "object_id": "Engine_Temperature",
                    "name": "Engine_Temperature",
                    "type": "Number"
                }
            ],
            "lazy": [],
            "commands": []
        }
    ]
}

[AlvaroVega]

Is there any way to create a tests (Proof of Concept) for iotagent-node-lib which reproduce the behavior that you report @walterwootz ? The usage of iotagent-node-lib by iotagent-json or iotagent-opcua should be equivalent, and that wrong behavior you are reporting is not observed in iotagent-json.

[walterwootz]

Ok thanks @fgalan we understand now. But there is still a problem regarding our IoTAgent OPC UA: when trying to perform provisioning through API we have tried to add a custom attribute ("mappings", see below) in the body with info about the southbound connection, but it does not get propagated to the deviceProvisioningHandler function.

POST http://localhost:4041/iot/devices

{
    "devices": [
        {
            "device_id": "age01_Car",
            "entity_name": "age01_Car",
            "entity_type": "Device",
            "apikey": "iot",
            "endpoint": "opc.tcp://localhost:5001/UA/CarServer",
            "mappings": [
                {
                    "ocb_id": "Engine_Temperature",
                    "opcua_id": "ns=3;s=Temperature",
                    "object_id": "ns=3;s=Temperature",
                    "inputArguments": []
                }
            ],
            "attributes": [
                {
                    "object_id": "Engine_Temperature",
                    "name": "Engine_Temperature",
                    "type": "Number"
                }
            ],
            "lazy": [],
            "commands": []
        }
    ]
}

This is the test, the "mappings" array is not forwarded to the device provisioning handler function (see the function here, line 72).

[AlvaroVega]

It seems that mappings is something not handled by iotagent-node-lib and maybe is only just related with opcua iotagent.

[walterwootz]

Resolved using internal_attributes in the provisioning body request. In that field you can pass any additional data. Thank you