GeoQueries seems to be slow

[Manifesto24]

First of all congratulations for this amazing project! I was recently started looking at the source code and looking forward to contribute!
So I came up to this issue :
I have about 1000 entities with the location attribute (GeoJson)
When doing a geospatial query:
/entities?georel=near;maxDistance==2000&geometry=point&coordinates=[-2.35, 40.78]
It seems to be very slow, and it takes 3 seconds to get a response.
Is this normal ?

[kzangeli]

Hello there, and welcome!

The geo-part of the queries isn't implemented by the broker itself, but by mongodb, which is the DB that Orion/Orion-LD use.
While 3 seconds for a query makes me sad, there's really nothing I can do about it.
It is a little surprising though, shouldn't be THAT slow.
I will try to look into this, starting by asking colleagues/competitors from NEC/EGM (they implement their own NGSI-LD brokers Scorpio/Stellio) that they run a similar test to see what numbers they get.

FYI: for geo-queries to even be able to use the "near" georel, the broker MUST create an index for the attribute (in this case the field is "attrs.location"), so, that is not the problem. Mongo would give an error if the index was missing.

Now, if you are interested in contributing (would be the first one in six years ...:)), I'm more than happy to meet and explain what you need to know.

I'll add comments to this issue once I know more about the performance issue you've detected.
I will also, of course, run a similar test myself in my local machine.
For that, why don't you post here a typical entity, so I test something as similar as possible to what you do?
One interesting point is, how many hits do you get for your query? (i.e. how many entities are inside your query area)

Lastly, thanks for reporting!

[kzangeli]

I just thought of one thing you can try.
Start the broker with -idIndex or: export ORIONLD_MONGO_ID_INDEX=TRUE.
That's an extra db index on "_id.id" and we've seen it help in other cases. I'm thinking on turning it on by default.
(indices typically make queries faster but updates slower ...)

[kzangeli]

So, I created a functest doing this:

# 01. Create 1000 entities with a location that is the center of Madrid
# 02. Create 5 entities with a location that is the center of Alcobendas
# 03. Geo-Query for all entities in Alcobendas, and time it

I timed the query and got this result:

real	0m0.064s
user	0m0.054s
sys	0m0.012s

This is how my entities look:

{
    "id": "urn:ngsi-ld:madrid:Exxx,
    "type": "Device",
    "location": {
      "type": "GeoProperty",
      "value": {
        "type": "Point",
        "coordinates": [40.418889, -3.691944]
      }
    }
  }

1000 of those (xxx goinf from 1 to 1000 in "id").
Plus 5 of these:

{
    "id": "urn:ngsi-ld:alcobendas:E'$eNo'",
    "type": "Device",
    "location": {
      "type": "GeoProperty",
      "value": {
        "type": "Point",
        "coordinates": [ 40.533333, -3.633333 ]
      }
    }
  }

This is the query:

orionCurl --url "/ngsi-ld/v1/entities?geometry=Point&georel=near;maxDistance==100&coordinates=\[40.533333,-3.633333\]"

And, well, 64 milliseconds seems like a reasonable query time.
(I'm not 100% sure of the order lat/long, but, that doesn't matter for the experiment)

So, I'm not able to reproduce the slow geo-queries.
Let me know how to change my test to make it more similar to what you have.

Also, I did not use the "-idIndex" CLI param for the broker.

[kzangeli]

(I reran with maxDistance==2000 and the query took 72ms, so, still nowhere close to what you see)

[Manifesto24]

After deploying the docker on Amazon AWS seems like it takes only couple of milliseconds to get a response. I guess I had some limitation from my docker setup or something else. The issue was not the broker. Thanks for your help !

[kzangeli]

ok!