Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Pagination doesn't work properly (used together with sorting and filtering) #4394

Closed
mapedraza opened this issue Aug 21, 2023 · 7 comments
Closed

Pagination doesn't work properly (used together with sorting and filtering) #4394

mapedraza opened this issue Aug 21, 2023 · 7 comments
Milestone
3.11.0

Comments

@mapedraza
Copy link
Collaborator

mapedraza commented Aug 21, 2023 •
edited by fgalan
Loading

Bug description

When querying to the CB paginated entities chucks, sorted by a certain attribute, with a q filter, some entities are not returned.

Taking as an examples a list of 57 entities, then, querying entities with limit=1 and offset from 0 to 60. For this purpose, a simple bash script is used:

url="http://localhost:1026/v2/entities?type=VerticalInstance&limit=1&offset=%of&orderBy=kEnvironmentService&q=status:SUCCESS&attrs=status,id2"

for i in {0..60}

do
  step_url="${url/"%of"/$i}"
  echo "\noffset=${i}"

  wget --no-check-certificate -qO- --method GET --header 'Fiware-Service: test'  --header 'Fiware-ServicePath: /pruebas3'  --header 'X-Auth-Token: null' $step_url

done

Data example bellow

  • Orion version: 3.10.1
  • MongoDB version 4.4.6
  • Env variables or CLI parameters
  • Operating System or Docker Image used

How to reproduce it

Create entities

curl --location 'http://localhost:1026/v2/op/update' \
--header 'Fiware-Service: test' \
--header 'Fiware-ServicePath: /pruebas3' \
--header 'X-Auth-Token: null' \
--header 'Content-Type: application/json' \
--data '{
	"actionType": "append",
	"entities": [{
			"id": "E0",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-02-01T12:39:57.567Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "al",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E1",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-02-22T17:28:11.300Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "ta",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E10",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-04-04T10:43:02.175Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "jc",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "FAILURE",
				"metadata": {}
			}
		},
		{
			"id": "E11",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-04-20T08:47:05.766Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "ca",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			},
			"version": {
				"type": "Text",
				"value": "dumps_3.1.0",
				"metadata": {}
			}
		},
		{
			"id": "E12",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-04-19T07:40:07.773Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "ca",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			},
			"version": {
				"type": "Text",
				"value": "pools_3.0.0",
				"metadata": {}
			}
		},
		{
			"id": "E13",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-04-19T16:16:15.408Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "ca",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			},
			"version": {
				"type": "Text",
				"value": "interactiveKiosk_2.0.0_totem",
				"metadata": {}
			}
		},
		{
			"id": "E14",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-04-20T09:11:19.178Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "ba",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			},
			"version": {
				"type": "Text",
				"value": "fireDetection_2.0.0",
				"metadata": {}
			}
		},
		{
			"id": "E15",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-04-20T15:38:09.196Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "ba",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			},
			"version": {
				"type": "Text",
				"value": "interactiveKiosk_2.0.0",
				"metadata": {}
			}
		},
		{
			"id": "E16",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-07-31T03:50:40.741Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "me",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E17",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-04-27T15:31:46.772Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "se",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E18",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-04-28T09:40:13.025Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "se",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E19",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-05-22T15:57:03.042Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "sa",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E2",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-02-28T09:28:40.254Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "al",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "FAILURE",
				"metadata": {}
			}
		},
		{
			"id": "E20",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-05-23T11:17:06.237Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "se",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E21",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-08-08T20:56:13.959Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "tv",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E22",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-05-24T16:19:10.869Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "ud",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E23",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-05-26T17:13:30.794Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "sa",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E24",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-06-18T02:05:42.006Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "tv",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "START",
				"metadata": {}
			}
		},
		{
			"id": "E25",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-06-02T08:16:32.792Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "ba",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E26",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-06-05T15:29:33.853Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "jc",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E27",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-06-06T08:53:10.018Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "ba",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E28",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-06-06T09:20:45.566Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "ca",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E29",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-06-07T14:35:09.098Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "sa",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E3",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-03-07T08:44:41.286Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "al",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E30",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-06-08T08:32:39.484Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "al",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E31",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-06-09T10:39:40.338Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "ba",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E32",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-06-13T16:00:44.803Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "se",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E33",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-06-14T09:36:32.361Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "ba",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E34",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-07-31T02:54:33.036Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "me",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "FAILURE",
				"metadata": {}
			}
		},
		{
			"id": "E35",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-06-30T04:52:16.601Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "tv",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E36",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-06-26T15:36:00.434Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "al",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E37",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "None",
				"value": null,
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "se",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "FAILURE",
				"metadata": {}
			}
		},
		{
			"id": "E38",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-06-26T09:36:38.508Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "se",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E39",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-06-29T11:08:07.804Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "al",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E4",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-03-09T16:14:49.389Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "ta",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "FAILURE",
				"metadata": {}
			}
		},
		{
			"id": "E40",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-07-18T09:24:08.467Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "se",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E41",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-06-29T12:23:32.411Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "al",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E42",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-06-30T11:48:15.084Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "se",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E43",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-07-03T14:48:33.203Z",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "FAILURE",
				"metadata": {}
			}
		},
		{
			"id": "E44",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-07-04T16:49:33.484Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "tv",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "FAILURE",
				"metadata": {}
			}
		},
		{
			"id": "E45",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-07-07T12:08:45.700Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "se",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E46",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-07-25T07:29:03.626Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "se",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E47",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-07-21T08:46:47.085Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "se",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E48",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-07-21T07:51:34.710Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "ba",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E49",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-07-27T06:17:10.361Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "me",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E5",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-04-28T12:03:57.020Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "al",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E50",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-08-09T00:12:42.507Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "al",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E51",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-08-07T09:18:25.339Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "se",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E52",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-08-08T06:34:03.305Z",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E53",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-08-11T10:37:06.796Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "al",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E54",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-08-14T12:57:48.272Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "al",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E55",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-08-16T06:47:16.308Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "sa",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E56",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-08-16T06:32:15.976Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "al",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E6",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-03-17T00:28:42.789Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "jc",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E7",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-03-17T08:56:19.763Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "al",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E8",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-03-21T15:13:58.292Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "jc",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		},
		{
			"id": "E9",
			"type": "VerticalInstance",
			"TimeInstant": {
				"type": "DateTime",
				"value": "2023-06-09T12:38:59.997Z",
				"metadata": {}
			},
			"kEnvironmentService": {
				"type": "Text",
				"value": "jc",
				"metadata": {}
			},
			"status": {
				"type": "Text",
				"value": "SUCCESS",
				"metadata": {}
			}
		}
	]
}'

Bash script output

 ./loop.sh

offset=0
[{"id":"E52","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=1
[{"id":"E3","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=2
[{"id":"E3","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=3
[{"id":"E5","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=4
[{"id":"E3","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=5
[{"id":"E30","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=6
[{"id":"E30","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=7
[{"id":"E5","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=8
[{"id":"E30","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=9
[{"id":"E3","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=10
[{"id":"E3","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=11
[{"id":"E3","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=12
[{"id":"E3","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=13
[{"id":"E33","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=14
[{"id":"E33","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=15
[{"id":"E31","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=16
[{"id":"E25","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=17
[{"id":"E27","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=18
[{"id":"E15","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=19
[{"id":"E25","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=20
[{"id":"E11","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=21
[{"id":"E13","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=22
[{"id":"E13","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=23
[{"id":"E28","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=24
[{"id":"E9","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=25
[{"id":"E6","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=26
[{"id":"E26","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=27
[{"id":"E8","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=28
[{"id":"E49","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=29
[{"id":"E16","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=30
[{"id":"E19","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=31
[{"id":"E23","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=32
[{"id":"E23","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=33
[{"id":"E29","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=34
[{"id":"E18","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=35
[{"id":"E38","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=36
[{"id":"E40","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=37
[{"id":"E20","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=38
[{"id":"E42","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=39
[{"id":"E45","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=40
[{"id":"E46","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=41
[{"id":"E51","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=42
[{"id":"E47","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=43
[{"id":"E32","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=44
[{"id":"E17","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=45
[{"id":"E1","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=46
[{"id":"E21","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=47
[{"id":"E35","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=48
[{"id":"E22","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
offset=49
[]
offset=50
[]
offset=51
[]
offset=52
[]
offset=53
[]
offset=54
[]
offset=55
[]
offset=56
[]
offset=57
[]
offset=58
[]
offset=59
[]
offset=60
[]%
@mapedraza mapedraza added the bug label Aug 21, 2023
@mapedraza
Copy link
Collaborator Author

CC: @danielvillalbamota

@fgalan
Copy link
Member

fgalan commented Aug 25, 2023

It seems the problem is related with &orderBy=kEnvironmentService. If that token is removed from URL, the behaviour works as expected.

@fgalan
Copy link
Member

fgalan commented Aug 25, 2023

As reference, this should be the right order, as seen at DB:

> db.entities.find({"attrs.status.value": "SUCCESS"}, {_id: 1}).sort({"attrs.kEnvironment.value": 1})
{ "_id" : { "id" : "E0", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E1", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E11", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E12", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E13", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E14", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E15", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E16", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E17", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E18", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E19", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E20", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E21", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E22", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E23", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E25", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E26", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E27", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E28", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E29", "type" : "VerticalInstance", "servicePath" : "/" } }
Type "it" for more
> it
{ "_id" : { "id" : "E3", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E30", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E31", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E32", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E33", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E35", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E36", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E38", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E39", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E40", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E41", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E42", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E45", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E46", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E47", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E48", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E49", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E5", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E50", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E51", "type" : "VerticalInstance", "servicePath" : "/" } }
Type "it" for more
> it
{ "_id" : { "id" : "E52", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E53", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E54", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E55", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E56", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E6", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E7", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E8", "type" : "VerticalInstance", "servicePath" : "/" } }
{ "_id" : { "id" : "E9", "type" : "VerticalInstance", "servicePath" : "/" } }

@fgalan
Copy link
Member

fgalan commented Aug 25, 2023

Running only the first three iterations (offset 0, 1, 2):

http://localhost:1026/v2/entities?type=VerticalInstance&limit=1&offset=0&orderBy=kEnvironmentService&q=status:SUCCESS&attrs=status,id2
[{"id":"E52","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
http://localhost:1026/v2/entities?type=VerticalInstance&limit=1&offset=1&orderBy=kEnvironmentService&q=status:SUCCESS&attrs=status,id2
[{"id":"E3","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]
http://localhost:1026/v2/entities?type=VerticalInstance&limit=1&offset=2&orderBy=kEnvironmentService&q=status:SUCCESS&attrs=status,id2
[{"id":"E3","type":"VerticalInstance","status":{"type":"Text","value":"SUCCESS","metadata":{}}}]

Logs at CB are:

time=2023-08-25T10:32:27.546Z | lvl=DEBUG | corr=b002b0fa-4332-11ee-a106-000c29583ca5 | trans=1692959502-558-00000000001 | from=0.0.0.0 | srv= | subsrv= | comp=Orion | op=connectionOperations.cpp[159]:collectionRangedQuery | msg=query() in 'orion.entities' collection limit=1, offset=0, query='{ "_id.type" : "VerticalInstance", "attrs.status.value" : { "$eq" : "SUCCESS" } }', count='{ "_id.type" : "VerticalInstance", "attrs.status.value" : { "$eq" : "SUCCESS" } }'
time=2023-08-25T10:32:27.559Z | lvl=DEBUG | corr=b002b0fa-4332-11ee-a106-000c29583ca5 | trans=1692959502-558-00000000001 | from=0.0.0.0 | srv= | subsrv= | comp=Orion | op=MongoGlobal.cpp[1638]:entitiesQuery | msg=retrieved document [1]: '{ "_id" : { "id" : "E52", "type" : "VerticalInstance", "servicePath" : "/" }, "attrNames" : [ "TimeInstant", "status" ], "attrs" : { "TimeInstant" : { "value" : 1691476443.3050000668, "type" : "DateTime", "creDate" : 1692953756.725143671, "modDate" : 1692953756.725143671, "mdNames" : [ ] }, "status" : { "value" : "SUCCESS", "type" : "Text", "creDate" : 1692953756.725143671, "modDate" : 1692953756.725143671, "mdNames" : [ ] } }, "creDate" : 1692953756.725143671, "modDate" : 1692953756.725143671, "lastCorrelator" : "33b5bafe-4325-11ee-9779-000c29583ca5" }'

time=2023-08-25T10:32:27.632Z | lvl=DEBUG | corr=b00fc3da-4332-11ee-b554-000c29583ca5 | trans=1692959502-558-00000000002 | from=0.0.0.0 | srv= | subsrv= | comp=Orion | op=connectionOperations.cpp[159]:collectionRangedQuery | msg=query() in 'orion.entities' collection limit=1, offset=1, query='{ "_id.type" : "VerticalInstance", "attrs.status.value" : { "$eq" : "SUCCESS" } }', count='{ "_id.type" : "VerticalInstance", "attrs.status.value" : { "$eq" : "SUCCESS" } }'
time=2023-08-25T10:32:27.641Z | lvl=DEBUG | corr=b00fc3da-4332-11ee-b554-000c29583ca5 | trans=1692959502-558-00000000002 | from=0.0.0.0 | srv= | subsrv= | comp=Orion | op=MongoGlobal.cpp[1638]:entitiesQuery | msg=retrieved document [1]: '{ "_id" : { "id" : "E3", "type" : "VerticalInstance", "servicePath" : "/" }, "attrNames" : [ "TimeInstant", "kEnvironmentService", "status" ], "attrs" : { "TimeInstant" : { "value" : 1678178681.2859992981, "type" : "DateTime", "creDate" : 1692953756.1760201454, "modDate" : 1692953756.1760201454, "mdNames" : [ ] }, "kEnvironmentService" : { "value" : "al", "type" : "Text", "creDate" : 1692953756.1760201454, "modDate" : 1692953756.1760201454, "mdNames" : [ ] }, "status" : { "value" : "SUCCESS", "type" : "Text", "creDate" : 1692953756.1760201454, "modDate" : 1692953756.1760201454, "mdNames" : [ ] } }, "creDate" : 1692953756.1760201454, "modDate" : 1692953756.1760201454, "lastCorrelator" : "33b5bafe-4325-11ee-9779-000c29583ca5" }'

time=2023-08-25T10:32:27.696Z | lvl=DEBUG | corr=b0199180-4332-11ee-8da3-000c29583ca5 | trans=1692959502-558-00000000003 | from=0.0.0.0 | srv= | subsrv= | comp=Orion | op=connectionOperations.cpp[159]:collectionRangedQuery | msg=query() in 'orion.entities' collection limit=1, offset=2, query='{ "_id.type" : "VerticalInstance", "attrs.status.value" : { "$eq" : "SUCCESS" } }', count='{ "_id.type" : "VerticalInstance", "attrs.status.value" : { "$eq" : "SUCCESS" } }'
time=2023-08-25T10:32:27.702Z | lvl=DEBUG | corr=b0199180-4332-11ee-8da3-000c29583ca5 | trans=1692959502-558-00000000003 | from=0.0.0.0 | srv= | subsrv= | comp=Orion | op=MongoGlobal.cpp[1638]:entitiesQuery | msg=retrieved document [1]: '{ "_id" : { "id" : "E3", "type" : "VerticalInstance", "servicePath" : "/" }, "attrNames" : [ "TimeInstant", "kEnvironmentService", "status" ], "attrs" : { "TimeInstant" : { "value" : 1678178681.2859992981, "type" : "DateTime", "creDate" : 1692953756.1760201454, "modDate" : 1692953756.1760201454, "mdNames" : [ ] }, "kEnvironmentService" : { "value" : "al", "type" : "Text", "creDate" : 1692953756.1760201454, "modDate" : 1692953756.1760201454, "mdNames" : [ ] }, "status" : { "value" : "SUCCESS", "type" : "Text", "creDate" : 1692953756.1760201454, "modDate" : 1692953756.1760201454, "mdNames" : [ ] } }, "creDate" : 1692953756.1760201454, "modDate" : 1692953756.1760201454, "lastCorrelator" : "33b5bafe-4325-11ee-9779-000c29583ca5" }'

It seems the query at DB doesn't include the sort statement... weird.

@fgalan
Copy link
Member

fgalan commented Aug 25, 2023

It seems the query at DB doesn't include the sort statement... weird.

Log were incomplete. After fixing logs:

time=2023-08-25T10:49:18.087Z | lvl=DEBUG | corr=0a570126-4335-11ee-93ff-000c29583ca5 | trans=1692960509-529-00000000001 | from=0.0.0.0 | srv= | subsrv= | comp=Orion | op=connectionOperations.cpp[161]:collectionRangedQuery | msg=query() in 'orion.entities' collection limit=1, offset=0, query='{ "_id.type" : "VerticalInstance", "attrs.status.value" : { "$eq" : "SUCCESS" } }', count='{ "_id.type" : "VerticalInstance", "attrs.status.value" : { "$eq" : "SUCCESS" } }', sort='{ "attrs.kEnvironmentService.value" : 1 }'
time=2023-08-25T10:49:18.101Z | lvl=DEBUG | corr=0a570126-4335-11ee-93ff-000c29583ca5 | trans=1692960509-529-00000000001 | from=0.0.0.0 | srv= | subsrv= | comp=Orion | op=MongoGlobal.cpp[1638]:entitiesQuery | msg=retrieved document [1]: '{ "_id" : { "id" : "E52", "type" : "VerticalInstance", "servicePath" : "/" }, "attrNames" : [ "TimeInstant", "status" ], "attrs" : { "TimeInstant" : { "value" : 1691476443.3050000668, "type" : "DateTime", "creDate" : 1692953756.725143671, "modDate" : 1692953756.725143671, "mdNames" : [ ] }, "status" : { "value" : "SUCCESS", "type" : "Text", "creDate" : 1692953756.725143671, "modDate" : 1692953756.725143671, "mdNames" : [ ] } }, "creDate" : 1692953756.725143671, "modDate" : 1692953756.725143671, "lastCorrelator" : "33b5bafe-4325-11ee-9779-000c29583ca5" }'

time=2023-08-25T10:49:18.155Z | lvl=DEBUG | corr=0a6172dc-4335-11ee-87be-000c29583ca5 | trans=1692960509-529-00000000002 | from=0.0.0.0 | srv= | subsrv= | comp=Orion | op=connectionOperations.cpp[161]:collectionRangedQuery | msg=query() in 'orion.entities' collection limit=1, offset=1, query='{ "_id.type" : "VerticalInstance", "attrs.status.value" : { "$eq" : "SUCCESS" } }', count='{ "_id.type" : "VerticalInstance", "attrs.status.value" : { "$eq" : "SUCCESS" } }', sort='{ "attrs.kEnvironmentService.value" : 1 }'
time=2023-08-25T10:49:18.163Z | lvl=DEBUG | corr=0a6172dc-4335-11ee-87be-000c29583ca5 | trans=1692960509-529-00000000002 | from=0.0.0.0 | srv= | subsrv= | comp=Orion | op=MongoGlobal.cpp[1638]:entitiesQuery | msg=retrieved document [1]: '{ "_id" : { "id" : "E3", "type" : "VerticalInstance", "servicePath" : "/" }, "attrNames" : [ "TimeInstant", "kEnvironmentService", "status" ], "attrs" : { "TimeInstant" : { "value" : 1678178681.2859992981, "type" : "DateTime", "creDate" : 1692953756.1760201454, "modDate" : 1692953756.1760201454, "mdNames" : [ ] }, "kEnvironmentService" : { "value" : "al", "type" : "Text", "creDate" : 1692953756.1760201454, "modDate" : 1692953756.1760201454, "mdNames" : [ ] }, "status" : { "value" : "SUCCESS", "type" : "Text", "creDate" : 1692953756.1760201454, "modDate" : 1692953756.1760201454, "mdNames" : [ ] } }, "creDate" : 1692953756.1760201454, "modDate" : 1692953756.1760201454, "lastCorrelator" : "33b5bafe-4325-11ee-9779-000c29583ca5" }'

time=2023-08-25T10:49:18.240Z | lvl=DEBUG | corr=0a6e617c-4335-11ee-874f-000c29583ca5 | trans=1692960509-529-00000000003 | from=0.0.0.0 | srv= | subsrv= | comp=Orion | op=connectionOperations.cpp[161]:collectionRangedQuery | msg=query() in 'orion.entities' collection limit=1, offset=2, query='{ "_id.type" : "VerticalInstance", "attrs.status.value" : { "$eq" : "SUCCESS" } }', count='{ "_id.type" : "VerticalInstance", "attrs.status.value" : { "$eq" : "SUCCESS" } }', sort='{ "attrs.kEnvironmentService.value" : 1 }'
time=2023-08-25T10:49:18.249Z | lvl=DEBUG | corr=0a6e617c-4335-11ee-874f-000c29583ca5 | trans=1692960509-529-00000000003 | from=0.0.0.0 | srv= | subsrv= | comp=Orion | op=MongoGlobal.cpp[1638]:entitiesQuery | msg=retrieved document [1]: '{ "_id" : { "id" : "E3", "type" : "VerticalInstance", "servicePath" : "/" }, "attrNames" : [ "TimeInstant", "kEnvironmentService", "status" ], "attrs" : { "TimeInstant" : { "value" : 1678178681.2859992981, "type" : "DateTime", "creDate" : 1692953756.1760201454, "modDate" : 1692953756.1760201454, "mdNames" : [ ] }, "kEnvironmentService" : { "value" : "al", "type" : "Text", "creDate" : 1692953756.1760201454, "modDate" : 1692953756.1760201454, "mdNames" : [ ] }, "status" : { "value" : "SUCCESS", "type" : "Text", "creDate" : 1692953756.1760201454, "modDate" : 1692953756.1760201454, "mdNames" : [ ] } }, "creDate" : 1692953756.1760201454, "modDate" : 1692953756.1760201454, "lastCorrelator" : "33b5bafe-4325-11ee-9779-000c29583ca5" }'

@fgalan
Copy link
Member

fgalan commented Aug 25, 2023 •
edited
Loading

After analysing this, I think is expected behaviour.

If we use &orderBy=kEnvironmentService and the value of kEnvironmentService is not unique in the entities, then the same query executed at DB level may result in different ordering for the entities which kEnvironmentService is the same. This is explained with more detail in MongoDB official documentation at https://www.mongodb.com/docs/manual/reference/method/cursor.sort/

Documents are returned in alphabetical order by borough, but the order of those documents with duplicate values for borough might not be the same across multiple executions of the same sort.

Moreover it is also said:

To achieve a consistent sort, add a field which contains exclusively unique values to the sort.

In fact, if we use in the query &orderBy=kEnvironmentService,dateCreated the results are consistent (as dateCreated makes results unique).

What should we do at CB?

  • (Easy) Document this behaviour in the "Ordering results" section of orion-api.md
  • (Not so easy, but not complex) Modify Orion sorting implementation so if orderBy is used in a query, then _id is added as last token in the ordering document to make results unique (this is the example shown in MongoDB documentation referenced above)

@fgalan fgalan mentioned this issue Aug 28, 2023
Merged
@fgalan fgalan added this to the 3.11.0 milestone Aug 28, 2023
@fgalan
Copy link
Member

fgalan commented Aug 28, 2023

By the moment it seems that with documentation improvements (PR #4405) is enough.

@fgalan fgalan closed this as completed Aug 28, 2023
@fgalan fgalan removed the bug label Aug 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
3.11.0
Development

No branches or pull requests
2 participants
@fgalan @mapedraza