sometimes the map doesn't show all the markers

[wbazant]

Spotted with maples in Clemson University (again when testing something else).

First observation, these are different dots:
http://beta.fallingfruit.org/map/@34.676566103490984,-82.83368184902955,16z
http://fallingfruit.org/?z=16&y=34.67628&x=-82.83776&m=true&t=roadmap&l=false&locale=en&c=forager,freegan

Secondly, playing around with the filter suggests that a map has a quiet maximum number of markers that it's willing to display, and the "Tree inventories" feature is enough to trigger it. See these selections of various maples:

[wbazant]

In src/redux/viewChange.js we decide whether to show clusters or not by:

export const getIsShowingClusters = (state) =>
  state.map.view.zoom <= VISIBLE_CLUSTER_ZOOM_LIMIT

Proposed resolution strategy:

• find out where the marker limit comes from and whether it can be configured
• find out how to recognise when the app gets more data than it can show on the map: maybe the result of /counts?
• if the app the realises it has too much data to show, it asks for clusters instead of locations

Actually it could also work entirely on counts. For rare species, we don't need to show clusters in a zoomed out location, for example this could just be a dot instead of a cluster with one location:

but the UX of zooming into the cluster after clicking on it is very helpful, so some consideration would need to be given to that.

[wbazant]

related to #133

[ezwelty]

@wbazant The logic for this is that the tree inventory data can be so dense that transferring all the locations in a request, and rendering the locations on the map, would slow or crash the app. So we devised the pre-computed clusters, which reduce the locations in an area to a count and a center of mass. They are computed on a nested quadtree down currently to zoom level 13.

Theoretically I might be able to precompute them at a variable depth, but easiest might be to first see down to what zoom level I would need to go for it to be sufficient in even the densest areas. Then maybe the response from types/counts endpoint could be used to decide whether to request clusters or locations?

[wbazant]

I think what's left of the problem is now the presentation aspect - the map can't draw all the markers, or they are placed so densely they are hard to read. I was thinking about having the client display some of the received locations as markers, until I realised it's not possible to do that on the edge of the map, it might be slow, and the result would move as the map moves, which is not good

Then maybe the response from types/counts endpoint could be used to decide whether to request clusters or locations?

Does the problem get easier if we allow the response to produce a mix of clusters and locations? It could overall be nice also for sparsely annotated areas, and even on the furthest zoom there are sometimes clusters of size 1, which could be just a location marker.

[ezwelty]

Does the problem get easier if we allow the response to produce a mix of clusters and locations? It could overall be nice also for sparsely annotated areas, and even on the furthest zoom there are sometimes clusters of size 1, which could be just a location marker.

The clusters are computed on the same (but I believe denser by a factor of 2) quad-tree which is used for the square base map tiles in web maps. I can imagine a scheme where, for each grid-cell, the API returns either locations or the single cluster. I should run some tests with the database to see what is possible performance-wise.

[wbazant]

I still like the dynamic clusters / locations improvement, but I also had a thought about this issue when looking at the code that fetches the locations in the map slice - it uses a limit:

src/redux/mapSlice.js
33:    await getLocations(selectParams(getState(), { limit: 250 })),

The limit should probably stay, because returning arbitrarily large payloads is leading to all sorts of problems, but I realised we could reduce severity of the bug by toasting a warning about the map being incomplete whenever getLocations returns 250 entries - then the user can zoom in or adjust type filters.