Unifying max memory settings between Docker and ROBOT

[gouttegd]

Currently, users working with large ontologies might need to increase two different memory settings:

• the maximal amount of memory that a Docker container is allowed to use (typically configured in the configuration panel of Docker Desktop);
• the maximal amount of memory that ROBOT is allowed to use when running within a ODK container (typically configured with the robot_java_args option in the -odk.yaml configuration file).

Changing only one of those settings is not enough: allowing Docker to use 40GB of RAM will be of little help if ROBOT is still limited to use only 8GB (this will only help for ODK workflow steps that do not use ROBOT, e.g. OAK-dependent steps), and conversely allowing ROBOT to use 40GB of RAM will have no effect if the container itself is limited to a smaller amount.

It would be nice if users only had to perform one configuration step (changing just one setting) and not have to worry about making sure that all components of the ODK can benefit from the new setting.

One way to achieve that would be to make the default ROBOT setting dependent on the Docker setting: that is, the run.sh wrapper script should detect the maximal amount of memory that Docker is configured to use, and set the ROBOT Java options accordingly. E.g., if Docker is configured to use at most 40GB of RAM, we can allow ROBOT to use, say, 36GB (90% of the total amount of memory allocated to the container).

One way to get the Docker setting from the wrapper script:

docker system info | sed -nre 's,^ Total Memory: ,,p'

[matentzn]

I think this is a good idea, but it should be possible to set them individually as well. There are scenarios, albeit a bit outlandish (involving eg parallelisation) that would require the sub process (robot) to have much less than 90% of the overall available memory.

[gouttegd]

Agreed. I'm only talking about a default setting: if there is no robot_java_args option in the ODK configuration, then we default to a value that is derived from the Docker setting -- otherwise we use whatever value the user has explicitly requested.