This commit introduces the concept of separate worker and web server processes. The worker is a distinct docker container that is respnsible for carrying out a simulation. The web server is within a separate container is only responsible for serving http requests. A redis database is used to communicate between the worker and web server. The web server and worker may scale indpendently and it is expected that a deployment would involve many workers to manage multiple simultaneous simulations/tests.

This commit is organized such that it adds to the existing boptest project layout, but it does not disturb any of the existing project layout. The worker depends on and makes extensive use of the existing boptest libraries, especially those APIs in testcase.py as well as the forecast and kpi source code directories.

Currently the web api presented by the web server is conceptually equilvant, but not api compatiable with the existing boptest/master API. A future commit will resolve this difference.

To test this commit:

cd <project-root>
docker-compose build
docker-compose up web worker

After starting the services defined by docker-compose, a web application should be viewable in the browser at http://localhost

* There is a graphql https://graphql.org api viewable at http://localhost/graphql
* This architecture depends on bulk file storage such as Amazon S3. In development mode a mino https://min.io container stands in for S3, and is available at http://localhost:9000. From this interface it is possible to view raw file (test case) uploads and simulation files. The default authentication can be found in the .env file that is included in this commit
* A forecast API has been added, which was not previously available in Alfalfa

Working (although incomplete) twoday example

Update API and add example controller

There is global attribution at the project root

Use redis messages to generate results as they are requested, instead of
the very time consuming approach of computing them on every step

Remove some unwanted Alfalfa isms

The exceptions are the min/max test, plus julia and javascript tests

When setting a new scenario only initialize a new job/test when there is
a new time period

Now an error will be returned for invalid control inputs

Most notably there is a more clear boundary around a "job" and they are
named clearly "boptest_add_testcase" and "boptest_run_test". Also, jobs
are each proper python modules.

Some Alfalfa terminology scrubed.

In the process of this, also remove all traces of graphql, and cleanup
route organization

This new api is used to select a testcase and returns a test id.
Also there is significant refactor of the messaging between web and
worker.

A previous commit created a regression where advance would not return
null at the end time

Previously every command opened two new redis connections between web
and worker. Now there is only a single set of connections that are
persistent during the life of the web process.

If no requests are made on a test for two minutes, it will timeout and
the worker will cleanup the test and go back to waiting for another job.

Also update testcase1 based on assumption that the testcase is already
available

The subscriptions are now more targeted instead of generic pattern based
subscriptions, the sub / unsub calls that are removed here are reminants
of the past

When waiting for a response from the worker for API calls there is a
built in timeout (while waiting for a response over redis). Previously
this timeout was 30 seconds, however some API calls such as set_scenario
were noticed to take signficant time for some models. In response, the
timeout has been increased to 2 minutes.

ref #343

This simply makes the code cleaner, and now that there is a Redis class
encapsulating the redis client, there is a logical place to put these
promisified functions.

This is a primitive function, that has the testcases hard coded. Can be
improved.

ref #445

ref #448

Issue448 service submit

Issue424 v030 service

This change was inadvertantly removed in a previous commit, and is now
being reintroduced.

close #474

close #484

close #486

The environment variable named SERVICE_TIMEOUT now controls inactivity
timeout as well as the timeout while waiting to select a testcase.

In the previous commit, the canonical BOPTEST dependency was added as a
git subtree. In this commit, the pieces of BOPTEST (unrelated to BOPTEST-service),
which have been made redundant by the subtree are removed.

At this stage BOPTEST-Service is functional while consuming BOPTEST in
the new subtree location. The BOPTEST tests still need to be updated in
order to work with the service because the subtree and therefore the
tests are based on the BOPTEST master branch. A future commit needs to
reintroduce the BOPTEST-Service flavor of the tests.

All of this is an effort to retain history while still migrating
BOPTEST-Service to a dedicated repository and introducing BOPTEST
(canonical) as a subtree.

close #13

close #15

ref #9

ref #9

…test-service into issue/9-user-test-cases