Spreadsheet table schema management

[celiot-IDM]

The PACE-HRH R package (pacehrh) does a lot of reading information from Excel workbook sheets, and saving the data in internal structures. Meikang and Rui have done a lot of work on a validation tool for the Excel spreadsheet content, but we should consider doing more to guarantee internal consistency inside the pacehrh code.

Things we could be thinking about:

• As part of some experiments done with Bee to use Shiny as a front-end to the model, we have APIs (in pace_scenarios_manage.R) to manage Scenarios (add, delete, update, read) without directly manipulating the Scenarios table at GPE$scenarios. This work anticipates the ability to completely insulate pacehrh code from the underlying data store implementation (data frame, data table, SQL Lite database, etc); to have schema-driven data stores (e.g. know that the "Label" field is a string); and to completely decouple pacehrh code from the Excel spreadsheet format (e.g. if a column name changes in Excel, all that needs to change in pacehrh is a line in the schema meta-data, and all other code stays the same.)
• This kind of abstraction is essential if we want to consider translation into other languages (French, etc).

This kind of work would make the codebase more complex (multiple extra abstraction layers to support data store management and robust localization). We should be very sure we need these features before we make the code base harder to understand.