Enhancements to CSV Format Handling

[PierreRaybaut]

This work is done in develop/large_csv_files branch.

Below is a detailed outline of the planned improvements:

CSV Format Handling Enhancement:

• Current Progress: We are in the process of validating improvements to the handling of CSV formats. Add improvements to the handling of CSV formats allowing for direct reading of 1GB CSV files, though the reading time takes several seconds (tens of seconds) with a non-responsive GUI during this period. While not entirely satisfactory, this enhancement ensures that signals are correctly displayed in DataLab and are fully usable as is, despite having 10M points per signal.

User Experience Adjustments when opening signals or importing them using the import wizard:

• We plan to further enhance the user experience by optimizing processing times. This will be achieved by loading data in batches instead of a single block when directly opening signals or using the import assistant from a text file.

• Additionally, we aim to introduce progress bars that allow for the cancellation of ongoing actions, enhancing interaction with DataLab during longer loading or processing operations.

Interpolation Option Prior to Data Loading:

• An interpolation option will be made available before data loading into DataLab for the aforementioned methods (direct file opening and using the import assistant). This feature is intended to refine the data preparation process, making it more flexible and efficient for users.