Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Type variables #17

Open
berningh opened this issue Feb 27, 2020 · 1 comment
Open

Type variables #17

berningh opened this issue Feb 27, 2020 · 1 comment
Assignees
Labels
enhancement New feature or request

Comments

@berningh
Copy link

@pyZerrenner Great job, thanks a lot for your effort.
I think the variables have a precision which is too good to be true. It's may be correctly read but then stored as a wrong type (float64 or so).

@pyZerrenner
Copy link
Owner

Hello @berningh ,

Thanks for the regards.

As these are measurement data, I do not think the numerical precision in memory should be connected to the measurement accuracy. What advantage do you see in setting specific data types for each quantity (I assume you talk about numpy arrays, since the precision of scalars cannot be set natively in Python). I can mainly think of an optimized memory usage .

A bit more information: I have simply used the default data type for numpy arrays (float64). It is correct, that e.g. the DOP3000 files contains time in a precision of 10-4 seconds and depth in 10-1 millimeter (the latter can be calculated with higher accuracy). The echo signal is an unsigned byte which is potentially upscaled 4 times, so that could be an uint16. The velocity is more complicated, as it is stored as a signed byte but is then processed with multiple other quantities (including emitting frequency, speed of sound, Doppler angle and PRF). I am not sure what would be the the correct precision for the resulting values in meter/second. float32 can store precisions of micrometer/second though in extreme cases even that might be too coarse - I rather err on the side of caution and take float64.

@pyZerrenner pyZerrenner self-assigned this Mar 21, 2020
@pyZerrenner pyZerrenner added the enhancement New feature or request label Mar 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants