Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

qlib在client存储featrues的股票字段数据使用np进行二进制文件的存储,为何不使用HDF5操作数据呢? #1807

Open
mrvegazhou opened this issue Jun 13, 2024 · 1 comment
Labels
question Further information is requested

Comments

@mrvegazhou
Copy link

mrvegazhou commented Jun 13, 2024

❓ Questions and Help

很想知道qlib的工程师为何要以bin的方式存储,而不用HDF5呢?

@mrvegazhou mrvegazhou added the question Further information is requested label Jun 13, 2024
@mrvegazhou mrvegazhou changed the title qlib在存储featrues的股票字段数据使用np进行二进制文件的存储,为何不使用HDF5操作数据呢? qlib在client存储featrues的股票字段数据使用np进行二进制文件的存储,为何不使用HDF5操作数据呢? Jun 13, 2024
@jimrok
Copy link

jimrok commented Jun 24, 2024

因为np的数据结构简单,就是float数据的序列化,有固定的长度,想要哪一日的数据,可以计算出偏移的位置,直接读取。理论上hdf5或者其他的库不能再更快了。单这个结构太难为维护了,基础数据稍微有些错误,你很难维护,他们没有提供完整的修复工具。看懂源代码是可以自己去修复的。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants