You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
You use 1 page size buffer for each splice call. Its fine if second splice (to pipe) allows kernel to just mark memory pages as pagecache (thus, its fast). But fails if underiying pip is slow. For example if it belongs to file opened with O_DIRECT.
Simple test to splice data between 1GB files opened with O_DIRECT each:
Tuned pipe buf size for both ends (fcntl F_SETPIPE_SZ) for same value as buf_size
Pipe buf is 64k by default in modern linux. Everything above 1M requires CAP_SYS_RESOURCE non-root users. 8M is maximum I was able to test, everything above returns ENOMEM for root.
Thanks!
The text was updated successfully, but these errors were encountered:
mocksoul
changed the title
Please made buffer size tunable
Please make buffer size tunable
Nov 22, 2022
You use 1 page size buffer for each splice call. Its fine if second splice (to pipe) allows kernel to just mark memory pages as pagecache (thus, its fast). But fails if underiying pip is slow. For example if it belongs to file opened with O_DIRECT.
Simple test to splice data between 1GB files opened with O_DIRECT each:
(2 x SATA3 RAID0 ssd)
What did I do above:
buf_size
accrordingly for each runfcntl F_SETPIPE_SZ
) for same value asbuf_size
Pipe buf is 64k by default in modern linux. Everything above 1M requires
CAP_SYS_RESOURCE
non-root users. 8M is maximum I was able to test, everything above returns ENOMEM for root.Thanks!
The text was updated successfully, but these errors were encountered: