]> git.ardis.lu - csvio/commit
git.ardis.lu / csvio / commit
? search:
summary | shortlog | log | commit | commitdiff | tree
(parent: a7027c2) | patch
Increase default chunk size from 1 KiB to 64 KiB
authorArdis Lu <ardisglu@gmail.com>
Tue, 1 Jul 2025 04:19:16 +0000 (21:19 -0700)
committerArdis Lu <ardisglu@gmail.com>
Tue, 1 Jul 2025 04:19:16 +0000 (21:19 -0700)
commit052903464cb8c42416952ed255fb1eaa3bc0667f
tree0f9d92fb08eba67cde9366979cd3c6df9e8cc628tree | snapshot (zip tar.gz)
parenta7027c2d2a654a0deffd1355f7503166c1446001commit | diff
Increase default chunk size from 1 KiB to 64 KiB

Benchmarks showed improvements or no change across all tests. Notably, 100MB Read improved from ~13s to ~3s.

The magic value of 64 KiB was selected based on Node.js's default of 16 KiB in similar code [1] and manually testing some multiples over that (Node.js tends to be conservative).

[1] https://github.com/nodejs/node/blob/f69f90d97a30cf9a5f7cbe0260e487ae4d457ed5/lib/internal/fs/promises.js#L317
src/core.js diff | blob | blame | history
CSV I/O (csvio) is a Node.js library for processing CSV files.
This page took 0.037702 seconds and 4 git commands to generate.