From: Ardis Lu <ardisglu@gmail.com>
Date: Tue, 1 Jul 2025 04:19:16 +0000 (-0700)
Subject: Increase default chunk size from 1 KiB to 64 KiB
X-Git-Tag: v0.1.2~10
X-Git-Url: https://git.ardis.lu/?a=commitdiff_plain;h=052903464cb8c42416952ed255fb1eaa3bc0667f;p=csvio

Increase default chunk size from 1 KiB to 64 KiB

Benchmarks showed improvements or no change across all tests. Notably, 100MB Read improved from ~13s to ~3s.

The magic value of 64 KiB was selected based on Node.js's default of 16 KiB in similar code [1] and manually testing some multiples over that (Node.js tends to be conservative).

[1] https://github.com/nodejs/node/blob/f69f90d97a30cf9a5f7cbe0260e487ae4d457ed5/lib/internal/fs/promises.js#L317
---

diff --git a/src/core.js b/src/core.js
index 60132d6..cce5ca4 100644
--- a/src/core.js
+++ b/src/core.js
@@ -44,7 +44,7 @@ export function createFileStream(path) {
 
   return new ReadableStream({
     type: 'bytes',
-    autoAllocateChunkSize: 1024,
+    autoAllocateChunkSize: 65536, // 64 KiB
     async start() {
       handle = await open(path, "r");
       resolveReady();