]>
git.ardis.lu - csvio/log
Ardis Lu [Tue, 25 Feb 2025 09:36:27 +0000 (01:36 -0800)]
Rename `options.value` to `options.placeholder`
Ardis Lu [Tue, 25 Feb 2025 09:32:49 +0000 (01:32 -0800)]
Refactor extra arguments into options object
Ardis Lu [Sat, 22 Feb 2025 09:45:03 +0000 (01:45 -0800)]
Rollback null-prototype to object prototype
Name collision is unlikely and enumeration may be useful. And it is easier for JSDoc to parse normal object initialization than `Object.create`.
Ardis Lu [Fri, 21 Feb 2025 08:18:38 +0000 (00:18 -0800)]
Pin Node.js version
Ardis Lu [Fri, 21 Feb 2025 08:12:33 +0000 (00:12 -0800)]
Add tests for ErrorStrategies
Ardis Lu [Fri, 21 Feb 2025 06:43:01 +0000 (22:43 -0800)]
Freeze ErrorStrategies and use null prototype
This configuration is the most restrictive and fit for its intended purpose as a namespace only.
Ardis Lu [Thu, 20 Feb 2025 07:49:23 +0000 (23:49 -0800)]
Add initial error strategies
Ardis Lu [Mon, 17 Feb 2025 07:01:01 +0000 (23:01 -0800)]
Add JSDoc for benchmark utils
Ardis Lu [Mon, 17 Feb 2025 06:54:29 +0000 (22:54 -0800)]
Add links to input and output CSVs
Ardis Lu [Sun, 16 Feb 2025 07:38:41 +0000 (23:38 -0800)]
Refactor to simplify functions
Ardis Lu [Sun, 16 Feb 2025 07:37:23 +0000 (23:37 -0800)]
Use package name for imports
Ardis Lu [Sun, 16 Feb 2025 07:31:43 +0000 (23:31 -0800)]
Make test input paths relative to the file instead of command
Before, `npm test` only worked in the root directory. Now it can run in any directory.
Ardis Lu [Sat, 15 Feb 2025 07:06:00 +0000 (23:06 -0800)]
Unlink file on keyboard interrupt
Ardis Lu [Sat, 15 Feb 2025 06:43:02 +0000 (22:43 -0800)]
Reorder test name for easier skimming
Ardis Lu [Fri, 14 Feb 2025 07:43:04 +0000 (23:43 -0800)]
Add 50MB benchmarks
Ardis Lu [Fri, 14 Feb 2025 07:41:54 +0000 (23:41 -0800)]
Use unique pseudorandom seed on each generation
Ardis Lu [Fri, 14 Feb 2025 07:39:37 +0000 (23:39 -0800)]
Use consistent name length to make it easier to compare outputs
Ardis Lu [Thu, 13 Feb 2025 07:45:40 +0000 (23:45 -0800)]
Replace passthrough options with typeCastOnly
To replicate old behavior (passthrough as string), simply use `type: "string"`.
Ardis Lu [Tue, 11 Feb 2025 06:41:02 +0000 (22:41 -0800)]
Assert non-nullable type instead of using nullish coalesce
There will be undefined `decimal` only if the input to `formatToParts` does not contain a decimal. But there will always be a decimal because the input is hardcoded.
The TS language server is not yet intelligent enough to assert it is non-nullable from the hardcoded input, so we need to pass the type hint.
Using nullish coalesce results in an uncovered branch which is impossible to test, so it should be removed.
Ardis Lu [Tue, 11 Feb 2025 06:33:25 +0000 (22:33 -0800)]
Add additional tests for uncovered branches
Ardis Lu [Mon, 10 Feb 2025 04:21:02 +0000 (20:21 -0800)]
Delete unnecessary token
Originally intended to match the remaining characters of the rest of the test name, but this is not needed since we only need to match the "large:" prefix.
Ardis Lu [Mon, 10 Feb 2025 04:12:26 +0000 (20:12 -0800)]
Assert example 1.6 logs to console
Ardis Lu [Mon, 10 Feb 2025 04:06:12 +0000 (20:06 -0800)]
Add 1MB benchmark
Ardis Lu [Sun, 9 Feb 2025 06:58:27 +0000 (22:58 -0800)]
Silence console functions on stdout and instead assert they occurred
This approach makes the test runner output cleaner, and also tests the console calls occurred as expected.
Ardis Lu [Sun, 9 Feb 2025 06:04:59 +0000 (22:04 -0800)]
Explicitly close file handle
Dismisses "Closing a FileHandle object on garbage collection" warnings in tests.
Ardis Lu [Sun, 9 Feb 2025 06:03:39 +0000 (22:03 -0800)]
Add cleanup on abort
`close()` is NOT called if CSVWriter errors, only `abort()`. So need to call `close()` within `abort()` to finish cleanup.
Ardis Lu [Sun, 9 Feb 2025 05:18:27 +0000 (21:18 -0800)]
Simplify ReadableStream constructor
Ardis Lu [Sat, 8 Feb 2025 09:47:24 +0000 (01:47 -0800)]
Fix temp files not being deleted
The test runner will run code inside a test suite block even if the tests inside are skipped, which means the temp file was created but the cleanup `after()` call is skipped. Putting the creation within `before()` makes the temp file creation skipped it the tests are skipped.
Ardis Lu [Sat, 8 Feb 2025 09:31:46 +0000 (01:31 -0800)]
Update prefix to match new project name
Ardis Lu [Fri, 7 Feb 2025 08:29:49 +0000 (00:29 -0800)]
Add test for saving progress on uncaught error
Ardis Lu [Fri, 7 Feb 2025 08:20:38 +0000 (00:20 -0800)]
Delete leftover flag from debugging
Ardis Lu [Fri, 7 Feb 2025 07:39:55 +0000 (23:39 -0800)]
Add initial benchmarks
Ardis Lu [Wed, 5 Feb 2025 06:26:12 +0000 (22:26 -0800)]
Update example to demonstrate more transformations
Ardis Lu [Tue, 4 Feb 2025 07:45:43 +0000 (23:45 -0800)]
Fix regex to match examples
This change allows for the file name convention "wip_ex1_1.js" for in-progress examples.
Ardis Lu [Mon, 3 Feb 2025 07:51:34 +0000 (23:51 -0800)]
Add "Abort a transformation" example
Ardis Lu [Thu, 30 Jan 2025 05:28:20 +0000 (21:28 -0800)]
Improve type documentation
Ardis Lu [Thu, 30 Jan 2025 05:07:51 +0000 (21:07 -0800)]
Fix broken test
Was previously passing by coincidence. Added an additional check to compare the actual rows vs. status rows.
Ardis Lu [Tue, 28 Jan 2025 07:26:05 +0000 (23:26 -0800)]
Add npm script for running test coverage report
Ardis Lu [Tue, 28 Jan 2025 07:22:04 +0000 (23:22 -0800)]
Add initial large CSV tests
Ardis Lu [Tue, 28 Jan 2025 06:33:02 +0000 (22:33 -0800)]
Add test for changing elapsed status
Ardis Lu [Tue, 28 Jan 2025 06:22:55 +0000 (22:22 -0800)]
Add tests for writing raw CSV strings
Ardis Lu [Tue, 28 Jan 2025 06:10:24 +0000 (22:10 -0800)]
Add test for raising non-error rejection
Ardis Lu [Tue, 28 Jan 2025 06:09:35 +0000 (22:09 -0800)]
Add test for handleHeaders error handling
Ardis Lu [Mon, 27 Jan 2025 07:04:03 +0000 (23:04 -0800)]
Note that rows includes the header row
Ardis Lu [Mon, 27 Jan 2025 07:03:41 +0000 (23:03 -0800)]
Add an example of using CSVWriter status
Ardis Lu [Mon, 27 Jan 2025 06:43:02 +0000 (22:43 -0800)]
Add public status property to CSVWriter for tracking progress
Ardis Lu [Sun, 26 Jan 2025 07:20:21 +0000 (23:20 -0800)]
Use consistent casing for types
Lowercase for all primitive types except Object and Array, TitleCase for all other types.
Ardis Lu [Sat, 25 Jan 2025 08:03:47 +0000 (00:03 -0800)]
Add example to combine retry and graceful failure strategies
Ardis Lu [Fri, 24 Jan 2025 04:23:55 +0000 (20:23 -0800)]
Add header row deletion test
Ardis Lu [Fri, 24 Jan 2025 04:23:21 +0000 (20:23 -0800)]
Clarify handleHeaders intended usage
Setting it to an Array<any> is preferred for simple scenarios.
Ardis Lu [Fri, 24 Jan 2025 04:21:43 +0000 (20:21 -0800)]
Add example to demonstrate passing a function to handleHeaders
Ardis Lu [Fri, 24 Jan 2025 03:47:14 +0000 (19:47 -0800)]
Allow Array<any> and TransformationFunction as inputs to handle headers
This change simplifies the common tasks of (1) renaming headers (just pass the new header array) and (2) appending headers (pass a one-line TransformationFunction). Setting the option to a boolean is the same behavior as includeHeaders.
Ardis Lu [Thu, 23 Jan 2025 06:36:29 +0000 (22:36 -0800)]
Passthrough encoding option for CSVReader
Ardis Lu [Thu, 23 Jan 2025 06:21:20 +0000 (22:21 -0800)]
Add missing semicolon
Ardis Lu [Wed, 22 Jan 2025 06:55:43 +0000 (22:55 -0800)]
Prepare for npm publish
Ardis Lu [Wed, 22 Jan 2025 05:59:15 +0000 (21:59 -0800)]
Renumber and categorize examples
Headers make the examples easier to skim. Using minor numberings is also useful to later add more examples without having to renumber existing groupings each time.
Ardis Lu [Wed, 22 Jan 2025 05:35:38 +0000 (21:35 -0800)]
Add graceful failure example
Ardis Lu [Tue, 21 Jan 2025 05:34:59 +0000 (21:34 -0800)]
Refactor CSVNormalizer.fixExcelDate suite to vectors
Ardis Lu [Tue, 21 Jan 2025 05:23:46 +0000 (21:23 -0800)]
Refactor CSVNormalizer.fixExcelBigInt suite to vectors
Ardis Lu [Tue, 21 Jan 2025 05:19:40 +0000 (21:19 -0800)]
Refactor CSVNormalizer.fixExcelNumber suite to vectors
Ardis Lu [Tue, 21 Jan 2025 05:19:16 +0000 (21:19 -0800)]
Run formatter
Ardis Lu [Tue, 21 Jan 2025 05:12:53 +0000 (21:12 -0800)]
Refactor expandScientificNotation suite to vectors
Ardis Lu [Tue, 21 Jan 2025 04:51:12 +0000 (20:51 -0800)]
Use CRLF line endings
Ardis Lu [Mon, 20 Jan 2025 09:27:27 +0000 (01:27 -0800)]
Refactor toCamelCase suite to vectors
Ardis Lu [Mon, 20 Jan 2025 07:44:36 +0000 (23:44 -0800)]
Add note about maxConcurrent for transformation function
Ardis Lu [Mon, 20 Jan 2025 07:40:42 +0000 (23:40 -0800)]
Simplify with early returns
Ardis Lu [Mon, 20 Jan 2025 07:30:06 +0000 (23:30 -0800)]
Rename to csvio
Ardis Lu [Mon, 20 Jan 2025 07:26:04 +0000 (23:26 -0800)]
Use "concurrent" instead of "parallel" to avoid confusion with threads
Ardis Lu [Mon, 20 Jan 2025 07:11:23 +0000 (23:11 -0800)]
Add maxConcurrent option
Ardis Lu [Sun, 19 Jan 2025 08:24:25 +0000 (00:24 -0800)]
Tweak wording
Ardis Lu [Sun, 19 Jan 2025 08:16:39 +0000 (00:16 -0800)]
Expose fix functions as static methods of CSVNormalizer
These functions may be useful to create custom normalizers.
Ardis Lu [Sun, 19 Jan 2025 07:56:42 +0000 (23:56 -0800)]
Add JSDoc type to output rows
Ardis Lu [Sun, 19 Jan 2025 07:52:59 +0000 (23:52 -0800)]
Refactor createCSVDenormalizationStream to CSVDenormalizer
Ardis Lu [Sun, 19 Jan 2025 07:47:34 +0000 (23:47 -0800)]
Refactor createCSVNormalizationStream to CSVNormalizer
Ardis Lu [Sun, 19 Jan 2025 07:35:50 +0000 (23:35 -0800)]
Make transform and flush private methods
Should've been private from the start to match pre-refactor.
Ardis Lu [Sun, 19 Jan 2025 07:25:37 +0000 (23:25 -0800)]
Move arrayToCSVString to be a static method of CSVWriter
This function is useful as part of the external API to make it easier to generate raw outputs for low-level transformations.
Ardis Lu [Sun, 19 Jan 2025 07:17:16 +0000 (23:17 -0800)]
Refactor createCSVWritableStream to CSVWriter
Ardis Lu [Sun, 19 Jan 2025 06:55:36 +0000 (22:55 -0800)]
Refactor createCSVReadableStream to CSVReader
Ardis Lu [Sun, 19 Jan 2025 06:24:33 +0000 (22:24 -0800)]
Populate JSDoc descriptions for TransformationFunction
Ardis Lu [Sun, 19 Jan 2025 06:22:29 +0000 (22:22 -0800)]
Refactor createCSVTransformStream to CSVTransformer
Using classes is more aligned with existing stream APIs such as TextEncoder and CompressionStream.
Ardis Lu [Sat, 18 Jan 2025 07:52:29 +0000 (23:52 -0800)]
Add maxBatchSize option
Although this functionality could be implemented by end-users by returning null in the transformation function until a batch is accumulated, baking it into the transform stream simplifies this common use case.
Ardis Lu [Sat, 18 Jan 2025 07:18:03 +0000 (23:18 -0800)]
Split out TransformationInput and TransformationOutput types
Ardis Lu [Sat, 18 Jan 2025 07:12:13 +0000 (23:12 -0800)]
Split out TransformationFunction callback function signature
Ardis Lu [Wed, 15 Jan 2025 08:28:40 +0000 (00:28 -0800)]
Add onError callback option
Ardis Lu [Tue, 14 Jan 2025 07:30:12 +0000 (23:30 -0800)]
Align example to match others
Ardis Lu [Mon, 13 Jan 2025 04:09:40 +0000 (20:09 -0800)]
Simplify and add test cases for parsePathLike
fileURLToPath already throws on unsupported URLs, no need to re-throw.
Ardis Lu [Sun, 12 Jan 2025 08:11:41 +0000 (00:11 -0800)]
Break out parsePathLike utility function
Ardis Lu [Sat, 11 Jan 2025 09:33:42 +0000 (01:33 -0800)]
Make file URL parser more robust
Ardis Lu [Sat, 11 Jan 2025 04:30:24 +0000 (20:30 -0800)]
Tweak wording
Ardis Lu [Fri, 10 Jan 2025 07:13:13 +0000 (23:13 -0800)]
Use PascalCase for type names
Ardis Lu [Fri, 10 Jan 2025 06:11:13 +0000 (22:11 -0800)]
Allow transformation function to return multiple rows
Ardis Lu [Fri, 10 Jan 2025 05:54:10 +0000 (21:54 -0800)]
Add table of contents
Ardis Lu [Fri, 10 Jan 2025 05:51:38 +0000 (21:51 -0800)]
Delete redundant tests
Use examples.test.js for end-to-end tests instead
Ardis Lu [Fri, 10 Jan 2025 05:48:27 +0000 (21:48 -0800)]
Add test suite to test all examples
Ardis Lu [Fri, 10 Jan 2025 05:45:15 +0000 (21:45 -0800)]
Explicitly await for stream close
Otherwise, a race condition occurs when running these examples in test cases
Ardis Lu [Thu, 9 Jan 2025 07:29:35 +0000 (23:29 -0800)]
Use paths relative to JS file location, not execution location
Allows the example files to be run from any directory, not just /examples
Ardis Lu [Thu, 9 Jan 2025 07:27:33 +0000 (23:27 -0800)]
Accept PathLike input for createCSVWritableStream
To be consistent with createCSVReadableStream and also allow paths relative to JS file (vs. execution directory only).
Ardis Lu [Thu, 9 Jan 2025 05:44:24 +0000 (21:44 -0800)]
Add example for returning null in transformation
Ardis Lu [Thu, 9 Jan 2025 05:38:02 +0000 (21:38 -0800)]
Allow transformation function to consume input row without output
Ardis Lu [Thu, 9 Jan 2025 05:16:36 +0000 (21:16 -0800)]
Specify "transformation function" instead of "logic function" to be clear
This page took 0.072098 seconds and 4 git commands to generate.