@radzor/csv-import

Parse, validate, and batch-import CSV files. Supports streaming for large files, schema validation with typed coercion, error row collection, and progress tracking. Works with file uploads, S3 streams, and local paths.

Data & Databasev0.1.0typescriptServercsvimportparsebatchdataetlvalidationstreamingby Radzor

Install

View source on GitHub →

$npx radzor@latest add csv-import

⚠ Constraints: For files over 50MB, always use parseStream() instead of parseBuffer() to avoid memory issues. The schema coercion casts strings to types but does not validate ranges or formats — add your own business logic after parsing. date fields are coerced with new Date() — ensure dates are in ISO 8601 or MM/DD/YYYY format.

Inputs

Name	Type	Default	Description
delimiter	string	,	Field delimiter character. Use '\t' for TSV.
hasHeader	boolean	true	Whether the first row contains column headers.
encoding	string	utf-8	File encoding (utf-8, latin1, utf-16le).
batchSize	number	100	Number of rows to emit per onBatch event and database insert.
skipEmptyRows	boolean	true	Skip rows where all fields are empty.
schema	Record<string, 'string' \| 'number' \| 'boolean' \| 'date'>	—	Column type map for coercion and validation. Keys are column names.

{ "llm": { "constraints": "For files over 50MB, always use parseStream() instead of parseBuffer() to avoid memory issues. The schema coercion casts strings to types but does not validate ranges or formats — add your own business logic after parsing. date fields are coerced with new Date() — ensure dates are in ISO 8601 or MM/DD/YYYY format.", "usageExamples": "llm/examples.md", "integrationPrompt": "llm/integration.md" }, "name": "@radzor/csv-import", "tags": [ "csv", "import", "parse", "batch", "data", "etl", "validation", "streaming" ], "events": [ { "name": "onRow", "payload": { "row": "Record<string, unknown>", "index": "number" }, "description": "Fired for each parsed row." }, { "name": "onBatch", "payload": { "rows": "Record<string, unknown>[]", "batchNumber": "number" }, "description": "Fired when a batch of rows is ready for processing." }, { "name": "onComplete", "payload": { "errorRows": "number", "totalRows": "number", "durationMs": "number" }, "description": "Fired when the entire file has been parsed." }, { "name": "onError", "payload": { "raw": "string", "code": "string", "line": "number", "message": "string" }, "description": "Fired for each row that fails validation or parsing." } ], "inputs": [ { "name": "delimiter", "type": "string", "default": ",", "required": false, "description": "Field delimiter character. Use '\\t' for TSV." }, { "name": "hasHeader", "type": "boolean", "default": true, "required": false, "description": "Whether the first row contains column headers." }, { "name": "encoding", "type": "string", "default": "utf-8", "required": false, "description": "File encoding (utf-8, latin1, utf-16le)." }, { "name": "batchSize", "type": "number", "default": 100, "required": false, "description": "Number of rows to emit per onBatch event and database insert." }, { "name": "skipEmptyRows", "type": "boolean", "default": true, "required": false, "description": "Skip rows where all fields are empty." }, { "name": "schema", "type": "Record<string, 'string' | 'number' | 'boolean' | 'date'>", "required": false, "description": "Column type map for coercion and validation. Keys are column names." } ], "radzor": "1.0.0", "actions": [ { "name": "parseBuffer", "params": [ { "name": "buffer", "type": "Buffer", "description": "CSV file content as Buffer." } ], "returns": "Promise<Record<string, unknown>[]>", "description": "Parse an in-memory CSV buffer and return all rows." }, { "name": "parseStream", "params": [ { "name": "stream", "type": "ReadableStream | NodeJS.ReadableStream", "description": "Readable stream of CSV data." } ], "returns": "Promise<ParseStats>", "description": "Stream-parse a CSV file, emitting onRow and onBatch events progressively." }, { "name": "validate", "params": [ { "name": "rows", "type": "Record<string, unknown>[]", "description": "Pre-parsed rows to validate against the schema." } ], "returns": "ValidationResult", "description": "Validate rows against the configured schema, returning valid rows and error details." }, { "name": "getHeaders", "params": [ { "name": "buffer", "type": "Buffer", "description": "CSV content to inspect." } ], "returns": "Promise<string[]>", "description": "Peek at the first row and return column header names." } ], "outputs": [ { "name": "rows", "type": "Record<string, unknown>[]", "description": "Parsed and validated rows." }, { "name": "parseStats", "type": "ParseStats", "fields": [ { "name": "totalRows", "type": "number" }, { "name": "validRows", "type": "number" }, { "name": "errorRows", "type": "number" }, { "name": "durationMs", "type": "number" } ], "description": "Total rows, valid rows, error rows, and processing duration." } ], "runtime": "server", "version": "0.1.0", "category": "data", "languages": [ "typescript" ], "description": "Parse, validate, and batch-import CSV files. Supports streaming for large files, schema validation with typed coercion, error row collection, and progress tracking. Works with file uploads, S3 streams, and local paths.", "dependencies": { "packages": { "csv-parse": "^5.5.0" } }, "composability": { "connectsTo": [ { "output": "rows", "compatibleWith": [ "@radzor/background-job.action.enqueue.payload" ] }, { "output": "rows", "description": "Feed imported CSV data to embeddings, search, LLM analysis, or re-export.", "compatibleWith": [ "@radzor/embeddings-store.action.add.text", "@radzor/search-index.action.index.documents", "@radzor/llm-completion.action.complete.prompt", "@radzor/csv-export.action.generate.data" ] } ] } }

@radzor/csv-import

Install

Inputs

Outputs

Actions

Events

Composability

radzor.manifest.json

Version History

Name	Type	Description
rows	Record<string, unknown>[]	Parsed and validated rows.
parseStats	ParseStats	Total rows, valid rows, error rows, and processing duration. .totalRowsnumber .validRowsnumber .errorRowsnumber .durationMsnumber