How does KNIME handle CSV files of ~2GB?
Dear KNIME Support Team,
When I try to read a ~2GB CSV file directly in KNIME 5.4.4, I get the following error:
Execute failed: Memory limit per column exceeded.
Please adapt the according setting.To work around this, I switched to the File Reader node and enabled:
Table specification → Limit data rows scanned = 10,000While the node executes successfully, the actual output only contains 10,000 rows, so the full CSV data is not read.
If I disable the row limit, the memory error persists.
The JVM heap has been increased to -Xmx6144m, and the CSV contains multiple long String columns.
I would appreciate KNIME official support guidance on:
How to handle such a large CSV file without upgrading KNIME?
Recommended usage or configuration for File Reader / CSV Reader nodes?
Reliable methods to ensure full reading of a ~2GB CSV file?
Thank you very much!