How to Automatically Refresh Data in KNIME Workflow with CSV/Excel/Table Readers

Hello KNIME community,

I am currently working on a KNIME workflow where I need to continuously read data from external files (such as CSV, Excel, or tables) that are being updated iteratively. Essentially, data is added to the same file or table after each iteration, and I need to automatically refresh the data being read without manually reloading the file every time new data is appended.

The issue I’m facing is that the nodes like CSV Reader, Excel Reader, or Table Reader do not seem to automatically refresh the data when new rows are added. Currently, the only way I can refresh the data is by manually reloading the table, which is not very practical for continuous workflows.

To give you a specific example, I’m working on a process that scrapes job listings from different pages and appends new URLs to a table (let’s say from pages 2, 3, 4, etc.). After each iteration, new data is added to the file, and I need to update my workflow to pick up these new rows automatically.

Has anyone here encountered this issue and found a way to automate the refresh of these data reading nodes so that they can handle data that is being added in real-time during each iteration?

Any insights or solutions would be greatly appreciated!

Thanks in advance!