Feedback on Big Data column filtering changes in recent versions

Hi everyone,

I wanted to raise a concern and start a discussion regarding a change in behavior when handling big data or large datasets in newer versions compared to version 5.8.3.

In KNIME 5.8.3, we were able to filter and select only the specific columns we were interested in early on when working with large datasets. However, in versions, this capability appears to have been removed or altered.

For large-scale workflows and big data processing, being able to prune unwanted columns early is critical for performance, memory management, and keeping execution times manageable. Without this direct column filtering capability at the source/reader level, workflows pulling massive datasets have to process unnecessary columns down the line until a separate filter node is hit.

Could the development team shed some light on why this change was made, or may I know if there is an intended workaround or equivalent node configuration in the newer versions to achieve early column selection?

Thanks for your time and for continuously improving the platform!