Row Filter on Database Level to only get incremental load of data

Hi KNIMErs,

I have a workflow where I load a huge amount of data initially from an Oracle DB.

Additionally I want to set up a workflow where I do daily incremental loads of newly added rows.

I have a column called LASTMODIFIEDDATE.

So my idea was to get the max of this column and turn it into a flow variable. Then I use this very flow variable in the DB Row Filter to only send “new rows” to the DB Reader.

Yes, I know, I could basically load the whole DB initial load again and then do filtering in the KNIME table. But I don’t want to do this, because for me it does not make sense to load 100,000s rows of data just to throw 99% of away directly afterwards. I feel that one of the big advantages of KNIME DB nodes is that we can do the data manipulation on DB level (for faster speed and execution).

However, I really don’t know what to put where when it comes to the FlowVariable tab in the DB Row Filter Node.

I have a var called ‘Max(LASTMODIFIEDDATE)’ but where do I put it and where do I tell KNIME which column it should compare to and with which operator.

I am a bit confused :wink: