SQLite - Bad value for type BigDecimal
Hi everyone,
I am encountering an issue when transferring column types from SQLite to Apache Spark. I utilize the SQLite Connector to import my SQLite db file. Using the DB Table Selector Node, I execute a Custom Query to directly load my final table from the related tables in the database using an SQL statement. In this process, I also carry out an aggregation to reduce the row count, which is crucial for my task. However, when I attempt to transfer the data into Spark using the DB to Spark method, I consistently receive the error message: “Bad value for type BigDecimal.” Despite experimenting with various settings in the Input Type Mapping for Numeric Types within the DB Table Selector Node, the same error persists. Could anyone offer assistance with this problem?
DB type according to SQL statement (via DB Table Selector)
Error message (DB to Spark)
Regards