Job Finder Chatbot : Replacing OpenAI with GPT4All

Hi everyone,

I’m currently working on reproducing the KNIME workflow from the blog post, how-to-build-custom-ai-powered-chatbot

but with a key constraint: I want to use only open-source models, without relying on OpenAI services.

Let me summarize my progress and the problem I’m facing:

Step 1 – Vector Store Creation with Open-Source Embeddings

I was able to import a dataset of job offers, and extract embeddings of the job descriptions using the GPT4All Embeddings Connector node with a local embedding model. I tested several models, and some didn’t work for embedding generation, but eventually I managed to generate valid embeddings and create a FAISS vector store using the FAISS Vector Store Creator node.

Step 2 – Creating the Agent with Open-Source Chat Model (Problem)

This is where I’m stuck.

According to the workflow, the Agent Prompter node must be connected to an agent created by the OpenAI Functions Agent Creator node. However, I can’t use this node with a GPT4All chat model — it only supports OpenAI nodes.

I tried using the GPT4All Chat Model Connector with the Agent Prompter node, but the nodes are not compatible.

The provided GPT4All Chat Model is incompatible with the expected type Agent. Connect one of the following nodes: OpenAI Functions Agent Creator.

What I’m trying to achieve:

  • A local conversational agent (e.g., with GPT4All) that queries the FAISS vector store and returns relevant job suggestions to the user.

  • A user-friendly UI (eventually) to interact with the chatbot.

My questions:

  • Is there any way to build a working conversational retrieval agent in KNIME using GPT4All instead of OpenAI models?

  • Are there alternative nodes or techniques to simulate agent behavior (like query + generation with context) with open-source chat models in KNIME?

Thanks a lot for your help!

I’d love to complete this without OpenAI and showcase what can be done with entirely local open-source models in KNIME.

Let me know if you’d like me to share the workflow file.

Best regards,