LLM prompter with local models issues

Hi, I am trying to work with A beginner’s guide to build your own LLM-based solutions | KNIME workflow. I use it as is, but try to change prompts and models. So, what I have.

I downlad several models from GPT4all and have following results:

  • GPT4All Falcon: gpt4all-falcon-newbpe-q4_0.gguf. LLM prompter crashed with: ERROR LLM Prompter 3:35 Execute failed: Error while sending a command.

  • Mistral Open Orca:mistral-7b-openorca.Q4_0.gguf. LLM prompter returns empty strings

  • Orca 2 (Medium):orca-2-7b.Q4_0.gguf. LLM prompter returns something. Some answers are the labels. But some answers looks like attempts to chat.

  • Mistral Instruct: mistral-7b-instruct-v0.1.Q4_0.gguf. It works. For two records I haven’t labels, but for others answers like “Label: negative”. Quality is bad, but it works.

As a result I had some questions. Please, look on it:

  • What’s happened with GPT4All Falcon? Could I do something to make it work?

  • Mistral Open Orca - have I a chance to make it work?

  • Orca 2 (Medium). Could I prepare initial prompt and then, using context, made prompts with task. In the CPT4All application I made this in following way:

Prompt: I want you work as text evaluator. Assign to request a sentiment label. Labels can be one of [“positive”, “negative”, “neutral”]. My request: In a media advisory , the NTSB said that after subsequent testing , `` the train detection system intermittently failed . ‘’

Response: To evaluate this text, I will analyze its sentiment based on the words and context. Step 1: Identify important aspects of the text The media advisory is a formal announcement from an authority (NTSB) The train detection system intermittently failed implies that there was a problem with the safety feature of the trains, which could have serious consequences Subsequent testing indicates that this issue was not resolved easily or quickly Step 2: Analyze sentiment based on aspects The formal announcement suggests that the situation is important and needs to be communicated clearly The intermittently failed train detection system implies a lack of reliability, which could cause concern for passengers and authorities alike Subsequent testing indicates that this issue was not resolved easily or quickly shows frustration and dissatisfaction with the solution Step 3: Assign sentiment label based on analysis Based on my analysis, I would assign a negative sentiment label to this text. The words and context convey a sense of worry, inconvenience, and disappointment. Prompt: Assign sentiment label to: In a media advisory , the NTSB said that after subsequent testing , `` the train detection system intermittently failed . ‘’ Response: Negative

Or, may be, somene can help with prompt?

  • May be someone can help me with Mistral Instruct to improve results? Now I have only 30% accuracy. I use prompt: “I want you to act as text evaluator. Assign to every next prompt a sentiment label. Labels can be one of [“positive”, “negative”, “neutral”]. My prompt:”

At all, I have only one working LLM at this time from four. That’s crash for me.

Thank you!