LLM model Node

Overview

The LLM Model Node integrates Large Language Models into your flow, enabling natural language processing, generation, and understanding. It supports multiple model providers, temperature control, and a fallback mechanism to ensure reliability.

Usage cost: 1 credit

Configuration

Settings

1. Model Selection

   • Primary Model*: Select the main LLM model

   • Fallback Model: Optional backup model if primary fails

   • Temperature (0-1): Controls response randomness and creativity

     • Lower values (closer to 0): More focused, deterministic responses

     • Higher values (closer to 1): More creative, varied responses

2. Prompts

   • System Prompt: Instructions/context for the model's behavior

   • User Prompt: The main input to be processed

   • Past Message History: Optional chat history for context

3. Output format

   • Output as JSON (Toggle):

     • Off (Default): The node outputs the model's response as a plain string.

     • On: Instructs the model to format its response as JSON and attempts to parse the output string. If parsing succeeds, the response output port will contain a JSON object/array. If it fails, response will contain the original string.

Note: JSON structure and parsing success depend heavily on the model's ability to follow instructions. Clearly prompting for JSON format is recommended when this is enabled.

Output Ports

• response (string): The model's generated response

Best Practices

1. Model Selection

   • Choose models based on your specific needs (cost, speed, capabilities)

   • Always configure a fallback model for critical flows

2. Temperature Settings

   • Use lower temperatures (0.1-0.3) for:

     • Factual responses

     • Structured output

     • Consistent results

   • Use higher temperatures (0.6-0.9) for:

     • Creative writing

     • Brainstorming

     • Conversational responses

3. Prompt Engineering

   • Keep system prompts clear and specific

   • Use variables in prompts to make them dynamic

   • Include relevant context in the prompt

   • Structure prompts with clear input/output expectations

4. Message History

   • Consider memory limitations of the model

   • Clean or truncate long conversation histories

5. Using JSON Output

   • Enable 'Output as JSON' when structured data is needed for downstream nodes or the final workflow output.

   • Instruct the model clearly in the prompt (User or System) to output only valid JSON. Specifying the exact desired keys and structure enhances reliability (e.g., "Respond ONLY with a valid JSON object containing 'name' (string) and 'items' (array of strings). Do not include any other text.").

   • Lower temperatures often improve the reliability of JSON generation.

Common Issues

• High temperature settings may lead to inconsistent outputs

• Missing or poorly formatted system prompts can result in unexpected responses

• Token limits may be exceeded with long prompts or chat histories

• Rate limiting may affect response times

• JSON Output Failures: Even when requested, the model might produce invalid JSON, include explanatory text around the JSON, or fail to follow formatting instructions.