# LLM model Node

### Overview

The LLM Model Node integrates Large Language Models into your flow, enabling natural language processing, generation, and understanding. It supports multiple model providers, temperature control, and a fallback mechanism to ensure reliability.

Usage cost: 1 credit

### Configuration

#### Settings

1. **Model Selection**
   * Primary Model\*: Select the main LLM model
   * Fallback Model: Optional backup model if primary fails
   * Temperature (0-1): Controls response randomness and creativity
     * Lower values (closer to 0): More focused, deterministic responses
     * Higher values (closer to 1): More creative, varied responses
2. **Prompts**
   * System Prompt: Instructions/context for the model's behavior
   * User Prompt: The main input to be processed
   * Past Message History: Optional chat history for context
3. **Output format**
   * **Output as JSON** (Toggle):
     * **Off (Default):** The node outputs the model's response as a plain string.
     * **On:** Instructs the model to format its response as JSON and attempts to parse the output string. If parsing succeeds, the `response` output port will contain a JSON object/array. If it fails, `response` will contain the original string.

*Note:* JSON structure and parsing success depend heavily on the model's ability to follow instructions. Clearly prompting for JSON format is recommended when this is enabled.

#### Output Ports

* `response` (string): The model's generated response

### Best Practices

1. **Model Selection**
   * Choose models based on your specific needs (cost, speed, capabilities)
   * Always configure a fallback model for critical flows
2. **Temperature Settings**
   * Use lower temperatures (0.1-0.3) for:
     * Factual responses
     * Structured output
     * Consistent results
   * Use higher temperatures (0.6-0.9) for:
     * Creative writing
     * Brainstorming
     * Conversational responses
3. **Prompt Engineering**
   * Keep system prompts clear and specific
   * Use variables in prompts to make them dynamic
   * Include relevant context in the prompt
   * Structure prompts with clear input/output expectations
4. **Message History**
   * Consider memory limitations of the model
   * Clean or truncate long conversation histories
5. **Using JSON Output**
   * Enable 'Output as JSON' when structured data is needed for downstream nodes or the final workflow output.
   * Instruct the model clearly in the prompt (User or System) to output *only* valid JSON. Specifying the exact desired keys and structure enhances reliability (e.g., "Respond ONLY with a valid JSON object containing 'name' (string) and 'items' (array of strings). Do not include any other text.").
   * Lower temperatures often improve the reliability of JSON generation.

### Common Issues

* High temperature settings may lead to inconsistent outputs
* Missing or poorly formatted system prompts can result in unexpected responses
* Token limits may be exceeded with long prompts or chat histories
* Rate limiting may affect response times
* JSON Output Failures: Even when requested, the model might produce invalid JSON, include explanatory text around the JSON, or fail to follow formatting instructions.