# LLM model Node

### Overview

The LLM Model Node integrates Large Language Models into your flow, enabling natural language processing, generation, and understanding. It supports multiple model providers, temperature control, and a fallback mechanism to ensure reliability.

Usage cost: 1 credit

### Configuration

#### Settings

1. **Model Selection**
   * Primary Model\*: Select the main LLM model
   * Fallback Model: Optional backup model if primary fails
   * Temperature (0-1): Controls response randomness and creativity
     * Lower values (closer to 0): More focused, deterministic responses
     * Higher values (closer to 1): More creative, varied responses
2. **Prompts**
   * System Prompt: Instructions/context for the model's behavior
   * User Prompt: The main input to be processed
   * Past Message History: Optional chat history for context
3. **Output format**
   * **Output as JSON** (Toggle):
     * **Off (Default):** The node outputs the model's response as a plain string.
     * **On:** Instructs the model to format its response as JSON and attempts to parse the output string. If parsing succeeds, the `response` output port will contain a JSON object/array. If it fails, `response` will contain the original string.

*Note:* JSON structure and parsing success depend heavily on the model's ability to follow instructions. Clearly prompting for JSON format is recommended when this is enabled.

#### Output Ports

* `response` (string): The model's generated response

### Best Practices

1. **Model Selection**
   * Choose models based on your specific needs (cost, speed, capabilities)
   * Always configure a fallback model for critical flows
2. **Temperature Settings**
   * Use lower temperatures (0.1-0.3) for:
     * Factual responses
     * Structured output
     * Consistent results
   * Use higher temperatures (0.6-0.9) for:
     * Creative writing
     * Brainstorming
     * Conversational responses
3. **Prompt Engineering**
   * Keep system prompts clear and specific
   * Use variables in prompts to make them dynamic
   * Include relevant context in the prompt
   * Structure prompts with clear input/output expectations
4. **Message History**
   * Consider memory limitations of the model
   * Clean or truncate long conversation histories
5. **Using JSON Output**
   * Enable 'Output as JSON' when structured data is needed for downstream nodes or the final workflow output.
   * Instruct the model clearly in the prompt (User or System) to output *only* valid JSON. Specifying the exact desired keys and structure enhances reliability (e.g., "Respond ONLY with a valid JSON object containing 'name' (string) and 'items' (array of strings). Do not include any other text.").
   * Lower temperatures often improve the reliability of JSON generation.

### Common Issues

* High temperature settings may lead to inconsistent outputs
* Missing or poorly formatted system prompts can result in unexpected responses
* Token limits may be exceeded with long prompts or chat histories
* Rate limiting may affect response times
* JSON Output Failures: Even when requested, the model might produce invalid JSON, include explanatory text around the JSON, or fail to follow formatting instructions.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.waterflai.ai/studio/studio-builders/flow-components-nodes/llm-model-node.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.