Skip to content

LLM Nodes (Agent Nodes)

LLM (Large Language Model) nodes, also called Agent nodes, allow you to interact with AI models like GPT-5.1, Claude, or other language models within your workflows. These nodes support streaming output, tool usage, image attachments, and chat history.

Configuration

Model Selection

  • Model: Choose from available models (e.g., openai/gpt-5.1, anthropic/claude-3-opus). The model list is loaded from the pi SDK and OpenRouter.
  • Default Model: If no model is selected, the default is openai/gpt-5.1 (GPT-5.1).
  • Model Resolution: The selected model ID is normalized via the pi SDK's resolveModel function. If resolution fails, the raw ID is used.

Prompt (Content)

  • Prompt: The main user prompt that is sent to the LLM. This is the content field of the node.
  • Template Variables: Use ${1}, ${2} etc. for positional inputs from parent nodes. The prompt is processed with replaceIndexedInputs using outputs from connected nodes.
  • Empty Prompt: If the prompt is empty after processing, the node will throw an error.

System Prompt Override

  • Optional: You can override the default system prompt for this node.
  • Default: If not set, the pi SDK's default system prompt is used.
  • Editing: Click "Edit system prompt..." to open a text editor.

Reasoning Effort

  • Values: minimal, low, medium, high
  • Default: medium
  • Effect: Controls the amount of reasoning the model performs. Higher effort may produce more thorough responses but uses more tokens.

Tools (Builtin Tools)

  • Tool Selection: You can enable or disable specific builtin tools (e.g., Bash, Read, Edit, Grep, etc.).
  • Default: All tools are enabled by default.
  • Disabling: Tools can be disabled individually or all at once. Disabled tools are not available to the LLM for that node.
  • Important: The Bash tool must be enabled for the dangerouslyAllowAll option to work.

Dangerously Allow All

  • Purpose: Allows the LLM to execute any command without user approval (e.g., Bash commands).
  • Requirement: The Bash tool must be enabled.
  • Safety: When enabled, the node will automatically approve tool calls that would otherwise require approval. Use with extreme caution.

Timeout

  • Default: 300 seconds (5 minutes)
  • Setting: You can set a custom timeout in seconds. 0 means no timeout.
  • Behavior: If the LLM request exceeds the timeout, it will be aborted.

Image Attachments

  • Supported: You can attach images (files or URLs) that will be sent along with the prompt.
  • Usage: Useful for multimodal models that accept image inputs.
  • Adding: Provide a file path (relative to workspace root) or a URL.

Input/Output

Inputs

  • Previous Node Output: The output from connected parent nodes is used as context.
  • Positional Inputs: Use ${1}, ${2} etc. in the prompt to reference inputs by order.
  • Structured Data: Supports any text output from previous nodes.

Outputs

  • Generated Text: The final text response from the LLM (excluding tool calls and thinking blocks).
  • Streaming: The node streams assistant content (text, thinking, tool use, tool results) in real-time to the webview.
  • Thread ID: For chat interactions, a thread ID is maintained to preserve conversation history.
  • Token Usage: Token counts are not directly exposed but can be calculated via the calculate_tokens command.

Variable Usage

Template Variables

Use positional inputs in the prompt:

Analyze the following data: ${1}
Provide a summary in ${2} sentences.

Context Variables

Access workflow context (not yet implemented in the current version).

Node References

Reference previous node outputs by order: - ${1}: Output from the first connected parent node - ${2}: Output from the second connected parent node - ${n}: Output from the n-th connected parent node

Examples

Basic Text Generation

You are a helpful assistant. Write a welcome message for new users.

Data Transformation

Convert the following data to JSON format:
${1}

Code Generation

You are a code assistant. Write a Python function to ${1}.

Analysis

Analyze the following data and provide insights:
${1}

Best Practices

Prompt Engineering

  1. Be Specific: Clear, detailed prompts yield better results.
  2. Provide Examples: Include examples when possible.
  3. Use Delimiters: Separate different parts of your prompt.
  4. Specify Format: Request specific output formats (e.g., JSON, Markdown).
  5. Set Constraints: Define length and style requirements.

Error Handling

  1. Validate Input: Check data before sending to LLM.
  2. Handle Failures: Use condition nodes to detect errors.
  3. Retry Logic: Implement retries for transient failures.
  4. Timeouts: Set appropriate timeout values.

Performance

  1. Batch Prompts: Combine related queries when possible.
  2. Cache Results: Store frequently used responses.
  3. Use Smaller Models: For simple tasks, use faster models.
  4. Optimize Tokens: Be concise to reduce costs.

Security

  1. Never Send Secrets: Don't include API keys or passwords.
  2. Sanitize Input: Clean user data before processing.
  3. Rate Limiting: Prevent abuse with rate limits.
  4. Audit Logging: Log all LLM interactions.

Common Patterns

Chain of Thought

Start → LLM (Reasoning) → LLM (Refinement) → End

Data Extraction

Input → LLM (Extract) → Transform → End

Content Generation

Prompt → LLM (Generate) → Validate → LLM (Edit) → End

Analysis Pipeline

Data → LLM (Analyze) → Condition → LLM (Report) → End

Troubleshooting

Model Not Available

  • Check pi authentication for the selected provider.
  • Verify model name spelling.
  • Ensure the model is supported by the pi SDK or OpenRouter.
  • Check API rate limits.

Poor Output Quality

  • Adjust reasoning effort (lower for consistency).
  • Improve prompt clarity.
  • Add examples to prompt.
  • Use system prompt for constraints.

High Token Usage

  • Reduce reasoning effort.
  • Use more concise prompts.
  • Implement summarization.
  • Cache repeated queries.

Timeout Errors

  • Increase timeout setting.
  • Check network connectivity.
  • Verify API endpoint.
  • Consider model response time.

pi SDK Not Available

  • Ensure project dependencies are installed with npm install.
  • The extension bundles the SDK; if missing, run npm install in the extension directory.

No Authenticated Pi Model

  • Configure pi /login, auth.json, or the selected provider environment variable before launching VS Code.
  • The LLM node requires valid authentication for its selected provider.

Integration with Other Nodes

Before LLM

  • CLI Node: Fetch external data.
  • Condition Node: Validate input.
  • Variable Node: Provide dynamic values.

After LLM

  • Condition Node: Validate output.
  • CLI Node: Use generated content.
  • Loop Node: Process multiple items.
  • Subflow Node: Delegate to a subflow.

Configuration Example

llm_node:
  title: "My Agent"
  model: "openai/gpt-5.1"
  content: "Analyze the following data: ${1}"
  systemPromptTemplate: "You are a data analyst."
  reasoningEffort: "medium"
  disabledTools: ["Bash"]  # Disable Bash tool
  dangerouslyAllowAll: false
  timeoutSec: 300
  attachments:
    - kind: "image"
      source: "file"
      path: "screenshots/error.png"

Next Steps