Debugging Pipelines

Debugging is essential for building reliable AI pipelines. FlowMason provides powerful debugging tools integrated with VSCode.

What You’ll Learn

• Set and manage breakpoints
• Step through pipeline execution
• Inspect variables and outputs
• Iterate on prompts during debugging
• Use conditional breakpoints

Prerequisites

• Completed the Building Your First Pipeline tutorial
• VSCode with FlowMason extension installed

Setting Up

Open the content-summarizer pipeline from the previous tutorial:

code pipelines/content-summarizer.pipeline.json

Step 1: Set a Breakpoint

Using the Gutter

Click in the left margin (gutter) next to the “summarize” stage. A red dot appears.

Using Keyboard

1. Place cursor on the stage line
2. Press F9

Using DAG Editor

1. Open the DAG view
2. Right-click on a stage
3. Select “Toggle Breakpoint”

Step 2: Start Debugging

Press F5 to start debugging. Enter your test input:

{"url": "https://example.com/article"}

The pipeline executes until it hits your breakpoint.

Step 3: Inspect Variables

When paused, the Variables panel shows:

Input

{
  "text": "The extracted content from the webpage..."
}

Context

{
  "pipeline": "content-summarizer",
  "execution_id": "abc123",
  "llm_provider": "anthropic"
}

Previous Stages

{
  "fetch": {
    "status": 200,
    "body": {...}
  },
  "extract": {
    "output": "Extracted text..."
  }
}

Step 4: Debug Controls

Press F10 to execute the summarize stage and see its output.

Step 5: Use the Prompt Editor

For AI stages, the Prompt Editor panel lets you iterate on prompts:

1. Find “Prompt Editor” in the sidebar
2. See the rendered prompt with substituted values
3. Edit the prompt text
4. Click “Re-run Stage” to test your changes
5. Compare outputs side-by-side

This is incredibly useful for refining AI behavior without restarting the pipeline.

Step 6: Watch Expressions

Add expressions to monitor specific values:

1. In the Watch panel, click ”+”
2. Add expressions like:
   • stages.fetch.output.status
   • input.url.length
   • stages.extract.output.substring(0, 100)

Step 7: Conditional Breakpoints

Right-click on a breakpoint to add a condition:

stages.fetch.output.status === 404

The breakpoint only triggers when the condition is true.

Example Conditions

// Break on error status
stages.fetch.output.status >= 400

// Break on long content
stages.extract.output.length > 10000

// Break on specific content
stages.extract.output.includes("error")

Step 8: Debug Console

Evaluate expressions while paused:

> stages.fetch.output.headers
{"content-type": "application/json", ...}

> stages.extract.output.split('\n').length
42

> input.url.replace('http:', 'https:')
"https://example.com/article"

Common Debugging Scenarios

API Returns Unexpected Data

1. Set breakpoint after fetch stage
2. Inspect stages.fetch.output
3. Check status codes and response format
4. Adjust extract expression if needed

AI Output Not as Expected

1. Set breakpoint on AI stage
2. Use Prompt Editor to view rendered prompt
3. Iterate on prompt wording
4. Test with “Re-run Stage”

Loop Running Too Long

1. Set conditional breakpoint: context.iteration > 10
2. Check loop items and conditions
3. Add early exit conditions

Intermittent Failures

1. Enable “Break on All Errors” in Debug panel
2. Run multiple times to catch failures
3. Inspect state when error occurs

Best Practices

1. Start with breakpoints at key stages - Don’t try to debug everything at once
2. Use the Prompt Editor - It’s the fastest way to iterate on AI behavior
3. Add watch expressions - Monitor important values automatically
4. Use conditional breakpoints - Skip to interesting cases
5. Check previous stages - Errors often originate upstream

What’s Next?

You’re now equipped to debug any FlowMason pipeline. In the next tutorial, you’ll learn how to write comprehensive tests.

Key concepts covered:

• Breakpoint management
• Debug controls
• Variable inspection
• Prompt iteration
• Conditional breakpoints
• Debug console

Continue to Testing Pipelines.