Secrets Management

Securely store and manage API keys, credentials, and sensitive configuration for your pipelines.

FlowMason provides secure secrets management for storing API keys, credentials, and sensitive configuration that your pipelines need at runtime.

Overview

Secrets Management provides:

• Encrypted Storage - Secrets encrypted at rest
• Access Control - Scoped access per pipeline or org
• Rotation Support - Rotate secrets without pipeline changes
• Audit Logging - Track all secret access
• Environment Isolation - Separate secrets per environment
• Reference Syntax - Easy secret references in configs

Quick Start

Create a Secret

POST /api/v1/secrets
Content-Type: application/json
Authorization: Bearer your-api-key

{
  "name": "openai_api_key",
  "value": "sk-...",
  "description": "OpenAI API key for content generation",
  "scope": "org",
  "metadata": {
    "provider": "openai",
    "environment": "production"
  }
}

Use in Pipeline

Reference secrets in your pipeline configuration:

{
  "stages": [
    {
      "id": "generator",
      "component_type": "generator",
      "config": {
        "api_key": "{{secrets.openai_api_key}}",
        "model": "gpt-4"
      }
    }
  ]
}

CLI Commands

Create Secret

fm secrets set openai_api_key "sk-..." --description "OpenAI API key"

List Secrets

fm secrets list

NAME              SCOPE    CREATED           LAST_ROTATED
openai_api_key    org      2024-01-15        2024-01-15
anthropic_key     org      2024-01-10        2024-01-14
webhook_token     pipeline 2024-01-12        -

Get Secret Metadata

fm secrets get openai_api_key

name: openai_api_key
scope: org
created_at: 2024-01-15T10:00:00Z
created_by: [email protected]
last_rotated: 2024-01-15T10:00:00Z
description: OpenAI API key for content generation
metadata:
  provider: openai
  environment: production

Rotate Secret

fm secrets rotate openai_api_key "sk-new-key..."

Delete Secret

fm secrets delete openai_api_key

Secret Scopes

Create Pipeline-Scoped Secret

POST /api/v1/secrets
Content-Type: application/json

{
  "name": "webhook_token",
  "value": "secret-token",
  "scope": "pipeline",
  "pipeline_id": "my-pipeline-id"
}

Reference Syntax

Use the {{secrets.name}} syntax in configurations:

{
  "config": {
    "api_key": "{{secrets.openai_api_key}}",
    "webhook_url": "https://api.example.com/hook?token={{secrets.webhook_token}}",
    "database_url": "{{secrets.database_connection_string}}"
  }
}

In HTTP Request Headers

{
  "id": "api_call",
  "component_type": "http_request",
  "config": {
    "url": "https://api.example.com/data",
    "headers": {
      "Authorization": "Bearer {{secrets.api_token}}",
      "X-API-Key": "{{secrets.service_key}}"
    }
  }
}

Secret Rotation

Manual Rotation

fm secrets rotate my_secret "new-value"

API Rotation

POST /api/v1/secrets/my_secret/rotate
Content-Type: application/json

{
  "new_value": "new-secret-value",
  "reason": "Scheduled rotation"
}

Rotation History

GET /api/v1/secrets/my_secret/history

[
  {
    "rotated_at": "2024-01-15T10:00:00Z",
    "rotated_by": "[email protected]",
    "reason": "Scheduled rotation"
  },
  {
    "rotated_at": "2024-01-01T10:00:00Z",
    "rotated_by": "[email protected]",
    "reason": "Initial creation"
  }
]

Audit Logging

All secret access is logged:

GET /api/v1/secrets/audit?secret_name=openai_api_key&days=7

[
  {
    "timestamp": "2024-01-15T12:30:00Z",
    "action": "access",
    "secret_name": "openai_api_key",
    "pipeline_id": "content-generator",
    "run_id": "run_abc123",
    "ip_address": "10.0.0.1"
  },
  {
    "timestamp": "2024-01-15T10:00:00Z",
    "action": "rotate",
    "secret_name": "openai_api_key",
    "user": "[email protected]"
  }
]

Environment Isolation

Set Environment-Specific Secrets

# Development
fm secrets set db_url "postgres://localhost/dev" --env development

# Staging
fm secrets set db_url "postgres://staging.db/app" --env staging

# Production
fm secrets set db_url "postgres://prod.db/app" --env production

Reference with Environment

{
  "config": {
    "database_url": "{{secrets.db_url}}"
  }
}

The correct secret is automatically used based on the execution environment.

Best Practices

1. Use Descriptive Names

# Good
fm secrets set stripe_live_api_key "sk_live_..."
fm secrets set sendgrid_production_key "SG...."

# Avoid
fm secrets set key1 "..."
fm secrets set api "..."

2. Add Metadata

fm secrets set openai_key "sk-..." \
  --description "OpenAI API key for content generation" \
  --metadata '{"provider": "openai", "tier": "team"}'

3. Rotate Regularly

Set up rotation reminders:

fm secrets set my_key "value" --rotation-days 90

4. Use Minimum Scope

# If only one pipeline needs it, use pipeline scope
fm secrets set webhook_token "token" --scope pipeline --pipeline my-pipeline

5. Never Log Secrets

Secrets are automatically redacted in logs:

[INFO] Using API key: sk-****...****

Python Integration

from flowmason_core.secrets import SecretsManager

secrets = SecretsManager()

# Get a secret value (in execution context)
api_key = await secrets.get("openai_api_key")

# List available secrets (metadata only)
all_secrets = await secrets.list()

# Check if secret exists
exists = await secrets.exists("my_secret")

In Custom Components

@node(name="custom_api_call")
class CustomAPINode:
    async def execute(self, input, context):
        # Access secrets via context
        api_key = await context.secrets.get("my_api_key")

        # Use the secret
        response = await self.call_api(api_key, input.data)
        return self.Output(result=response)

Security Notes

1. Encryption - All secrets encrypted with AES-256-GCM at rest
2. Transport - TLS required for all secret operations
3. Access - Secrets only decrypted at runtime, never logged
4. Isolation - Secrets isolated per organization
5. Backup - Encrypted backups with separate key management