SKILL.md
---
name: Workflow Automation
description: Master workflow design, n8n patterns, automation triggers, error handling, and monitoring for reliable business process automation
metadata:
author: cosmicstack-labs
version: 1.0.0
category: automation
tags:
- workflow-automation
- n8n
- triggers
- error-handling
- business-process
- orchestration
- monitoring
---
# Workflow Automation
## Core Principles
### 1. Reliability First
Every workflow must handle failure gracefully. Automations should degrade predictably, not silently. A workflow that fails noisily is better than one that fails silently — at least you know to fix it.
### 2. Idempotency
Design every action to be safe to run multiple times. If a workflow retries mid-way, running the same step twice should produce the same result as running it once. This is the single most important property for building reliable automations.
### 3. Observability
You cannot improve what you cannot see. Every workflow must log key events, expose execution traces, and alert on failures. Treat your workflows as production systems — because they are.
### 4. Loose Coupling
Workflow steps should communicate through well-defined interfaces (files, databases, APIs), not through shared mutable state. This allows steps to be replaced, tested, and scaled independently.
### 5. Fail Fast, Recover Gracefully
Validate inputs immediately at the start of a workflow. Catch known error conditions early. For unexpected errors, have a fallback path — don't let a single failure cascade through the entire system.
## Automation Maturity Model
| Level | Name | Description |
|-------|------|-------------|
| 0 | Ad-hoc | Manual processes, no automation. Everything done by hand. |
| 1 | Basic | Simple single-step automations. No error handling. Manual retries. |
| 2 | Structured | Multi-step workflows with basic error handling. Logs exist but aren't monitored. |
| 3 | Reliable | Idempotent steps, retry with backoff, dead letter queues. Alerts on failure. |
| 4 | Observable | Full execution tracing, dashboards, performance metrics. Proactive alerting. |
| 5 | Self-healing | Automatic rollback, compensating transactions, adaptive error handling. |
**Target at least Level 3** for any workflow that touches production data.
---
## Workflow Design Patterns
### Triggers → Actions → Error Handling
Every workflow follows a three-phase structure:
```javascript
// Conceptual workflow structure
Phase 1: TRIGGER — webhook receives event, cron fires, form submitted
Phase 2: ACTION — transform data, call APIs, update databases, send notifications
Phase 3: HANDLE — on success: log, confirm. On error: retry, notify, dead-letter
```
**Pattern: Guard Clause at Entry**
Before executing any actions, validate that you have everything you need:
```javascript
// n8n pseudocode
if (!input.payload.email) {
throw new Error('Missing required field: email');
// This routes to error handler, not the success path
}
```
### Idempotency
Make every operation idempotent by design:
- **Create operations**: Check if a record already exists before creating. Use upsert patterns.
- **API calls**: Include an idempotency key in headers. If the call retries, the server recognizes the duplicate.
- **File operations**: Use atomic writes — write to a temp file, then rename into place.
```javascript
// Idempotent webhook handler pattern
// n8n: Before creating a record, search for duplicates
const existing = await searchDatabase({ email: $json.email });
if (existing) {
// Update existing record instead of creating duplicate
return { id: existing.id, action: 'updated' };
}
return { id: await createRecord($json), action: 'created' };
```
### Fan-Out / Fan-In
Distribute work across parallel paths, then aggregate results:
- **Fan-Out**: Split a batch of items into individual items, process each in parallel.
- **Fan-In**: Collect results from parallel branches, merge, and proceed.
```javascript
// n8n pattern: Loop Over Items node
// Input: [{ id: 1, name: 'Alice' }, { id: 2, name: 'Bob' }]
// Each item is processed independently by subsequent nodes
// Results merge back automatically in n8n's item-based execution model
// For explicit fan-in with aggregation:
const results = $input.all();
const summary = {
total: results.length,
succeeded: results.filter(r => r.success).length,
failed: results.filter(r => !r.success).length
};
```
**Best Practice**: Set concurrency limits on fan-outs. Don't fire 10,000 requests at once — use batch sizes of 10–50.
### Saga Pattern for Distributed Workflows
For operations spanning multiple systems, use the Saga pattern to maintain data consistency:
**Choreography-based Saga**: Each service publishes events that trigger the next step. If a step fails, each previous step runs a compensating action.
```javascript
// Saga transaction example
Steps:
1. Order Service: Reserve inventory → publish "InventoryReserved"
2. Payment Service: Charge customer → publish "PaymentCharged"
3. Shipping Service: Create shipment → publish "ShipmentCreated"
4. Notification: Send confirmation → complete
// On failure at step 2:
// → Trigger compensating transactions:
// - Payment service: Void charge
// - Order service: Release inventory reservation
```
**Orchestration-based Saga**: A central coordinator (your n8n workflow) calls each service and manages compensation.
```javascript
// n8n orchestration pattern
try {
await reserveInventory(orderId, items);
await chargeCustomer(orderId, amount);
await createShipment(orderId);
} catch (error) {
// Compensating transactions in reverse order
await voidShipment(orderId); // if created
await refundCustomer(orderId); // if charged
await releaseInventory(orderId); // if reserved
throw error; // Re-raise after cleanup
}
```
**Rule**: Compensating transactions must themselves be idempotent and reliable.
---
## n8n-Specific Patterns
### Webhook Triggers
```javascript
// n8n Webhook node — receive and respond
Configuration:
- HTTP Method: POST
- Path: /orders/new
- Response: Respond to Webhook node
// Best practice: Always validate webhook signatures
function verifyWebhookSignature(payload, signature, secret) {
const crypto = require('crypto');
const expected = crypto
.createHmac('sha256', secret)
.update(JSON.stringify(payload))
.digest('hex');
return crypto.timingSafeEqual(Buffer.from(signature), Buffer.from(expected));
}
```
**Production Checklist for Webhooks**:
- Respond quickly (under 10s) — return `202 Accepted` and process async
- Validate signatures to prevent replay attacks
- Log the raw payload before any transformation
- Send an error workflow as the response if validation fails
### Cron Schedules
```
// Cron expression examples in n8n Schedule node
Every hour: 0 * * * *
Daily at 9 AM: 0 9 * * *
Weekdays only: 0 9 * * 1-5
Every 15 min: */15 * * * *
First of month: 0 9 1 * *
```
**Best Practice**: Add a 5-minute buffer for time-sensitive jobs. Use timezone-aware scheduling. Avoid "every minute" in production.
### Error Workflows
n8n supports dedicated error workflows that execute when a regular workflow fails:
```javascript
// Error workflow receives:
// {
// _error: { message, description, timestamp, workflowId, executionId },
// input_data: { ... } // snapshot of input when error occurred
// }
// Error workflow actions:
// 1. Log to monitoring system
// 2. Send notification (Slack, Email, PagerDuty)
// 3. Write to dead letter queue
// 4. Optionally: attempt recovery or rollback
```
**Every workflow should have an error workflow assigned.**
### Sub-Workflows
Break large workflows into reusable sub-workflows:
```javascript
// Main workflow calls sub-workflow
// Sub-workflow ("Send Notification") receives inputs and returns outputs
// Benefits: reusable, testable in isolation, cleaner main flow
// Sub-workflow pattern:
// Input: { to: string, subject: string, body: string }
// Process: validate → format → send → log
// Output: { sent: boolean, messageId: string, timestamp: string }
```
**Sub-workflow guidelines**:
- Keep sub-workflows focused on one thing
- Document inputs and outputs clearly
- Version your sub-workflows (n8n supports versioning)
- Test sub-workflows independently before using them in production
### Binary Data Handling
When working with files, images, or attachments:
```javascript
// n8n binary data pattern
// Read File node or HTTP Request node (response format: file)
// Process with: Extract from File, Spreadsheet File, etc.
// Binary data considerations:
// - Memory: Large files (>50MB) can cause OOM errors
// - Use temp storage for large payloads
// - Stream where possible instead of loading into memory
// - Clean up temp files after processing
// Example: Process uploaded CSV
const items = $input.all();
for (const item of items) {
const binaryData = item.binary?.file;
if (!binaryData) continue;
// binaryData.data is already available as a buffer
const rows = parseCSV(binaryData.data.toString());
// ... process rows
}
```
---
## Trigger Types
### Webhooks
- Real-time, event-driven triggers
- Requires public endpoint (use ngrok for testing)
- Supports GET, POST, PUT, PATCH, DELETE
- Can respond immediately or defer processing
```javascript
// Webhook trigger checklist
□ Public endpoint accessible
□ SSL/TLS enabled (HTTPS)
□ Authentication configured (API key, Basic Auth, JWT)
□ Payload validation in place
□ Response configured (200, 202, or custom)
□ Error workflow assigned
□ Rate limiting considered
```
### Schedules (Cron)
- Time-based triggers for batch processing
- Use cron expressions for flexibility
- Consider timezone implications
- Avoid scheduling many workflows at the same minute (thundering herd)
### Event-Driven (Polling)
- Check external systems for changes at intervals
- Use state tracking to only process new/updated items
- Store last-checked timestamp for incremental processing
```javascript
// Polling pattern: incremental fetch
const lastChecked = await getLastCheckedTimestamp();
const newItems = await fetchChangesSince(lastChecked);
await setLastCheckedTimestamp(Date.now());
return newItems;
```
### Form Submissions
- n8n forms provide built-in UI for data collection
- Supports validation, file uploads, conditional fields
- Results flow directly into workflow execution
### Queue-Based Triggers
- Process items from message queues (RabbitMQ, SQS, Redis)
- Supports parallel processing and back-pressure
- Ideal for handling uneven workloads
---
## Error Handling
### Retries with Backoff
```javascript
// Exponential backoff configuration
// Retry 1: wait 1s
// Retry 2: wait 2s
// Retry 3: wait 4s
// Retry 4: wait 8s
// Retry 5: wait 16s (cap here)
// n8n Error Trigger settings:
// - Retry on failure: YES
// - Max retries: 3-5
// - Wait between retries: exponential
// - Error workflow: [your error workflow]
// Custom backoff in code:
function shouldRetry(attempt, error) {
if (attempt >= 5) return false; // max attempts
if (error.status >= 400 && error.status < 500) return false; // client errors don't retry
return true; // server errors and network issues → retry
}
```
**Retry Policy Rules**:
- Don't retry 4xx errors (client mistakes won't fix themselves)
- Do retry 5xx errors, rate limits (429), and network timeouts
- Add jitter to prevent thundering herd on retries
- Cap maximum retries (5 is a good default)
### Dead Letter Queues
Items that fail all retry attempts go to a dead letter queue (DLQ):
```javascript
// n8n DLQ pattern using Error Workflow
// Error workflow writes to:
// 1. A spreadsheet or database table marked as "failed"
// 2. A dedicated Slack channel
// 3. An SQS/S3 dead letter bucket
// DLQ record structure:
{
originalPayload: { ... },
error: { message: "...", stack: "..." },
attempts: 5,
timestamp: "2025-01-15T10:30:00Z",
workflowId: "123",
executionId: "456"
}
```
**DLQ Best Practices**:
- Review DLQ contents regularly (daily minimum)
- Set up alerts when DLQ grows beyond threshold
- Build a replay mechanism to reprocess DLQ items after fixes
- Never lose items — the DLQ is a safety net, not a trash bin
### Error Notifications
Multi-channel alerting ensures someone is always notified:
```javascript
// Error notification channels (ordered by severity)
// Critical (production data loss risk):
// - PagerDuty/OpsGenie
// - SMS
// - Phone call
// Warning (retryable or non-critical):
// - Slack/Teams channel
// - Email to team
// - Ticket in help desk system
// Informational (non-urgent):
// - Log to monitoring dashboard
// - Daily digest email
```
### Rollback Strategies
When a workflow fails partway through, you need to undo partial work:
| Strategy | When to Use | Example |
|----------|-------------|---------|
| Compensating Transaction | Saga pattern, distributed systems | Refund payment after shipping fails |
| State Restoration | Idempotent operations | Restore original field value |
| Skip & Log | Non-critical side effects | Failed notification? Log and continue |
| Manual Intervention | Complex or risky rollback | Financial reconciliation |
---
## Monitoring and Logging
### Workflow Execution History
Every workflow execution generates a record. Use these for debugging:
```javascript
// Key metrics to track per workflow:
{
executionId: "uuid",
workflowId: "uuid",
workflowName: "Order Processing",
status: "success" | "error" | "running" | "waiting",
startedAt: "2025-01-15T10:00:00Z",
finishedAt: "2025-01-15T10:00:03Z",
duration: 3042, // ms
nodeExecutions: [
{ node: "Webhook", duration: 120, status: "success" },
{ node: "HTTP Request", duration: 2800, status: "success" },
{ node: "Send Email", duration: 122, status: "success" }
],
error: null
}
```
### Alerting Configuration
Set up alerts for these conditions:
```javascript
// Alert thresholds
□ Workflow failure rate > 1% in 5-minute window
□ Any workflow stuck in "running" state for > 30 minutes
□ Execution count drops to 0 (trigger may be down)
□ Average duration increases by > 2x from baseline
□ Dead letter queue size exceeds threshold
```
### Dashboard
Build a monitoring dashboard with these panels:
1. **Workflow Health**: Green/red status per workflow, last 24h
2. **Failure Rate**: Percentage of failed executions over time
3. **Execution Volume**: Count of executions per workflow
4. **Latency**: P50, P95, P99 execution duration
5. **DLQ Size**: Number of items waiting in dead letter queues
6. **Error Breakdown**: Most common error messages
---
## Security Considerations
### Credential Management
```javascript
// DO NOT hardcode credentials
// ❌ Bad:
const apiKey = 'sk-live-abc123';
// ✅ Good: Use n8n credentials system
// Store in: Settings → Credentials
// Reference by name in nodes
// n8n handles encryption at rest and in transit
// ✅ Good: Use environment variables for deployment-specific values
// process.env.SLACK_WEBHOOK_URL
```
**Credential Rules**:
- Rotate keys every 90 days minimum
- Use separate credentials for dev/staging/production
- Never log or expose credential values in error messages
- Use OAuth where available instead of API keys
- Restrict credential access by user role in n8n
### Data Privacy
```javascript
// PII handling in workflows
// 1. Mask or redact sensitive fields in logs
const safeLog = { ...payload, password: '***', ssn: '***-***-1234' };
// 2. Use field-level encryption for sensitive data
const encrypted = encrypt(payload.ssn, encryptionKey);
// 3. Set data retention policies
// - Delete workflow execution data after 30 days
// - Never store raw PII in error logs
// 4. Be GDPR/CCPA aware
// - Support data deletion requests
// - Document what data flows through each workflow
// - Get consent before processing personal data
```
**Security Checklist**:
```
□ All webhooks use HTTPS
□ Credentials stored in n8n credential store (not in code)
□ Error messages don't leak sensitive data
□ Input validation prevents injection attacks
□ Rate limiting on exposed endpoints
□ Audit logging for sensitive operations
□ Regular credential rotation scheduled
□ Network segmentation (don't expose n8n to the internet directly)
```
---
## Common Mistakes
### 1. No Error Workflow
**Problem**: Workflow fails silently. No one knows until a customer complains.
**Fix**: Every workflow must have an error workflow assigned. This is non-negotiable.
### 2. Missing Input Validation
**Problem**: A malformed payload causes cryptic errors deep in the workflow.
**Fix**: Validate inputs in the first node after the trigger. Fail fast with clear messages.
### 3. Not Handling Rate Limits
**Problem**: Workflow calls an API 100 times per second and gets throttled.
**Fix**: Add rate limiting awareness — check `Retry-After` headers, add delays between calls.
### 4. Tight Coupling Between Steps
**Problem**: Changing one API requires updating every workflow that calls it.
**Fix**: Use sub-workflows for shared logic. One change propagates automatically.
### 5. Ignoring Idempotency
**Problem**: A retry creates duplicate records, sends duplicate emails, or charges twice.
**Fix**: Design every mutation to be safe to run multiple times.
### 6. Overly Complex Workflows
**Problem**: A single workflow has 50+ nodes and is impossible to debug.
**Fix**: Break into sub-workflows (10-15 nodes max per workflow). Each sub-workflow does one thing well.
### 7. No Monitoring
**Problem**: Workflow has been silently failing for 3 days, processing zero items.
**Fix**: Set up basic health monitoring — failure alerts, execution count tracking, duration metrics.
### 8. Hardcoded Configuration
**Problem**: API URLs, email addresses, and thresholds embedded in code.
**Fix**: Use environment variables, n8n settings, or a configuration workflow.
### 9. Binary Data Memory Issues
**Problem**: Processing large files (100MB+) causes out-of-memory errors.
**Fix**: Stream data, use external processing services, or implement chunking.
### 10. Testing Only Happy Path
**Problem**: Workflow works perfectly in tests with perfect data, fails on the first real payload.
**Fix**: Test with invalid data, missing fields, slow APIs, timeouts, and duplicate inputs.