ai_agent_workflow.md
# AI Agent Workflow Pattern
**Use Case**: An AI agent with tool access, memory, and reasoning sits inside a larger workflow — trigger feeds it, it decides and acts, output flows on.
> **For agent design depth, use the `n8n-agents` skill.** This file covers where an AI agent sits in a workflow's architecture (trigger → agent → output, the `ai_*` sub-node connection types). The `n8n-agents` skill owns the design rules: tool selection and `$fromAI` parameters, the system-prompt vs tool-description split, structured output with autoFix, memory and sessionId, human-in-the-loop review, RAG, and chat shell/core/sub-agent topologies. Start there when building or debugging an agent.
---
## Pattern Structure
```
Trigger → AI Agent (Model + Tools + Memory + optional Output Parser) → [Process Response] → Output
```
**Key Characteristic**: AI-powered decision making with tool use. From the *workflow* angle, an agent is one node with a main input/output plus `ai_*` sub-node slots — it slots into the same trigger → process → deliver spine as every other pattern.
---
## Core AI Connection Types
Agent workflows wire sub-nodes into the agent with dedicated `ai_*` connection types — **not** the regular `main` connection. This is the single most important architectural fact: a tool wired to `main` is invisible to the agent (and `validate_workflow` flags it as disconnected).
| Connection type | Wires in | Into slot |
|---|---|---|
| `ai_languageModel` | The LLM (OpenAI, Anthropic, Gemini, Ollama…) | model (required) |
| `ai_tool` | Any node the agent can call | tools |
| `ai_memory` | Conversation context store | memory |
| `ai_outputParser` | Structured-output parser | output parser |
| `ai_embedding` | Vector embeddings | RAG chain |
| `ai_vectorStore` | Vector database | RAG chain |
| `ai_document` | Document loaders | RAG ingest |
| `ai_textSplitter` | Text chunking | RAG ingest |
**Wiring direction**: a sub-node connects FROM itself TO the agent, and the connection lives on the sub-node keyed by its `ai_*` type. With `n8n_update_partial_workflow` you add each with an `addConnection` op using `sourceOutput: "ai_tool"` (or `"ai_languageModel"`, etc.). Multiple tools all stack on the same `ai_tool` index 0.
---
## Core Components
The agent has a **main input** (the user message / prompt) and up to four sub-node slots:
1. **Trigger** — Chat Trigger (chat UI/streaming), Webhook (API), Manual (testing), or Schedule (periodic). Feeds the agent's main input.
2. **Language Model** (`ai_languageModel`, required) — the reasoning engine. One chat-model sub-node; a second can be wired as a fallback.
3. **Tools** (`ai_tool`, optional but the whole point) — **ANY node can be a tool.** HTTP Request, a database node, a sub-workflow, Code, or a pre-built tool node connects via the `ai_tool` port and the agent calls it by name.
4. **Memory** (`ai_memory`, optional) — maintains conversation context across turns, keyed by a `sessionKey`.
5. **Output Parser** (`ai_outputParser`, optional) — forces structured JSON instead of free text.
**Critical output fact**: the AI Agent node puts its final answer in **`$json.output`** — not `$json.text` or `$json.response`. Downstream nodes reference `{{ $json.output }}`.
**Fan-out tip**: when several agents run in parallel (e.g. multiple research agents feeding one report), avoid funneling them into a Merge node — Merge `combineAll` does a cross-product and mishandles inputs arriving at different times (often yielding 0 output). Either have each agent deliver its own output directly, or collect same-shaped items with an **Aggregate** node followed by a Code node for formatting.
For the deep slot mechanics — tool types, `$fromAI` parameters, memory configuration, parser schemas — see **n8n-agents**.
---
## Common Use Cases
Short architecture sketches. Each is a trigger → agent → output spine; the agent's sub-nodes are listed under it.
### 1. Conversational Chatbot
```
Webhook (chat message) → AI Agent → Webhook Response
├─ Chat Model (ai_languageModel)
├─ HTTP Request Tool — search knowledge base (ai_tool)
├─ Database node — query orders (ai_tool)
└─ Window Buffer Memory, keyed on session_id (ai_memory)
```
### 2. Document Q&A (RAG)
```
Setup (run once): Read Files → Text Splitter → Embeddings → Vector Store
Query (recurring): Webhook → AI Agent → Webhook Response
├─ Chat Model (ai_languageModel)
├─ Vector Store Tool — search docs (ai_tool)
└─ Buffer Memory (ai_memory)
```
### 3. Data Analysis Assistant
```
Webhook (data question) → AI Agent → Code (chart data) → Webhook Response
├─ Chat Model (ai_languageModel)
├─ Postgres node, read-only user (ai_tool)
└─ Code Tool — analysis (ai_tool)
```
### 4. Workflow Automation Agent
```
Slack (slash command) → AI Agent → Slack (status)
├─ Chat Model (ai_languageModel)
├─ HTTP Request Tool — GitHub API (ai_tool)
├─ HTTP Request Tool — Deploy API (ai_tool)
└─ Postgres node — deployment logs (ai_tool)
```
### 5. Email Processing Agent
```
Email Trigger → AI Agent → Email (auto-response) → Slack (notify team)
├─ Chat Model (ai_languageModel)
├─ Vector Store Tool — similar tickets (ai_tool)
└─ HTTP Request Tool — create Jira ticket (ai_tool)
```
For the *content* of these (tool descriptions, system prompts, schema design), see **n8n-agents** `EXAMPLES.md`.
---
## What the deep design lives in n8n-agents
This file is the workflow-architecture view. The design depth below is owned by **n8n-agents** — go there, don't duplicate it here:
- **Tool configuration** (the four tool types, native vs `.toolWorkflow` vs HTTP Request Tool vs MCP Client, `$fromAI()` anatomy, tool names/descriptions as prompt) → **n8n-agents** `TOOLS.md`, and `SUBWORKFLOW_AS_TOOL.md` for wiring a sub-workflow as a tool.
- **Memory configuration** (buffer/window/postgres/redis, `contextWindowLength`, sessionId handling per trigger) → **n8n-agents** `MEMORY.md`.
- **Agent vs chain vs classifier choice, prompt engineering, system-prompt vs tool-description split** → **n8n-agents** `SYSTEM_PROMPT.md` (and the SKILL.md "Pick the right node" table).
- **RAG chains, structured output, streaming, fallback models** → **n8n-agents** `RAG.md` and `STRUCTURED_OUTPUT.md`.
- **Human review / gating destructive tools** → **n8n-agents** `HUMAN_REVIEW.md`.
- **Error handling** (tool failures, LLM API errors, retries, error workflows) → **n8n-error-handling**, plus the agent-specific notes in **n8n-agents**.
- **Performance, security, testing, common gotchas** → **n8n-agents** (anti-patterns table and quick-reference checklist) for the agent-specific ones; the workflow lifecycle (test → validate → activate) is in this skill's SKILL.md "Workflow lifecycle" section.
One workflow-architecture safety note worth restating here: **any tool that fetches third-party content** (HTTP Request, web search, MCP Client, scrapers) can return attacker-controlled text that reaches the agent's context — indirect prompt injection. If the agent can both *read the internet* AND *take an action the user can't undo*, put a guardrail (human review, read-only scopes) between them. The detail lives in **n8n-agents** `HUMAN_REVIEW.md` and the **n8n-agents** anti-patterns.
---
## Checklist for AI Agent Workflows
Architecture-level checks (the design-level checklist lives in **n8n-agents**):
- [ ] Trigger feeds the agent's **main** input
- [ ] Language model wired via **`ai_languageModel`** (required)
- [ ] Tools wired via **`ai_tool`** ports — NOT `main` (a tool on `main` is disconnected from the agent)
- [ ] Memory wired via **`ai_memory`**, keyed on a stable `sessionKey` from the trigger — when conversation context is needed
- [ ] Output parser wired via **`ai_outputParser`** — when downstream needs strict JSON
- [ ] Downstream nodes read the response from **`{{ $json.output }}`**
- [ ] Parallel agents collected with **Aggregate**, not Merge `combineAll`
- [ ] Validated with `validate_workflow` (confirms sub-nodes sit on `ai_*`, not `main`)
- [ ] Tested and activated per the lifecycle (see SKILL.md "Workflow lifecycle" section)
---
## Summary
**Key Points**:
1. An agent is **one node** with a main input/output plus `ai_*` sub-node slots — it fits the standard trigger → process → deliver spine.
2. **8 AI connection types** — wire the model with `ai_languageModel`, tools with `ai_tool`, memory with `ai_memory`, parsers with `ai_outputParser`. Never `main`.
3. **ANY node can be a tool** — connect it via the `ai_tool` port.
4. The response is in **`$json.output`**.
5. For all design depth — tools, memory, prompts, structured output, RAG, human review, chat topologies — go to **n8n-agents**.
**Pattern**: Trigger → AI Agent (Model + Tools + Memory + optional Parser) → Output
**Related**:
- **n8n-agents** — the deep agent design guide (tools, memory, prompts, structured output, RAG, human review, chat topologies)
- [webhook_processing.md](webhook_processing.md) — receiving chat messages
- [http_api_integration.md](http_api_integration.md) — tools that call APIs
- [database_operations.md](database_operations.md) — database tools for agents
- SKILL.md "Workflow lifecycle" section — test, validate, and activate the workflow
- **n8n-error-handling** — tool-failure and LLM-error handling
database_operations.md
# Database Operations Pattern
**Use Case**: Read, write, sync, and manage database data in workflows.
---
## Pattern Structure
```
Trigger → [Query/Read] → [Transform] → [Write/Update] → [Verify/Log]
```
**Key Characteristic**: Data persistence and synchronization
---
## Core Components
### 1. Trigger
**Options**:
- **Schedule** - Periodic sync/maintenance (most common)
- **Webhook** - Event-driven writes
- **Manual** - One-time operations
### 2. Database Read Nodes
**Supported databases**:
- Postgres
- MySQL
- MongoDB
- Microsoft SQL
- SQLite
- Redis
- And more via community nodes
### 3. Transform
**Purpose**: Map between different database schemas or formats
**Typical nodes**:
- **None** - Map fields in the write node's own column slots
- **Edit Fields (Set)** - A named step when several nodes read the mapped values
- **Merge / Compare Datasets** - Combine sources, diff them for a sync
- **Summarize / Remove Duplicates** - Totals, dedupe
- **Code** - Only the closed list (see `n8n-code-javascript`)
### 4. Database Write Nodes
**Operations**:
- INSERT - Create new records
- UPDATE - Modify existing records
- UPSERT - Insert or update
- DELETE - Remove records
### 5. Verification
**Purpose**: Confirm operations succeeded
**Methods**:
- Query to verify records
- Count rows affected
- Log results
---
## Common Use Cases
### 1. Data Synchronization
**Flow**: Schedule → Read Source DB → Transform → Write Target DB → Log
**Example** (Postgres to MySQL sync):
```
1. Schedule (every 15 minutes)
2. Postgres "Get changed users" (SELECT * FROM users WHERE updated_at > $1; queryReplacement = {{ $json.last_sync }})
3. MySQL (INSERT or UPDATE users; map columns in the node's column slots)
4. Postgres (UPDATE sync_log SET last_sync = NOW(); executeOnce: true)
5. Slack (notify: "Synced {{ $('Get changed users').all().length }} users"; executeOnce: true)
```
No IF for "records exist": with no items, the downstream nodes don't run.
**Incremental sync query**:
```sql
SELECT *
FROM users
WHERE updated_at > $1
ORDER BY updated_at ASC
LIMIT 1000
```
**Parameters**:
```javascript
{
"parameters": [
"={{$node['Get Last Sync'].json.last_sync}}"
]
}
```
### 2. ETL (Extract, Transform, Load)
**Flow**: Extract from multiple sources → Transform → Load into warehouse
**Example** (Consolidate data):
```
1. Schedule (daily at 2 AM)
2. [Parallel branches]
├─ Postgres (SELECT orders)
├─ MySQL (SELECT customers)
└─ MongoDB (SELECT products)
3. Merge (combine by fields: orders.customer_id = customers.id, then a second Merge for products; or one SQL-mode Merge with numberInputs 3)
4. Postgres (warehouse - INSERT into fact_sales; columns mapped in the node)
6. Email (send summary report)
```
### 3. Data Validation & Cleanup
**Flow**: Schedule → Query → Validate → Update/Delete invalid records
**Example** (Clean orphaned records):
```
1. Schedule (weekly)
2. Postgres (SELECT users WHERE email IS NULL OR email = '')
3. IF (invalid records exist)
4. Postgres (UPDATE users SET status='inactive' WHERE email IS NULL)
5. Postgres (DELETE FROM users WHERE created_at < NOW() - INTERVAL '1 year' AND status='inactive')
6. Slack (alert: "Cleaned X invalid records")
```
### 4. Backup & Archive
**Flow**: Schedule → Query → Export → Store
**Example** (Archive old records):
```
1. Schedule (monthly)
2. Postgres (SELECT * FROM orders WHERE created_at < NOW() - INTERVAL '2 years')
3. Convert To File (operation toJson)
4. Read/Write Files from Disk (save to archive.json)
5. Google Drive (upload archive)
6. Postgres (DELETE FROM orders WHERE created_at < NOW() - INTERVAL '2 years')
```
### 5. Real-time Data Updates
**Flow**: Webhook → Parse → Update Database
**Example** (Update user status):
```
1. Webhook (receive status update)
2. Postgres (UPDATE users SET status = {{$json.body.status}} WHERE id = {{$json.body.user_id}})
3. IF (rows affected > 0)
4. Redis (SET user:{{$json.body.user_id}}:status {{$json.body.status}})
5. Webhook Response ({"success": true})
```
---
## Database Node Configuration
### Postgres
#### SELECT Query
```javascript
{
operation: "executeQuery",
query: "SELECT id, name, email FROM users WHERE created_at > $1 LIMIT $2",
parameters: [
"={{$json.since_date}}",
"100"
]
}
```
#### INSERT
```javascript
{
operation: "insert",
table: "users",
columns: "id, name, email, created_at",
values: [
{
id: "={{$json.id}}",
name: "={{$json.name}}",
email: "={{$json.email}}",
created_at: "={{$now}}"
}
]
}
```
#### UPDATE
```javascript
{
operation: "update",
table: "users",
updateKey: "id",
columns: "name, email, updated_at",
values: {
id: "={{$json.id}}",
name: "={{$json.name}}",
email: "={{$json.email}}",
updated_at: "={{$now}}"
}
}
```
#### UPSERT (INSERT ... ON CONFLICT)
```javascript
{
operation: "executeQuery",
query: `
INSERT INTO users (id, name, email)
VALUES ($1, $2, $3)
ON CONFLICT (id)
DO UPDATE SET name = $2, email = $3, updated_at = NOW()
`,
parameters: [
"={{$json.id}}",
"={{$json.name}}",
"={{$json.email}}"
]
}
```
### MySQL
#### SELECT with JOIN
```javascript
{
operation: "executeQuery",
query: `
SELECT u.id, u.name, o.order_id, o.total
FROM users u
LEFT JOIN orders o ON u.id = o.user_id
WHERE u.created_at > ?
`,
parameters: [
"={{$json.since_date}}"
]
}
```
#### Bulk INSERT
```javascript
{
operation: "insert",
table: "orders",
columns: "user_id, total, status",
values: $json.orders // Array of objects
}
```
### MongoDB
#### Find Documents
```javascript
{
operation: "find",
collection: "users",
query: JSON.stringify({
created_at: { $gt: new Date($json.since_date) },
status: "active"
}),
limit: 100
}
```
#### Insert Document
```javascript
{
operation: "insert",
collection: "users",
document: JSON.stringify({
name: $json.name,
email: $json.email,
created_at: new Date()
})
}
```
#### Update Document
```javascript
{
operation: "update",
collection: "users",
query: JSON.stringify({ _id: $json.user_id }),
update: JSON.stringify({
$set: {
status: $json.status,
updated_at: new Date()
}
})
}
```
---
## Batch Processing
### Pattern 1: Split In Batches
**Use when**: Processing large datasets to avoid memory issues
```
Postgres (SELECT 10000 records)
→ Split In Batches (100 items per batch)
→ Transform
→ MySQL (write batch)
→ Loop (until all processed)
```
### Pattern 2: Paginated Queries
**Use when**: Database has millions of records
```
Set (initialize: offset=0, limit=1000)
→ Loop Start
→ Postgres (SELECT * FROM large_table LIMIT {{$json.limit}} OFFSET {{$json.offset}})
→ IF (records returned)
├─ Process records
├─ Set (increment offset by 1000)
└─ Loop back
└─ [No records] → End
```
**Query**:
```sql
SELECT * FROM large_table
ORDER BY id
LIMIT $1 OFFSET $2
```
### Pattern 3: Cursor-Based Pagination
**Better performance for large datasets**:
```
Set (initialize: last_id=0)
→ Loop Start
→ Postgres (SELECT * FROM table WHERE id > {{$json.last_id}} ORDER BY id LIMIT 1000)
→ IF (records returned)
├─ Process records
├─ Summarize (max of id) → the next query's last_id
└─ Loop back
└─ [No records] → End
```
**Query**:
```sql
SELECT * FROM table
WHERE id > $1
ORDER BY id ASC
LIMIT 1000
```
---
## Transaction Handling
### Pattern 1: BEGIN/COMMIT/ROLLBACK
**For databases that support transactions**:
```javascript
// Node 1: Begin Transaction
{
operation: "executeQuery",
query: "BEGIN"
}
// Node 2-N: Your operations
{
operation: "executeQuery",
query: "INSERT INTO ...",
continueOnFail: true
}
// Node N+1: Commit or Rollback
{
operation: "executeQuery",
query: "={{$node['Operation'].json.error ? 'ROLLBACK' : 'COMMIT'}}"
}
```
### Pattern 2: Atomic Operations
**Use database features for atomicity**:
```sql
-- Upsert example (atomic)
INSERT INTO inventory (product_id, quantity)
VALUES ($1, $2)
ON CONFLICT (product_id)
DO UPDATE SET quantity = inventory.quantity + $2
```
### Pattern 3: Error Rollback
**Manual rollback on error**:
```
Try Operations:
Postgres (INSERT orders)
MySQL (INSERT order_items)
Error Trigger:
Postgres (DELETE FROM orders WHERE id = {{$json.order_id}})
MySQL (DELETE FROM order_items WHERE order_id = {{$json.order_id}})
```
---
## Data Transformation
### Schema Mapping, type conversion, aggregation
These are native jobs:
| Job | Native way |
|---|---|
| Rename / map fields | the write node's column slots, each a short expression (`{{ $json.first_name + ' ' + $json.last_name }}`) |
| Type conversion | in the same expression: `{{ $json.user_id.toNumber() }}`, `{{ $json.created_at.toDateTime().toISO() }}`, `{{ $json.metadata.parseJson() }}`, `{{ $json.active === 1 }}` |
| Defaults | `{{ $ifEmpty($json.status, 'active') }}` |
| Totals per day / group | Summarize: split by the date field, `count` and `sum` of `amount` — or `GROUP BY` in the query itself |
| Group by a derived key (the date part of a timestamp) | Edit Fields adds `day = {{ $json.created_at.toDateTime().format('yyyy-MM-dd') }}`, then Summarize splits by `day` |
A Code node is right here only when a mapping needs loops (one source row becomes a variable number of target rows in
ways Split Out can't express) — with a `Why Code:` note.
---
## Performance Optimization
### 1. Use Indexes
Ensure database has proper indexes:
```sql
-- Add index for sync queries
CREATE INDEX idx_users_updated_at ON users(updated_at);
-- Add index for lookups
CREATE INDEX idx_orders_user_id ON orders(user_id);
```
### 2. Limit Result Sets
Always use LIMIT:
```sql
-- ✅ Good
SELECT * FROM large_table
WHERE created_at > $1
LIMIT 1000
-- ❌ Bad (unbounded)
SELECT * FROM large_table
WHERE created_at > $1
```
### 3. Use Prepared Statements
Parameterized queries are faster:
```javascript
// ✅ Good - prepared statement
{
query: "SELECT * FROM users WHERE id = $1",
parameters: ["={{$json.id}}"]
}
// ❌ Bad - string concatenation
{
query: "SELECT * FROM users WHERE id = '={{$json.id}}'"
}
```
### 4. Batch Writes
Write multiple records at once:
```javascript
// ✅ Good - batch insert
{
operation: "insert",
table: "orders",
values: $json.items // Array of 100 items
}
// ❌ Bad - individual inserts in loop
// 100 separate INSERT statements
```
### 5. Connection Pooling
Configure in credentials:
```javascript
{
host: "db.example.com",
database: "mydb",
user: "user",
password: "pass",
// Connection pool settings
min: 2,
max: 10,
idleTimeoutMillis: 30000
}
```
---
## Error Handling
### Pattern 1: Check Rows Affected
```
Database Operation (UPDATE users...)
→ IF ({{$json.rowsAffected === 0}})
└─ Alert: "No rows updated - record not found"
```
### Pattern 2: Constraint Violations
```javascript
// Database operation with continueOnFail: true
{
operation: "insert",
continueOnFail: true
}
// Next node: Check for errors
IF ({{$json.error !== undefined}})
→ IF ({{$json.error.includes('duplicate key')}})
└─ Log: "Record already exists - skipping"
→ ELSE
└─ Alert: "Database error: {{$json.error}}"
```
### Pattern 3: Rollback on Error
```
Try Operations:
→ Database Write 1
→ Database Write 2
→ Database Write 3
Error Trigger:
→ Rollback Operations
→ Alert Admin
```
---
## Security Best Practices
### 1. Use Parameterized Queries (Prevent SQL Injection)
```javascript
// ✅ SAFE - parameterized
{
query: "SELECT * FROM users WHERE email = $1",
parameters: ["={{$json.email}}"]
}
// ❌ DANGEROUS - SQL injection risk
{
query: "SELECT * FROM users WHERE email = '={{$json.email}}'"
}
```
### 2. Least Privilege Access
**Create dedicated workflow user**:
```sql
-- ✅ Good - limited permissions
CREATE USER n8n_workflow WITH PASSWORD 'secure_password';
GRANT SELECT, INSERT, UPDATE ON orders TO n8n_workflow;
GRANT SELECT ON users TO n8n_workflow;
-- ❌ Bad - too much access
GRANT ALL PRIVILEGES TO n8n_workflow;
```
### 3. Validate Input Data
Check before the write with an IF (or Filter) node, one condition per rule — `{{ $json.email.isEmail() }}` is true,
`{{ ($json.name ?? '').trim().length >= 2 }}` is true — and send failures to their own branch. Normalize in the write
node's slots: `{{ $json.email.toLowerCase().trim() }}`. For per-field error messages back to a caller, see the Code
validator exception in `n8n-error-handling`.
### 4. Encrypt Sensitive Data
Use the Crypto node (typeVersion 2), action `encrypt`: the passphrase or key comes from a `crypto` credential, never from
the workflow JSON (`$credentials` doesn't exist inside a Code node). Configure its mode and cipher from `get_node` on the
user's version. Prefer encryption the database provides (pgcrypto, column encryption) when it's available.
---
## Common Gotchas
### 1. ❌ Wrong: Unbounded queries
```sql
SELECT * FROM large_table -- Could return millions
```
### ✅ Correct: Use LIMIT
```sql
SELECT * FROM large_table
ORDER BY created_at DESC
LIMIT 1000
```
### 2. ❌ Wrong: String concatenation in queries
```javascript
query: "SELECT * FROM users WHERE id = '{{$json.id}}'"
```
### ✅ Correct: Parameterized queries
```javascript
query: "SELECT * FROM users WHERE id = $1",
parameters: ["={{$json.id}}"]
```
### 3. ❌ Wrong: No transaction for multi-step operations
```
INSERT into orders
INSERT into order_items // Fails → orphaned order record
```
### ✅ Correct: Use transaction
```
BEGIN
INSERT into orders
INSERT into order_items
COMMIT (or ROLLBACK on error)
```
### 4. ❌ Wrong: Processing all items at once
```
SELECT 1000000 records → Process all → OOM error
```
### ✅ Correct: Batch processing
```
SELECT records → Split In Batches (1000) → Process → Loop
```
### 5. ❌ Wrong: Expecting output items from write operations
```
INSERT INTO table ... → Next node never executes (0 output items)
```
Database write operations (INSERT, UPDATE, DELETE) may return **0 result rows** from the database engine — reliably so for raw query execution (e.g. `executeQuery` with an INSERT), while some database nodes return the affected rows instead.
When 0 rows come back, n8n translates this to **0 output items**, silently breaking any downstream chain.
### ✅ Correct: Set `alwaysOutputData` on write nodes
```
INSERT INTO table ... (alwaysOutputData: true) → Next node executes with 1 empty item
```
Set `alwaysOutputData: true` on any node that executes INSERT, UPDATE, or DELETE.
This ensures at least 1 empty item (`{json: {}}`) flows downstream.
> **Tip:** If downstream nodes need actual data (not the empty passthrough item),
> reference the upstream node directly: `$('DataSource Node').all()`
---
## Real Template Examples
From n8n template library (456 database templates):
**Data Sync**:
```
Schedule → Postgres (SELECT new records) → Transform → MySQL (INSERT)
```
**ETL Pipeline**:
```
Schedule → [Multiple DB reads] → Merge → Transform → Warehouse (INSERT)
```
**Backup**:
```
Schedule → Postgres (SELECT all) → JSON → Google Drive (upload)
```
Use `search_templates({query: "database"})` to find more!
---
## Checklist for Database Workflows
### Planning
- [ ] Identify source and target databases
- [ ] Understand schema differences
- [ ] Plan transformation logic
- [ ] Consider batch size for large datasets
- [ ] Design error handling strategy
### Implementation
- [ ] Use parameterized queries (never concatenate)
- [ ] Add LIMIT to all SELECT queries
- [ ] Use appropriate operation (INSERT/UPDATE/UPSERT)
- [ ] Configure credentials properly
- [ ] Test with small dataset first
### Performance
- [ ] Add database indexes for queries
- [ ] Use batch operations
- [ ] Implement pagination for large datasets
- [ ] Configure connection pooling
- [ ] Monitor query execution times
### Security
- [ ] Use parameterized queries (SQL injection prevention)
- [ ] Least privilege database user
- [ ] Validate and sanitize input
- [ ] Encrypt sensitive data
- [ ] Never log sensitive data
### Reliability
- [ ] Add transaction handling if needed
- [ ] Check rows affected
- [ ] Handle constraint violations
- [ ] Implement retry logic
- [ ] Add Error Trigger workflow
---
## Summary
**Key Points**:
1. **Always use parameterized queries** (prevent SQL injection)
2. **Batch processing** for large datasets
3. **Transaction handling** for multi-step operations
4. **Limit result sets** to avoid memory issues
5. **Validate input data** before writes
**Pattern**: Trigger → Query → Transform → Write → Verify
**Related**:
- [http_api_integration.md](http_api_integration.md) - Fetching data to store in DB
- [scheduled_tasks.md](scheduled_tasks.md) - Periodic database maintenance
http_api_integration.md
# HTTP API Integration Pattern
**Use Case**: Fetch data from REST APIs, transform it, and use it in workflows.
---
## Pattern Structure
```
Trigger → HTTP Request → [Transform] → [Action] → [Error Handler]
```
**Key Characteristic**: External data fetching with error handling
---
## Core Components
### 1. Trigger
**Options**:
- **Schedule** - Periodic fetching (most common)
- **Webhook** - Triggered by external event
- **Manual** - On-demand execution
### 2. HTTP Request Node
**Purpose**: Call external REST APIs
**Configuration**:
```javascript
{
method: "GET", // GET, POST, PUT, DELETE, PATCH
url: "https://api.example.com/users",
authentication: "predefinedCredentialType",
sendQuery: true,
queryParameters: {
"page": "={{$json.page}}",
"limit": "100"
},
sendHeaders: true,
headerParameters: {
"Accept": "application/json",
"X-API-Version": "v1"
}
}
```
### 3. Response Processing
**Purpose**: Extract and transform API response data
**Typical flow**:
```
Service node or HTTP Request → Split Out (when the body wraps an array) → Filter → Action
```
Map fields in the action node's own slots; add an Edit Fields node only when several nodes read the same mapped values.
### 4. Action
**Common actions**:
- Store in database
- Send to another API
- Create notifications
- Update spreadsheet
### 5. Error Handler
**Purpose**: Handle API failures gracefully
**Error Trigger Workflow**:
```
Error Trigger → Log Error → Notify Admin → Retry Logic (optional)
```
---
## Common Use Cases
### 1. Data Fetching & Storage
**Flow**: Schedule → HTTP Request → Transform → Database
**Example** (Fetch GitHub issues):
```
1. Schedule (every hour)
2. GitHub (Repository → Get Issues, Return All; filters: state = open, labels = bug)
3. Postgres (upsert issues; map id, title, created_at in the node's column slots)
```
The service node pages, authenticates and filters by label itself. With HTTP Request instead (only when the GitHub
node lacks the operation — note it as `Why HTTP:`), use the pagination option and a Filter node with the condition
`{{ $json.labels.pluck('name').includes('bug') }}` is true.
### 2. API to API Integration
**Flow**: Trigger → Fetch from API A → Transform → Send to API B
**Example** (Jira to Slack):
```
1. Schedule (every 15 minutes)
2. Jira (Issue → Get Many, Return All; options.jql = `updated >= startOfDay()`)
3. Slack (post the message; the text maps the issue fields with expressions)
```
No IF is needed for "tickets exist": when Jira returns no items, the Slack node doesn't run.
### 3. Data Enrichment
**Flow**: Trigger → Fetch base data → Call enrichment API → Combine → Store
**Example** (Enrich contacts with company data):
```
1. Postgres "Get new contacts" (SELECT new contacts)
2. Clearbit (Company → Enrich; domain = `{{ $json.email.extractDomain() }}`)
3. Postgres (UPDATE contacts; contact fields from `$('Get new contacts').item.json`, company fields from `$json`)
```
The domain is one helper expression, and each enrichment pairs back to its contact through `$('Node').item`, so
neither a Code nor a Set node is needed.
### 4. Monitoring & Alerting
**Flow**: Schedule → Check API health → IF unhealthy → Alert
**Example** (API health check):
```
1. Schedule (every 5 minutes)
2. HTTP Request (GET /health endpoint)
3. IF (status !== 200 OR response time > 2000ms)
4. Slack (alert #ops-team)
5. PagerDuty (create incident)
```
### 5. Batch Processing
**Flow**: Trigger → Fetch large dataset → Split in Batches → Process → Loop
**Example** (Process all users):
```
1. Manual Trigger
2. HTTP Request (GET /api/users?limit=1000)
3. Split In Batches (100 items per batch)
4. HTTP Request (POST /api/process for each batch)
5. Wait (2 seconds between batches - rate limiting)
6. Loop (back to step 4 until all processed)
```
---
## Authentication Methods
### 1. None (Public APIs)
```javascript
{
authentication: "none"
}
```
### 2. Bearer Token (Most Common)
**Setup**: Create credential
```javascript
{
authentication: "predefinedCredentialType",
nodeCredentialType: "httpHeaderAuth",
headerAuth: {
name: "Authorization",
value: "Bearer YOUR_TOKEN"
}
}
```
**Access in workflow**:
```javascript
{
authentication: "predefinedCredentialType",
nodeCredentialType: "httpHeaderAuth"
}
```
### 3. API Key (Header or Query)
**Header auth**:
```javascript
{
sendHeaders: true,
headerParameters: {
"X-API-Key": "={{$credentials.apiKey}}"
}
}
```
**Query auth**:
```javascript
{
sendQuery: true,
queryParameters: {
"api_key": "={{$credentials.apiKey}}"
}
}
```
### 4. Basic Auth
**Setup**: Create "Basic Auth" credential
```javascript
{
authentication: "predefinedCredentialType",
nodeCredentialType: "httpBasicAuth"
}
```
### 5. OAuth2
**Setup**: Create OAuth2 credential with:
- Authorization URL
- Token URL
- Client ID
- Client Secret
- Scopes
```javascript
{
authentication: "predefinedCredentialType",
nodeCredentialType: "oAuth2Api"
}
```
---
## Handling API Responses
### Success Response (200-299)
**Default**: Data flows to next node
**Access response**:
```javascript
// Entire response
{{$json}}
// Specific fields
{{$json.data.id}}
{{$json.results[0].name}}
```
### Pagination
Use HTTP Request's built-in pagination option — not a Set → HTTP → Code → IF loop. It fetches every page in one node
and returns the combined items. (A service node's "Return All" usually pages for you; prefer the service node.)
| API style | `paginationMode` | Next page | Stop when (`completeExpression`) |
|---|---|---|---|
| Page number / offset | `updateAParameterInEachRequest` | `qs` param `page` = `{{ $pageCount + 1 }}` (offset: `{{ $pageCount * 100 }}`) | `{{ !$response.body.has_more }}`, or `responseIsEmpty` |
| Cursor in the body | `updateAParameterInEachRequest` | `qs` param `cursor` = `{{ $response.body.next_cursor }}` | `{{ !$response.body.next_cursor }}` |
| Next URL in the body | `responseContainsNextURL` | `nextURL` = `{{ $response.body.next }}` | `{{ !$response.body.next }}` |
| Link header | `responseContainsNextURL` | `nextURL` = `{{ $response.headers.link?.match(/<([^>]+)>;\s*rel="next"/)?.[1] }}` | `{{ !$response.headers.link?.includes('rel="next"') }}` |
- **Always set `limitPagesFetched: true` and `maxRequests`.** Without a cap, a stop condition that never fires pages
forever.
- A page whose body is an object (`{ "data": [...], "has_more": true }`) arrives as one item per page: add a Split Out on
`data`. An array body is split into items automatically.
- Full JSON for each mode, the variables (`$pageCount`, `$response`, `$request`) and the traps are in
`n8n-node-configuration` → NATIVE_NODES.md → "HTTP Request pagination".
### Error Responses (400-599)
**Configure HTTP Request**:
```javascript
{
continueOnFail: true, // Don't stop workflow on error
ignoreResponseCode: true // Get response even on error
}
```
**Handle errors**:
```
HTTP Request (continueOnFail: true)
→ IF (check error)
├─ [Success Path]
└─ [Error Path] → Log → Retry or Alert
```
**IF condition**:
```javascript
{{$json.error}} is empty
// OR
{{$json.statusCode}} < 400
```
---
## Rate Limiting
### Pattern 1: Wait Between Requests
HTTP Request's batching option throttles without a loop: `options.batching.batch` with `batchSize: 1` and
`batchInterval: 1000` (ms between batches). Use a loop only when several nodes per item must be throttled together:
```
Loop Over Items (batch size = your limit)
→ HTTP Request
→ Wait (1 second)
→ back to Loop Over Items
```
### Pattern 2: Retries and Backoff
Start with the node settings: `retryOnFail: true`, `maxTries: 3`, `waitBetweenTries: 2000` retries with a fixed delay
(see `n8n-error-handling`). When the API needs a growing delay, route the error output (`onError:
"continueErrorOutput"`) to a Wait node whose amount is `{{ 2 ** $runIndex }}` seconds and back into the request, with
an IF that stops after N runs (`{{ $runIndex < 4 }}`).
### Pattern 3: Respect Rate Limit Headers
Turn on the HTTP Request's "Include Response Headers and Status" response option, then:
```
HTTP Request (full response)
→ IF {{ Number($json.headers['x-ratelimit-remaining'] ?? 999) < 10 }}
├─ true → Wait (until {{ DateTime.fromSeconds(Number($json.headers['x-ratelimit-reset'])).toISO() }}) → next request
└─ false → next request
```
---
## Request Configuration
### GET Request
```javascript
{
method: "GET",
url: "https://api.example.com/users",
sendQuery: true,
queryParameters: {
"page": "1",
"limit": "100",
"filter": "active"
}
}
```
### POST Request (JSON Body)
```javascript
{
method: "POST",
url: "https://api.example.com/users",
sendBody: true,
bodyParametersJson: JSON.stringify({
name: "={{$json.name}}",
email: "={{$json.email}}",
role: "user"
})
}
```
### POST Request (Form Data)
```javascript
{
method: "POST",
url: "https://api.example.com/upload",
sendBody: true,
bodyParametersUi: {
parameter: [
{ name: "file", value: "={{$json.fileData}}" },
{ name: "filename", value: "={{$json.filename}}" }
]
},
sendHeaders: true,
headerParameters: {
"Content-Type": "multipart/form-data"
}
}
```
### PUT/PATCH Request (Update)
```javascript
{
method: "PATCH",
url: "https://api.example.com/users/={{$json.userId}}",
sendBody: true,
bodyParametersJson: JSON.stringify({
status: "active",
last_updated: "={{$now}}"
})
}
```
### DELETE Request
```javascript
{
method: "DELETE",
url: "https://api.example.com/users/={{$json.userId}}"
}
```
---
## Error Handling Patterns
### Pattern 1: Retry on Failure
```
HTTP Request (continueOnFail: true)
→ IF (error occurred)
└→ Wait (5 seconds)
└→ HTTP Request (retry)
```
### Pattern 2: Fallback API
```
HTTP Request (Primary API, continueOnFail: true)
→ IF (failed)
└→ HTTP Request (Fallback API)
```
### Pattern 3: Error Trigger Workflow
**Main Workflow**:
```
HTTP Request → Process Data
```
**Error Workflow**:
```
Error Trigger
├─→ Slack (alert team; the message reads the trigger payload with expressions)
└─→ Database (log error for analysis; same fields)
```
### Pattern 4: Circuit Breaker
A failure count that survives between executions is state across runs, one of the closed-list Code cases. Keep the
counter in `$getWorkflowStaticData('global')` and note the node `Why Code: state across runs (failure counter)`.
```javascript
// Code node - circuit breaker logic
const failures = $json.recentFailures || 0;
const threshold = 5;
if (failures >= threshold) {
throw new Error('Circuit breaker open - too many failures');
}
return [{ json: { canProceed: true } }];
```
---
## Response Transformation
### Extract Nested Data
Split Out on the array (`fieldToSplitOut: "data.items"`), then read the nested fields where they're used
(`{{ $json.attributes.contact.email }}`), or in one Edit Fields node when several nodes read them. For one value in one
field, `$jmespath()` does it inline (see `n8n-expression-syntax`).
### Flatten Arrays
Split Out keeps parent fields when you ask it to: `fieldToSplitOut: "results"`, `include: "selectedOtherFields"`,
`fieldsToInclude: "id"` gives one item per result with the parent's `id` alongside. Arrays nested more than one level
deep take one Split Out per level; arbitrary depth (a tree) is a recursion case for Code.
### Combine Multiple API Responses
```
HTTP Request 1 (users)
├─→ HTTP Request 2 (orders for each user)
└─→ Merge input 1
HTTP Request 2 → Merge input 2 (Combine by Fields: users.id = orders.user_id, joinMode enrichInput1)
```
---
## Testing & Debugging
### 1. Test with Manual Trigger
Replace Schedule with Manual Trigger for testing
### 2. Use Postman/Insomnia First
- Test API outside n8n
- Understand response structure
- Verify authentication
### 3. Log Responses
```javascript
// Code node - log for debugging
console.log('API Response:', JSON.stringify($input.first().json, null, 2));
return $input.all();
```
### 4. Check Execution Data
- View node output in n8n UI
- Check headers, body, status code
- Verify data structure
### 5. Use Binary Data Properly
For file downloads:
```javascript
{
method: "GET",
url: "https://api.example.com/download/file.pdf",
responseFormat: "file", // Important for binary data
outputPropertyName: "data"
}
```
---
## Performance Optimization
### 1. Parallel Requests
Each input item becomes one request, and HTTP Request sends a batch of them together. Cap the concurrency with the
batching option (`options.batching.batch.batchSize`, plus `batchInterval` in ms between batches) — no Split In Batches
loop needed. With one item holding an array of IDs, put a Split Out on the array first.
### 2. Caching
```
IF (check cache exists)
├─ [Cache Hit] → Use cached data
└─ [Cache Miss] → HTTP Request → Store in cache
```
### 3. Conditional Fetching
Only fetch if data changed:
```
HTTP Request (GET with If-Modified-Since header)
→ IF (status === 304)
└─ Use existing data
→ IF (status === 200)
└─ Process new data
```
### 4. Batch API Calls
If API supports batch operations:
```javascript
{
method: "POST",
url: "https://api.example.com/batch",
bodyParametersJson: JSON.stringify({
requests: $json.items.map(item => ({
method: "GET",
url: `/users/${item.id}`
}))
})
}
```
---
## Common Gotchas
### 1. ❌ Wrong: Hardcoded URLs
```javascript
url: "https://api.example.com/prod/users"
```
### ✅ Correct: Use environment variables
```javascript
url: "={{$env.API_BASE_URL}}/users"
```
### 2. ❌ Wrong: Credentials in parameters
```javascript
headerParameters: {
"Authorization": "Bearer sk-abc123xyz" // ❌ Exposed!
}
```
### ✅ Correct: Use credentials system
```javascript
authentication: "predefinedCredentialType",
nodeCredentialType: "httpHeaderAuth"
```
### 3. ❌ Wrong: No error handling
```javascript
HTTP Request → Process (fails if API down)
```
### ✅ Correct: Handle errors
```javascript
HTTP Request (continueOnFail: true) → IF (error) → Handle
```
### 4. ❌ Wrong: Blocking on large responses
Processing 10,000 items synchronously
### ✅ Correct: Use batching
```
Split In Batches (100 items) → Process → Loop
```
---
## Real Template Examples
From n8n template library (892 API integration templates):
**GitHub to Notion**:
```
Schedule → HTTP Request (GitHub API) → Transform → HTTP Request (Notion API)
```
**Weather to Slack**:
```
Schedule → HTTP Request (Weather API) → Set (format) → Slack
```
**CRM Sync**:
```
Schedule → HTTP Request (CRM A) → Transform → HTTP Request (CRM B)
```
Use `search_templates({query: "http api"})` to find more!
---
## Checklist for API Integration
### Planning
- [ ] Test API with Postman/curl first
- [ ] Understand response structure
- [ ] Check rate limits
- [ ] Review authentication method
- [ ] Plan error handling
### Implementation
- [ ] Use credentials (never hardcode)
- [ ] Configure proper HTTP method
- [ ] Set correct headers (Content-Type, Accept)
- [ ] Handle pagination if needed
- [ ] Add query parameters properly
### Error Handling
- [ ] Set continueOnFail: true if needed
- [ ] Check response status codes
- [ ] Implement retry logic
- [ ] Add Error Trigger workflow
- [ ] Alert on failures
### Performance
- [ ] Use batching for large datasets
- [ ] Add rate limiting if needed
- [ ] Consider caching
- [ ] Test with production load
### Security
- [ ] Use HTTPS only
- [ ] Store secrets in credentials
- [ ] Validate API responses
- [ ] Use environment variables
---
## Summary
**Key Points**:
1. **Authentication** via credentials system (never hardcode)
2. **Error handling** is critical (continueOnFail + IF checks)
3. **Pagination** for large datasets
4. **Rate limiting** to respect API limits
5. **Transform responses** to match your needs
**Pattern**: Trigger → HTTP Request → Transform → Action → Error Handler
**Related**:
- [webhook_processing.md](webhook_processing.md) - Receiving HTTP requests
- [database_operations.md](database_operations.md) - Storing API data
README.md
# n8n Workflow Patterns
Proven architectural patterns for building n8n workflows.
---
## Purpose
Teaches architectural patterns for building n8n workflows. Provides structure, best practices, and proven approaches for common use cases.
## Activates On
- build workflow
- workflow pattern
- workflow architecture
- workflow structure
- webhook processing
- http api
- api integration
- database sync
- ai agent
- chatbot
- scheduled task
- automation pattern
## File Count
7 files
## Priority
**HIGH** - Addresses 813 webhook searches (most common use case)
## Dependencies
**n8n-mcp tools**:
- search_nodes (find nodes for patterns)
- get_node (understand node operations)
- search_templates (find example workflows)
- tools_documentation with topic "ai_agents_guide" (AI pattern guidance)
**Related skills**:
- n8n MCP Tools Expert (find and configure nodes)
- n8n Expression Syntax (write expressions in patterns)
- n8n Node Configuration (configure pattern nodes)
- n8n Validation Expert (validate pattern implementations)
## Coverage
### The 5 Core Patterns
1. **Webhook Processing** (Most Common - 813 searches)
- Receive HTTP requests → Process → Respond
- Critical gotcha: Data under $json.body
- Authentication, validation, error handling
2. **HTTP API Integration** (892 templates)
- Fetch from REST APIs → Transform → Store/Use
- Authentication methods, pagination, rate limiting
- Error handling and retries
3. **Database Operations** (456 templates)
- Read/Write/Sync database data
- Batch processing, transactions, performance
- Security: parameterized queries, read-only access
4. **AI Agent Workflow** (234 templates, 270 AI nodes)
- AI agents with tool access and memory
- 8 AI connection types
- ANY node can be an AI tool
5. **Scheduled Tasks** (28% of all workflows)
- Recurring automation workflows
- Cron schedules, timezone handling
- Monitoring and error handling
### Cross-Cutting Concerns
- Data flow patterns (linear, branching, parallel, loops)
- Error handling strategies
- Performance optimization
- Security best practices
- Testing approaches
- Monitoring and logging
## Evaluations
5 scenarios (100% coverage expected):
1. **eval-001**: Webhook workflow structure
2. **eval-002**: HTTP API integration pattern
3. **eval-003**: Database sync pattern
4. **eval-004**: AI agent workflow with tools
5. **eval-005**: Scheduled report generation
## Key Features
✅ **5 Proven Patterns**: Webhook, HTTP API, Database, AI Agent, Scheduled tasks
✅ **Complete Examples**: Working workflow configurations for each pattern
✅ **Best Practices**: Proven approaches from real-world n8n usage
✅ **Common Gotchas**: Documented mistakes and their fixes
✅ **Integration Guide**: How patterns work with other skills
✅ **Template Examples**: Real examples from 2,653+ n8n templates
## Files
- **SKILL.md** - Pattern overview, selection guide, checklist
- **webhook_processing.md** - Webhook patterns, data structure, auth
- **http_api_integration.md** - REST APIs, pagination, rate limiting
- **database_operations.md** - DB operations, batch processing, security
- **ai_agent_workflow.md** - AI agents, tools, memory, 8 connection types
- **scheduled_tasks.md** - Cron schedules, timezone, monitoring
- **README.md** (this file) - Skill metadata
## Success Metrics
**Expected outcomes**:
- Users select appropriate pattern for their use case
- Workflows follow proven structural patterns
- Common gotchas avoided (webhook $json.body, SQL injection, etc.)
- Proper error handling implemented
- Security best practices followed
## Pattern Selection Stats
Common workflow composition:
**Trigger Distribution**:
- Webhook: 35% (most common)
- Schedule: 28%
- Manual: 22%
- Service triggers: 15%
**Transformation Nodes**:
- Set: 68%
- Code: 42%
- IF: 38%
- Switch: 18%
**Output Channels**:
- HTTP Request: 45%
- Slack: 32%
- Database: 28%
- Email: 24%
**Complexity**:
- Simple (3-5 nodes): 42%
- Medium (6-10 nodes): 38%
- Complex (11+ nodes): 20%
## Critical Insights
**Webhook Processing**:
- 813 searches (most common use case!)
- #1 gotcha: Data under $json.body (not $json directly)
- Must choose response mode: onReceived vs lastNode
**API Integration**:
- Authentication via credentials (never hardcode!)
- Pagination essential for large datasets
- Rate limiting prevents API bans
- continueOnFail: true for error handling
**Database Operations**:
- Always use parameterized queries (SQL injection prevention)
- Batch processing for large datasets
- Read-only access for AI tools
- Transaction handling for multi-step operations
**AI Agents**:
- 8 AI connection types (ai_languageModel, ai_tool, ai_memory, etc.)
- ANY node can be an AI tool (connect to ai_tool port)
- Memory essential for conversations (Window Buffer recommended)
- Tool descriptions critical (AI uses them to decide when to call)
**Scheduled Tasks**:
- Set workflow timezone explicitly (DST handling)
- Prevent overlapping executions (use locks)
- Error Trigger workflow for alerts
- Batch processing for large data
## Workflow Creation Checklist
Every pattern follows this checklist:
### Planning Phase
- [ ] Identify the pattern (webhook, API, database, AI, scheduled)
- [ ] List required nodes (use search_nodes)
- [ ] Understand data flow (input → transform → output)
- [ ] Plan error handling strategy
### Implementation Phase
- [ ] Create workflow with appropriate trigger
- [ ] Add data source nodes
- [ ] Configure authentication/credentials
- [ ] Add transformation nodes (native first: Filter, IF, Summarize, Merge…; Code only with a `Why Code:` note)
- [ ] Add output/action nodes
- [ ] Configure error handling
### Validation Phase
- [ ] Validate each node configuration
- [ ] Validate complete workflow
- [ ] Test with sample data
- [ ] Handle edge cases
### Deployment Phase
- [ ] Review workflow settings
- [ ] Activate workflow
- [ ] Monitor first executions
- [ ] Document workflow
## Real Template Examples
**Weather to Slack** (Template #2947):
```
Schedule (daily 8 AM) → HTTP Request (weather) → Slack (message built with expressions)
```
**Webhook Processing**: 1,085 templates
**HTTP API Integration**: 892 templates
**Database Operations**: 456 templates
**AI Workflows**: 234 templates
Use `search_templates` to find examples for your use case!
## Integration with Other Skills
**Pattern Selection** (this skill):
1. Identify use case
2. Select appropriate pattern
3. Follow pattern structure
**Node Discovery** (n8n MCP Tools Expert):
4. Find nodes for pattern (search_nodes)
5. Understand node operations (get_node)
**Implementation** (n8n Expression Syntax + Node Configuration):
6. Write expressions ({{$json.body.field}})
7. Configure nodes properly
**Validation** (n8n Validation Expert):
8. Validate workflow structure
9. Fix validation errors
## Last Updated
2025-10-20
---
**Part of**: n8n-skills repository
**Conceived by**: Romuald Członkowski - [aiadvisors.pl/en](https://aiadvisors.pl/en)
scheduled_tasks.md
# Scheduled Tasks Pattern
**Use Case**: Recurring automation workflows that run automatically on a schedule.
---
## Pattern Structure
```
Schedule Trigger → [Fetch Data] → [Process] → [Deliver] → [Log/Notify]
```
**Key Characteristic**: Time-based automated execution
---
## Core Components
### 1. Schedule Trigger
**Purpose**: Execute workflow at specified times
**Modes**:
- **Interval** - Every X minutes/hours/days
- **Cron** - Specific times (advanced)
- **Days & Hours** - Simple recurring schedule
### 2. Data Source
**Common sources**:
- HTTP Request (APIs)
- Database queries
- File reads
- Service-specific nodes
### 3. Processing
**Typical operations**:
- Filter/transform data
- Aggregate statistics
- Generate reports
- Check conditions
### 4. Delivery
**Output channels**:
- Email
- Slack/Discord/Teams
- File storage
- Database writes
### 5. Logging
**Purpose**: Track execution history
**Methods**:
- Database log entries
- File append
- Monitoring service
---
## Schedule Configuration
### Interval Mode
**Best for**: Simple recurring tasks
**Examples**:
```javascript
// Every 15 minutes
{
mode: "interval",
interval: 15,
unit: "minutes"
}
// Every 2 hours
{
mode: "interval",
interval: 2,
unit: "hours"
}
// Every day at midnight
{
mode: "interval",
interval: 1,
unit: "days"
}
```
### Days & Hours Mode
**Best for**: Specific days and times
**Examples**:
```javascript
// Weekdays at 9 AM
{
mode: "daysAndHours",
days: ["monday", "tuesday", "wednesday", "thursday", "friday"],
hour: 9,
minute: 0
}
// Every Monday at 6 PM
{
mode: "daysAndHours",
days: ["monday"],
hour: 18,
minute: 0
}
```
### Timezone Gotcha (applies to all modes)
`triggerAtHour` / `hour` values use the **instance timezone, not UTC**. n8n resolves it from the `GENERIC_TIMEZONE` env var (or the workflow's timezone setting); when neither is set, it falls back to the host system timezone. A trigger set to hour 21 on a server in `America/Edmonton` fires at 9 PM MST, not 21:00 UTC. Always confirm the instance timezone before scheduling, or set the workflow timezone explicitly.
### Cron Mode (Advanced)
**Best for**: Complex schedules
**Examples**:
```javascript
// Every weekday at 9 AM
{
mode: "cron",
expression: "0 9 * * 1-5"
}
// First day of every month at midnight
{
mode: "cron",
expression: "0 0 1 * *"
}
// Every 15 minutes during business hours (9 AM - 5 PM) on weekdays
{
mode: "cron",
expression: "*/15 9-17 * * 1-5"
}
```
**Cron format**: `minute hour day month weekday`
- `*` = any value
- `*/15` = every 15 units
- `1-5` = range (Monday-Friday)
- `1,15` = specific values
**Cron examples**:
```
0 */6 * * * Every 6 hours
0 9,17 * * * At 9 AM and 5 PM daily
0 0 * * 0 Every Sunday at midnight
*/30 * * * * Every 30 minutes
0 0 1,15 * * 1st and 15th of each month
```
---
## Common Use Cases
### 1. Daily Reports
**Flow**: Schedule → Fetch data → Aggregate → Format → Email
**Example** (Sales report):
```
1. Schedule (daily at 9 AM)
2. Postgres (query yesterday's sales)
SELECT date, SUM(amount) as total, COUNT(*) as orders
FROM orders
WHERE date = CURRENT_DATE - INTERVAL '1 day'
GROUP BY date
3. Email (send to team@company.com)
Subject: Daily Sales Report - {{ $json.date }}
Body: HTML built with expressions, e.g. average order value {{ ($json.total / $json.orders).toFixed(2) }}
4. Slack (post summary to #sales)
```
The query already returns the totals, so no transform node is needed. For a day-over-day comparison, return both days
from the same SQL (or Summarize over two days' rows) rather than computing it in Code.
### 2. Data Synchronization
**Flow**: Schedule → Fetch from source → Transform → Write to target
**Example** (CRM to data warehouse sync):
```
1. Schedule (every hour)
2. Postgres (get last sync time)
SELECT MAX(synced_at) AS last_sync FROM sync_log
3. HTTP Request (fetch new CRM contacts since last sync)
GET /api/contacts?updated_since={{$json.last_sync}}
4. IF (check if new records exist)
5. Postgres (warehouse - INSERT new contacts; map CRM fields to warehouse columns in the node's column slots)
7. Postgres (UPDATE sync_log SET synced_at = NOW())
8. IF (error occurred)
└─ Slack (alert #data-team)
```
### 3. Monitoring & Health Checks
**Flow**: Schedule → Check endpoints → Alert if down
**Example** (Website uptime monitor):
```
1. Schedule (every 5 minutes)
2. HTTP Request (GET https://example.com/health)
- timeout: 10 seconds
- continueOnFail: true
3. IF (status !== 200 OR response_time > 2000ms)
4. Redis (check alert cooldown - don't spam)
- Key: alert:website_down
- TTL: 30 minutes
5. IF (no recent alert sent)
6. [Alert Actions]
├─ Slack (notify #ops-team)
├─ PagerDuty (create incident)
├─ Email (alert@company.com)
└─ Redis (set alert cooldown)
7. Postgres (log uptime check result)
```
### 4. Cleanup & Maintenance
**Flow**: Schedule → Find old data → Archive/Delete → Report
**Example** (Database cleanup):
```
1. Schedule (weekly on Sunday at 2 AM)
2. Postgres (find old records)
SELECT * FROM logs
WHERE created_at < NOW() - INTERVAL '90 days'
LIMIT 10000
3. IF (records exist)
4. Convert To File (operation toJson: all rows into one archive file)
5. Google Drive (upload archive file)
- Filename: logs_archive_{{ $now.format('yyyy-MM-dd') }}.json
6. Postgres (DELETE archived records)
DELETE FROM logs
WHERE id IN ({{$json.archived_ids}})
7. Slack (report: "Archived X records, deleted Y records")
```
### 5. Data Enrichment
**Flow**: Schedule → Find incomplete records → Enrich → Update
**Example** (Enrich contacts with company data):
```
1. Schedule (nightly at 3 AM)
2. Postgres (find contacts without company data)
SELECT id, email, domain FROM contacts
WHERE company_name IS NULL
AND created_at > NOW() - INTERVAL '7 days'
LIMIT 100
3. Loop Over Items (10 contacts per batch)
4. Clearbit (Company → Enrich; domain = {{ $json.email.extractDomain() }})
5. Postgres (UPDATE contacts with company data; map the Clearbit fields in the column slots)
6. Wait (1 second - rate limiting)
7. Loop (back to step 4 until all batches processed)
9. Email (summary: "Enriched X contacts")
```
### 6. Backup Automation
**Flow**: Schedule → Export data → Compress → Store → Verify
**Example** (Database backup):
```
1. Schedule (daily at 2 AM)
2. Execute Command (pg_dump -h db.example.com mydb | gzip > /backups/backup.sql.gz)
- Self-hosted only, and blocked by default: the instance must re-enable the node (see n8n-self-hosting SECURITY.md).
A Code node can't do this either: child_process is not available in the sandbox.
- Often better outside n8n: a cron job or the database's own backup tool, with n8n only checking the result.
3. Read/Write Files from Disk (read /backups/backup.sql.gz as binary)
- For a file that isn't compressed yet, the Compression node gzips it.
4. AWS S3 (upload compressed backup)
- Bucket: backups
- Key: db/backup-{{$now.format('YYYY-MM-DD')}}.sql.gz
5. AWS S3 (list old backups)
- Keep last 30 days only
6. AWS S3 (delete old backups)
7. IF (error occurred)
├─ PagerDuty (critical alert)
└─ Email (backup failed!)
ELSE
└─ Slack (#devops: "✅ Backup completed")
```
### 7. Content Publishing
**Flow**: Schedule → Fetch content → Format → Publish
**Example** (Automated social media posts):
```
1. Schedule (every 3 hours during business hours)
- Cron: 0 9,12,15,18 * * 1-5
2. Google Sheets (read content queue)
- Sheet: "Scheduled Posts"
- Filter: status=pending AND publish_time <= NOW()
3. IF (posts available)
4. HTTP Request (shorten URLs in post)
5. HTTP Request (POST to Twitter API)
6. HTTP Request (POST to LinkedIn API)
7. Google Sheets (update status=published)
8. Slack (notify #marketing: "Posted: {{$json.title}}")
```
---
## Timezone Considerations
### Set Workflow Timezone
```javascript
// In workflow settings
{
timezone: "America/New_York" // EST/EDT
}
```
### Common Timezones
```
America/New_York - Eastern (US)
America/Chicago - Central (US)
America/Denver - Mountain (US)
America/Los_Angeles - Pacific (US)
Europe/London - GMT/BST
Europe/Paris - CET/CEST
Asia/Tokyo - JST
Australia/Sydney - AEDT
UTC - Universal Time
```
### Handle Daylight Saving
**Best practice**: Use timezone-aware scheduling
```javascript
// ❌ Bad: UTC schedule for "9 AM local"
// Will be off by 1 hour during DST transitions
// ✅ Good: Set workflow timezone
{
timezone: "America/New_York",
schedule: {
mode: "daysAndHours",
hour: 9 // Always 9 AM Eastern, regardless of DST
}
}
```
---
## Error Handling
### Pattern 1: Error Trigger Workflow
**Main workflow**: Normal execution
**Error workflow**: Alerts and recovery
**Main**:
```
Schedule → Fetch → Process → Deliver
```
**Error**:
```
Error Trigger (for main workflow)
├─→ Slack (#ops-team: "❌ Scheduled job failed"; details from the trigger payload with expressions)
├─→ Email (admin alert)
└─→ Postgres (log error for analysis)
```
### Pattern 2: Retry with Backoff
```
Schedule → HTTP Request (continueOnFail: true)
→ IF (error)
├─ Wait (5 minutes)
├─ HTTP Request (retry 1)
└─ IF (still error)
├─ Wait (15 minutes)
├─ HTTP Request (retry 2)
└─ IF (still error)
└─ Alert admin
```
### Pattern 3: Partial Failure Handling
```
Schedule → Process (onError: continueErrorOutput)
├─ success output → Summarize (count) ─┐
└─ error output → Summarize (count) ─┴→ Merge (combine by position) → Report:
"✅ Processed: 95/100"
"❌ Failed: 5/100"
```
---
## Performance Optimization
### 1. Batch Processing
For large datasets:
```
Schedule → Query (LIMIT 10000)
→ Split In Batches (100 items)
→ Process batch
→ Loop
```
### 2. Parallel Processing
When operations are independent:
```
Schedule
├─ [Branch 1: Update DB]
├─ [Branch 2: Send emails]
└─ [Branch 3: Generate report]
→ Merge (wait for all) → Final notification
```
### 3. Skip if Already Running
Prevent overlapping executions:
```
Schedule → Redis (check lock)
→ IF (lock exists)
└─ End (skip this execution)
→ ELSE
├─ Redis (set lock, TTL 30 min)
├─ [Execute workflow]
└─ Redis (delete lock)
```
### 4. Early Exit on No Data
Don't waste time if nothing to process:
```
Schedule → Query (check if work exists)
→ IF (no results)
└─ End workflow (exit early)
→ ELSE
└─ Process data
```
---
## Monitoring & Logging
### Pattern 1: Execution Log Table
```sql
CREATE TABLE workflow_executions (
id SERIAL PRIMARY KEY,
workflow_name VARCHAR(255),
started_at TIMESTAMP,
completed_at TIMESTAMP,
status VARCHAR(50),
records_processed INT,
error_message TEXT
);
```
**Log execution**:
```
Schedule
→ Set (record start)
→ [Workflow logic]
→ Postgres (INSERT execution log)
```
### Pattern 2: Metrics Collection
```
Schedule → [Execute]
→ Summarize (records processed, successes)
→ HTTP Request (send to monitoring system; duration = {{ $now.diffTo($('Record start').item.json.started_at, 'seconds') }})
- Datadog, Prometheus, etc.
```
### Pattern 3: Summary Notifications
Daily/weekly execution summaries:
```
Schedule (daily at 6 PM) → Query execution logs
→ Summarize (count executions, split by status)
→ Email (summary report)
"Today's Workflow Executions:
- 24/24 successful
- 0 failures
- Avg duration: 2.3 min"
```
---
## Testing Scheduled Workflows
### 1. Use Manual Trigger for Testing
**Development pattern**:
```
Manual Trigger (for testing)
→ [Same workflow logic]
→ [Outputs]
// Once tested, replace with Schedule Trigger
```
### 2. Test with Different Times
Pin a constant item instead of the trigger: an Edit Fields node with `currentTime` = `2024-01-15T09:00:00Z`, and
downstream expressions read `$json.currentTime` instead of `$now` while testing.
### 3. Dry Run Mode
```
Schedule → Set (dryRun: true)
→ IF (dryRun)
└─ Log what would happen (don't execute)
→ ELSE
└─ Execute normally
```
### 4. Shorter Interval for Testing
```javascript
// Testing: every 1 minute
{
mode: "interval",
interval: 1,
unit: "minutes"
}
// Production: every 1 hour
{
mode: "interval",
interval: 1,
unit: "hours"
}
```
---
## Common Gotchas
### 1. ❌ Wrong: Ignoring timezone
```javascript
Schedule (9 AM) // 9 AM in which timezone?
```
### ✅ Correct: Set workflow timezone
```javascript
// Workflow settings
{
timezone: "America/New_York"
}
```
### 2. ❌ Wrong: Overlapping executions
```
Schedule (every 5 min) → Long-running task (10 min)
// Two executions running simultaneously!
```
### ✅ Correct: Add execution lock
```
Schedule → Redis (check lock)
→ IF (locked) → Skip
→ ELSE → Execute
```
### 3. ❌ Wrong: No error handling
```
Schedule → API call → Process (fails silently)
```
### ✅ Correct: Add error workflow
```
Main: Schedule → Execute
Error: Error Trigger → Alert
```
### 4. ❌ Wrong: Processing all data at once
```
Schedule → SELECT 1000000 records → Process (OOM)
```
### ✅ Correct: Batch processing
```
Schedule → SELECT with pagination → Split In Batches → Process
```
### 5. ❌ Wrong: Hardcoded dates
```javascript
query: "SELECT * FROM orders WHERE date = '2024-01-15'"
```
### ✅ Correct: Dynamic dates
```javascript
query: "SELECT * FROM orders WHERE date = CURRENT_DATE - INTERVAL '1 day'"
```
---
## Real Template Examples
From n8n template library:
**Template #2947** (Weather to Slack):
```
Schedule (daily 8 AM)
→ HTTP Request (weather API)
→ Set (format message)
→ Slack (post to #general)
```
**Daily backup**:
```
Schedule (nightly 2 AM)
→ Postgres (export data)
→ Google Drive (upload)
→ Email (confirmation)
```
**Monitoring**:
```
Schedule (every 5 min)
→ HTTP Request (health check)
→ IF (down) → PagerDuty alert
```
Use `search_templates({query: "schedule"})` to find more!
---
## Checklist for Scheduled Workflows
### Planning
- [ ] Define schedule frequency (interval, cron, days & hours)
- [ ] Set workflow timezone
- [ ] Estimate execution duration
- [ ] Plan for failures and retries
- [ ] Consider timezone and DST
### Implementation
- [ ] Configure Schedule Trigger
- [ ] Set workflow timezone in settings
- [ ] Add early exit for no-op cases
- [ ] Implement batch processing for large data
- [ ] Add execution logging
### Error Handling
- [ ] Create Error Trigger workflow
- [ ] Implement retry logic
- [ ] Add alert notifications
- [ ] Log errors for analysis
- [ ] Handle partial failures gracefully
### Monitoring
- [ ] Log each execution (start, end, status)
- [ ] Track metrics (duration, records, success rate)
- [ ] Set up daily/weekly summaries
- [ ] Alert on consecutive failures
- [ ] Monitor resource usage
### Testing
- [ ] Test with Manual Trigger first
- [ ] Verify timezone behavior
- [ ] Test error scenarios
- [ ] Check for overlapping executions
- [ ] Validate output quality
### Deployment
- [ ] Document workflow purpose
- [ ] Set up monitoring
- [ ] Configure alerts
- [ ] Activate workflow in n8n UI ⚠️ **Manual activation required** (API/MCP cannot activate)
- [ ] Test in production (short interval first)
- [ ] Monitor first few executions
---
## Advanced Patterns
### Dynamic Scheduling
**Change schedule based on conditions**:
```
Schedule (check every hour)
→ IF ({{ $now.weekday <= 5 && $now.hour >= 9 && $now.hour < 17 }})
└─ Execute workflow
→ ELSE
└─ Skip
```
### Dependent Schedules
**Chain workflows**:
```
Workflow A (daily 2 AM): Data sync
→ On completion → Trigger Workflow B
Workflow B: Generate report (depends on fresh data)
```
### Conditional Execution
**Skip based on external factors**:
```
Schedule → HTTP Request (check feature flag)
→ IF (feature enabled)
└─ Execute
→ ELSE
└─ Skip
```
---
## Summary
**Key Points**:
1. **Set workflow timezone** explicitly
2. **Batch processing** for large datasets
3. **Error handling** is critical (Error Trigger + retries)
4. **Prevent overlaps** with execution locks
5. **Monitor and log** all executions
**Pattern**: Schedule → Fetch → Process → Deliver → Log
**Schedule Modes**:
- **Interval**: Simple recurring (every X minutes/hours)
- **Days & Hours**: Specific days and times
- **Cron**: Advanced complex schedules
**Related**:
- [http_api_integration.md](http_api_integration.md) - Fetching data on schedule
- [database_operations.md](database_operations.md) - Scheduled database tasks
- [webhook_processing.md](webhook_processing.md) - Alternative to scheduling
SKILL.md
---
name: n8n-workflow-patterns
description: Proven workflow architectural patterns from real n8n workflows. Use when building new workflows, designing workflow structure, choosing workflow patterns, planning workflow architecture, or asking about webhook processing, HTTP API integration, database operations, AI agent workflows, batch processing, or scheduled tasks. Always consult this skill when the user asks to create, build, or design an n8n workflow, automate a process, or connect services — even if they don't explicitly mention 'patterns'. Covers webhook, API, database, AI, batch processing, and scheduled automation architectures. Also use when optimizing a slow workflow or speeding up large-item-count processing (node count, batchSize, all-items vs per-item).
---
# n8n Workflow Patterns
Proven architectural patterns for building n8n workflows.
---
## The 6 Core Patterns
Based on analysis of real workflow usage:
1. **[Webhook Processing](webhook_processing.md)** (Most Common)
- Receive HTTP requests → Process → Output
- Pattern: Webhook → Validate → Transform → Respond/Notify
2. **[HTTP API Integration](http_api_integration.md)**
- Fetch from REST APIs → Transform → Store/Use
- Pattern: Trigger → HTTP Request → Transform → Action → Error Handler
3. **[Database Operations](database_operations.md)**
- Read/Write/Sync database data
- Pattern: Schedule → Query → Transform → Write → Verify
4. **[AI Agent Workflow](ai_agent_workflow.md)**
- AI agents with tools and memory
- Pattern: Trigger → AI Agent (Model + Tools + Memory) → Output
5. **[Scheduled Tasks](scheduled_tasks.md)**
- Recurring automation workflows
- Pattern: Schedule → Fetch → Process → Deliver → Log
6. **Batch Processing** (below)
- Process large datasets in chunks with API rate limits
- Pattern: Prepare → SplitInBatches → Process per batch → Accumulate → Aggregate
---
## Pattern Selection Guide
### When to use each pattern:
**Webhook Processing** - Use when:
- Receiving data from external systems
- Building integrations (Slack commands, form submissions, GitHub webhooks)
- Need instant response to events
- Example: "Receive Stripe payment webhook → Update database → Send confirmation"
**HTTP API Integration** - Use when:
- Fetching data from external APIs
- Synchronizing with third-party services
- Building data pipelines
- Example: "Fetch GitHub issues → Transform → Create Jira tickets"
**Database Operations** - Use when:
- Syncing between databases
- Running database queries on schedule
- ETL workflows
- Example: "Read Postgres records → Transform → Write to MySQL"
**AI Agent Workflow** - Use when:
- Building conversational AI
- Need AI with tool access
- Multi-step reasoning tasks
- Example: "Chat with AI that can search docs, query database, send emails"
**Scheduled Tasks** - Use when:
- Recurring reports or summaries
- Periodic data fetching
- Maintenance tasks
- Example: "Daily: Fetch analytics → Generate report → Email team"
**Batch Processing** - Use when:
- Processing large datasets that exceed API batch limits
- Need to accumulate results across multiple API calls
- Nested loops (e.g., multiple categories × paginated API calls per category)
- Example: "Fetch products for 4 markets × 1000 per API call → Aggregate all results"
---
## Common Workflow Components
All patterns share these building blocks:
### 1. Triggers
- **Webhook** - HTTP endpoint (instant)
- **Schedule** - Cron-based timing (periodic)
- **Manual** - Click to execute (testing)
- **Polling** - Check for changes (intervals)
### 2. Data Sources
- **HTTP Request** - REST APIs
- **Database nodes** - Postgres, MySQL, MongoDB
- **Service nodes** - Slack, Google Sheets, etc.
- **Extract From File** - CSV / XLSX / PDF from a downloaded or uploaded file
### 3. Transformation (native first)
- **Filter / IF / Switch** - Drop items, route
- **Edit Fields (Set)** - Named steps: constant items, payload objects, error and return items
- **Summarize / Aggregate / Split Out** - Group and total, collect into an array, explode an array
- **Merge / Compare Datasets** - Join two lists, diff them for a sync
- **Remove Duplicates / Sort / Limit** - Dedupe (also across runs), top-N
- **Date & Time, XML, HTML, Crypto** - Dates, markup, hashes and signatures
- **Code** - Only the closed list (loops or recursion no node expresses, state across runs, parsing no node handles), with a `Why Code:` note
The job-to-node table in `using-n8n-mcp-skills` maps jobs to these nodes; configs are in `n8n-node-configuration` → NATIVE_NODES.md. Reading a filtered slice of a nested API response for **one field**? A single `{{ $jmespath($json, "…") }}` in that field does it without extra nodes (quoting rules in n8n-expression-syntax).
### 4. Outputs
- **HTTP Request** - Call APIs
- **Database** - Write data
- **Communication** - Email, Slack, Discord
- **Storage** - Files, cloud storage
### 5. Error Handling
- **Error Trigger** - Catch workflow errors
- **IF** - Check for error conditions
- **Stop and Error** - Explicit failure
- **Continue On Fail** - Per-node setting
---
## Workflow Creation Checklist
When building ANY workflow, follow this checklist:
### Planning Phase
- [ ] Identify the pattern (webhook, API, database, AI, scheduled)
- [ ] List required nodes (use search_nodes)
- [ ] Understand data flow (input → transform → output)
- [ ] Plan error handling strategy
### Implementation Phase
- [ ] Create workflow with appropriate trigger
- [ ] Add data source nodes
- [ ] Configure authentication/credentials
- [ ] Add transformation nodes (native first: Filter, IF, Summarize, Merge…; Code only with a `Why Code:` note)
- [ ] Add output/action nodes
- [ ] Configure error handling
### Validation Phase
- [ ] Validate each node configuration (validate_node)
- [ ] Validate complete workflow (validate_workflow)
- [ ] Test with sample data
- [ ] Handle edge cases (empty data, errors)
### Deployment Phase
- [ ] Review workflow settings (execution order, timeout, error handling)
- [ ] Activate workflow using `activateWorkflow` operation
- [ ] Monitor first executions
- [ ] Document workflow purpose and data flow
---
## Workflow lifecycle: validate, verify, test before activating
Building the nodes is the start, not the finish. Before a workflow goes live, run it through four gates — and remember the headline rule: **validation passing is necessary, not sufficient.** A workflow can validate clean and still drop items, pick the wrong Merge input, or post Slack messages as plain text. Clean validation means the *shapes* are right, not that the *logic* is.
1. **Validate.** Run `validate_workflow` on the full JSON during build, or `n8n_validate_workflow({ id })` once the workflow exists on the instance. Fix every error and re-validate. This catches schema, node-config, expression, and reference errors — the structural layer.
2. **Verify the connections.** Pull the workflow with `n8n_get_workflow({ id })` and read the `connections` object directly. Validation confirms connections aren't *broken*; it doesn't confirm they're *correct*. This is where you catch the valid-but-wrong wiring: a Merge whose `useDataOfInput` doesn't line up with the connection slot, a Switch fallback that connects to nothing, a fan-out branch that was never wired onward, an error output that goes nowhere. (See the n8n Node Configuration skill's NODE_FAMILY_GOTCHAS.md for the silent ones.)
3. **Test.** Run `n8n_test_workflow` and inspect the output via `n8n_executions`. Confirm the output shape matches what consumers expect, fan-outs all produced data, and (for webhook APIs) the status/body/headers are right. **Real side effects fire during a test** — writes commit, messages send, external APIs are called. If any node has a user-visible side effect, confirm with the user before running, or test against safe data first.
4. **Activate** only after the first three pass — using `n8n_update_partial_workflow` with the `activateWorkflow` operation. Don't activate straight off a clean validation; an active workflow that drops data or double-sends is worse than one that never started.
Skipping any gate trades a few minutes now for debugging a live, possibly stateful, possibly traffic-bearing workflow later. The trade is never worth it.
---
## Data Flow Patterns
### Linear Flow
```
Trigger → Transform → Action → End
```
**Use when**: Simple workflows with single path
### Branching Flow
```
Trigger → IF → [True Path]
└→ [False Path]
```
**Use when**: Different actions based on conditions
### Parallel Processing
```
Trigger → [Branch 1] → Merge
└→ [Branch 2] ↗
```
**Use when**: Independent operations that can run simultaneously
### Loop Pattern
```
Trigger → Split in Batches → Process → Loop (until done)
```
**Use when**: Processing large datasets in chunks
### Error Handler Pattern
```
Main Flow → [Success Path]
└→ [Error Trigger → Error Handler]
```
**Use when**: Need separate error handling workflow
---
## Batch Processing Pattern
### SplitInBatches Loop
The SplitInBatches node splits a large dataset into smaller chunks for processing. Understanding its outputs is critical:
- `main[0]` = **done** — fires ONCE after all batches complete
- `main[1]` = **each batch** — fires per batch (this is the loop body)
```
Prepare Items → SplitInBatches → [main[1]: Process Batch] → (loops back)
[main[0]: Done] → Limit 1 → Aggregate
```
Always add a **Limit 1** node after the done output.
### Choosing batchSize (the cost lever)
A SplitInBatches loop re-runs its whole body once per iteration — ~0.8 ms/iteration of engine overhead plus the body's own cost — so total ≈ `⌈items / batchSize⌉ × (overhead + body)`. batchSize is a direct speed dial:
- Pick the **largest batch your real constraint allows** (API page size, rate limit, memory). Bigger batches = fewer iterations = less overhead; the body still sees every item.
- `batchSize: 1` is the expensive extreme — one full engine pass per item. Use it only when you must act on a single item at a time (nested-loop control, or an API that takes exactly one id).
- If you're looping only to "go over the items" with no external constraint, you don't need the loop: connect the source to the consumer. Per-item execution handles it, and native transform nodes process all items in one call.
### Cross-Iteration Data
After the loop, `$('Node Inside Loop').all()` returns **ONLY the last batch's items**. To accumulate across all iterations, use `$getWorkflowStaticData('global')` in a Code node inside the loop. See the n8n Code JavaScript skill for the full pattern.
### Nested Loops
When processing N categories × M items per category (where an API has a batch limit):
```
Define Categories (N items)
→ Outer Loop (SplitInBatches, batchSize=1)
→ Prepare category data
→ Inner Loop (SplitInBatches, batchSize=1000)
→ API Call → Verify → (loops back to Inner Loop via main[1])
→ Inner done[0] → Rate Limit Delay → back to Outer Loop
→ Outer done[0] → Limit 1 → Final Aggregate
```
**Wiring gotcha**: The inner done[0] must connect back to the OUTER loop input, not to the aggregate. The outer done[0] feeds the final aggregate.
### API Pagination
Paging through one endpoint is HTTP Request's pagination option, not a loop (page-number, offset, cursor and next-URL APIs, with a page cap — see `n8n-node-configuration` → NATIVE_NODES.md). The loop below is for sweeps the option can't express, such as an API without multi-ID filtering that you walk with `id_from` + date windows, each window its own query:
```
Schedule → Set Date Window → Fetch Page → Process
→ IF has more? → [true] Update id_from → Fetch Page (loop)
→ [false] → Aggregate → Output
```
### Dry-Run / Verification Tolerance
When testing with API write nodes disabled (for dry runs), downstream verification nodes receive the request body instead of the response. Make verification tolerant:
```javascript
// In verification Code node
const body = $input.first().json;
const looksLikeRequest = body.method && body.parameters && !body.status;
if (looksLikeRequest) {
return [{ json: { status: 'SKIPPED', message: 'Upstream disabled for testing' }}];
}
// Normal response verification below...
```
---
## Performance on the hot path
When a workflow processes **thousands of items** with little I/O, its speed is set by how many times n8n crosses a per-item / per-iteration boundary — each crossing sets up an execution context and copies the items. Four architecture choices dominate:
1. **Consolidating a hot chain into one Code node is an exception, not a default.** Every node→node hop re-copies all items, so on a measured hot path — thousands of items per run, transform work rather than I/O dominating — one All-Items Code node can beat a long chain. That is the closed list's "heavy all-items work" case: record it in the node's `Why Code:` note. Below that, native nodes win on readability.
2. **If Code stays, use "Run Once for All Items," not "Each Item."** Each Item pays a full sandbox per item; a chain of *Each-Item* Code nodes is the worst case, because the per-item tax multiplies by node count.
3. **Maximize batchSize** in SplitInBatches loops (see the Batch Processing pattern above) — iterations are the cost.
4. **Don't micro-optimize expressions** — complexity is free; node and iteration count are what you pay for.
**But profile first.** Most production workflows are I/O-bound — sequential HTTP / DB / Sheets calls (hundreds of ms each) dwarf all of the above. These rules matter when transform work is the floor, or when an anti-pattern (Each-Item Code, batchSize 1, long per-item chains) turns a cheap operation into a slow one. Below a few hundred items, none of it matters. The **n8n Code JavaScript** skill has the full measured model.
---
## Integration-Specific Gotchas
### Google Sheets
- **NEVER use `append`** on sheets with formula columns — it breaks formulas. Use Google Sheets API `values.update` (PUT) via HTTP Request node with a `googleApi` credential
- **Write numbers, not strings** for formula-dependent columns — string "4.98" breaks `ADD()` formulas. Convert in the column's expression: `{{ $json.price.toNumber() }}` or `{{ parseFloat($json.price) }}`
- **Per-item execution trap**: Google Sheets nodes execute once per input item. If you need a single bulk write, collect the rows into one item with an Aggregate node first
- **UNFORMATTED_VALUE returns numbers**, not text like "N/A" — filter explicitly with a Filter node
### Google Drive
- **`convertToGoogleDocument: true` creates a Google Doc (text)**, NOT a Google Sheet — to upload a CSV for download, omit this option entirely
- **CSV download link format**: `https://drive.google.com/uc?id={fileId}&export=download` — use instead of `/view` links
### Bidirectional Threshold Checking
When comparing values (prices, quantities, metrics), always check both directions:
```javascript
// ❌ Only catches increases
if (diff > threshold) { flag(); }
// ✅ Catches both spikes AND crashes — both are data-quality signals
if (Math.abs(diff) > threshold) { flag(); }
```
---
## Common Gotchas
### 1. Webhook Data Structure
**Problem**: Can't access webhook payload data
**Solution**: Data is nested under `$json.body`
```javascript
❌ {{$json.email}}
✅ {{$json.body.email}}
```
See: n8n Expression Syntax skill
### 2. Multiple Input Items
**Problem**: Node processes all input items, but I only want one
**Solution**: Use "Execute Once" mode or process first item only
```javascript
{{$json[0].field}} // First item only
```
### 3. Authentication Issues
**Problem**: API calls failing with 401/403
**Solution**:
- Configure credentials properly
- Use the "Credentials" section, not parameters
- Test credentials before workflow activation
### 4. Node Execution Order
**Problem**: Nodes executing in unexpected order
**Solution**: Check workflow settings → Execution Order
- v0: Top-to-bottom (legacy)
- v1: Connection-based (recommended)
### 5. Expression Errors
**Problem**: Expressions showing as literal text
**Solution**: Use {{}} around expressions
- See n8n Expression Syntax skill for details
---
## Integration with Other Skills
These skills work together with Workflow Patterns:
**n8n MCP Tools Expert** - Use to:
- Find nodes for your pattern (search_nodes)
- Understand node operations (get_node)
- Create workflows (n8n_create_workflow)
- Deploy templates (n8n_deploy_template)
- Use `tools_documentation({topic: "ai_agents_guide", depth: "full"})` for AI pattern guidance
- Manage data tables with `n8n_manage_datatable`
- Organize workflows into folders with `n8n_manage_folders`
**n8n Expression Syntax** - Use to:
- Write expressions in transformation nodes
- Access webhook data correctly ({{$json.body.field}})
- Reference previous nodes ({{$node["Node Name"].json.field}})
**n8n Node Configuration** - Use to:
- Configure specific operations for pattern nodes
- Understand node-specific requirements
**n8n Validation Expert** - Use to:
- Validate workflow structure
- Fix validation errors
- Ensure workflow correctness before deployment
---
## Pattern Statistics
Common workflow patterns:
**Most Common Triggers**:
1. Webhook - 35%
2. Schedule (periodic tasks) - 28%
3. Manual (testing/admin) - 22%
4. Service triggers (Slack, email, etc.) - 15%
**Most Common Transformations**:
1. Set (field mapping) - 68%
2. Code (custom logic) - 42%
3. IF (conditional routing) - 38%
4. Switch (multi-condition) - 18%
**Most Common Outputs**:
1. HTTP Request (APIs) - 45%
2. Slack - 32%
3. Database writes - 28%
4. Email - 24%
**Average Workflow Complexity**:
- Simple (3-5 nodes): 42%
- Medium (6-10 nodes): 38%
- Complex (11+ nodes): 20%
---
## Quick Start Examples
### Example 1: Simple Webhook → Slack
```
1. Webhook (path: "form-submit", POST)
2. Slack (post message to #notifications; the text maps $json.body fields with expressions)
```
### Example 2: Scheduled Report
```
1. Schedule (daily at 9 AM)
2. HTTP Request (fetch analytics)
3. Summarize (totals per group)
4. Email (send formatted report)
5. Error Trigger → Slack (notify on failure)
```
### Example 3: Database Sync
```
1. Schedule (every 15 minutes)
2. Postgres (query new records)
3. IF (check if records exist)
4. MySQL (insert records)
5. Postgres (update sync timestamp)
```
### Example 4: AI Assistant
```
1. Webhook (receive chat message)
2. AI Agent
├─ OpenAI Chat Model (ai_languageModel)
├─ HTTP Request Tool (ai_tool)
├─ Database Tool (ai_tool)
└─ Window Buffer Memory (ai_memory)
3. Webhook Response (send AI reply)
```
### Example 5: API Integration
```
1. Manual Trigger (for testing)
2. HTTP Request (GET /api/users; pagination option if the API pages)
3. Postgres (upsert users; map the fields in the node's column slots)
```
No loop and no Set: Postgres processes every item, and the column slots take the mapping expressions.
---
## Detailed Pattern Files
For comprehensive guidance on each pattern:
- **[webhook_processing.md](webhook_processing.md)** - Webhook patterns, data structure, response handling
- **[http_api_integration.md](http_api_integration.md)** - REST APIs, authentication, pagination, retries
- **[database_operations.md](database_operations.md)** - Queries, sync, transactions, batch processing
- **[ai_agent_workflow.md](ai_agent_workflow.md)** - AI agents, tools, memory, langchain nodes
- **[scheduled_tasks.md](scheduled_tasks.md)** - Cron schedules, reports, maintenance tasks
---
## Real Template Examples
From n8n template library:
**Template #2947**: Weather to Slack
- Pattern: Scheduled Task
- Nodes: Schedule → HTTP Request (weather API) → Set → Slack
- Complexity: Simple (4 nodes)
**Webhook Processing**: Most common pattern
- Most common: Form submissions, payment webhooks, chat integrations
**HTTP API**: Common pattern
- Most common: Data fetching, third-party integrations
**Database Operations**: Common pattern
- Most common: ETL, data sync, backup workflows
**AI Agents**: Growing in usage
- Most common: Chatbots, content generation, data analysis
Use `search_templates` and `get_template` from n8n-mcp tools to find examples!
---
## Best Practices
### ✅ Do
- Start with the simplest pattern that solves your problem
- Plan your workflow structure before building
- Use error handling on all workflows
- Test with sample data before activation
- Follow the workflow creation checklist
- Use descriptive node names
- Document complex workflows (notes field)
- Monitor workflow executions after deployment
### ❌ Don't
- Build workflows in one shot (iterate! avg 56s between edits)
- Skip validation before activation
- Ignore error scenarios
- Use complex patterns when simple ones suffice
- Hardcode credentials in parameters
- Forget to handle empty data cases
- Mix multiple patterns without clear boundaries
- Deploy without testing
---
## Summary
**Key Points**:
1. **6 core patterns** cover 90%+ of workflow use cases
2. **Webhook processing** is the most common pattern
3. Use the **workflow creation checklist** for every workflow
4. **Plan pattern** → **Select nodes** → **Build** → **Validate** → **Deploy**
5. Integrate with other skills for complete workflow development
**Next Steps**:
1. Identify your use case pattern
2. Read the detailed pattern file
3. Use n8n MCP Tools Expert to find nodes
4. Follow the workflow creation checklist
5. Use n8n Validation Expert to validate
**Related Skills**:
- n8n MCP Tools Expert - Find and configure nodes
- n8n Expression Syntax - Write expressions correctly
- n8n Validation Expert - Validate and fix errors
- n8n Node Configuration - Configure specific operations
webhook_processing.md
# Webhook Processing Pattern
**Use Case**: Receive HTTP requests from external systems and process them instantly.
---
## Pattern Structure
```
Webhook → [Validate] → [Transform] → [Action] → [Response/Notify]
```
**Key Characteristic**: Instant event-driven processing
---
## Core Components
### 1. Webhook Node (Trigger)
**Purpose**: Create HTTP endpoint to receive data
**Configuration**:
```javascript
{
path: "form-submit", // URL path: https://n8n.example.com/webhook/form-submit
httpMethod: "POST", // GET, POST, PUT, DELETE
responseMode: "onReceived", // or "lastNode" for custom response
responseData: "allEntries" // or "firstEntryJson"
}
```
**Critical Gotcha**: Data is nested under `$json.body`
```javascript
❌ {{$json.email}}
✅ {{$json.body.email}}
```
### 2. Validation (Optional but Recommended)
**Purpose**: Verify incoming data before processing
**Options**:
- **IF node** - Check required fields exist (one condition per rule)
- **Stop and Error** - Fail gracefully with message
- **Code node** - Only when the caller needs per-field error messages (a loop over a schema; see `n8n-error-handling`)
**Example**:
```javascript
// IF node condition
{{$json.body.email}} is not empty AND
{{$json.body.name}} is not empty
```
### 3. Transformation
**Purpose**: Map webhook data to desired format
**Typical nodes**:
- **None** - Map fields in the action node's own slots (`{{ $json.body.email }}` straight in the CRM or database node)
- **Edit Fields (Set)** - A named step: a payload several nodes read, constants, an error or return item
- **Native transforms** - Split Out, Filter, Summarize, Date & Time (see the job-to-node table in `using-n8n-mcp-skills`)
**Example** (Edit Fields, when several nodes read these values):
```javascript
{
"user_email": "={{$json.body.email}}",
"user_name": "={{$json.body.name}}",
"timestamp": "={{$now}}"
}
```
### 4. Action
**Purpose**: Do something with the data
**Common actions**:
- Store in database (Postgres, MySQL, MongoDB)
- Send notification (Slack, Email, Discord)
- Call another API (HTTP Request)
- Update external system (CRM, support ticket)
### 5. Response (If responseMode: "lastNode")
**Purpose**: Send custom HTTP response
**Webhook Response Node**:
```javascript
{
statusCode: 200,
headers: {
"Content-Type": "application/json"
},
body: {
"status": "success",
"message": "Form received"
}
}
```
---
## Common Use Cases
### 1. Form Submissions
**Flow**: Form → Webhook → Validate → Database → Email Confirmation
**Example**:
```
1. Webhook (path: "contact-form", POST)
2. IF (check email & message not empty)
3. Postgres (insert into contacts table)
4. Email (send confirmation to user)
5. Slack (notify team in #leads)
6. Webhook Response ({"status": "success"})
```
**Real Data Access**:
```javascript
Name: {{$json.body.name}}
Email: {{$json.body.email}}
Message: {{$json.body.message}}
```
### 2. Payment Webhooks (Stripe, PayPal)
**Flow**: Payment Provider → Webhook → Verify → Update Database → Send Receipt
**Security**: Verify webhook signatures — with the provider's trigger node when one exists (Stripe Trigger, GitHub
Trigger), otherwise natively:
```
Webhook (options.rawBody: true)
→ Crypto (hmac SHA256 over the binary "data"; secret from a crypto credential)
→ IF (signature header equals the computed value) → process / Respond 401
```
The signature is computed over the **raw** body; re-serialising `$json.body` breaks it. Stripe signs
`<timestamp>.<raw body>` rather than the body alone, so its check needs that string built first. Full node JSON:
`n8n-node-configuration` → NATIVE_NODES.md → "Webhook signature".
### 3. Chat Platform Integrations (Slack, Discord, Teams)
**Flow**: Chat Command → Webhook → Process → Respond
**Example** (Slack slash command):
```
1. Webhook (path: "slack-command", POST)
2. HTTP Request (fetch data from API; query = {{ $json.body.text }})
3. Respond to Webhook (immediate Slack response; the message is built with expressions from $json and $('Webhook').item.json.body)
```
The Slack payload fields are read where they're used (`$json.body.text`, `$json.body.user_id`); no parsing node is needed.
**Slack Data Access**:
```javascript
Command: {{$json.body.command}}
Text: {{$json.body.text}}
User ID: {{$json.body.user_id}}
Channel ID: {{$json.body.channel_id}}
```
### 4. GitHub/GitLab Webhooks
**Flow**: Git Event → Webhook → Parse → Notify/Deploy
**Example** (new PR notification):
```
1. Webhook (path: "github", POST)
2. IF (check $json.body.action equals "opened")
3. Slack (notify #dev-team; title, author and URL read with the expressions below)
4. Webhook Response (200 OK)
```
The GitHub Trigger node registers the webhook and filters events for you; prefer it over a bare Webhook (service node
first).
**GitHub Data Access**:
```javascript
Event Type: {{$json.headers['x-github-event']}}
Action: {{$json.body.action}}
PR Title: {{$json.body.pull_request.title}}
Author: {{$json.body.pull_request.user.login}}
URL: {{$json.body.pull_request.html_url}}
```
### 5. IoT Device Data
**Flow**: Device → Webhook → Validate → Store → Alert (if threshold)
**Example** (temperature sensor):
```
1. Webhook (path: "sensor-data", POST)
2. Postgres (insert into sensor_readings; columns from $json.body.*)
3. IF (temperature > 80)
4. Email (alert admin)
```
---
## Webhook Data Structure
### Standard Structure
```json
{
"headers": {
"content-type": "application/json",
"user-agent": "...",
"x-custom-header": "..."
},
"params": {
"id": "123" // From URL: /webhook/form/:id
},
"query": {
"token": "abc" // From URL: /webhook/form?token=abc
},
"body": {
// ⚠️ YOUR DATA IS HERE!
"name": "John",
"email": "john@example.com"
}
}
```
### Accessing Different Parts
```javascript
// Headers
{{$json.headers['content-type']}}
{{$json.headers['x-api-key']}}
// URL Parameters
{{$json.params.id}}
// Query Parameters
{{$json.query.token}}
{{$json.query.page}}
// Body (MOST COMMON)
{{$json.body.email}}
{{$json.body.user.name}}
{{$json.body.items[0].price}}
```
---
## Authentication & Security
### 1. Header or Basic Auth (shared secret)
**Simple and enough for most internal callers**
Set the Webhook node's `authentication` to `headerAuth` (or `basicAuth` / `jwtAuth`) and attach a credential. n8n
rejects requests without the right header before the workflow runs. Never compare a token in an IF node: the secret
then sits in the workflow JSON, which breaks the "secrets never go in text fields" rule.
```json
{ "httpMethod": "POST", "path": "orders", "authentication": "headerAuth", "responseMode": "responseNode", "options": {} }
```
Query-string tokens leak into logs and browser history; prefer a header.
### 2. Signature Verification (HMAC)
**Best security** (for webhooks from services like GitHub, Shopify, Stripe)
```
Webhook (options.rawBody: true)
→ Crypto (typeVersion 2, action hmac, type SHA256, binaryData true, binaryPropertyName "data", encoding hex;
secret from a crypto credential)
→ IF {{ $('Webhook').item.json.headers['x-hub-signature-256'] }} equals {{ 'sha256=' + $json.computedSignature }}
├─ true → process
└─ false → Respond to Webhook 401
```
- Compute over the raw body (`rawBody: true` puts it in binary `data`); `JSON.stringify($json.body)` changes the bytes.
- Header name and prefix differ per provider (GitHub `x-hub-signature-256: sha256=…`, Shopify base64). Read their docs.
- Use the provider's trigger node if n8n has one (GitHub Trigger, Stripe Trigger) before building this by hand.
- Full JSON: `n8n-node-configuration` → NATIVE_NODES.md → "Webhook signature".
### 3. IP Allowlist
**Restrict access by IP** with the Webhook node's `options.ipWhitelist` ("IP(s) Allowlist"): comma-separated
addresses or CIDR ranges; empty allows all. Good for internal callers with fixed addresses, on top of auth.
---
## Response Modes
### onReceived (Default)
**Behavior**: Immediate 200 OK response, workflow continues in background
**Use when**:
- Long-running workflows
- Response doesn't depend on workflow result
- Fire-and-forget processing
**Configuration**:
```javascript
{
responseMode: "onReceived",
options: { responseCode: { values: { responseCode: 200 } } } // Webhook v2+: a nested collection
}
```
### lastNode (Custom Response)
**Behavior**: Wait for workflow completion, send custom response
**Use when**:
- Need to return data to caller
- Synchronous processing required
- Form submissions with confirmation
**Configuration**:
```javascript
{
responseMode: "lastNode"
}
```
**Then add Webhook Response node**:
```javascript
{
statusCode: 200,
headers: {
"Content-Type": "application/json"
},
body: {
"id": "={{$json.record_id}}",
"status": "success"
}
}
```
---
## Error Handling
### Pattern 1: Try-Catch with Error Trigger
```
Main Flow:
Webhook → [nodes...] → Success Response
Error Flow:
Error Trigger → Log Error → Slack Alert → Error Response
```
**Error Trigger Configuration**:
```javascript
{
workflowId: "current-workflow-id"
}
```
**Error Response** (if responseMode: "lastNode"):
```javascript
{
statusCode: 500,
body: {
"status": "error",
"message": "Processing failed"
}
}
```
### Pattern 2: Validation Early Exit
```
Webhook → IF (validate) → [True: Process]
└→ [False: Error Response]
```
**False Branch Response**:
```javascript
{
statusCode: 400,
body: {
"status": "error",
"message": "Invalid data: missing email"
}
}
```
### Pattern 3: Continue On Fail
**Per-node setting**: Continue even if node fails
**Use case**: Non-critical notifications
```
Webhook → Database (critical) → Slack (continueOnFail: true)
```
---
## Testing Webhooks
### 1. Use Manual Trigger
Replace Webhook with Manual Trigger for testing:
```
Manual Trigger → [set test data] → rest of workflow
```
### 2. Use curl
```bash
curl -X POST https://n8n.example.com/webhook/form-submit \
-H "Content-Type: application/json" \
-d '{"email": "test@example.com", "name": "Test User"}'
```
### 3. Use Postman/Insomnia
- Create request collection
- Test different payloads
- Verify responses
### 4. Webhook.site
- Use webhook.site for testing
- Copy webhook.site URL to your service
- View requests and debug
---
## Performance Considerations
### Large Payloads
- Webhook timeout: 120 seconds (default)
- For large data, consider async processing:
```
Webhook → Queue (Redis/DB) → Response (immediate)
Separate Workflow:
Schedule → Check Queue → Process
```
### High Volume
- Use "Execute Once" mode if processing all items together
- Consider rate limiting
- Monitor execution times
- Scale n8n instance if needed
### Retries
- Webhook calls typically don't retry automatically
- Implement retry logic on caller side
- Or use queue pattern for guaranteed processing
---
## Common Gotchas
### 1. ❌ Wrong: Accessing webhook data
```javascript
{{$json.email}} // Empty or undefined
```
### ✅ Correct
```javascript
{{$json.body.email}} // Data is under .body
```
### 2. ❌ Wrong: Response mode confusion
Using Webhook Response node with responseMode: "onReceived" (ignored)
### ✅ Correct
Set responseMode: "responseNode" to use a Respond to Webhook node (`lastNode` returns the last node's output instead)
### 3. ❌ Wrong: No validation
Assuming data is always present and valid
### ✅ Correct
Validate data early with an IF node (per-field messages: the Code validator in `n8n-error-handling`)
### 4. ❌ Wrong: Hardcoded paths
Using same path for dev/prod
### ✅ Correct
Use environment variables: `{{$env.WEBHOOK_PATH_PREFIX}}/form-submit`
---
## Real Template Examples
From n8n template library (1,085 webhook templates):
**Simple Form to Slack**:
```
Webhook → Set → Slack
```
**Payment Processing**:
```
Webhook → Verify Signature → Update Database → Send Receipt → Notify Admin
```
**Chat Bot**:
```
Webhook → Parse Command → AI Agent → Format Response → Webhook Response
```
Use `search_templates({query: "webhook"})` to find more!
---
## Checklist for Webhook Workflows
### Setup
- [ ] Choose descriptive webhook path
- [ ] Configure HTTP method (POST most common)
- [ ] Choose response mode (onReceived vs lastNode)
- [ ] Test webhook URL before connecting services
### Security
- [ ] Add authentication (token, signature, IP whitelist)
- [ ] Validate incoming data
- [ ] Sanitize user input (if storing/displaying)
- [ ] Use HTTPS (always)
### Data Handling
- [ ] Remember data is under $json.body
- [ ] Handle missing fields gracefully
- [ ] Transform data to desired format
- [ ] Log important data (for debugging)
### Error Handling
- [ ] Add Error Trigger workflow
- [ ] Validate required fields
- [ ] Return appropriate error responses
- [ ] Alert team on failures
### Testing
- [ ] Test with curl/Postman
- [ ] Test error scenarios
- [ ] Verify response format
- [ ] Monitor first executions
---
## Summary
**Key Points**:
1. **Data under $json.body** (most common mistake!)
2. **Validate early** to catch bad data
3. **Choose response mode** based on use case
4. **Secure webhooks** with auth
5. **Handle errors** gracefully
**Pattern**: Webhook → Validate → Transform → Action → Response
**Related**:
- [n8n Expression Syntax](../../n8n-expression-syntax/SKILL.md) - Accessing webhook data correctly
- [http_api_integration.md](http_api_integration.md) - Making HTTP requests in response