references/examples.md
# Examples and Patterns
Common browser automation patterns and complete examples.
## Basic Patterns
### Login Flow
```javascript
// 1. Navigate to login page
browser_navigate(url="https://example.com/login")
// 2. Get snapshot to find element refs
browser_snapshot()
// 3. Fill username (ref from snapshot)
browser_type(element="Username input", ref="ref-username", text="myusername")
// 4. Fill password (ref from snapshot)
browser_type(element="Password input", ref="ref-password", text="mypassword")
// 5. Click login button (ref from snapshot)
browser_click(element="Login button", ref="ref-login-btn")
// 6. Wait for login to complete
browser_wait_for(text="Welcome")
// 7. Verify with screenshot
browser_take_screenshot(filename="login-success.png")
```
### Search and Extract
```javascript
// Navigate and search
browser_navigate(url="https://example.com")
browser_snapshot()
// Find search box ref from snapshot, then type and submit
browser_type(element="Search box", ref="ref-search", text="query", submit=true)
// Wait for results
browser_wait_for(text="Results")
// Capture results
browser_take_screenshot(filename="results.png")
browser_snapshot() // Get new snapshot with results
```
### Form Filling
```javascript
browser_navigate(url="https://example.com/form")
browser_snapshot()
// Fill multiple fields
browser_type(element="Name", ref="ref-name", text="John Doe")
browser_type(element="Email", ref="ref-email", text="john@example.com")
browser_select_option(element="Country", ref="ref-country", values=["USA"])
browser_type(element="Message", ref="ref-message", text="Hello world")
// Submit form
browser_click(element="Submit", ref="ref-submit")
browser_wait_for(text="Thank you")
```
## Advanced Patterns
### Multi-Tab Workflow
```javascript
// Open first tab
browser_navigate(url="https://example.com/page1")
browser_snapshot()
browser_click(element="Link", ref="ref-link1")
// Open second tab in side panel
browser_tabs(action="new", position="side")
browser_navigate(url="https://example.com/page2", viewId="new-tab-id")
// Switch between tabs
browser_tabs(action="select", index=0)
browser_snapshot() // Work with first tab
browser_tabs(action="select", index=1)
browser_snapshot() // Work with second tab
```
### Dynamic Content Handling
```javascript
browser_navigate(url="https://example.com/dynamic")
browser_snapshot()
// Click button that loads content
browser_click(element="Load more", ref="ref-load-more")
// Wait for new content
browser_wait_for(textGone="Loading...")
browser_wait_for(text="New content loaded")
// Get new snapshot with updated elements
browser_snapshot()
// Interact with new elements
browser_click(element="New button", ref="ref-new-button")
```
### Error Debugging
```javascript
browser_navigate(url="https://example.com")
browser_snapshot()
// Check for console errors
const consoleMessages = browser_console_messages()
// Look for errors in output
// Check network requests
const networkRequests = browser_network_requests()
// Look for failed requests
// Take screenshot for visual debugging
browser_take_screenshot(filename="debug.png")
```
### Character-by-Character Input
For elements that require key event handlers:
```javascript
browser_snapshot()
browser_type(
element="Code editor",
ref="ref-editor",
text="console.log('hello')",
slowly=true // Types character by character
)
```
## Integration with Snapshot Query
Combine with snapshot-query for element discovery. See [references/snapshot-query.md](references/snapshot-query.md) for complete guide.
**Basic pattern:**
```javascript
// 1. Get snapshot
browser_snapshot()
// Snapshot saved to: C:\Users\{username}\.cursor\browser-logs\snapshot-{timestamp}.log
// 2. Query snapshot to find element refs
const result = mcp_snapshot-query_find_by_name(
file_path="snapshot-2026-01-09T15-00-42-849Z.log",
name="搜索"
)
// 3. Use ref from query result
browser_click(element="搜索", ref=result.ref)
```
**Using BM25 for better matching:**
```javascript
browser_snapshot()
const results = mcp_snapshot-query_find_by_name_bm25(
file_path="snapshot.log",
name="submit button",
top_k=1
)
browser_click(element="Submit", ref=results[0].ref)
```
## Best Practices
### Always Snapshot Before Interaction
```javascript
// ❌ Wrong: No snapshot
browser_navigate(url="https://example.com")
browser_click(element="Button", ref="ref-unknown") // Ref not available
// ✅ Correct: Snapshot first
browser_navigate(url="https://example.com")
browser_snapshot() // Get refs
browser_click(element="Button", ref="ref-from-snapshot")
```
### Wait for Dynamic Content
```javascript
// ❌ Wrong: No wait
browser_click(element="Load", ref="ref-load")
browser_click(element="Result", ref="ref-result") // May not exist yet
// ✅ Correct: Wait for content
browser_click(element="Load", ref="ref-load")
browser_wait_for(text="Content loaded")
browser_snapshot() // Get new refs
browser_click(element="Result", ref="ref-result")
```
### Handle Page Changes
```javascript
// After navigation or page change, get new snapshot
browser_navigate(url="https://example.com/page1")
browser_snapshot()
browser_click(element="Link", ref="ref-link")
// Page changed, get new snapshot
browser_snapshot() // New refs needed
browser_click(element="New page button", ref="ref-new-button")
```
### Error Handling Pattern
```javascript
try {
browser_navigate(url="https://example.com")
browser_snapshot()
browser_click(element="Button", ref="ref-button")
browser_wait_for(text="Success")
} catch (error) {
// Debug on failure
browser_console_messages()
browser_take_screenshot(filename="error.png")
throw error
}
```
references/snapshot-format.md
# Snapshot File Format
Detailed documentation of snapshot log file structure and format.
## File Location
Snapshot files are saved to:
```
C:\Users\{username}\.cursor\browser-logs\snapshot-{timestamp}.log
```
## File Naming
Format: `snapshot-{ISO 8601 timestamp}.log`
Examples:
- `snapshot-2026-01-09T15-00-42-849Z.log`
- `snapshot-2026-01-09T16-30-15-123Z.log`
Timestamp format: `YYYY-MM-DDTHH-MM-SS-millisecondsZ` (UTC)
## File Format
Snapshot files use **YAML format** representing the page's accessibility tree structure.
### Basic Structure
Each element contains:
```yaml
- role: {element role}
ref: {unique reference identifier}
name: {optional element name/text}
children:
- {child elements}
```
### Field Descriptions
#### `role` (required)
Element role type following WAI-ARIA role specification.
Common values:
- `generic`: Generic container element
- `link`: Link
- `button`: Button
- `textbox`: Text input field
- `img`: Image
- `list`: List container
- `listitem`: List item
- `heading`: Heading
- `pagedescription`: Page description
- And others following WAI-ARIA standards
#### `ref` (required)
Unique reference identifier for the element.
- Format: `ref-{random string}`
- Examples: `ref-zketxgetcys`, `ref-b8rs5tdhk3e`
- **Critical**: This `ref` value is used for all element interactions (click, type, etc.)
- **Page-specific**: Refs are only valid for the current page state
- **Temporary**: Refs change with each new snapshot
#### `name` (optional)
Element name or text content.
Typically contains:
- Button text
- Link text
- Input field labels
- Image alt text
- Other accessibility text (what screen readers would read)
#### `children` (optional)
List of child elements.
- If element has children, this field contains array of child elements
- Child elements follow the same structure
- Represents the accessibility tree hierarchy
## Example
```yaml
- role: generic
ref: ref-zketxgetcys
children:
- role: img
ref: ref-zd3798voq9
- role: pagedescription
name: 欢迎进入 腾讯网,盲人用户使用操作智能引导，请按快捷键Ctrl+Alt+R；阅读详细操作说明请按快捷键Ctrl+Alt+问号键。
ref: ref-us13t9giybd
children:
- role: img
ref: ref-z0obxnpx1y
- role: generic
ref: ref-p37ecs217hp
children:
- role: generic
ref: ref-6z2ca9bkkxf
children:
- role: generic
ref: ref-wuz1gvkset
children:
- role: generic
ref: ref-62u5o5sunu
children:
- role: generic
ref: ref-r2kez4jj9y
children:
- role: link
ref: ref-mzigpg3ijr
- role: generic
ref: ref-zhx4wavxy6q
children:
- role: generic
ref: ref-sh2bokrxotn
children:
- role: textbox
ref: ref-b8rs5tdhk3e
- role: button
name: 搜索
ref: ref-b9k8zlttiah
children:
- role: img
ref: ref-z4ue1duqv2
```
## File Size
- **Simple pages**: Few KB to tens of KB
- **Complex pages** (news portals, SPAs): Can reach 100+ KB
- Example: Tencent homepage snapshot ~88 KB (1590 lines)
## Use Cases
### 1. Element Discovery
Find element references for browser automation:
```yaml
# Search for button in snapshot file
- role: button
name: 搜索
ref: ref-b9k8zlttiah
```
### 2. Page Structure Analysis
Understand page hierarchy and element relationships.
### 3. Accessibility Analysis
Check element roles and names for accessibility compliance.
### 4. Historical Comparison
Compare snapshots from different times to track page changes.
## Querying Snapshots
For complete snapshot querying guide, see [references/snapshot-query.md](references/snapshot-query.md).
### Quick Examples
**Command line:**
```bash
# Find element by name
uvx snapshot-query snapshot.log find-name "搜索"
# Find by role
uvx snapshot-query snapshot.log find-role button
# Find by ref
uvx snapshot-query snapshot.log find-ref ref-b9k8zlttiah
```
**MCP tools:**
```javascript
mcp_snapshot-query_find_by_name(
file_path="snapshot-2026-01-09T15-00-42-849Z.log",
name="搜索"
)
```
### Using grep/ripgrep
**Windows (PowerShell):**
```powershell
Select-String -Path "snapshot-*.log" -Pattern "搜索"
Select-String -Path "snapshot-*.log" -Pattern "role: button"
```
**Linux/Mac:**
```bash
grep "搜索" snapshot-*.log
grep "role: button" snapshot-*.log
```
### Using Python
```python
import yaml
with open('snapshot.log', 'r', encoding='utf-8') as f:
data = yaml.safe_load(f)
def find_buttons(items):
buttons = []
for item in items:
if item.get('role') == 'button':
buttons.append(item)
if 'children' in item:
buttons.extend(find_buttons(item['children']))
return buttons
buttons = find_buttons(data)
for button in buttons:
print(f"Button: {button.get('name', 'N/A')}, ref: {button.get('ref')}")
```
## Important Notes
1. **Ref Validity**: Refs are only valid for the current page state. After navigation or page changes, get a new snapshot.
2. **File Growth**: Each `browser_snapshot()` call creates a new file. Clean up old logs periodically.
3. **Privacy**: Snapshot files may contain page content. Protect sensitive information.
4. **YAML Parsing**: Use proper YAML parser (e.g., `yaml.safe_load()` in Python) to avoid parsing errors.
5. **Encoding**: Files use UTF-8 encoding to support international characters.
references/snapshot-query.md
# Snapshot Query Guide
Query and analyze snapshot log files generated by cursor-ide-browser to find element references for browser automation.
## Installation
Run without installation using `uvx`:
```bash
uvx snapshot-query <file_path> <command> [args]
```
Or install via pip:
```bash
pip install snapshot-query
```
## Core Concepts
### Snapshot Files
Snapshot files are YAML-formatted accessibility tree structures saved by cursor-ide-browser:
- **Location**: `C:\Users\{username}\.cursor\browser-logs\`
- **Format**: `snapshot-{ISO 8601 timestamp}.log`
- **Structure**: Tree of elements with `role`, `ref`, `name`, and `children`
### Element Properties
- **`role`**: Element type (button, link, textbox, etc.)
- **`ref`**: Unique reference identifier (used for browser interactions)
- **`name`**: Element text/label (optional)
- **`children`**: Child elements (optional)
## Quick Start
Find button reference for browser automation:
```bash
browser_snapshot() # Get page snapshot
uvx snapshot-query snapshot.log find-name "search" # Find element
# Extract ref from output, then use: browser_click(element="Search", ref="ref-xxxxx")
```
Find input field:
```bash
uvx snapshot-query snapshot.log find-role textbox
# Or: uvx snapshot-query snapshot.log find-name "username"
```
List all interactive elements:
```bash
uvx snapshot-query snapshot.log interactive
```
## Commands
### Basic Queries
- `find-name "text"` - Fuzzy search by name
- `find-name-exact "text"` - Exact match
- `find-name-bm25 "text" [top_k]` - Relevance-ranked (BM25)
- `find-role button` - Find by role
- `find-ref ref-xxxxx` - Find by reference ID
### Advanced Queries
- `find-grep "pattern" [field]` - Regular expression (field: name/role/ref)
- `find-selector "selector"` - CSS/jQuery selector (e.g., `button[name='search']`)
- `find-text "text"` - Text content search
### Utilities
- `interactive` - List interactive elements
- `count` - Count by type
- `path ref-xxxxx` - Show element path
- `all-refs` - List all refs
## Python API
```python
from snapshot_query import SnapshotQuery
query = SnapshotQuery("snapshot.log")
buttons = query.find_by_role("button")
element = query.find_by_ref("ref-xxxxx")
# Advanced: BM25, CSS selectors, regex
results = query.find_by_name_bm25("search", top_k=5)
results = query.find_by_selector("button[name='search']")
results = query.find_by_regex("^search$", field="name")
```
## MCP Integration
Configure in `~/.cursor/mcp.json`:
```json
{
"mcpServers": {
"snapshot-query": {
"command": "snapshot-query-mcp",
"args": []
}
}
}
```
### Available MCP Tools
- `mcp_snapshot-query_find_by_name` - Find elements by name (fuzzy match)
- `mcp_snapshot-query_find_by_name_bm25` - Find by name with BM25 relevance ranking
- `mcp_snapshot-query_find_by_role` - Find elements by role
- `mcp_snapshot-query_find_by_ref` - Find element by reference ID
- `mcp_snapshot-query_find_by_text` - Find elements containing text
- `mcp_snapshot-query_find_by_regex` - Find using regular expression
- `mcp_snapshot-query_find_by_selector` - Find using CSS/jQuery selector
- `mcp_snapshot-query_find_interactive_elements` - List all interactive elements
- `mcp_snapshot-query_count_elements` - Count elements by type
- `mcp_snapshot-query_get_element_path` - Get element path in tree
- `mcp_snapshot-query_extract_all_refs` - Extract all reference IDs
### MCP Usage Examples
```javascript
// Find element by name
mcp_snapshot-query_find_by_name(
file_path="snapshot-2026-01-09T15-00-42-849Z.log",
name="搜索"
)
// Find with relevance ranking
mcp_snapshot-query_find_by_name_bm25(
file_path="snapshot.log",
name="login",
top_k=5
)
// Find by role
mcp_snapshot-query_find_by_role(
file_path="snapshot.log",
role="button"
)
// Find using CSS selector
mcp_snapshot-query_find_by_selector(
file_path="snapshot.log",
selector="button[name='search']"
)
```
## Integration with Browser Automation
### Complete Workflow
```javascript
// 1. Navigate and snapshot
browser_navigate(url="https://example.com")
browser_snapshot()
// Snapshot saved to: C:\Users\{username}\.cursor\browser-logs\snapshot-{timestamp}.log
// 2. Query snapshot to find element
const result = mcp_snapshot-query_find_by_name(
file_path="snapshot-2026-01-09T15-00-42-849Z.log",
name="搜索"
)
// 3. Use ref from query result
browser_click(element="搜索", ref=result.ref)
```
### Finding Multiple Elements
```javascript
browser_snapshot()
// Find username input
const username = mcp_snapshot-query_find_by_name(
file_path="snapshot.log",
name="username"
)
// Find password input
const password = mcp_snapshot-query_find_by_name(
file_path="snapshot.log",
name="password"
)
// Use refs
browser_type(element="Username", ref=username.ref, text="user")
browser_type(element="Password", ref=password.ref, text="pass")
```
### Using BM25 for Better Results
When element name is ambiguous or has variations:
```javascript
browser_snapshot()
// BM25 finds most relevant matches
const results = mcp_snapshot-query_find_by_name_bm25(
file_path="snapshot.log",
name="submit button",
top_k=3
)
// Use top result
browser_click(element="Submit", ref=results[0].ref)
```
## Best Practices
1. **Prefer BM25 for relevance ranking** - `find-name-bm25` is better than `find-name` for ambiguous queries
2. **Use CSS selectors for flexibility** - `find-selector "button[name='search']"` for precise queries
3. **Refs change with each snapshot** - Get a new snapshot before using refs
4. **Use path command to understand hierarchy** - `path ref-xxxxx` shows element location in tree
5. **Query after snapshot** - Always query the most recent snapshot file
## Common Patterns
### Find and Click Button
```bash
browser_snapshot()
uvx snapshot-query snapshot.log find-name "submit"
browser_click(element="Submit", ref="ref-from-output")
```
Or with MCP:
```javascript
browser_snapshot()
const button = mcp_snapshot-query_find_by_name_bm25(
file_path="snapshot.log",
name="submit",
top_k=1
)
browser_click(element="Submit", ref=button.ref)
```
### Fill Form
```bash
uvx snapshot-query snapshot.log find-name "username" # Get ref-username
uvx snapshot-query snapshot.log find-name "password" # Get ref-password
browser_type(element="Username", ref="ref-username", text="user")
browser_type(element="Password", ref="ref-password", text="pass")
```
Or with MCP:
```javascript
browser_snapshot()
const username = mcp_snapshot-query_find_by_name(file_path="snapshot.log", name="username")
const password = mcp_snapshot-query_find_by_name(file_path="snapshot.log", name="password")
browser_type(element="Username", ref=username.ref, text="user")
browser_type(element="Password", ref=password.ref, text="pass")
```
### Find All Buttons
```bash
uvx snapshot-query snapshot.log find-role button
```
Or with MCP:
```javascript
const buttons = mcp_snapshot-query_find_by_role(file_path="snapshot.log", role="button")
// buttons contains all button elements with their refs
```
### Find Interactive Elements
```bash
uvx snapshot-query snapshot.log interactive
```
Or with MCP:
```javascript
const interactive = mcp_snapshot-query_find_interactive_elements(file_path="snapshot.log")
// Returns all clickable, typeable, selectable elements
```
## Troubleshooting
- **Can't find element**: Try `find-name-bm25` or check spelling
- **Ref not working**: Refs change with each snapshot - get a new snapshot
- **Too many results**: Use `find-name-bm25` with `top_k` or more specific selectors
- **Element not found**: Verify snapshot file path is correct and file exists
- **Multiple matches**: Use BM25 with `top_k=1` to get best match, or use CSS selector for precision
## Command Line vs MCP
**Command line** (`uvx snapshot-query`):
- Good for quick one-off queries
- Outputs text that needs parsing
- Requires file path as argument
**MCP tools** (`mcp_snapshot-query_*`):
- Better for programmatic integration
- Returns structured data
- Can be used directly in automation workflows
- Recommended for browser automation integration
references/tools.md
# Complete Tool Reference
All available browser automation tools with full parameter details.
## Navigation Tools
### browser_navigate
Navigate to a URL.
**Parameters:**
- `url` (string, required): Target URL
- `viewId` (string, optional): Browser tab ID. Omit to use last interacted tab
- `position` (string, optional):
- `"active"` (default): Open in current editor group
- `"side"`: Open in side panel (use when user mentions "side", "beside", "side panel")
**Example:**
```javascript
browser_navigate(url="https://example.com", position="side")
```
### browser_navigate_back
Navigate back to previous page.
**Parameters:**
- `viewId` (string, optional): Browser tab ID
**Example:**
```javascript
browser_navigate_back()
```
## Page Information Tools
### browser_snapshot
Capture accessibility snapshot of current page. **Required before any element interaction.**
**Parameters:**
- `viewId` (string, optional): Browser tab ID
**Returns:**
- Accessibility tree structure
- Element references (`ref`) for all interactive elements
- Element roles, names, and states
- Snapshot also saved to local log file
**Snapshot Log Files:**
- Location: `C:\Users\{username}\.cursor\browser-logs\snapshot-{timestamp}.log`
- Format: YAML accessibility tree
- Filename: `snapshot-{ISO 8601 timestamp}.log`
**Example:**
```javascript
browser_snapshot()
```
### browser_take_screenshot
Capture page screenshot.
**Parameters:**
- `type` (string, optional): Image format, default `"png"`
- `filename` (string, optional): Save filename. Default: `page-{timestamp}.{png|jpeg}`
- `element` (string, optional): Element description (for element screenshot)
- `ref` (string, optional): CSS selector (for element screenshot)
- `fullPage` (boolean, optional): Capture full scrollable page, default `false`
- `viewId` (string, optional): Browser tab ID
**Example:**
```javascript
browser_take_screenshot(fullPage=true)
browser_take_screenshot(element="Login form", ref="form#login")
```
### browser_console_messages
Get all console messages from page.
**Parameters:**
- `viewId` (string, optional): Browser tab ID
**Returns:**
- All console messages (errors, warnings, logs)
**Example:**
```javascript
browser_console_messages()
```
### browser_network_requests
Get all network requests since page load.
**Parameters:**
- `viewId` (string, optional): Browser tab ID
**Returns:**
- Network request details (URL, method, status, response)
**Example:**
```javascript
browser_network_requests()
```
## Element Interaction Tools
### browser_click
Click a page element.
**Parameters:**
- `element` (string, required): Element description (for permission)
- `ref` (string, required): Element reference from snapshot
- `doubleClick` (boolean, optional): Double click, default `false`
- `button` (string, optional): Mouse button, default `"left"`
- `modifiers` (array, optional): Modifier keys (e.g., `["Control", "Shift"]`)
- `viewId` (string, optional): Browser tab ID
**Example:**
```javascript
browser_click(element="Login button", ref="ref-login-btn", doubleClick=false)
```
### browser_type
Type text into editable element.
**Parameters:**
- `element` (string, required): Element description (for permission)
- `ref` (string, required): Element reference from snapshot
- `text` (string, required): Text to type
- `submit` (boolean, optional): Submit (press Enter), default `false`
- `slowly` (boolean, optional): Type character by character (for key handlers), default `false`
- `viewId` (string, optional): Browser tab ID
**Example:**
```javascript
// Normal typing
browser_type(element="Username", ref="ref-username", text="myusername")
// Type and submit
browser_type(element="Search", ref="ref-search", text="query", submit=true)
// Character by character (triggers key events)
browser_type(element="Code editor", ref="ref-editor", text="console.log('hello')", slowly=true)
```
### browser_hover
Hover over element.
**Parameters:**
- `element` (string, required): Element description (for permission)
- `ref` (string, required): Element reference from snapshot
- `viewId` (string, optional): Browser tab ID
**Example:**
```javascript
browser_hover(element="Menu item", ref="ref-menu-item")
```
### browser_select_option
Select option in dropdown.
**Parameters:**
- `element` (string, required): Element description (for permission)
- `ref` (string, required): Element reference from snapshot
- `values` (array, required): Values to select (single or multiple)
- `viewId` (string, optional): Browser tab ID
**Example:**
```javascript
// Single select
browser_select_option(element="Country", ref="ref-country", values=["USA"])
// Multi-select
browser_select_option(element="Tags", ref="ref-tags", values=["tag1", "tag2"])
```
### browser_press_key
Press keyboard key.
**Parameters:**
- `key` (string, required): Key name (e.g., `"ArrowLeft"` or character `"a"`)
- `viewId` (string, optional): Browser tab ID
**Example:**
```javascript
browser_press_key(key="ArrowLeft")
browser_press_key(key="a")
// For key combinations, call multiple times
browser_press_key(key="Control")
browser_press_key(key="c")
```
## Synchronization Tools
### browser_wait_for
Wait for text to appear/disappear or wait for time.
**Parameters:**
- `time` (number, optional): Wait time in seconds
- `text` (string, optional): Text to wait for
- `textGone` (string, optional): Text to wait for to disappear
- `viewId` (string, optional): Browser tab ID
**Note:** At least one of `time`, `text`, or `textGone` must be provided.
**Example:**
```javascript
browser_wait_for(text="Loading complete")
browser_wait_for(textGone="Loading...")
browser_wait_for(time=3)
browser_wait_for(text="Page loaded", time=5)
```
## Window Management Tools
### browser_resize
Resize browser window.
**Parameters:**
- `width` (number, required): Window width
- `height` (number, required): Window height
- `viewId` (string, optional): Browser tab ID
**Example:**
```javascript
browser_resize(width=1920, height=1080)
```
### browser_tabs
List, create, close, or select browser tabs.
**Parameters:**
- `action` (string, required): Operation type
- `"list"`: List all tabs
- `"new"`: Create new tab
- `"close"`: Close tab
- `"select"`: Select tab
- `index` (number, optional):
- For `"select"`: Required, tab index to select
- For `"close"`: Optional, default closes current tab
- `position` (string, optional): Only for `"new"` action
- `"active"` (default): Open in current editor group
- `"side"`: Open in side panel
**Example:**
```javascript
browser_tabs(action="list")
browser_tabs(action="new", position="side")
browser_tabs(action="select", index=0)
browser_tabs(action="close")
```
SKILL.md
---
name: cursor-ide-browser-skills
description: Browser automation in Cursor IDE using MCP protocol server `cursor-ide-browser`. Use when (1) Automating web interactions in Cursor IDE, (2) Navigating web pages, (3) Clicking buttons or links, (4) Filling forms or input fields, (5) Taking screenshots or capturing page snapshots, (6) Debugging web pages by checking console messages or network requests, (7) Extracting information from web pages, (8) Testing web applications, (9) Interacting with web-based documentation or tools, (10) Any task requiring programmatic browser control within Cursor IDE
metadata:
short-description: Browser automation in Cursor IDE via MCP
---
# Cursor IDE Browser Automation
Browser automation tool for Cursor IDE using MCP (Model Context Protocol) server `cursor-ide-browser` and accessibility snapshots for precise element interaction.
## Core Mechanism
**Accessibility Snapshot First**: Always get a snapshot before interacting with elements. The snapshot provides structured page information with element references (`ref`) needed for all interactions.
```javascript
// Standard workflow
browser_navigate(url="https://example.com")
browser_snapshot() // Required: Get element references
browser_click(element="Button", ref="ref-from-snapshot")
```
## Essential Workflow
1. **Navigate** to target page
2. **Snapshot** to get element references (required before any interaction)
3. **Convert to Markdown** (⭐ Recommended) for easier searching, locating and reading
4. **Search with grep in md** to find information or locate interactive elements
5. **Interact** using refs from snapshot
6. **Wait** for dynamic content if needed
7. **Verify** with screenshots or console messages
**Quick example:**
```javascript
browser_navigate(url="https://example.com")
browser_snapshot() // Creates .log file
mcp_snapshot-query_convert_to_markdown(file_path="snapshot.log")
grep(pattern="button|登录", path="snapshot.md") // Find elements
browser_click(element="Login", ref="ref-from-grep-results")
```
## Key Tools
**Navigation:**
- `browser_navigate(url, position?)` - Navigate to URL
- `browser_navigate_back()` - Go back
**Page Information:**
- `browser_snapshot()` - **Required before interactions** - Get accessibility tree with element refs
- `browser_take_screenshot(fullPage?, filename?)` - Capture visual
- `browser_console_messages()` - Get console logs
- `browser_network_requests()` - Get network activity
**Element Interaction:**
- `browser_click(element, ref, doubleClick?, button?, modifiers?)` - Click element
- `browser_type(element, ref, text, submit?, slowly?)` - Type text
- `browser_hover(element, ref)` - Hover
- `browser_select_option(element, ref, values)` - Select dropdown
- `browser_press_key(key)` - Press key (supports PageDown, PageUp, ArrowDown, ArrowUp, Space, End, Home for scrolling)
**Synchronization:**
- `browser_wait_for(text?, textGone?, time?)` - Wait for text or time
**Tab Management:**
- `browser_tabs(action, index?, position?)` - Manage tabs (list/new/close/select)
## Element References
- `**element**`: Human-readable description (for permission confirmation)
- `**ref**`: Technical reference from snapshot (required for interaction)
- Refs are **page-state specific** - get a new snapshot after navigation or page changes
## Snapshot Files
Snapshots are automatically saved as YAML files:
- **Location**: `C:\Users\{username}\.cursor\browser-logs\snapshot-{timestamp}.log`
- **Format**: YAML accessibility tree with `role`, `ref`, `name`, `children`
- **Usage**: Extract `ref` values for element interactions
## Querying Snapshots
### ⭐ Recommended Workflow: Convert to Markdown + Grep
**Best practice for finding information and locating interactive elements:**
1. **Get snapshot** → Creates `.log` file
2. **Convert to Markdown** → More readable format with structured content
3. **Use grep** → Fast text search across the entire document
4. **Extract refs** → Use found refs for interactions
```javascript
// Step 1: Get page snapshot
browser_snapshot() // Creates: snapshot-2026-01-10T23-43-30-351Z.log
// Step 2: Convert to Markdown (RECOMMENDED)
mcp_snapshot-query_convert_to_markdown(
file_path="snapshot-2026-01-10T23-43-30-351Z.log",
include_ref=true
) # save to snapshot-2026-01-10T23-43-30-351Z.md
// Step 3: Search with grep (much easier than querying raw YAML)
grep(pattern="搜索|button|登录", path="snapshot.md", -i=true)
grep(pattern="^\\[.*\\]\\(ref-|^\\*\\*.*\\*\\* `ref-", path="snapshot.md") // Find all links/buttons
// Step 4: Use found refs for interaction
browser_click(element="Login button", ref="ref-found-from-grep")
```
**Why this workflow is preferred:**
- ✅ **More readable**: Markdown format is human-friendly
- ✅ **Faster search**: `grep` is more efficient than parsing YAML
- ✅ **Better context**: See surrounding content with `-C` flag
- ✅ **Easy element discovery**: Links and buttons clearly formatted
- ✅ **Preserves refs**: All element references included for interaction
**Alternative: Direct Query Tools**
For programmatic element finding, use snapshot-query MCP tools:
**Command line:**
```bash
browser_snapshot() # Generate snapshot
uvx snapshot-query snapshot.log find-name "search" # Find element
```
**MCP tools:**
```javascript
mcp_snapshot-query_find_by_name(file_path="snapshot.log", name="搜索")
mcp_snapshot-query_find_by_role(file_path="snapshot.log", role="button")
mcp_snapshot-query_find_by_text(file_path="snapshot.log", text="登录")
mcp_snapshot-query_find_by_regex(file_path="snapshot.log", pattern="\\d+\\s*ft", field="name")
mcp_snapshot-query_find_by_name_bm25(file_path="snapshot.log", name="search query", top_k=5)
mcp_snapshot-query_count_elements(file_path="snapshot.log")
mcp_snapshot-query_get_element_path(file_path="snapshot.log", ref="ref-xxx")
mcp_snapshot-query_extract_all_refs(file_path="snapshot.log")
```
**Integrated workflow:**
```javascript
browser_snapshot() // Creates snapshot file
// Query snapshot to find element ref
const result = mcp_snapshot-query_find_by_name(file_path="snapshot.log", name="Login")
browser_click(element="Login", ref=result.ref) // Use ref from query
```
**⭐ snapshot-query works with OCR results too:**
The snapshot-query tools can process OCR results from `fast-paddleocr-mcp`. After OCR processing, you get a `.snapshot.log` file that can be queried just like browser snapshots:
```javascript
// OCR generates webpage.png.snapshot.log
mcp_fast-paddleocr-mcp_ocr_image(image_path="webpage.png", language="ch")
// Query OCR results with snapshot-query
mcp_snapshot-query_find_by_text(
file_path="webpage.png.snapshot.log",
text="8 ft",
case_sensitive=false
)
// Use regex to find measurements
mcp_snapshot-query_find_by_regex(
file_path="webpage.png.snapshot.log",
pattern="\\d+\\s*ft|cm|meters?",
field="name"
)
// Semantic search for better results
mcp_snapshot-query_find_by_name_bm25(
file_path="webpage.png.snapshot.log",
name="height measurement",
top_k=5
)
// Convert to Markdown for analysis
mcp_snapshot-query_convert_to_markdown(
file_path="webpage.png.snapshot.log",
include_ref=true
)
```
See [references/snapshot-query.md](references/snapshot-query.md) for complete snapshot-query documentation.
## Common Patterns
**Login flow:**
```javascript
browser_navigate(url="https://example.com/login")
browser_snapshot()
// Find username input ref from snapshot
browser_type(element="Username", ref="ref-username", text="user")
// Find password input ref from snapshot
browser_type(element="Password", ref="ref-password", text="pass")
// Find login button ref from snapshot
browser_click(element="Login", ref="ref-login-btn")
browser_wait_for(text="Welcome")
```
**Search and extract (with Markdown workflow):**
```javascript
browser_navigate(url="https://www.baidu.com/s?wd=哈梅内伊有几个孩子")
browser_snapshot() // Creates snapshot.log
// Convert to Markdown for easier searching
mcp_snapshot-query_convert_to_markdown(
file_path="snapshot.log",
include_ref=true
)
// Search for information using grep
grep(pattern="六名|6个|子女", path="snapshot.md", -i=true, -C=3)
// Find interactive elements (links/buttons)
grep(pattern="^\\[.*\\]\\(ref-|^\\*\\*.*\\*\\* `ref-", path="snapshot.md")
// Click on found link using ref
browser_click(element="Article link", ref="ref-45py92vjdrs")
browser_wait_for(text="Results")
browser_take_screenshot(filename="results.png")
```
**Debug page issues:**
```javascript
browser_snapshot()
browser_console_messages() // Check for errors
browser_network_requests() // Check failed requests
```
**Scrolling web pages:**
```javascript
browser_press_key("PageDown") // Scroll down one page
browser_press_key("PageUp") // Scroll up one page
browser_press_key("ArrowDown") // Scroll down line by line
browser_press_key("ArrowUp") // Scroll up line by line
browser_press_key("Space") // Scroll down one screen
browser_press_key("End") // Scroll to bottom
browser_press_key("Home") // Scroll to top
browser_wait_for(time=1) // Wait after scrolling for content to load
```
**OCR processing with fast-paddleocr-mcp:**
```javascript
// Take screenshot of webpage
browser_take_screenshot(filename="webpage.png", fullPage=false)
// Process with OCR (generates .md and .snapshot.log files)
mcp_fast-paddleocr-mcp_ocr_image(
image_path="webpage.png",
language="ch" // Use "ch" for Chinese+English, "en" for English only
)
// Query OCR results with snapshot-query
mcp_snapshot-query_find_by_text(
file_path="webpage.png.snapshot.log",
text="tallest",
case_sensitive=false
)
// Use BM25 semantic search for better results
mcp_snapshot-query_find_by_name_bm25(
file_path="webpage.png.snapshot.log",
name="height tallest person",
top_k=5
)
// Convert OCR snapshot to Markdown for easier analysis
mcp_snapshot-query_convert_to_markdown(
file_path="webpage.png.snapshot.log",
include_ref=true
)
```
**Cross-verification workflow:**
```javascript
// Navigate to multiple sources for verification
browser_navigate(url="https://source1.com/article")
browser_snapshot()
// Extract information from source 1
browser_navigate(url="https://source2.com/article")
browser_snapshot()
// Extract information from source 2
// Compare and verify information consistency
// Prefer authoritative sources (Wikipedia, official records, etc.)
```
## Important Notes
1. **Always snapshot before interaction** - Refs are required and page-specific
2. **⭐ Convert to Markdown first** - Use `convert_to_markdown` + `grep` for finding information and elements (much easier than querying raw YAML)
3. **Wait for dynamic content** - Use `browser_wait_for()` for async operations
4. **Refs expire** - Get new snapshot after navigation or page changes
5. **Multi-tab support** - Use `viewId` parameter or `browser_tabs()` to manage tabs
6. **Position control** - Use `position="side"` when user mentions side panel
7. **OCR limitations** - OCR may merge adjacent text (e.g., "otherreliablesourcesccordingtoG"). Key information is usually extracted correctly, but verify important details
8. **Cross-verification** - For critical information, verify across multiple authoritative sources (Wikipedia, official records, etc.)
9. **Tool combination** - Combine browser automation + OCR + snapshot-query for comprehensive web content analysis
## Best Practices & Lessons Learned
### Workflow Optimization
1. **Standard workflow**: Navigate → Snapshot → Convert to Markdown → Search → Interact
2. **OCR workflow**: Screenshot → OCR → Query with snapshot-query → Extract information
3. **Verification workflow**: Multiple sources → Extract → Compare → Verify consistency
### Tool Integration
- **Browser + OCR**: Use `browser_take_screenshot()` + `fast-paddleocr-mcp` to extract text from visual content
- **OCR + snapshot-query**: OCR generates `.snapshot.log` files that can be queried with all snapshot-query tools
- **Markdown + grep**: Convert snapshots/OCR results to Markdown for easier searching
### Key Insights
- **snapshot-query is universal**: Works with both browser snapshots and OCR results
- **Markdown conversion is recommended**: Much easier to search and read than raw YAML
- **BM25 semantic search**: Use `find_by_name_bm25()` for better relevance when exact matches are unclear
- **Cross-verification**: Always verify critical information from multiple authoritative sources
- **OCR accuracy**: Works well for key information but may merge adjacent text - verify important details
## Detailed Reference
- **Complete tool reference**: See [references/tools.md](references/tools.md) for all tools with full parameters
- **Examples and patterns**: See [references/examples.md](references/examples.md) for detailed workflows
- **Snapshot file format**: See [references/snapshot-format.md](references/snapshot-format.md) for YAML structure details
- **Snapshot querying**: See [references/snapshot-query.md](references/snapshot-query.md) for querying snapshot files