Skip to main content

Browser Automation

📖 Best for: developers, ops, testers, data analysts who need to scrape data from API-less sites, run automated tests, or fill forms

📖 Reading time: 5 minutes

📖 In one sentence: YingClaw has a built-in headless browser that runs on remote servers without a GUI, drives 6 tools (screenshot / computer / web_fetch / web_search_tool / http_request / playwright) via natural language, no code required, auto-manages login state and basic captchas, gets smarter with Memory System. Available to all users (developers / ops / testers / data analysts), triggered by natural language or explicit tool calls, runs on headless Chromium, no GUI dependency, Linux / Docker ready.

I. Core Value​

ValueDescription
No GUIRuns on remote server / Docker, suits CI/CD and cloud deployment
Zero codeNatural-language describes intent; AI drives the browser; no need to learn CSS Selector / XPath
Auto login stateAuto-manages credentials / cookies / sessions, no re-login on revisit
Smart error toleranceElements change → AI finds alternatives intelligently, script doesn't break
Intelligent evolutionCombined with Memory System, gets more proficient the more you use it

II. Main Capabilities​

1. screenshot — Screen Capture​

Capture current screen, returns file path + base64 encoding; suits screenshot archiving, visual verification, debugging anomalies.

2. computer — Mouse & Keyboard Interaction​

Supports click, double-click, drag, scroll, key press, text input; normalized coordinates 0-999 (top-left [0,0], bottom-right [999,999]), consistent across resolutions; auto-switches strategy after 3 failed operations.

3. web_fetch — Web Scraping​

Scrapes web content, auto-converts to plain text (strips HTML tags); GET only, auto-follows redirects.

4. web_search_tool — Multi-Engine Search​

Parallel multi-engine search (exa / parallel), results merged and ranked; returns Markdown brief + main snippet.

5. http_request — General HTTP​

Supports GET / POST / PUT / DELETE / PATCH; customizable headers (auth / Content-Type); suits REST API calls, data submission.

6. playwright (Optional) — Complex Scripts​

Supports multi-tab, iframe nesting, network interception; suits complex SPAs, cross-domain operations; not installed by default, pip install playwright to enable.

7. Auto Login State Management​

Credential management via Memory System, no plaintext; cookie persistence — log in once, never re-log; session reuse — shared across tabs.

8. Basic Captcha Handling​

Simple cases auto-handled: pure numeric captcha, simple slider; complex cases need human: reCAPTCHA / behavior verification; suits internal systems / small sites, not large commercial platforms (strict anti-bot).

III. Typical Use Cases​

Use Case 1: Competitor Price Scraping​

"Log into JD with my account, scrape current prices of 5 SKUs, generate Excel comparison"
→ auto-login (creds from memory) → search → scrape prices → write Excel

Batch data scraping on no-API e-commerce sites, 10× efficiency boost.

Use Case 2: Auto Form Filling​

"Open this signup page, fill name/phone/email (from my memory), check agreement, submit"
→ computer tool: click input → type → check → submit

Repetitive forms (HR signup / surveys / info registration) processed in one go.

Use Case 3: Responsive Testing​

"Capture homepage at 1920x1080, 1366x768, 375x667 respectively, check layout"
→ screenshot switches 3 resolutions → capture each → AI compares

Frontend self-test responsive layout without opening multiple browser windows.

Use Case 4: Data Collection​

"Open Sina News homepage, scrape all headline titles + URLs + publish times"
→ web_fetch → parse HTML → extract fields → format output

Sentiment monitoring possible even on sites with no RSS / API.

Use Case 5: E2E Automation Testing​

"Use test account to: login → add to cart → checkout → pay → submit, screenshot each step"
→ computer simulates full flow → screenshot archive → AI verifies

QA E2E tests with no Playwright code, just describe in natural language.

IV. Usage Guide​

Step 1: Issue with natural language — say "log into XX with my account and scrape YY" or "auto-fill and submit this form"; YingClaw auto-picks tools + sequences steps.

Step 2: Provide credentials — don't write account/password in chat; store in Memory System first (memory_store key="jd_account"), YingClaw auto-retrieves from memory.

Step 3: View screenshots / results — YingClaw auto-screenshots key nodes during operations; after task completion returns result summary + screenshot file paths.

Step 4: Handle captchas — simple captchas auto-handled; complex captchas YingClaw pauses for you to intervene, then say "continue" to resume.

Step 5: Error handling and rollback — on failure, YingClaw auto-captures error screenshot + preserves operation history; one-click retry / skip / rollback to last step; critical actions (payment / delete) auto-pause for human confirmation.

V. Best Practices​

  • Store credentials in memory system — memory_store key="account_xxx", AI auto-retrieves, no plaintext in code
  • Rate-limit to prevent bans — add sleep 2-5s between batch operations; use http_request for high-frequency scraping (browsers get anti-bot blocked easily)
  • Use stable selectors — prefer name / data-testid / aria-label over dynamic class; elements change → script still works
  • Capture screenshots as evidence — screenshot key steps for archive, see the scene at a glance on failure
  • Prefer http_request for batch tasks — if API exists, call API, 100× faster than browser; reserve browser for human-interaction scenarios
  • Clear cookies regularly — test account cookies easily pollute, clean periodically to avoid state confusion
  • Human-confirm sensitive actions — operations involving payment / delete / submit must have human confirmation to avoid accidents
  • Set reasonable timeouts — long tasks need timeout protection to prevent headless browser from hanging
  • Detailed error logs — on failure, record screenshot + current URL + operation steps + error info for easy debugging
  • Use isolation for complex ops — complex / high-risk operations use isolated sessions (session_target="isolated"), don't pollute main conversation context