Skip to content
cccskills
FreePublished on GitHub

webcrawl

网页抓取与开放检索。Jina Reader（免费零配置）+ Firecrawl + Tavily + Exa + SerpAPI。题目背景涉及行业 / 政策 / 经济时启用；纯数学 / 物理机理题不启用。

The English summary is being prepared. Showing the original description.

woodfishhhh/EZ_math_model
See how to install

Files included (1)

  • SKILL.md2.3 KB

SKILL.md (original)

Review the instructions the agent will receive before you install.

webcrawl — 网页抓取与开放检索

何时使用

  • 题目纯数学 / 物理机理 → 不要启用。
  • 题目涉及行业 / 政策 / 地理 / 经济 → 启用。
  • 评价类题目需要业界基准 / 标准 → 启用。

入口决策树

有具体 URL 要读 → Jina Reader（免费，零配置）
  失败 → Firecrawl scrape（要 EZMM_FIRECRAWL_KEY）
  失败 → 让 LLM 用 WebFetch（最后兜底）

要先搜后读 → Tavily（要 EZMM_TAVILY_KEY）
            / Exa（要 EZMM_EXA_KEY）
            / SerpAPI（要 EZMM_SERPAPI_KEY）
全部不可用 → 让用户给具体 URL，再走单页抓取

命令模板

Jina Reader（免费）

# 任意 URL → markdown，零配置
$url = 'https://example.com/article'
Invoke-RestMethod -Uri "https://r.jina.ai/$url" -OutFile workdir/.../web/$(($url | Get-FileHash -Algorithm MD5).Hash.Substring(0,8)).md

Firecrawl

import requests, os
r = requests.post(
    "https://api.firecrawl.dev/v1/scrape",
    headers={"Authorization": f"Bearer {os.environ['EZMM_FIRECRAWL_KEY']}"},
    json={"url": "https://...", "formats": ["markdown"]},
    timeout=60,
).json()
markdown = r["data"]["markdown"]

Tavily / Exa / SerpAPI

参考 tools/webcrawl/SKILL.md 的代码段。

落盘规范

抓回的页面落到：

workdir/{task_id}/attachments/external/web/<sha8>.md

文件头三行写：

<!-- source: tavily | firecrawl | jina | exa | serp -->
<!-- fetched_at: 2026-05-20T10:00:00+08:00 -->
<!-- url: https://... -->

礼貌爬取

  • 单源间隔 ≥ 1s
  • 尊重 robots.txt（除非用户授权）
  • 不绕过登录 / 反爬
  • 大批量爬取走 Firecrawl，不裸 requests

失败诊断

情况处理
全部 key 缺失仅 Jina 可用；写诊断说明可用工具集
单页 4xx / 5xx退避重试 1 次；仍失败记诊断
抓回登录墙 / 验证码不绕过；写诊断
超时 > 60s中断，写诊断

Reviews

No reviews yet. Tell others what you thought.

Comments

No login needed. Share how it worked for you, ask a question, or note where it did not run.

No comments yet. Be the first to write one.

Comments are published right away. Do not include personal information such as email addresses or phone numbers. Up to 2 URLs.

0 / 1,000 characters

This form is protected by reCAPTCHA, and the Google Privacy Policy and Terms of Service apply.

More skills from this repository

agent-research-aggregator

Free

Pre-pipeline aggregator that scans AI agent cache directories (.claude, .cursor, .antigravity, .openclaw) or any user-specified directory for experimentation logs, extracts insights and numeric results, and formats them as PaperOrchestra-ready inputs (idea.md + experimental_log.md). TRIGGER when the user says "aggregate my agent logs for paper writing", "extract experiments from my coding agent history", "prepare PaperOrchestra inputs from my cache", "turn my agent logs into a paper", mentions a folder or directory they want to use as the basis for a paper, or wants to run PaperOrchestra but only has scattered agent experiment histories rather than structured inputs. Run this BEFORE paper-orchestra. Also called automatically by paper-orchestra when workspace/inputs/idea.md or workspace/inputs/experimental_log.md are missing.

The English summary is being prepared. Showing the original description.

woodfishhhh/EZ_math_model43Updated Jul 28, 2026

brainstorming

Free

Use when EZ_math_model model selection is unclear after the modeling decision tree, the problem spans multiple domains, or modeler needs several candidate approaches before writing modeling_plan.md.

The English summary is being prepared. Showing the original description.

woodfishhhh/EZ_math_model43Updated Jul 28, 2026

content-refinement-agent

Free

Step 5 of the PaperOrchestra pipeline (arXiv:2604.05018). Iteratively refine drafts/paper.tex by simulating peer review and applying targeted revisions, with strict accept/revert halt rules. Maintains a worklog and snapshots each iteration so revert is real, not symbolic. TRIGGER when the orchestrator delegates Step 5 or when the user asks to "refine the draft", "iterate on the paper", or "run peer review on this paper".

The English summary is being prepared. Showing the original description.

woodfishhhh/EZ_math_model43Updated Jul 28, 2026

dataset

Free

公开数据集发现入口。覆盖 Kaggle / UCI / HuggingFace / 天池。题目要求"自行 查找数据"或"补充外部数据"时启用；附件已含数据时**不调用**。

The English summary is being prepared. Showing the original description.

woodfishhhh/EZ_math_model43Updated Jul 28, 2026

dispatching-parallel-agents

Free

Use when EZ_math_model runs in multi or hybrid mode and has two or more independent subtasks that can be assigned to subagents without shared files, shared state, or dataflow dependencies.

The English summary is being prepared. Showing the original description.

woodfishhhh/EZ_math_model43Updated Jul 28, 2026

docx

Free

Use when EZ_math_model needs to convert paper.md to paper.docx, read a DOCX problem statement, preserve equations and tables, or package a modeling report as a Word document.

The English summary is being prepared. Showing the original description.

woodfishhhh/EZ_math_model43Updated Jul 28, 2026

See all skills by woodfishhhh

Report a problem with this skill