feat: align v2 wire format with scrapegraph-py PR #84

Rebase base URL, env vars, and auth header onto the new scrapegraph-py v2
SDK contract (ScrapeGraphAI/scrapegraph-py#84):

- Base URL: /api/v2 -> /v2 (default https://api.scrapegraphai.com/v2)
- Env: SGAI_API_URL (SCRAPEGRAPH_API_BASE_URL kept as legacy alias)
- Env: SGAI_TIMEOUT_S for httpx timeout (default 120s)
- Drop Authorization: Bearer; keep SGAI-APIKEY only (matches SDK)
- Update docstrings, resources, README, server.json, .agent docs to
  reference #84 and the /v2 base URL.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Author: VinciGit00

.agent/README.md

@@ -379,7 +379,7 @@ npx @modelcontextprotocol/inspector scrapegraph-mcp
 ## 📅 Changelog
 
 ### April 2026
-- ✅ Migrated MCP client and tools to **API v2** ([scrapegraph-py#82](https://github.com/ScrapeGraphAI/scrapegraph-py/pull/82)): base `https://api.scrapegraphai.com/api/v2`, Bearer + SGAI-APIKEY, new crawl/monitor/credits/history tools; removed sitemap, agentic_scrapper, status polling tools.
+- ✅ Migrated MCP client and tools to **API v2** ([scrapegraph-py#84](https://github.com/ScrapeGraphAI/scrapegraph-py/pull/84)): base `https://api.scrapegraphai.com/v2`, `SGAI-APIKEY` header (matches SDK wire format), new crawl/monitor/credits/history tools; removed sitemap, agentic_scrapper, status polling tools. Env vars aligned with SDK: `SGAI_API_URL`, `SGAI_TIMEOUT_S`.
 
 ### January 2026
 - ✅ Added `time_range` parameter to SearchScraper for filtering results by recency (v1-era; **ignored on API v2**)

.agent/system/project_architecture.md

@@ -20,11 +20,11 @@
 The ScrapeGraph MCP Server is a production-ready [Model Context Protocol](https://modelcontextprotocol.io/introduction) (MCP) server that provides seamless integration between AI assistants (like Claude, Cursor, etc.) and the [ScrapeGraphAI API](https://scrapegraphai.com). This server enables language models to leverage advanced AI-powered web scraping capabilities with enterprise-grade reliability.
 
 **Key Capabilities (API v2):**
-- **Scrape** (`markdownify`, `scrape`) — POST `/api/v2/scrape`
-- **Extract** (`smartscraper`) — POST `/api/v2/extract` (URL-only)
-- **Search** (`searchscraper`) — POST `/api/v2/search`
-- **Crawl** — POST/GET `/api/v2/crawl` (+ stop/resume); markdown/html crawl only
-- **Monitor, credits, history** — `/api/v2/monitor`, `/credits`, `/history`
+- **Scrape** (`markdownify`, `scrape`) — POST `/v2/scrape`
+- **Extract** (`smartscraper`) — POST `/v2/extract` (URL-only)
+- **Search** (`searchscraper`) — POST `/v2/search`
+- **Crawl** — POST/GET `/v2/crawl` (+ stop/resume); markdown/html crawl only
+- **Monitor, credits, history** — `/v2/monitor`, `/credits`, `/history`
 
 **Purpose:**
 - Bridge AI assistants (Claude, Cursor, etc.) with web scraping capabilities
@@ -130,7 +130,7 @@ AI Assistant (Claude/Cursor)
     ↓ (stdio via MCP)
 FastMCP Server (this project)
     ↓ (HTTPS API calls)
-ScrapeGraphAI API (default https://api.scrapegraphai.com/api/v2)
+ScrapeGraphAI API (default https://api.scrapegraphai.com/v2)
     ↓ (web scraping)
 Target Websites
 ```
@@ -140,9 +140,9 @@ Target Websites
 The server follows a simple, single-file architecture:
 
 **`ScapeGraphClient` Class:**
-- HTTP client wrapper for ScrapeGraphAI API v2 ([scrapegraph-py#82](https://github.com/ScrapeGraphAI/scrapegraph-py/pull/82))
-- Base URL: `https://api.scrapegraphai.com/api/v2` (override with env `SCRAPEGRAPH_API_BASE_URL`)
-- Auth: `Authorization: Bearer`, `SGAI-APIKEY`, `X-SDK-Version: scrapegraph-mcp@2.0.0`
+- HTTP client wrapper for ScrapeGraphAI API v2 ([scrapegraph-py#84](https://github.com/ScrapeGraphAI/scrapegraph-py/pull/84))
+- Base URL: `https://api.scrapegraphai.com/v2` (override with env `SGAI_API_URL`)
+- Auth: `SGAI-APIKEY`, `X-SDK-Version: scrapegraph-mcp@2.0.0` (matches scrapegraph-py v2)
 - v2 methods include `scrape_v2`, `extract`, `search_api`, `crawl_*`, `monitor_*`, `credits`, `history`, plus compatibility wrappers used by MCP tools
 
 **FastMCP Server:**
@@ -391,10 +391,10 @@ If status is "completed":
 
 ### ScrapeGraphAI API
 
-**Base URL:** `https://api.scrapegraphai.com/api/v2` (configurable via `SCRAPEGRAPH_API_BASE_URL`)
+**Base URL:** `https://api.scrapegraphai.com/v2` (configurable via `SGAI_API_URL`)
 
 **Authentication:**
-- Headers: `Authorization: Bearer <key>`, `SGAI-APIKEY: <key>`
+- Headers: `SGAI-APIKEY: <key>` (matches scrapegraph-py v2 wire format)
 - Obtain API key from: [ScrapeGraph Dashboard](https://dashboard.scrapegraphai.com)
 
 **Endpoints used (v2):**

README.md

@@ -28,9 +28,15 @@ A production-ready [Model Context Protocol](https://modelcontextprotocol.io/intr
 
 ## API v2
 
-This MCP server targets **ScrapeGraph API v2** (`https://api.scrapegraphai.com/api/v2`), aligned with
-[scrapegraph-py PR #82](https://github.com/ScrapeGraphAI/scrapegraph-py/pull/82). Auth sends both
-`Authorization: Bearer` and `SGAI-APIKEY`. Override the base URL with **`SCRAPEGRAPH_API_BASE_URL`** if needed.
+This MCP server targets **ScrapeGraph API v2** (`https://api.scrapegraphai.com/v2`), aligned 1:1 with
+[scrapegraph-py PR #84](https://github.com/ScrapeGraphAI/scrapegraph-py/pull/84). Auth uses the
+`SGAI-APIKEY` header. Environment variables mirror the Python SDK:
+
+- **`SGAI_API_URL`** — override the base URL (default `https://api.scrapegraphai.com/v2`)
+- **`SGAI_TIMEOUT_S`** — request timeout in seconds (default `120`)
+- **`SGAI_API_KEY`** — API key (can also be passed via MCP `scrapegraphApiKey` or `X-API-Key` header)
+
+> `SCRAPEGRAPH_API_BASE_URL` is still honored as a legacy alias for `SGAI_API_URL`.
 
 ## Key Features
 

server.json

@@ -24,11 +24,18 @@
           "name": "SGAI_API_KEY"
         },
         {
-          "description": "Override API base URL (default https://api.scrapegraphai.com/api/v2)",
+          "description": "Override API base URL (default https://api.scrapegraphai.com/v2)",
           "isRequired": false,
           "format": "string",
           "isSecret": false,
-          "name": "SCRAPEGRAPH_API_BASE_URL"
+          "name": "SGAI_API_URL"
+        },
+        {
+          "description": "Request timeout in seconds (default 120)",
+          "isRequired": false,
+          "format": "string",
+          "isSecret": false,
+          "name": "SGAI_TIMEOUT_S"
         }
       ]
     }

src/scrapegraph_mcp/server.py

@@ -2,7 +2,7 @@
 """
 MCP server for ScapeGraph API integration (API v2).
 
-Aligned with scrapegraph-py v2 ([ScrapeGraphAI/scrapegraph-py#82](https://github.com/ScrapeGraphAI/scrapegraph-py/pull/82)):
+Aligned with scrapegraph-py v2 ([ScrapeGraphAI/scrapegraph-py#84](https://github.com/ScrapeGraphAI/scrapegraph-py/pull/84)):
 - markdownify: Page content via POST /scrape (markdown by default)
 - smartscraper: Structured extraction via POST /extract (url + prompt; schema optional)
 - searchscraper: Web search via POST /search (supports numResults, schema, prompt,
@@ -22,7 +22,11 @@
 includePatterns, excludePatterns, contentTypes, webhookUrl, contentType).
 
 Removed on v2 (no API equivalent): sitemap, agentic_scrapper, markdownify_status, smartscraper_status.
-Optional base URL override: SCRAPEGRAPH_API_BASE_URL (default https://api.scrapegraphai.com/api/v2).
+
+Environment variables (match scrapegraph-py v2):
+- SGAI_API_URL (default https://api.scrapegraphai.com/v2) — base URL override
+- SGAI_TIMEOUT_S (default 120) — request timeout in seconds
+- SCRAPEGRAPH_API_BASE_URL — legacy alias for SGAI_API_URL (still honored)
 
 ## Parameter Validation and Error Handling
 
@@ -83,11 +87,26 @@
 logger = logging.getLogger(__name__)
 
 MCP_SERVER_VERSION = "2.0.0"
-DEFAULT_API_BASE_URL = "https://api.scrapegraphai.com/api/v2"
+# Matches scrapegraph-py v2 (env.py): https://api.scrapegraphai.com/v2
+DEFAULT_API_BASE_URL = "https://api.scrapegraphai.com/v2"
 
 
 def _api_base_url() -> str:
-    return os.environ.get("SCRAPEGRAPH_API_BASE_URL", DEFAULT_API_BASE_URL).rstrip("/")
+    # SGAI_API_URL mirrors scrapegraph-py v2; SCRAPEGRAPH_API_BASE_URL is a legacy alias.
+    return (
+        os.environ.get("SGAI_API_URL")
+        or os.environ.get("SCRAPEGRAPH_API_BASE_URL")
+        or DEFAULT_API_BASE_URL
+    ).rstrip("/")
+
+
+def _api_timeout_s() -> float:
+    # SGAI_TIMEOUT_S mirrors scrapegraph-py v2 (default 120s).
+    val = os.environ.get("SGAI_TIMEOUT_S")
+    try:
+        return float(val) if val else 120.0
+    except ValueError:
+        return 120.0
 
 
 DEFAULT_SCREENSHOT_FORMAT: Dict[str, Any] = {
@@ -144,19 +163,20 @@ def _build_json_format_entry(
 
 
 class ScapeGraphClient:
-    """HTTP client for ScrapeGraphAI API v2 (see scrapegraph-py PR #82)."""
+    """HTTP client for ScrapeGraphAI API v2 (see scrapegraph-py PR #84)."""
 
     def __init__(self, api_key: str, base_url: Optional[str] = None) -> None:
         self.api_key = api_key
         self.base_url = (base_url or _api_base_url()).rstrip("/")
+        # Match scrapegraph-py v2 wire format: single SGAI-APIKEY header. We keep
+        # Content-Type/accept for broker compatibility and X-SDK-Version for telemetry.
         self.headers = {
-            "Authorization": f"Bearer {api_key}",
             "SGAI-APIKEY": api_key,
             "Content-Type": "application/json",
             "accept": "application/json",
             "X-SDK-Version": f"scrapegraph-mcp@{MCP_SERVER_VERSION}",
         }
-        self.client = httpx.Client(timeout=httpx.Timeout(120.0))
+        self.client = httpx.Client(timeout=httpx.Timeout(_api_timeout_s()))
 
     def _parse_response(self, response: httpx.Response) -> Dict[str, Any]:
         if response.status_code >= 400:
@@ -608,7 +628,7 @@ def web_scraping_guide() -> str:
     """
     return """# ScapeGraph Web Scraping Guide (API v2)
 
-See [scrapegraph-py#82](https://github.com/ScrapeGraphAI/scrapegraph-py/pull/82) for the upstream SDK migration.
+See [scrapegraph-py#84](https://github.com/ScrapeGraphAI/scrapegraph-py/pull/84) for the upstream SDK migration.
 
 ## Core tools
 - **markdownify** — `POST /scrape` (markdown output)
@@ -627,7 +647,7 @@ def web_scraping_guide() -> str:
 1. Use **markdownify** or **scrape** before **smartscraper** when you only need readable text.
 2. Multi-page **AI** extraction: run **smartscraper** per URL, or use **monitor_create** on a schedule.
 3. Poll **smartcrawler_fetch_results** until the crawl finishes.
-4. Override API host with env **SCRAPEGRAPH_API_BASE_URL** if needed (default production v2 base URL).
+4. Override API host with env **SGAI_API_URL** if needed (default `https://api.scrapegraphai.com/v2`).
 """
 
 
@@ -677,7 +697,7 @@ def quick_start_examples() -> str:
 limit: 10
 ```
 
-Auth: `SGAI_API_KEY` or MCP `scrapegraphApiKey`. Optional: `SCRAPEGRAPH_API_BASE_URL`.
+Auth: `SGAI_API_KEY` or MCP `scrapegraphApiKey`. Optional: `SGAI_API_URL`, `SGAI_TIMEOUT_S`.
 """
 
 
@@ -691,9 +711,9 @@ def api_status() -> str:
     """
     return """# ScapeGraph API Status (MCP v2)
 
-- **MCP package version**: 2.0.0 (matches [scrapegraph-py#82](https://github.com/ScrapeGraphAI/scrapegraph-py/pull/82) API surface)
-- **Default API base**: `https://api.scrapegraphai.com/api/v2` (override with `SCRAPEGRAPH_API_BASE_URL`)
-- **Auth headers**: `Authorization: Bearer`, `SGAI-APIKEY`, `X-SDK-Version: scrapegraph-mcp@2.0.0`
+- **MCP package version**: 2.0.0 (matches [scrapegraph-py#84](https://github.com/ScrapeGraphAI/scrapegraph-py/pull/84) API surface)
+- **Default API base**: `https://api.scrapegraphai.com/v2` (override with `SGAI_API_URL`)
+- **Auth headers**: `SGAI-APIKEY`, `X-SDK-Version: scrapegraph-mcp@2.0.0`
 
 ## Tools
 markdownify, scrape, smartscraper, searchscraper, smartcrawler_initiate, smartcrawler_fetch_results, crawl_stop, crawl_resume, generate_schema, credits, sgai_history, monitor_create, monitor_list, monitor_get, monitor_pause, monitor_resume, monitor_delete