Web Search MCP Server: Local LLM Web Search Without API Keys

Web Search MCP Server

A Web Search MCP Server is a lightweight, TypeScript‑based MCP (Model Context Protocol) server that gives locally‑hosted LLMs real‑time, on‑premise web‑search capabilities. It eliminates the need for paid APIs by directly connecting to popular search engines—Bing, Brave, and DuckDuckGo—and extracting full page content or lightweight snippets.

Why Use It?

Requirement Why It Matters
No API keys Many LLM deployments avoid external costs and potential latency.
Multi‑engine strategy Falls back to alternative engines when one fails.
Full page extraction Pulls the entire article for high‑quality context.
Concurrent processing Retrieves multiple results in parallel, reducing total wait time.
Playwright support Simulates human browsing to bypass bot detection.
Configurable via environment variables Tweak timeout, browser count, and relevance thresholds without code changes.

Quick Installation

# 1. Clone the repo
git clone https://github.com/mrkrsl/web-search-mcp.git
cd web-search-mcp

# 2. Install Node.js v18+ and npm v8+
#    (Use official installers or nvm to manage versions.)

# 3. Install dependencies & Playwright browsers
npm install
npx playwright install

# 4. Build the TypeScript code
npm run build

# 5. Run the server (development mode)
npm start   # or "npm run dev" for hot‑reload

Tip: On Windows, replace forward slashes and use PowerShell commands.

Configuration

Create or edit mcp.json in your project root (or any location you prefer) to tell your MCP client where to find the server binary.

{
  "mcpServers": {
    "web-search": {
      "command": "node",
      "args": ["/path/to/web-search-mcp/dist/index.js"],
      "env": {
        "MAX_CONTENT_LENGTH": "10000",
        "BROWSER_HEADLESS": "true",
        "MAX_BROWSERS": "3",
        "BROWSER_FALLBACK_THRESHOLD": "3"
      }
    }
  }
}
  • command – typically node.
  • args – absolute path to dist/index.js.
  • env – optional environment variables for fine‑tuning.

Common Environment Variables

Variable Default Purpose
MAX_CONTENT_LENGTH 500000 Maximum characters to return from page extraction.
DEFAULT_TIMEOUT 6000 Request timeout in milliseconds.
MAX_BROWSERS 3 Max concurrent browser instances.
BROWSER_TYPES chromium,firefox Which browsers to launch.
ENABLE_RELEVANCE_CHECKING true Turns on search‑quality scoring.
RELEVANCE_THRESHOLD 0.3 Minimum quality score.
FORCE_MULTI_ENGINE_SEARCH false Force querying all engines even if one succeeds.

MCP Tool Overview

The server exposes three distinct tools that can be called through any MCP‑compatible client (LM Studio, LibreChat, or your own application).

Feature Detail
Search logic Prioritises Bing → Brave → DuckDuckGo.
Result payload Array of objects containing url, title, description, and optional content.
Customisation limit (1‑10), includeContent (boolean).

Example request

{
  "name": "full-web-search",
  "arguments": {
    "query": "TypeScript MCP server",
    "limit": 3,
    "includeContent": true
  }
}

2. get-web-search-summaries

Same search strategy as full-web-search but returns only the snippet/descriptions—no full page content. Perfect for quick lookup or when bandwidth is limited.

Example request

{
  "name": "get-web-search-summaries",
  "arguments": {
    "query": "best TypeScript web search",
    "limit": 5
  }
}

3. get-single-web-page-content

Useful when you already know the desired URL and just need the main content.

Example request

{
  "name": "get-single-web-page-content",
  "arguments": {
    "url": "https://example.com/article.html",
    "maxContentLength": 5000
  }
}

Using with a Local LLM Client

Below is a minimalist example for integrating the server into a LibreChat configuration.

# librechat.yaml
mcpServers:
  web-search:
    type: stdio
    command: node
    args:
      - /app/mcp/web-search-mcp/dist/index.js
    env:
      MAX_CONTENT_LENGTH: "10000"
    serverInstructions: true

In a chat message:

{{{ full-web-search(query="nodejs web search", limit=5, includeContent=true) }}}

The LLM will receive the structured JSON with URLs, titles, descriptions, and full content, which can be fed as context for generation.

Performance Tips

Setting Why it helps
DEFAULT_TIMEOUT = 4000 Faster failure on slow networks; reduces total round‑trip time.
MAX_BROWSERS = 1 Cuts memory usage; handy for low‑resource environments.
BROWSER_HEADLESS = true Eliminates rendering overhead for headless mode.
RELEVANCE_THRESHOLD = 0.5 Stricter filtering for higher quality results.

Remember: Tuning these values depends on your network, CPU, and the size of LLM requests.

Troubleshooting

Symptom Possible Cause Fix
npm install fails Outdated Node/NPM Upgrade to Node 18+ and NPM 8+
Browser install hangs Proxy/Network restriction Run npx playwright install --with-deps behind VPN or proxy config
Search results empty Engine blocked Increase BROWSER_FALLBACK_THRESHOLD or enable FORCE_MULTI_ENGINE_SEARCH
Out‑of‑memory Too many browsers Reduce MAX_BROWSERS
Unexpected HTTP/2 errors Protocol support issue Server auto‑falls back; ensure BROWSER_HEADLESS=true

Extending the Server

The codebase is modular. New tools can be added easily:

  1. Create a file under src/tools/ implementing a run function.
  2. Export it in src/index.ts.
  3. Update the package.json and mcp.json if needed.

Tip: Leverage the zod schema validation already present to enforce arguments.

Licensing

The project is licensed under the MIT License. Feel free to fork, modify, and incorporate into your own pipelines.

Final Words

The Web Search MCP Server closes a significant gap for local LLM developers: instant, API‑free web browsing. By combining multiple search engines, sophisticated content extraction, and a clean MCP interface, it offers both depth (full page content) and speed (summary mode) in a single repository. Whether you run a small research assistant or a production chatbot, this tool gives your LLM the up‑to‑date knowledge it needs without external costs.

Happy searching!

Original Article: View Original

Share this article