Apify MCP for Marketers: Turn Claude and OpenAI Agents Into a Web Scraping Stack

Most marketing teams treat AI agents as glorified chatbots. They prompt Claude or ChatGPT, paste the answer somewhere, and move on. The teams pulling ahead in 2026 wired their agents into a web scraping backbone, so the agent fetches live data instead of hallucinating from a stale training cutoff.

Apify is the missing piece. Through the Model Context Protocol, your agent gets access to thousands of pre-built scrapers for Google Maps, TikTok, LinkedIn, Instagram, Trustpilot, G2, ChatGPT, Perplexity, and Google Search. You stop writing brittle integrations and start shipping research workflows running on autopilot.

TL;DR

Apify exposes 6,000+ Actors as MCP tools at mcp.apify.com. Connect Claude Desktop, Claude Code, or the OpenAI Agents SDK with one config block and your agent runs SERP tracking, GEO monitoring, lead enrichment, ad library scraping, and review intelligence in plain English. Pay-per-result pricing means most workflows cost $1-$30 per run. This post walks through 6 production workflows with full Actor inputs and ready-to-use prompts.

Key Takeaways

Apify MCP turns 6,000+ scrapers into callable tools for Claude and OpenAI agents
Hosted endpoint at mcp.apify.com works with OAuth in Claude Desktop, Claude Code, Cursor, and VS Code
Real pricing: SERP at $0.25/1k results, Google Maps leads at $9/1k, Trustpilot at $0.75/1k reviews
The official AI Brand Visibility Actor tracks brand mentions across Gemini, ChatGPT, and Perplexity
Output handoff: pipe Apify datasets into Google Sheets, Airtable, Notion, or HubSpot natively
Skip Apify when you need ad-hoc one-off lookups, internal CRM data, or anything an MCP search tool already covers
Six full workflows in this post: quickstart, competitor intel, lead gen, review monitoring, GEO tracking, content audit

How Do You Get Started With Apify MCP in 30 Minutes?

Get started in five steps taking 30 minutes total. The hardest part is picking your first workflow, not the setup.

Run this exact sequence:

Sign up at Apify and grab your API token from the Integrations page in Apify Console
Install Claude Desktop or open Claude.ai web (paid plan required for custom integrations)
Add the Apify MCP server using the connector flow in Settings, pointing at https://mcp.apify.com
Authorize via OAuth in the browser tab Claude opens
Test with a one-shot prompt: “Search Google for ‘best CRM for real estate’ and return the top 10 organic results”

Within minutes Claude calls the Google Search Results Scraper Actor, runs it on Apify’s cloud, and returns structured results in chat. The free tier gives you $5 of monthly platform credit, enough for 20+ test runs.

For Claude Code users, the terminal command looks like this:

claude mcp add --transport http apify https://mcp.apify.com

For OpenAI Agents SDK users, connect through MCPServerStreamableHttp:

from agents import Agent, Runner
from agents.mcp import MCPServerStreamableHttp
import os

async with MCPServerStreamableHttp(
    name="Apify MCP Server",
    params={
        "url": "https://mcp.apify.com",
        "headers": {"Authorization": f"Bearer {os.environ['APIFY_TOKEN']}"},
        "timeout": 120,
    },
    cache_tools_list=True,
) as server:
    agent = Agent(
        name="MarketingResearcher",
        instructions="Pull competitive intel using available web tools.",
        mcp_servers=[server],
    )
    result = await Runner.run(agent, "Find the top 20 SaaS competitors of Gong on G2 with their ratings.")

Frameworks including LangGraph, CrewAI, Mastra, and the Vercel AI SDK all hook into the same endpoint.

Action item: Connect to mcp.apify.com today. Run the test prompt above. Confirm results come back before reading the rest of this post.

What Does Apify MCP Cost in 2026?

Apify MCP itself is free. You pay for Actor runs based on each Actor’s pricing model. Most marketing Actors use pay-per-result pricing, so you pay per row of data extracted instead of per minute of compute.

Real pricing pulled from the Apify Store today:

Apify Google Search Results Scraper: ~$0.25 per 1,000 organic results
Apify Google Maps Scraper: $2.50-$9 per 1,000 places depending on the variant
Google Maps Email Extractor: $9 per 1,000 leads with email enrichment
G2 Product Reviews Scraper: ~$1-$2 per 1,000 reviews
Trustpilot Scraper: $0.75 per 1,000 reviews
Twitter/X Scraper: $0.25-$0.40 per 1,000 tweets
TikTok Scraper: ~$0.50-$2 per hashtag or profile run
Facebook Ads Library Scraper: $0.75 per 1,000 ad records

A team running weekly competitive monitoring on 5 competitors typically spends $30-$80 per month. The same workflow replaces a $500-$2,000 monthly subscription to a competitive intelligence tool.

The free tier includes $5 of monthly platform credit. Pricing across Actors varies widely, so check each Actor’s Store page before scheduling production runs.

Action item: Pick 3 Actors aligned with your top weekly tasks. Note their pricing. Set a monthly budget cap in Apify Console under Billing.

Which Apify Actors Should Marketers Wire Into Their Agents?

Pick the Actors mapped to your weekly research questions. Loading all 6,000 into your tool list slows the agent down and confuses routing. A focused 5-10 Actor whitelist works better than a giant menu.

Use these starter packs for common marketing jobs:

Google Search Results Scraper for SERP tracking and competitive keyword research
Google Maps Email Extractor for local lead generation with contact details
TikTok Scraper and TikTok Ads Scraper for short-form trend analysis and ad creative research
LinkedIn Ads Scraper and Facebook Ads Library Scraper for paid social benchmarking
G2 Product Reviews Scraper and Trustpilot Scraper for review monitoring
Instagram Profile Scraper and Instagram Hashtag Scraper for influencer and content research
Reddit Scraper Lite for community sentiment and customer language mining
Smart Article Extractor for competitive content audits
AI Brand Visibility Actor for tracking mentions in ChatGPT, Gemini, and Perplexity

Each Actor accepts structured inputs (URLs, search queries, hashtags) and returns JSON your agent parses without extra glue code.

Action item: List your top 5 weekly research tasks. Match each to an Actor. Whitelist only those in your MCP config.

How Do You Build a Competitor Intelligence Agent With Claude and Apify?

Build a competitor intelligence agent by chaining three Actors inside one Claude conversation. The pattern works for any vertical: SaaS, e-commerce, local services, agencies.

Use this prompt structure with Claude Desktop or Claude Code connected to Apify MCP:

SYSTEM: You are a competitive intelligence analyst.

<context>
Our company: {{COMPANY_NAME}}
Top 3 competitors: {{COMPETITOR_1}}, {{COMPETITOR_2}}, {{COMPETITOR_3}}
Vertical: {{INDUSTRY}}
</context>

Run this workflow:

1. Use Google Search Results Scraper to pull the top 10 SERPs for each competitor name plus "review"
2. Use G2 Product Reviews Scraper to fetch the last 50 reviews for each competitor
3. Use Facebook Ads Library Scraper to fetch active ads for each competitor

Then deliver:

- Top 5 customer complaints per competitor (from reviews)
- Top 3 ad angles per competitor (from ad library)
- Positioning gaps we exploit

Output: Markdown report with sections per competitor and a synthesis at the end.

What Does the Actor Input JSON Look Like?

If MCP routing gets confused or you want a fallback, give the agent the raw Actor input. Here is the JSON for the Google Search Results Scraper step:

{
  "queries": "competitor1 reviews\ncompetitor2 reviews\ncompetitor3 reviews",
  "resultsPerPage": 10,
  "maxPagesPerQuery": 1,
  "countryCode": "us",
  "languageCode": "en"
}

A run finishes in 8-15 minutes and costs roughly $0.50-$2.00 in Actor credits depending on review volume. Time saved versus manual research: 6-10 hours per cycle.

Action item: Run the prompt above against your top 3 competitors this week. Save the output. Compare next week’s run to track shifts.

How Do You Use Apify Agents for Lead Generation?

Build a lead generation pipeline with the Google Maps Email Extractor and a Claude or OpenAI agent. The Actor scrapes Google Maps places, follows links to their websites, and extracts emails, phone numbers, and social profiles.

Use this prompt with your MCP-connected agent:

SYSTEM: You are a lead generation operator.

<context>
ICP: {{IDEAL_CUSTOMER_DESCRIPTION}}
Geography: {{CITIES_OR_REGIONS}}
Vertical search terms: {{KEYWORDS}}
</context>

MUST follow these steps:
1. Use Google Maps Email Extractor with each search term across each geography
2. Limit to businesses with rating above 4.0 and at least 20 reviews
3. Filter results to those with valid email and active website
4. Score each lead 1-10 based on ICP fit using the website description

Output: CSV with columns: business_name, website, email, phone, rating, review_count, ICP_score, reasoning.

Real Actor Input for Google Maps Email Extractor

The agent fills in this JSON when calling the Actor. Save it as a fallback if MCP misroutes:

{
  "searchStringsArray": ["dental clinic", "orthodontist", "pediatric dentist"],
  "locationQuery": "Austin, Texas",
  "maxCrawledPlacesPerSearch": 200,
  "skipClosedPlaces": true,
  "language": "en"
}

A single run pulls 500-2,000 qualified leads with contact details. Budget: roughly $9 per 1,000 leads with email enrichment, plus platform fees. Compare to $1-$5 per lead from a paid list provider.

Action item: Define one ICP and one geography. Run the prompt. Review the top 50 scored leads.

How Do You Run Review Monitoring With Claude Code?

Run review monitoring as a scheduled workflow inside Claude Code so the agent runs autonomously on a cron schedule. The Trustpilot, G2, and Amazon Reviews scrapers all return structured review data your agent diffs against last week’s pull.

Set up the workflow in three steps:

Connect Apify MCP to Claude Code with the claude mcp add command shown earlier
Create a project folder with a CLAUDE.md describing the monitoring task
Schedule a daily run with cron or a CI job invoking Claude Code in headless mode

The CLAUDE.md content tells the agent what to do every run:

SYSTEM: You are a review monitoring agent.

<context>
Brands tracked: {{BRAND_LIST}}
Review sources: G2, Trustpilot, Amazon
Threshold: Alert if average rating drops more than 0.2 points week-over-week
</context>

Daily workflow:
1. Pull last 7 days of reviews for each brand from each source
2. Compute average rating and sentiment
3. Compare to last week's snapshot in /reports/last_run.json
4. If threshold crossed, write an alert to /reports/alerts.md

Output: Updated /reports/snapshot.json with current week data and any alerts.

The agent stores state between runs as JSON files. You wire alerts into Slack with a webhook or post to a Notion database.

Action item: Set up Claude Code with Apify MCP, point it at one brand, and run the monitor manually. Then schedule it.

How Do You Track AI Search Visibility (GEO) With Apify?

Track AI search visibility by combining the AI Brand Visibility Actor with a Claude or OpenAI agent. The Apify-maintained Actor queries Gemini, ChatGPT, and Perplexity directly with prompts you define, then returns structured data on whether your brand appears in the response.

Note as of April 2026: ChatGPT no longer exposes query fan-out data via the web interface, so most GEO Actors rely on the OpenAI API or fall back to Perplexity and Gemini. Build your prompts assuming Perplexity and Gemini coverage as the floor.

The workflow:

Define 20-50 prompts your buyers ask AI assistants
The agent runs each prompt through the AI Brand Visibility Actor
The agent extracts cited sources and ranks them by frequency
The agent identifies which sources mention you and which omit you

Use this prompt to wire it up:

SYSTEM: You are a GEO (Generative Engine Optimization) analyst.

<context>
Brand: {{BRAND_NAME}}
Category: {{CATEGORY}}
Target prompts: {{LIST_OF_BUYER_QUERIES}}
Competitors to flag: {{COMPETITOR_LIST}}
</context>

Run AI Brand Visibility for each prompt. For each result:
1. Extract all citation sources (domains and URLs)
2. Tag whether OUR brand is mentioned
3. Tag which competitors are mentioned
4. Aggregate cited domains across all prompts

Output: Markdown table with columns:
- domain
- citation_frequency
- mentions_us (Y/N)
- mentions_competitors (list)
- outreach_priority (1-3)

Real Actor Input for AI Brand Visibility

The Actor takes a brand name, competitor list, and prompts. Sample JSON:

{
  "brandName": "yfxmarketer",
  "competitors": ["competitor1.com", "competitor2.com"],
  "prompts": [
    "best AI marketing automation tools 2026",
    "how to automate competitor research with AI",
    "Apify alternatives for marketers"
  ],
  "platforms": ["gemini", "perplexity"]
}

This builds a prioritized list of domains to target for guest posts, podcast appearances, or backlinks. The domains AI engines cite are the domains you want to be on. Run weekly and you spot trend shifts before they show up in traffic data.

Action item: Write 20 buyer prompts for your category. Run them through AI Brand Visibility. Build your AI SEO target list from the citation data.

How Do You Run a Content Gap Audit With Apify and Claude?

Run a content gap audit by combining Smart Article Extractor with the Google Search Results Scraper. Your agent finds what ranks for your target keywords, extracts the actual article bodies, and identifies the subtopics your competitors cover but you miss.

Use this prompt:

SYSTEM: You are a content strategy analyst.

<context>
Our domain: {{OUR_DOMAIN}}
Target keywords: {{KEYWORD_LIST}}
Our existing articles: {{ARTICLE_URLS}}
</context>

Workflow:
1. Use Google Search Results Scraper to find top 10 ranking pages for each keyword
2. Use Smart Article Extractor to pull the full content of each ranking page
3. Compare topics covered in ranking pages versus our existing articles
4. Identify subtopics, questions, and entities competitors cover that we miss

Output: Table with columns:
- keyword
- subtopics_we_cover
- subtopics_we_miss
- new_article_recommendations (with proposed H2 outline)

A single run typically costs $1-$5 in Actor credits and produces a content backlog of 10-30 article ideas grounded in real SERP data. Run quarterly to keep your editorial calendar tied to what AI search engines and Google reward.

Action item: Pick 10 keywords you want to rank for. Run this audit. Add the top 3 missed subtopics to next sprint.

How Do You Get Apify Data Into Sheets, Airtable, and Your CRM?

Get Apify data into your downstream tools by using the native Integrations tab on every Actor’s run page. You do not need Zapier or Make for the common destinations.

The integrations available out of the box:

Google Sheets: append rows from each run, with column mapping
Gmail: send the dataset as an attachment or HTML email
HubSpot: create or update contacts and companies
Airtable: push records into a base
Slack: post a summary message with key counts
Webhooks: send the dataset to any HTTP endpoint
Make and Zapier: trigger downstream automations
S3 and other cloud storage for archival

Set up the Google Sheets integration in three clicks:

Open the Actor’s last run in Apify Console
Click the Integrations tab
Connect Google Sheets, pick a sheet, map columns

Future runs from the Actor (manual or scheduled) auto-append rows. Wire your sales team’s pipeline directly to a daily Google Maps lead pull, no glue code required.

For agent-driven runs, the agent stores results in the Apify Dataset by default. Pull them into your stack with the dataset URL or the Apify SDK:

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")
dataset_items = client.dataset("DATASET_ID").list_items().items
# Push to your CRM, database, or BI tool from here

Action item: Set up the Google Sheets integration on one Actor run. Confirm rows append. Then schedule the Actor weekly.

When Should You NOT Reach for Apify MCP?

Apify MCP earns its place when your agent needs live web data. It is overkill for plenty of tasks marketers do every day.

Skip Apify when:

You need internal CRM, analytics, or product data: use the HubSpot MCP, Salesforce MCP, or Snowflake MCP instead
You want one-shot lookups already covered by Claude’s built-in web search or Perplexity
The data target offers a free official API: use the API directly through a custom MCP or function call
Your task is a single page summary: Claude’s web fetch tool handles it without an Actor
You need real-time pricing or stock data: use a financial data API instead
Your data lives behind authentication you control: a custom scraper inside your stack respects auth better

The rule: Apify shines for structured extraction at scale across known sites with anti-bot defenses. For everything else, simpler tools win.

Action item: List your top 5 data needs. Tag each as “Apify fits” or “use a different tool”. Stop forcing Apify into wrong-fit jobs.

How Do You Move From One-Off Runs to Production Workflows?

Move to production by separating ad-hoc exploration from scheduled automation. Use Claude Desktop or Claude.ai web for exploration. Use Apify Schedules, Claude Code in headless mode, or the OpenAI Agents SDK in a deployed app for production runs.

The split looks like this:

Exploration: Claude Desktop with full Apify MCP access, you guide each step
Recurring tasks: Claude Code with a CLAUDE.md spec, scheduled by cron
Customer-facing agents: OpenAI Agents SDK or Claude Agent SDK deployed on Vercel or AWS
Background jobs: Apify Schedules running specific Actors directly via API

Production workflows skip the LLM where possible. If you already know which Actor to run with which inputs, schedule the Actor directly in Apify Console and pipe results to Sheets or your CRM. The LLM adds value when interpretation or chaining is required, not when you are running the same scrape every Monday at 9am.

Action item: Categorize your top 10 marketing data tasks into one of the four buckets above. Build the simplest version of each.

What Are the Common Failure Modes and How Do You Fix Them?

Three failure modes show up in production. Address each upfront and your workflows stay reliable.

The first is vague prompts triggering runaway runs. An agent told to “research this company exhaustively” chains dozens of Actor calls. Fix this by adding hard limits to every prompt: max pages, max results, max retries.

The second is bloated tool lists slowing the agent down. If you whitelist 50 Actors, the agent spends tokens reading tool metadata and routing gets confused. Whitelist 5-10 Actors per use case and run separate agent sessions for separate workflows.

The third is stale data when Actors hit rate limits or anti-bot protections. Build in a verification step: after each Actor run, the agent checks output schema and re-runs with different inputs if data looks broken.

Action item: Audit your last 5 agent runs. Look for vague prompts, oversized tool lists, and silent failures. Fix the worst one this week.

How Do You Save Reusable Agent Prompts as a Library?

Save reusable prompts as Claude Projects, slash commands, or shared docs so your team runs the same playbook every time. Reusable prompts cut prep time from 10 minutes to 30 seconds and make agent runs reproducible.

Three patterns work well:

Claude Projects: create one Project per workflow (Competitor Intel, Lead Gen, GEO Audit) with the system prompt and Apify MCP attached. Share with the team.
Claude Code slash commands: drop prompts into .claude/commands/ as markdown files. Run /competitor-intel and the agent loads the prompt with variables.
A shared Notion or GitHub repo with each prompt as a markdown file with {{VARIABLE}} placeholders.

The slash command pattern is the most operator-friendly. Once a workflow earns its keep, freeze the prompt as a command and run it weekly.

Action item: Pick the workflow you ran most this month. Save it as a Claude Project or slash command this week.

Final Takeaways

Apify MCP is the cheapest way to give Claude and OpenAI agents real eyes on the internet. Connect it once and your agents fetch live data instead of guessing.

Pick 5-10 Actors mapped to your actual weekly questions. A small whitelist beats a sprawling menu every time.

Cap every prompt with hard limits on pages, results, and retries. Vague prompts burn budget fast.

Use Claude Desktop for exploration, Claude Code for scheduled jobs, and the OpenAI Agents SDK for deployed agents. Match the tool to the job.

Track AI search visibility, monitor reviews, scrape ad libraries, audit content gaps, and enrich leads with the same MCP connection. One server replaces five SaaS subscriptions.

Skip Apify when internal data tools, native APIs, or simpler MCPs already solve the problem. Tool fit beats tool stacking.

Save winning prompts as Projects or slash commands. The compound effect comes from reusing what works, not building new every time.