Web Agents for E-commerce Price Monitoring: MAP Enforcement, Competitive Intelligence, and Catalog Sync

The scale problem: why traditional scrapers can't keep up

You have 1,000 SKUs. Sold across 20 retailers. You need daily pricing data.

That's 20,000 data points to collect every morning — before your pricing team starts work.

Traditional scrapers were built for a simpler web. Static HTML, predictable selectors, no JavaScript. Today's retail sites are the opposite: prices rendered client-side, layout A/B tests that silently break your CSS selectors every few weeks, bot detection that blocks entire IP ranges. Maintaining that infrastructure isn't a one-time project. It becomes a full-time engineering responsibility.

The failure mode is quiet and expensive. Your extraction returns stale data with no error. A competitor drops below your Minimum Advertised Price (MAP) price. You find out three weeks later when a distributor emails to complain. By then, other retailers have matched the low price, and you're managing a channel conflict instead of preventing one.

Web agents change the architecture of the problem. Instead of maintaining brittle selectors across 20 retailer codebases, you describe the goal in plain English — "find the current price of this product" — and the agent navigates, renders, and extracts by reading page content directly. No selectors to maintain. No silent failures when layouts change.

This article walks through three production-ready use cases: competitor price monitoring, MAP violation detection, and dynamic pricing intelligence — with working code for each.

When does a web agent actually beat a extraction for price monitoring?

Before the use cases: if you're still deciding whether this applies to you, here's the honest decision framework.

Quick reference: when a web agent beats a extraction for price monitoring

1. Target sites render prices via JavaScript — anything a traditional extraction can't see after initial page load
2. Volume × frequency exceeds extraction maintenance capacity — roughly 50+ SKUs daily across 5+ retailers
3. You need schema-consistent output — downstream pricing systems consuming structured JSON
4. Your existing tooling stops working on sites with strict access requirements — behavioral fingerprinting, session management, IP variability
5. You need an audit trail — each agent run produces a timestamped record, not just a number

Use case 1: Competitor price monitoring at scale

The pattern: give the agent a list of product URLs, a structured output schema in the goal prompt, and run them concurrently. Total time equals the slowest single task — not the sum of all tasks.

Installation

pip install tinyfish
export TINYFISH_API_KEY=sk-tinyfish-*****

The code

import asyncio
import json
from datetime import datetime, timezone
from tinyfish import AsyncTinyFish, BrowserProfile

client = AsyncTinyFish()  # Reads TINYFISH_API_KEY from environment

PRODUCTS = [
    {"product_id": "airpods-pro-3", "url": "https://www.bestbuy.com/site/..."},
    {"product_id": "airpods-pro-3", "url": "https://www.amazon.com/dp/..."},
    {"product_id": "sony-wh1000xm6", "url": "https://www.target.com/p/..."},
    # Add up to 1,000 product URLs
]

async def extract_price(product_id: str, url: str) -> dict:
    goal = (
        "Find the current listed price of the product on this page. "
        "Return a JSON object with these fields only: "
        "price (number, no currency symbol), currency (ISO 4217 code), in_stock (boolean). "
        "If no price is visible, set price to null."
    )

    response = await client.agent.run(
        url=url,
        goal=goal,
        browser_profile=BrowserProfile.STEALTH,
    )

    # For debugging: response.streaming_url contains a live browser replay of this run (valid 24h)
    # response.result is a dict shaped by your goal prompt — or None if the run failed
    # at the infrastructure level (browser crash, timeout, etc.)
    result = response.result or {}

    # Two distinct failure modes to handle:
    # 1. Infrastructure failure: response.result is None → caught by `or {}`
    # 2. Goal failure: run completed but agent couldn't achieve the task →
    #    result contains {"status": "failure", "reason": "..."} instead of your data
    #    This is the most common production surprise — COMPLETED status ≠ goal achieved
    if result.get("status")  "failure":
        return {
            "product_id": product_id,
            "price": None,
            "error": result.get("reason", "goal_failed"),
            "scraped_at": datetime.now(timezone.utc).isoformat(),
            "source_url": url,
        }

# (continued)

    # Happy path: result contains the fields you specified in the goal prompt
    return {
        "product_id": product_id,
        "price": result.get("price"),        # number, as specified in goal
        "currency": result.get("currency", "USD"),
        "in_stock": result.get("in_stock"),  # boolean, as specified in goal
        "scraped_at": datetime.now(timezone.utc).isoformat(),
        "source_url": url,
    }

async def main():
    # All requests fire concurrently — total time = slowest single task
    tasks = [extract_price(p["product_id"], p["url"]) for p in PRODUCTS]
    results = await asyncio.gather(*tasks)
    print(json.dumps(results, indent=2))

asyncio.run(main())

Note on concurrency: When you send more requests than your plan's concurrent session limit, TinyFish queues the excess runs automatically (status: PENDING) — they start as sessions free up. Size your batches to your plan's concurrency limit for predictable run times.

Note on result handling: status: "COMPLETED" means the browser ran successfully — not that your goal succeeded. A run that hit an Access Denied page will also return COMPLETED, but result will contain {"status": "failure", "reason": "..."} instead of your data. The code above handles both cases explicitly. This is the most common source of silent failures in production price monitors.

Output schema

[
  {
    "product_id": "airpods-pro-3",
    "price": 249.99,
    "currency": "USD",
    "in_stock": true,
    "scraped_at": "2026-03-27T14:32:01Z",
    "source_url": "https://www.bestbuy.com/site/..."
  },
  {
    "product_id": "airpods-pro-3",
    "price": 239.00,
    "currency": "USD",
    "in_stock": true,
    "scraped_at": "2026-03-27T14:32:04Z",
    "source_url": "https://www.amazon.com/dp/..."
  }
]

The parallel execution advantage

The math: total time equals the slowest single task, not the sum. At 1,000 concurrent agents, a batch that took 50 minutes sequentially completes in the time it takes to process one page.

Success rates by retailer category

Use case 2: MAP violation detection with evidence reports

MAP enforcement has a discovery problem. Your team can't manually check every authorized retailer every day — and by the time a violation surfaces through a distributor complaint, the channel damage is done.

A scheduled agent run closes the gap: extract prices, compare against your MAP database, generate a timestamped evidence record per violation. The whole workflow runs while your team sleeps.

The code

import asyncio
import json
from datetime import datetime, timezone
from tinyfish import AsyncTinyFish, BrowserProfile

client = AsyncTinyFish()

# Your MAP pricing database
MAP_PRICES = {
    "airpods-pro-3": 249.00,
    "sony-wh1000xm6": 299.00,
}

# Authorized retailers to monitor per product
RETAILERS = {
    "airpods-pro-3": [
        {"name": "BestBuy",  "url": "https://www.bestbuy.com/site/..."},
        {"name": "Target",   "url": "https://www.target.com/p/..."},
        {"name": "Walmart",  "url": "https://www.walmart.com/ip/..."},
    ],
}

async def check_retailer(product_id: str, map_price: float, retailer: dict) -> dict | None:
    response = await client.agent.run(
        url=retailer["url"],
        goal=(
            "Return a JSON object with: price (number, no symbol), currency (ISO 4217). "
            "If no price is found, set price to null."
        ),
        browser_profile=BrowserProfile.STEALTH,
    )

    result = response.result or {}
    advertised = result.get("price")

    if advertised is None or result.get("status")  "failure":
        return None  # Could not extract price — log separately

    if advertised < map_price:
        return {
            "product_id": product_id,
            "retailer": retailer["name"],
            "map_price": map_price,
            "advertised_price": advertised,
            "violation_amount": round(map_price - advertised, 2),
            "evidence_url": retailer["url"],
            "detected_at": datetime.now(timezone.utc).isoformat(),
        }

    return None  # Compliant

async def run_map_check():
    all_tasks = []
    for product_id, map_price in MAP_PRICES.items():
        for retailer in RETAILERS.get(product_id, []):
            all_tasks.append(check_retailer(product_id, map_price, retailer))

    results = await asyncio.gather(*all_tasks)

# (continued)
    violations = [r for r in results if r is not None]

    print(json.dumps(violations, indent=2))
    return violations

asyncio.run(run_map_check())

Evidence report output

{
  "product_id": "airpods-pro-3",
  "retailer": "Target",
  "map_price": 249.00,
  "advertised_price": 219.99,
  "violation_amount": 29.01,
  "evidence_url": "https://www.target.com/p/...",
  "detected_at": "2026-03-27T14:35:22Z"
}

This structure pipes directly into your brand protection workflow. Route violations to Slack for same-day retailer outreach, to your ERP for automated distributor notification, or to a compliance dashboard. The timestamped URL is your legal evidence record — captured at the exact moment of detection.

Use case 3: Dynamic pricing intelligence

Static MAP enforcement catches violations after the fact. The harder problem is building a pricing system that reacts to market changes before they compound.

The structural issue: prices on food delivery, travel, and marketplace platforms load dynamically per user session, vary by location, and change multiple times per day. A extraction built on CSS selectors breaks every time a vendor updates their layout — and in high-velocity markets, that's often. An agent built on a goal — "find the current price of this item at this location" — adapts to layout changes automatically, because it's reading the page content and layout directly, not pattern-matching against a selector you wrote last quarter.

The output that makes this useful downstream isn't a report — it's a structured feed with enough granularity to drive pricing decisions:

{
  "market_id": "sf-94102",
  "restaurant_id": "R_8821",
  "item_id": "burger-classic",
  "base_price": 12.99,
  "delivery_fee": 1.99,
  "promo_price": null,
  "scraped_at": "2026-03-27T18:00:00Z"
}

The code structure to produce this is identical to Use case 1 — AsyncTinyFish + asyncio.gather() across a list of URLs, with a goal prompt that specifies the schema above. The only difference is the schema itself. If you've already built Use case 1, this is a goal prompt change, not an architecture change.

Note: This represents a generalized deployment pattern, not a published case study. Specific customers are not identified.

A major food delivery platform uses this pattern to track millions of pricing variables per month across thousands of markets. The scale is unusual; the architecture isn't.

Handling access requirements on retail sites: an honest assessment

The majority of major consumer retail platforms runs at up to 85% success rate on sites with strict automation requirements. For most teams, this covers 80%+ of the retailers they care about.

The exception is sites running enterprise behavioral analysis systems. These systems don't look for a missing header — they model whether the entire session pattern looks human. Success rates are lower and inconsistent across all automation tools. No vendor publishes reliable numbers for the hardest-protected sites, and you should be skeptical of any that do.

Practical approach for protected retailers:

from tinyfish import TinyFish, BrowserProfile, ProxyConfig, ProxyCountryCode

client = TinyFish()

result = None
with client.agent.stream(
    url="https://protected-retailer.com/product/...",
    goal="Extract the current price and stock status",
    browser_profile=BrowserProfile.STEALTH,
    proxy_config=ProxyConfig(enabled=True, country_code=ProxyCountryCode.US),
) as stream:
    for event in stream:
        # SDK CompleteEvent: result lives in event.result_json
        if getattr(event, "type", None)  "COMPLETE":
            # Layer 1: infrastructure failure
            result = event.result_json
            break

# Layer 2: goal failure
if result and isinstance(result, dict) and result.get("status")  "failure":
    result = None

TinyFish handles detection at the infrastructure level — not through JS injection applied after browser start. Adding a matching country-code proxy handles geo-specific access requirements. For sites that still reliably block: weigh whether the data value justifies ongoing engineering time, or whether an alternative data source exists.

Cost model: what does daily monitoring actually cost?

TinyFish uses Wallet-based billing rather than recurring plans. Search and Fetch are free. Web Agent costs $0.016 per step, while Browser costs $0.002 per minute. New accounts receive $8 in Wallet funds, equivalent to up to 500 Agent steps if used only for Agent.

Estimated price-monitoring costs

The Agent estimates assume exactly one step per price check and should be treated only as a lower-bound illustration. Actual Agent cost equals:

Monthly checks × average steps per check × $0.016

For stable, known product URLs, use Fetch instead. It is free and supports up to 150 URLs per minute on standard accounts.

For context on extraction maintenance costs: a junior engineer spending 25% of their time keeping selectors current across 20 retailer sites is a real line item. That's not in the table above.

Build vs. buy: when your existing extraction is still the right answer

The honest answer is that many teams don't need a web agent for price monitoring. If your requirements are under 50 products, static pages, and a handful of stable low-protection retailers — Scrapy or a basic Playwright script is cheaper. Build it yourself.

The crossover point isn't a product decision, it's a maintenance math question: when does the engineering time spent keeping selectors current across 20 retailer sites exceed the cost of outsourcing that infrastructure? That typically happens around 100+ products, 5+ retailers, or the first time a retailer redesign takes down your monitoring for a week without anyone noticing.

Web agents also make sense when extraction accuracy has downstream consequences. MAP violation reports submitted with incorrect prices are worse than no report — they expose you to retailer disputes you can't back up. A extraction that silently returns the wrong price is a liability. An agent run with a failure status is just a gap in your data, which is recoverable.

Get started

New accounts receive $8 in Wallet funds, equivalent to up to 500 Agent steps. This does not necessarily equal 500 price checks because an Agent workflow may require multiple steps.

If you already have the product URLs, use Fetch instead: it can scan those 500 URLs for free without drawing from the Wallet, subject to the standard rate limit.

For teams monitoring 10,000+ products or needing SLA guarantees, contact our enterprise team for volume pricing and dedicated support.

FAQ

Can web agents handle price monitoring on major e-commerce platforms?

Yes. Major e-commerce platforms run at up to 85% success rate on sites with strict automation requirements. Standard browser profile covers most product pages; switch to managed browser profile for high-volume or well-protected retailers.

How fresh is the price data?

Agents run on demand, so freshness is whatever schedule you set. Daily is the most common pattern. For flash-sale categories, teams commonly run 4× daily or trigger runs on inventory alerts.

Does this work for international retailers?

Yes. Include a currency field (ISO 4217) in your goal schema. For geo-restricted content, add proxy_config with the matching country code (US, GB, CA, DE, FR, JP, AU supported).

What does `COMPLETED` status actually mean?

Infrastructure success only — the browser launched and finished. It does not mean your goal succeeded. Always check the result field: if it contains {"status": "failure"}, the agent ran but couldn't extract the data. This is the most common production gotcha.

What if one agent in a batch fails?

Each run is independent — one failure doesn't affect others. Every run includes a streaming_url for debugging (valid 24 hours). Failed runs where infrastructure succeeded are not billed.

Is there a concurrency limit?

Yes. Exceeding it triggers automatic queuing — no 429 errors, but later requests take longer. Check your plan's limit in the dashboard.

See It in Action

New accounts include $8 in Wallet funds, which can cover up to 500 Agent steps. The number of complete e-commerce monitoring runs this supports depends on how many steps each workflow requires. Known product URLs can be monitored with the free Fetch API instead.

Start free, no credit card →

Related Reading

• Pillar: What Can AI Web Agents Actually Do? 10 Real-World Use Cases
• How to Monitor 1,000 Websites in Parallel with the TinyFish API
• Web Agents for Procurement: Multi-Vendor Portal Automation
• When Web Agents Fail: Debugging Goal-Based Automation