How to Check Source Freshness in a Live Web Search Pipeline

Building a pipeline that answers from the live web? Start free with TinyFish. TinySearch and TinyFetch are free within their limits.
TL;DR
Data freshness is not one date. A page has a publication date, an update date, the time you retrieved it, and the date of the event it describes, and they answer different questions. Retrieval time only tells you when you looked. An update date tells you someone touched the page, not that every sentence on it is current.

In this post I build a small freshness check for a live web search pipeline: a policy that says how old evidence may be for each kind of question, rules for how much to trust each source and when a second source is needed, a reusable Python function that reads every date signal from a fetched page and returns fresh, stale or unknown with reasons, and a rule for when the pipeline should search again or answer with a qualifier such as "as of October 28, 2026, according to the Federal Reserve".
The example is real. I asked a pipeline for the current US federal funds rate on November 9, 2026, and the top search result was a page updated two weeks earlier whose own metadata still stated the old rate.
A quick disclosure: I work at TinyFish, and the searching and fetching below use TinySearch and TinyFetch. The checking code itself is plain Python and works with any search and fetch layer.
Why data freshness breaks live web pipelines
Most articles about data freshness in RAG are about keeping a vector index in sync with its source systems: re-embedding changed documents, streaming updates, expiring old chunks. That matters, but a live web pipeline has a different problem. It fetches pages at question time, so the index is never stale. The pages are.
A page fetched one second ago can state a fact that stopped being true months ago. Search engines rank evergreen explainers highly, those explainers get "updated" without every sentence changing, and snippets are often built from metadata that nobody updates at all. If your pipeline treats "I just retrieved this" as "this is current", it will answer confidently and wrongly.
So the check has to look at the page's own dates, decide which of them actually bears on the question, and then look at the claim itself.
The question: what is the federal funds rate right now?
I picked a question whose answer changes on a known schedule: "What is the current target range for the US federal funds rate?" The Federal Open Market Committee (FOMC) sets it at eight scheduled meetings a year. Between meetings it normally doesn't change, so any source written before the most recent meeting is suspect, however recently it was fetched.
On the day I ran this, November 9, 2026, the most recent decision was the FOMC statement of October 28, 2026, which kept the target range at 3-3/4 to 4 percent. The previous change was a 25 basis point increase at the September 15-16 meeting, which the Fed's own open market operations table lists under September 17 at 3.75-4.00.
That is the ground truth the checker has to reach on its own.
Four dates that get confused
Every page your pipeline retrieves can carry up to four different dates. They come from different places and prove different things.
| Date | What it means | Where it comes from | What it proves |
|---|---|---|---|
| Publication date | When the page was first published | Open Graph article:published_time, JSON-LD datePublished, a byline, TinyFetch's published_date field | When the page started existing. Nothing about later changes |
| Update date | When someone last edited the page | JSON-LD dateModified, a visible "Updated" or "Last Update" line, the HTTP Last-Modified header | That the page was touched. Not that every claim on it was revised |
| Retrieval time | When your pipeline fetched it | Your own clock (and whether the fetch was live or cached) | When you looked. Never that the content is current |
| Event date | When the thing the page describes happened | Dates in the text itself: "the Committee decided on October 28" | What state of the world the claim describes |
For a time-sensitive question, the event date is what you actually care about. The update date is a useful proxy, the publication date is a weaker one, and retrieval time is not evidence of freshness at all.
Even the event date can be ambiguous. NerdWallet says the September increase came at the "Sept. 15-16 meeting". The Fed's table lists it under September 17, the date it took effect. Both are right. Your policy has to say which one counts.
What one real search returned
Here is what the pipeline saw, step by step. I ran these calls through the TinyFish search and fetch tools on November 9, 2026, at about 5:29 PM PT.
Step 1: search. A TinySearch news query for the FOMC decision returned, among others, NerdWallet's "What Is the Federal Funds Rate?" with a relative date of "2 weeks ago" and this snippet:

The federal funds rate — or Fed rate — is the interest rate banks pay one another to borrow or loan money overnight. The current target range is 3.50% to...A pipeline that answers from snippets would stop here and say 3.50% to 3.75%. That was the range before the September increase.
Step 2: fetch. I fetched the page live with TinyFetch, with ttl set to 0 so the result could not come from cache, and with include_etag_and_last_modified and page metadata turned on. The relevant fields:

published_date: 2026-10-28T07:00:00Z
last_modified: (not returned; the origin didn't send one)
description: "... The current target range is 3.50% to 3.75%."
og:description: "... The current target range is 3.50% to 3.75%."
body: "Updated Oct 28, 2026"
"The current Fed rate is 3.75% to 4.00%."A newly retrieved page with outdated information
This is the case freshness checks most often miss. Everything about the NerdWallet page says "fresh": it was retrieved seconds ago, its publication metadata is October 28, 2026, and its visible byline says "Updated Oct 28, 2026", the day of the latest decision. The body is correct.
But the page's meta description, its Open Graph description and its Twitter description all still say "The current target range is 3.50% to 3.75%." That sentence was true before September 2026. The writers updated the article and not the metadata, and the search snippet was built from the stale metadata.
Two lessons for the pipeline:
- A fresh date doesn't certify every sentence on the page. You have to check the claim itself. When the same page states two different values for the same fact, treat the page as unreliable for that fact until something else breaks the tie.
- Snippets are not evidence. Fetch the page and read the body before you answer a time-sensitive question.
The Fed's own open market operations page shows the opposite trap. It states the correct current range, 3.75-4.00, but its "Last Update" line says September 16, 2026, before the October 28 meeting. On its own it can't tell you whether October changed anything, so a correct value still fails the freshness check. That's the right call: the pipeline should look for something dated after October 28.
A simple freshness policy
The policy answers one question per kind of fact: how recent does the evidence need to be, and which date counts?
| Kind of question | Examples | Rule | Date that counts |
|---|---|---|---|
| Live value | Prices, stock levels, latest library version | No older than 7 days | Update date, then publication date |
| Scheduled value | Policy rates, official statistics, scheduled releases | Dated on or after the last change point | Event date or update date |
| Officeholder | Who runs an agency, a company's CEO | No older than 30 days | Update date, then publication date |
| Stable fact | Definitions, how-tos, history | No older than 3 years | Publication date is fine |
The scheduled-value row is the useful trick. A fixed age limit is the wrong tool for a rate that only changes at meetings: a statement from 13 days ago is perfectly current if there hasn't been a meeting since, and a page from yesterday is stale if it was written before today's decision. Give the checker the last change point instead. For the federal funds rate, that's the most recent FOMC decision, which you can look up from the Fed's meeting calendar.
The numbers above are starting points, not standards. Tune them to your domain and to how costly a stale answer is.
Source reliability and corroboration
Age is only one input. A fresh blog post can repeat an old number, and an official page can be both correct and slow to update. So the decision also asks who is saying it and whether anyone else agrees:
| Tier | What counts | Rule for a current answer |
|---|---|---|
| Primary | The body that sets or publishes the fact: a central bank's statement, a company's own pricing page, an official register | One fresh primary source is enough |
| Secondary | Reputable outlets that report the fact: major news sites, established finance or reference sites | Needs a second fresh secondary source from a different site stating the same value |
| Other | Aggregators, forums, AI-written summaries, undated pages | Never enough on its own. Use it to find better sources |
You assign the tier per domain for your use case. The checker doesn't guess it, which keeps the rule explicit and easy to audit.
Recency is a filter, not a score
It's tempting to rank sources by date and take the newest. Don't. Recency only matters relative to the question: for a policy rate, a statement from two weeks ago is current if nothing has happened since, and for a definition, a ten-year-old textbook page can beat yesterday's rewrite. Sorting by date also tends to surface summaries and reposts ahead of the primary source they copied. Use the age policy to rule sources out, then choose among the ones left by reliability and agreement.
Handling missing or ambiguous dates
Real pages are messy. These are the cases the checker has to handle, and how mine does.
- No date at all. Return unknown, not fresh. Retrieval time can't stand in for a content date.
- Only a copyright year. "© 2026" in a footer is usually a template, not an edit date. Ignore it.
- Conflicting values across sources. If two fresh sources disagree, don't pick the newer one. Search again, and prefer a primary source to settle it.
- Conflicting dates. Prefer the most specific signal: a visible "Updated" or "Last Update" line, then the publication metadata, then the HTTP Last-Modified header. Record which one you used so the answer can cite it.
- Relative dates. TinySearch returned "2 weeks ago" for the NerdWallet result. Convert relative dates against the time of the search, and treat them as approximate. They're fine for filtering, not for citing.
- Dates without a time. "Oct 28, 2026" can't be compared to a decision announced at 2:00 p.m. EDT that day. My checker compares date-only values by calendar day, so a page updated "Oct 28" isn't marked stale against a same-day decision.
- Time zones. Store everything as UTC. When a date has no zone, assume UTC and say so in the reasons.
- Dates in the future. Usually a parsing error or a scheduled post. Treat them as unreliable.
- Cached fetches. If your fetch layer can serve cached copies, your retrieval time is not the time the page was read from the origin. With TinyFetch, set ttl to 0 for a live fetch when freshness matters.
The reusable checker
Here's the whole thing in one file. It has no dependencies beyond the Python standard library. extract_dates collects the date signals from one fetch result, check_freshness applies the age policy, claim_values pulls every stated value for the claim from the metadata and the body, and decide applies the reliability and corroboration rules and turns everything into an action.
Inputs: one fetched page as a dict (url, published_date, last_modified, description, text), the kind of question, the current time, the last change point if there is one, and a reliability tier you assign to each source.
Outputs: a FreshnessResult per page (fresh, stale or unknown, the date used and the reasons), then one decision: answer with a qualifier, or search again with what to try next and a dated fallback.
"""Check whether a retrieved source is fresh enough to answer a time-sensitive question."""
from __future__ import annotations
import re
from dataclasses import dataclass, field
from datetime import datetime, timedelta, timezone
from email.utils import parsedate_to_datetime
# How old the evidence may be, by kind of question.
# None means "no fixed age": the evidence must be dated on or after the last change point.
POLICY: dict[str, timedelta | None] = {
"live_value": timedelta(days=7), # prices, stock levels, latest versions
"scheduled_value": None, # policy rates, scheduled releases: check the last change point
"officeholder": timedelta(days=30), # who holds a role
"stable_fact": timedelta(days=365 * 3), # definitions, how-tos, history
}
MONTHS = "Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Sept|Oct|Nov|Dec"
VISIBLE_DATE = re.compile(
rf"(Updated|Last [Uu]pdated?|Last [Mm]odified|Published)[:\s]*\n?\s*"
rf"((?:{MONTHS})[a-z]*\.? \d{{1,2}},? \d{{4}})"
)
@dataclass
class SourceDates:
url: str
retrieved_at: datetime # when we fetched it: never proof the fact is current
published: datetime | None = None # page metadata (TinyFetch published_date)
modified: datetime | None = None # visible "Updated ..." / "Last Update" line
http_last_modified: datetime | None = None
date_only: set[str] = field(default_factory=set) # fields with no time of day
notes: list[str] = field(default_factory=list)
@dataclass
class FreshnessResult:
status: str # "fresh" | "stale" | "unknown"
date_used: datetime | None
date_field: str | None
reasons: list[str]
def parse_date(value: str | None) -> datetime | None:
"""Parse ISO 8601, HTTP dates and 'October 28, 2026'. Naive dates are assumed UTC."""
if not value:
return None
try:
dt = datetime.fromisoformat(value.replace("Z", "+00:00"))
except ValueError:
try:
dt = parsedate_to_datetime(value)
except (TypeError, ValueError):
text = re.sub(r"\bSept\b", "Sep", value.replace(".", ""))
for fmt in ("%B %d, %Y", "%b %d, %Y", "%B %d %Y", "%b %d %Y"):
try:
dt = datetime.strptime(text, fmt)
break
except ValueError:
continue
else:
return None
return dt if dt.tzinfo else dt.replace(tzinfo=timezone.utc)
def extract_dates(page: dict, retrieved_at: datetime) -> SourceDates:
"""Collect the date signals from one TinyFetch result."""
src = SourceDates(url=page["url"], retrieved_at=retrieved_at)
src.published = parse_date(page.get("published_date"))
src.http_last_modified = parse_date(page.get("last_modified"))
match = VISIBLE_DATE.search(page.get("text") or "")
if match:
src.modified = parse_date(match.group(2))
src.date_only.add("modified")
src.notes.append(f'visible date: "{match.group(1)} {match.group(2)}"')
if not (src.published or src.modified or src.http_last_modified):
src.notes.append("no publication or update date found")
return src
def check_freshness(src: SourceDates, kind: str, now: datetime,
last_change: datetime | None = None) -> FreshnessResult:
"""Is this source dated recently enough for this kind of question?"""
reasons = list(src.notes)
order = [("modified", src.modified), ("published", src.published),
("http_last_modified", src.http_last_modified)]
name, used = next(((n, d) for n, d in order if d), (None, None))
if used is None:
reasons.append("only the retrieval time is known: that is when we looked, not when the page changed")
return FreshnessResult("unknown", None, None, reasons)
if used > now + timedelta(days=1):
reasons.append(f"{name} {used:%Y-%m-%d} is in the future, so it is unreliable")
return FreshnessResult("unknown", used, name, reasons)
if last_change is not None:
# A date with no time of day can only be compared by calendar day.
before = used.date() < last_change.date() if name in src.date_only else used < last_change
if before:
reasons.append(f"{name} {used:%Y-%m-%d} is before the last change point {last_change:%Y-%m-%d}")
return FreshnessResult("stale", used, name, reasons)
max_age = POLICY[kind]
if max_age is None and last_change is None:
reasons.append(f"{kind} needs a last change point to judge freshness")
return FreshnessResult("unknown", used, name, reasons)
if max_age is not None and now - used > max_age:
reasons.append(f"{name} is {(now - used).days} days old; {kind} allows {max_age.days}")
return FreshnessResult("stale", used, name, reasons)
reasons.append(f"{name} {used:%Y-%m-%d} passes the {kind} policy")
return FreshnessResult("fresh", used, name, reasons)
def claim_values(page: dict, pattern: str) -> set[str]:
"""Every value the page states for the claim, in the metadata and in the body."""
blob = (page.get("description") or "") + "\n" + (page.get("text") or "")
return set(re.findall(pattern, blob))
# How much to trust a source for a current fact. You assign the tier per domain.
TIERS = {"primary": 3, "secondary": 2, "other": 1}
def decide(results: list[tuple[str, FreshnessResult, set[str], str]]) -> dict:
"""Answer, answer with a qualifier, or search again.
results: (url, freshness result, values stated on the page, tier).
A current answer needs a fresh primary source, or two fresh secondary
sources from different sites that state the same value.
"""
usable = [(url, r, next(iter(vals)), tier) for url, r, vals, tier in results
if r.status == "fresh" and len(vals) == 1]
primary = [u for u in usable if u[3] == "primary"]
if primary:
url, r, value, _ = max(primary, key=lambda x: x[1].date_used)
return {"action": "answer", "value": value,
"qualifier": f"as of {r.date_used:%B %-d, %Y}, according to {url}"}
by_value: dict[str, list] = {}
for u in usable:
if u[3] == "secondary":
by_value.setdefault(u[2], []).append(u)
for value, group in by_value.items():
sites = {re.sub(r"^https?://(www\.)?([^/]+).*", r"\2", u[0]) for u in group}
if len(sites) >= 2:
newest = max(group, key=lambda x: x[1].date_used)
return {"action": "answer", "value": value,
"qualifier": f"as of {newest[1].date_used:%B %-d, %Y}, according to {len(sites)} independent sources"}
conflicting = [url for url, r, vals, _ in results if r.status == "fresh" and len(vals) > 1]
if conflicting:
return {"action": "search_again", "why": f"fresh pages disagree with themselves: {conflicting}",
"next": "consult a primary source"}
if usable:
url, r, value, tier = max(usable, key=lambda x: x[1].date_used)
return {"action": "search_again", "why": f"only one {tier} source supports {value}",
"next": "consult a primary source, or answer with a qualifier if retries run out",
"fallback": f"{value}, as of {r.date_used:%B %-d, %Y}, according to {url} (not confirmed by a primary source)"}
return {"action": "search_again", "why": "no source is dated after the last change point",
"next": "tighten the search: after_date at the last change point, or official domains only"}A few design choices worth calling out:
- The result always carries the date it used and the reasons. That makes the final answer citable and makes failures debuggable.
- Unknown is a real outcome, separate from stale. A page with no dates isn't necessarily wrong, it just can't be trusted for a time-sensitive answer on its own.
- The claim check is deliberately dumb: a regex for the value you care about. For rates it catches "3.50% to 3.75%", "3.75-4.00" and "3-3/4 to 4 percent". In production you'd normalize those to one form before comparing, or ask a model to extract the value. The point is to look at the claim, not just the dates.
Running it on the real pages
I saved the fields from the three pages I fetched (NerdWallet, the Fed's open market operations page and the October 28 FOMC statement) and ran the checker against them, with the last change point set to the October 28 decision. I tagged federalreserve.gov as primary and NerdWallet as secondary:
search result: https://www.nerdwallet.com/banking/news/what-is-the-fed-rate | date: 2 weeks ago
snippet ends: ight. The current target range is 3.50% to...
https://www.nerdwallet.com/banking/news/what-is-the-fed-rate [secondary]
status: fresh (used modified)
- visible date: "Updated Oct 28, 2026"
- modified 2026-10-28 passes the scheduled_value policy
values on page: ['3.50% to 3.75%', '3.75% to 4.00%']
https://www.federalreserve.gov/monetarypolicy/openmarket.htm [primary]
status: stale (used modified)
- visible date: "Last Update September 16, 2026"
- modified 2026-09-16 is before the last change point 2026-10-28
values on page: ['3.75-4.00']
https://www.federalreserve.gov/newsevents/pressreleases/monetary20261028a.htm [primary]
status: fresh (used modified)
- visible date: "Last Update October 28, 2026"
- modified 2026-10-28 passes the scheduled_value policy
values on page: ['3-3/4 to 4 percent']
decision: {'action': 'answer', 'value': '3-3/4 to 4 percent', 'qualifier': 'as of October 28, 2026, according to https://www.federalreserve.gov/newsevents/pressreleases/monetary20261028a.htm'}Each page lands where it should:
- NerdWallet passes the date check but states two different values, so decide() won't answer from it.
- The Fed's open market operations page has the right value but is dated before the last change point, so it's stale for this question.
- The FOMC statement is a primary source, dated on the change point, and states one value, so the pipeline answers from it, with a qualifier that names the date and the source.
A testable example
The same rules fit in four small tests you can drop into your own suite. They use made-up pages, so they run offline:
from datetime import datetime, timezone
from freshness_check import extract_dates, check_freshness, claim_values, decide
NOW = datetime(2026, 11, 10, 1, 29, tzinfo=timezone.utc)
LAST_CHANGE = datetime(2026, 10, 28, 18, 0, tzinfo=timezone.utc)
RATE = r"\d\.\d{2}%? ?(?:to|-) ?\d\.\d{2}%?"
def run(page, tier):
res = check_freshness(extract_dates(page, NOW), "scheduled_value", NOW, last_change=LAST_CHANGE)
return (page["url"], res, claim_values(page, RATE), tier)
def test_fresh_date_but_stale_sentence_means_search_again():
page = {"url": "https://example-news.com/fed-rate",
"published_date": "2026-10-28T07:00:00Z",
"description": "The current target range is 3.50% to 3.75%.",
"text": "Updated Oct 28, 2026\nThe current Fed rate is 3.75% to 4.00%."}
decision = decide([run(page, "secondary")])
assert decision["action"] == "search_again"
assert decision["next"] == "consult a primary source"
def test_page_with_no_dates_is_unknown_not_fresh():
page = {"url": "https://example.com/rates", "text": "The rate is 3.75% to 4.00%."}
assert run(page, "secondary")[1].status == "unknown"
def test_two_independent_secondary_sources_corroborate():
a = {"url": "https://www.site-a.com/x", "text": "Updated Oct 29, 2026\nThe range is 3.75% to 4.00%."}
b = {"url": "https://site-b.org/y", "text": "Updated Oct 30, 2026\nNow 3.75% to 4.00%."}
decision = decide([run(a, "secondary"), run(b, "secondary")])
assert decision["action"] == "answer"
assert decision["value"] == "3.75% to 4.00%"
def test_one_secondary_source_is_not_enough():
a = {"url": "https://www.site-a.com/x", "text": "Updated Oct 29, 2026\nThe range is 3.75% to 4.00%."}
decision = decide([run(a, "secondary")])
assert decision["action"] == "search_again"
assert "fallback" in decisionRunning them:
PASS test_fresh_date_but_stale_sentence_means_search_again
PASS test_one_secondary_source_is_not_enough
PASS test_page_with_no_dates_is_unknown_not_fresh
PASS test_two_independent_secondary_sources_corroborateThe first test is the NerdWallet case in miniature: a page with a fresh update date that still states the old value in its metadata. The checker refuses to answer from it and asks for a primary source.
Want to run this against your own questions? Get a free TinyFish API key and plug TinySearch and TinyFetch into the checker.
Wiring it into TinySearch and TinyFetch
One thing to be clear about first: a live fetch gets you the page as it is right now, not a guarantee that what the page says is current. TinyFetch can tell you when you retrieved a page and what dates it carries. Whether the claim on it is still true is the checker's job, and yours. The checker needs two things from your web layer: search results you can filter by date and source, and fetched pages with their date metadata. Both are parameters in the TinySearch reference and the TinyFetch reference.
On the search side:
- recency_minutes limits results to a window relative to now, from 1 minute up to 10 years.
- after_date and before_date set calendar cut-offs in YYYY-MM-DD form. They can't be combined with recency_minutes.
- include_domains restricts results to sources you trust for the fact, such as federalreserve.gov.
- domain_type set to "news" returns news results with a publisher and a date.
On the fetch side:
- ttl set to 0 forces a live fetch instead of a cached copy.
- include_etag_and_last_modified returns the origin's ETag and Last-Modified validators when it sends them. None of the three pages above sent a Last-Modified header, which is common, so don't rely on it alone.
- if_modified_since replays a saved Last-Modified value. If the page hasn't changed, the result comes back with not_modified set to true and you can skip re-checking it.
- Every result includes published_date when it can be detected, plus the page text, where visible "Updated" lines live.
Here's how the pieces connect, using the parameter names from those references. In my runs I made the same calls through the TinyFish search and fetch tools, so treat this as a sketch to adapt rather than a tested script:
from datetime import datetime, timezone
from tinyfish import TinyFish
from freshness_check import extract_dates, check_freshness, claim_values, decide, parse_date
client = TinyFish() # reads TINYFISH_API_KEY from the environment
RATE = r"\d\.\d{2}%? ?(?:to|-) ?\d\.\d{2}%?|\d-\d/\d to \d percent"
last_change = parse_date("2026-10-28T18:00:00Z")
def fresh_answer(query: str, domains: str | None = None, after: str | None = None) -> dict:
hits = client.search.query(query=query, include_domains=domains, after_date=after)
urls = [r.url for r in hits.results[:5]]
pages = client.fetch.get_contents(urls=urls, format="markdown", ttl=0,
include_etag_and_last_modified=True)
now = datetime.now(timezone.utc)
results = []
for page in pages.results:
p = {"url": page.url, "published_date": page.published_date,
"last_modified": getattr(page, "last_modified", None),
"description": page.description, "text": page.text}
res = check_freshness(extract_dates(p, now), "scheduled_value", now, last_change)
tier = "primary" if "federalreserve.gov" in page.url else "secondary"
results.append((page.url, res, claim_values(p, RATE), tier))
return decide(results)
answer = fresh_answer("federal funds rate target range")
if answer["action"] == "search_again":
# Tighten the search: official source only, dated after the last change point.
answer = fresh_answer("FOMC statement federal funds rate target range",
domains="federalreserve.gov", after="2026-10-27")
print(answer)That second, tighter search is exactly what I ran. Restricted to federalreserve.gov and dated after October 27, its top result was the October 28 FOMC statement.
When to search again and when to qualify the answer
The decide() function encodes three outcomes: answer, search again with a next step, or fall back to a qualified answer. Here's when each one fits.
Search again when:
- No source passes the freshness policy.
- The fresh sources disagree, with each other or with themselves.
- The only fresh source is a snippet you haven't fetched.
- Only one secondary source supports the value, with no primary source and nothing to corroborate it.
Each retry should be tighter than the last: add after_date at the last change point or a short recency window, restrict include_domains to official or primary sources, or switch to news results for very recent events. Cap the retries. Two or three rounds is usually enough to know whether a fresh answer exists.
Answer with a qualifier whenever the fact is time-sensitive, even when everything checks out. "The target range is 3.75% to 4.00% as of October 28, 2026, according to the Federal Reserve" is more useful than a bare number, because it tells the reader exactly when it could change.
Say you don't know when the retries run out. A good fallback is the most recent value you have, clearly dated, plus a note that you couldn't confirm it's still current. That's honest, and it's still useful.
For facts you check often, stop re-searching from scratch. Watch the primary source instead: TinyMonitor can watch a page or a search topic on a schedule and tell you when it changes, so your pipeline knows when to refresh. More on that in the Monitor docs.
What this check doesn't solve
It's worth being straight about the limits:
- It trusts dates that publishers set. A site can stamp "Updated today" on an unchanged page. The claim check catches some of that, but not all.
- The claim check only sees what your pattern or extractor sees. If the stale value is phrased differently, a regex will miss it.
- Change points have to come from somewhere. For scheduled facts that's easy. For prices, a short age limit is the best you can do.
- Reliability tiers are only as good as your list. The checker trusts the tier you give each domain. Review the list, especially for secondary sources.
For the broader picture of building search into agents, see our guides to agentic search, production agentic search architecture, giving agents better context rather than more context, production-grade web fetching and the best web search APIs for AI agents.
Give your pipeline the live web, with the dates to judge it. Start free with TinyFish: TinySearch and TinyFetch are free within their limits, and new accounts get $8 for TinyAgent and TinyBrowser.
AI disclosure
Content on this website may be created or refined with the assistance of AI tools and is subject to human editorial review.



