Clair API
  • All APIs
  • Pricing
  • Docs
Web Scraping
  • Web Scraper API
E-commerce
  • Amazon API
    • Product
    • Reviews
    • Search
    • Price
    • Best sellers
    • Seller
    • ASIN lookup
  • Google Shopping API
SERP & Search
  • Google Search API
  • Google Maps API
    • Scraper
    • Place
AI Search
  • ChatGPT API
  • Google AI Overview API
  • Google AI Mode API
Jobs & Hiring
  • Glassdoor API
    • Jobs
    • Reviews
    • Salary
  • Indeed API
Company Data & Reviews
  • Trustpilot API
  • Crunchbase API
    • Company
    • People
Video & Social
  • YouTube API
  • TikTok API
  • Instagram API
News & Media
  • Google News API
  • Reuters API
  • Bloomberg API
  • Associated Press API
  • Use cases
  • Comparisons
  • Glossary
Ecommerce and reviews
  • Amazon price monitoring
  • Trustpilot review monitoring
  • MAP enforcement on Amazon
  • Amazon rating monitoring
  • Amazon category research
  • Trustpilot competitor benchmarking
  • Google Shopping price tracking
Search and local
  • Google rank tracking
  • Local lead generation
  • Local SEO competitor audit
  • Grounding LLM answers in search results
  • Content research from People also ask
AI search
  • AI visibility tracking
  • Find the sites AI answers cite
  • AI Overview tracking for SEO keywords
Leads and companies
  • Account research briefs
Jobs and hiring
  • Salary benchmarking
  • Job market analytics
  • Employer reputation tracking
News
  • Brand and media monitoring
  • Financial news tracking
  • News corpus for research and RAG
Social and video
  • YouTube transcripts for RAG
  • Instagram engagement rate
  • Influencer campaign tracking
Comparisons
  • Clair vs SerpApi
  • Clair vs Google Places API
  • Clair vs Google Custom Search API
GlossarySERPSERP APIPeople also askGoogle place IDASINBuy BoxGTINSalary percentilesTrustScoreRating distributionAll 47 terms →
PricingDocsStatus
Dashboard
  • Use cases
  • Associated Press API
  • Wire service

News corpus for research and RAG

Kaan Ersoy·1 min read·Oct 29, 2026

{ }
Collect the full text of news stories on a topic, with titles, authors, and dates, as clean JSON. Search AP for the topic, read each article, and write one line per story. The same pattern works with Reuters sections and Bloomberg search.
Try the Associated Press API →

On this page

  1. Who
  2. Pipeline
  3. Code
  4. Cost
  5. Limits
  6. FAQ

Who runs this, and what for

TeamWhat they decide with it
ResearchersHow a topic was covered over a period, from the text rather than the headlines.
AI engineersWhat current, sourced text to put in a retrieval index so a model can cite news.
AnalystsWhich events and entities recur in coverage of a market or region.

The calls, step by step

  1. 1. Associated Press world news

    GET /v1/ap/search

  2. 2. Associated Press article

    GET /v1/ap/article

The endpoints this pipeline calls, in order. Each step below shows a recorded response from one of them.
  1. 1. Search the topic

    AP search takes q with sort=newest and up to 20 pages. Each result has the URL, publish time, and content type; keep articles and skip videos and galleries.

    The apnews.com page this step reads
    The page on apnews.com that this call reads, as a signed-out visitor sees it. The response below is what Clair returns from it.

    GET /v1/ap/search?q=hurricane&sort=newest&page=1 · results.0

    {
      "position": 1,
      "title": "Powerful Hurricane Polo drenches Mexico’s coast as Nolo takes aim at Hawaii",
      "url": "https://apnews.com/article/hurricane-polo-mexico-nolo-hawaii-a036e08f59c3f3842aaba2ee69bd9d37",
      "published_at": "2026-09-24T08:20:28.000Z",
      "authors": [],
      "content_type": "article",
      "premium": false
    }
    Recorded response. Live values differ.
  2. 2. Read each article

    The article endpoint returns the story as paragraphs, with authors and section. AP stories are free to read, so body_truncated is false and the text is complete.

    GET /v1/ap/article?url=https%3A%2F%2Fapnews.com%2Farticle%2Fhurricane-polo-mexico-nolo-hawaii-a036e08f59c3f3842aaba2ee69bd9d37 · article

    {
      "title": "Powerful Hurricane Polo drenches Mexico’s coast as Nolo takes aim at Hawaii",
      "url": "https://apnews.com/article/hurricane-polo-mexico-nolo-hawaii-a036e08f59c3f3842aaba2ee69bd9d37",
      "authors": [
        {
          "name": "Desiree Brooks",
          "url": null
        },
        {
          "name": "Megan Janetsky",
          "url": null
        }
      ],
      "section": "World News",
      "access": "free",
      "body_truncated": false,
      "paragraphs": [
        "Hurricane Polo dumped heavy rain on Mexico’s Pacific coast."
      ]
    }
    Recorded response. Live values differ.
  3. 3. Store it with its source

    Keep the URL and publish time with every document. A retrieval system that cites the article, with its date, is one readers can check.

    GET /v1/ap/article?url=https%3A%2F%2Fapnews.com%2Farticle%2Fhurricane-polo-mexico-nolo-hawaii-a036e08f59c3f3842aaba2ee69bd9d37 · article.url

    "https://apnews.com/article/hurricane-polo-mexico-nolo-hawaii-a036e08f59c3f3842aaba2ee69bd9d37"
    Recorded response. Live values differ.

A script to start from

news_corpus.py

# Build a news corpus for a topic from AP: search, then read each article,
# and write one JSON line per story for analysis or a retrieval index.
import json, os, requests

API = "https://api.clair.im"
HEADERS = {"Authorization": f"Bearer {os.environ['CLAIR_API_KEY']}"}
QUERY = "hurricane"
PAGES = 5

def get(path, **params):
    r = requests.get(f"{API}{path}", headers=HEADERS, timeout=60, params=params)
    r.raise_for_status()
    return r.json()

urls = []
for page in range(1, PAGES + 1):
    body = get("/v1/ap/search", q=QUERY, sort="newest", page=page)
    urls += [r["url"] for r in body["results"] if r["content_type"] == "article"]
    if not body["next_cursor"]:
        break

with open("corpus.jsonl", "w") as f:
    for url in dict.fromkeys(urls):  # keep order, drop repeats
        a = get("/v1/ap/article", url=url)["article"]
        f.write(json.dumps({
            "url": a["url"], "title": a["title"], "section": a["section"],
            "authors": [x["name"] for x in a["authors"]],
            "text": "\n\n".join(a["paragraphs"]),
        }) + "\n")

Set CLAIR_API_KEY to your Clair key; one plan covers the Associated Press API. Each call spends that API's rate from the plan's monthly balance.

What it costs per month

ScheduleRequestsPlanPer monthPer 1,000
One topic, 100 articles5 search pages plus 100 article calls = 105 requests.105Starter$30$285.71
10 topics, 200 articles each10 × (10 search pages + 200 articles) = 2,100 requests.2,100Starter$30$14.29
Daily top-up: 30 new articles a dayOne search page and 30 articles a day for 30 days = 930 requests.930Starter$30$32.26
The cheapest Clair plan whose monthly balance covers each volume, every API spending from the same balance. USD, before tax. The free tier is $1 of usage a month, for trying the pipeline.

What to plan for

  • Rights stay with AP. Analysis, research, and retrieval with citations are the usual uses; republishing stories needs an AP license.
  • AP search goes back as far as AP's site search does, up to 20 pages per query. For longer periods, run narrower queries.
  • Section and topic hub pages mix in most-read and right-rail cards. Search results are cleaner for a corpus.

Common questions

Can I use Reuters or Bloomberg the same way?

Yes, with limits. Reuters has no keyword search, so list sections or authors. Bloomberg search works, but metered stories return only their public paragraphs.

What format should the corpus be?

JSON lines, one story per line with url, title, date, and text, loads into most analysis and embedding tools without conversion.

How do I avoid duplicates?

Deduplicate on the article URL. AP updates stories in place, so re-reading a URL later gives the latest version.

Keep reading

APIs

  • Associated Press API

Glossary

  • Wire service
    Clair API

    Live data APIs for search, news, jobs, marketplaces, companies, reviews, and social video, plus a scraper for any public page. Each call fetches the page when you ask.

    contact@serinlabs.com@clair_api

    APIs

    Web Scraping
    • Web Scraper API
    E-commerce
    • Amazon API
      • Amazon Product API
      • Amazon Reviews API
      • Amazon Search API
      • Amazon Price API
      • Amazon Best Sellers API
      • Amazon Seller API
      • Amazon ASIN Lookup API
    • Google Shopping API
    SERP & Search
    • Google Search API
    • Google Maps API
      • Google Maps Scraper API
      • Google Maps Place API
    AI Search
    • ChatGPT API
    • Google AI Overview API
    • Google AI Mode API
    Jobs & Hiring
    • Glassdoor API
      • Glassdoor Jobs API
      • Glassdoor Reviews API
      • Glassdoor Salary API
    • Indeed API
    Company Data & Reviews
    • Trustpilot API
    • Crunchbase API
      • Crunchbase Company API
      • Crunchbase People API
    Video & Social
    • YouTube API
    • TikTok API
    • Instagram API
    News & Media
    • Google News API
    • Reuters API
    • Bloomberg API
    • Associated Press API

    Use cases

    Ecommerce and reviews
    • Amazon price monitoring
    • Trustpilot review monitoring
    • MAP enforcement on Amazon
    • Amazon rating monitoring
    • Amazon category research
    • Trustpilot competitor benchmarking
    • Google Shopping price tracking
    Search and local
    • Google rank tracking
    • Local lead generation
    • Local SEO competitor audit
    • Grounding LLM answers in search results
    • Content research from People also ask
    AI search
    • AI visibility tracking
    • Find the sites AI answers cite
    • AI Overview tracking for SEO keywords
    Leads and companies
    • Account research briefs
    Jobs and hiring
    • Salary benchmarking
    • Job market analytics
    • Employer reputation tracking
    News
    • Brand and media monitoring
    • Financial news tracking
    • News corpus for research and RAG
    Social and video
    • YouTube transcripts for RAG
    • Instagram engagement rate
    • Influencer campaign tracking

    Compare and learn

    Comparisons
    • Clair vs SerpApi
    • Clair vs Google Places API
    • Clair vs Google Custom Search API
    Glossary
    • SERP
    • SERP API
    • People also ask
    • Google place ID
    • ASIN
    • Buy Box
    • All 47 terms
    Developers
    • Documentation
    • Quickstart
    • API catalog
    • Pricing
    • Status
    • All resources
    Featured onCITABLEClair API - Featured on Startup FameFeatured on Twelve Tools Listed on Turbo0 Featured on Wired BusinessClair API on Nick Launches Featured on neeed.directory Featured on DailyPings ShareSpeak on SaaSGrow Startup Benchmarks Featured on LaunchVault Featured on EasyLaunch Featured on SubmitMySaas Featured on DodoDirectory Featured on LaunchZone Featured on LemonLaunch 'LiftOff' Default v2 badge Launched on LaunchBoard - Product Launch Platform Listed on CurlShip aListed onamaze.directory Featured on Brick Launch Featured on saas.fyi Featured on Wall of Tools Featured onfreeboard.lolFeatured on Yo.directory Featured on TheDevToolsIndex
    Featured onCITABLEClair API - Featured on Startup FameFeatured on Twelve Tools Listed on Turbo0 Featured on Wired BusinessClair API on Nick Launches Featured on neeed.directory Featured on DailyPings ShareSpeak on SaaSGrow Startup Benchmarks Featured on LaunchVault Featured on EasyLaunch Featured on SubmitMySaas Featured on DodoDirectory Featured on LaunchZone Featured on LemonLaunch 'LiftOff' Default v2 badge Launched on LaunchBoard - Product Launch Platform Listed on CurlShip aListed onamaze.directory Featured on Brick Launch Featured on saas.fyi Featured on Wall of Tools Featured onfreeboard.lolFeatured on Yo.directory Featured on TheDevToolsIndex
    © 2026 Serin Technologies, LLC
    • About Clair
    • Terms of Service
    • Privacy Policy