AI Build · Case study
The Intelligence Times
Tracking competitors means hours of reading news, repos and papers. Teams need a cited brief, not 40 open tabs.
- My role
- Solo builder (product + engineering)
- Outcomes
- • 4 specialist agents produce a cited competitive-intelligence brief in ~25 seconds
- • Measured 79% citation coverage and 80% number fidelity on real runs
- • Runs entirely on free tiers ($0/month) as a live public web app
- Multi-agent
- RAG
- Pinecone
- BM25
- Groq
- Python
User problem
Keeping up with competitors is slow, manual work. The signal is spread across news articles, GitHub activity and research papers, and a generic chatbot will confidently summarize things it never read. People doing competitive research need a short brief they can trust: every claim traced to a source.
Who it’s for
Product managers, founders and strategy folks who need a weekly read on a market or competitor set without spending a day collecting sources. [ADD DETAIL: any specific users you tested with]
Success metrics
- Citation coverage: share of claims in a brief backed by a retrieved source (measured: 79%)
- Number fidelity: share of numbers in a brief that match their source (measured: 80%)
- Latency: time to a finished brief (measured: ~25s)
- Cost: must run at $0/month on free tiers (achieved)
Solution & key trade-offs
A multi-agent RAG system: four specialist LLM agents each generate their own search queries, retrieve evidence with hybrid search, and write cited sections of a brief.
- Hybrid retrieval over pure vector search. Semantic search misses exact names, tickers and version numbers, while BM25 misses paraphrases. I run both and merge the two rankings with Reciprocal Rank Fusion, which needs no score calibration between systems.
- LLM-as-judge re-ranking. I added an extra LLM pass that scores retrieved chunks for relevance before they reach the writer. It trades a little latency for fewer off-topic citations.
- Groq for inference. Fast, free-tier-friendly inference keeps the full multi-agent run around 25s.
- Pinecone-hosted embeddings. No local ML dependencies, so it deploys anywhere for free.
How it works
- Ingest: in-app ingestion pulls from NewsAPI, GitHub and arXiv and indexes the content.
- Plan: 4 specialist agents self-generate queries for their slice of the brief.
- Retrieve: each query hits Pinecone (semantic) and BM25 (keyword); results are fused with RRF.
- Re-rank: an LLM judge scores the fused candidates and keeps the most relevant.
- Write: agents draft cited sections on Groq; the final brief is assembled with sources.
Deployed on Streamlit / Hugging Face Spaces.
Results & evaluation
- 79% citation coverage and 80% number fidelity, measured on real runs
- ~25 seconds end to end
- $0/month infrastructure
[ADD DETAIL: how many runs or queries the evaluation covered, and how you scored them]
What I’d do next
- [ADD NEXT STEP: e.g. close the gap on uncited claims]
- [ADD NEXT STEP]