AI Build · Case study
Medical Note Summarizer
Clinicians lose time reading long, unstructured notes, and a summary that hallucinates is worse than no summary.
- My role
- Solo builder (product + engineering)
- Outcomes
- • Fine-tuned Llama 2 7B on clinical notes via Replicate
- • Evaluated with ROUGE, BLEU and hallucination checks: [ADD METRIC]
- • Deployed as a public demo on Streamlit / Hugging Face
- Fine-tuning
- Llama 2
- Replicate
- Evaluation
- Streamlit
Live Demo [ADD LINK]GitHub [ADD LINK]
User problem
Clinical notes are long and inconsistent. A summarizer can save clinicians time, but only if it is faithful: in healthcare, an invented medication or dosage is a safety issue.
Who it’s for
Clinicians and care teams who need to scan a patient’s history quickly. [ADD DETAIL]
Success metrics
- ROUGE / BLEU: overlap with reference summaries ([ADD METRIC])
- Hallucination rate: share of summaries containing facts not in the source note ([ADD METRIC])
- Latency per note: [ADD METRIC]
Solution & key trade-offs
- Fine-tune a 7B model instead of prompting a large one. [ADD DETAIL: why, e.g. cost, control, privacy]
- Replicate for training and hosting. No GPU setup needed, pay-per-use.
- ROUGE wasn’t enough. Overlap metrics reward copying and can’t catch invented facts, so I added explicit hallucination checks.
How it works
[ADD ARCHITECTURE DIAGRAM]
- Clinical note dataset prepared for instruction fine-tuning ([ADD DETAIL: dataset])
- Llama 2 7B fine-tuned via Replicate
- Evaluation harness: ROUGE, BLEU, hallucination checks
- Streamlit app on Hugging Face for live demos
Results & evaluation
- [ADD METRIC: ROUGE-L, BLEU]
- [ADD METRIC: hallucination rate before vs. after]
What I’d do next
- [ADD NEXT STEP]