PHAROS: 4 agents, 60 seconds, 1 missed drug safety signal away from disaster

I split the system because the jobs are different enough that a single agent would be mediocre at all of them. Monitoring for volume spikes is not the same skill as computing statistical ratios, which is not the same as writing regulatory documents, which is not the same as deciding who to page at 2 a.m. Each agent gets a system prompt tuned to its specific task and temperature settings that match: ANALYST runs at 0.0 because you don't want creative PRR numbers. SCRIBE runs at 0.2 for controlled text generation. SENTINEL at 0.1.

SENTINEL watches the pharos-adverse-events index for volume spikes. It uses ES|QL to compare the last 7 days of report volume against a 90-day baseline. If a drug shows a 3x jump, SENTINEL fires an Elastic workflow that kicks off ANALYST. In the CARDIVEX run, it caught a 15x spike.

ANALYST is where the real detection happens. It runs the WHO PRR calculation entirely in ES|QL — STATS for counts, EVAL for the ratio math, and WHERE for thresholds — across drug-reaction pairs. Then, it runs temporal analysis with BUCKET(report_date, 1 week) to catch weekly clustering, geographic aggregation on geo.country_code, and a hybrid BM25 + dense vector search to find similar historical signals. Severity classification is tiered: PRR ≥ 5.0 with 5+ cases is CRITICAL, PRR ≥ 2.0 with 3+ cases is HIGH, and anything above 1.5 goes to MONITORING. Confirmed signals get written to pharos-signals.

SCRIBE picks up confirmed signals and generates three document types: MedWatch 3500A, PSUR Section VI, and a case narrative. It pulls up to 100 supporting case reports from the adverse events index and produces the documents and indexes them into pharos-regulatory-reports.

HERALD is the action layer. CRITICAL signals get a Slack alert (Block Kit formatting), a Jira P1 ticket, and emails to the safety officer and VP of Safety. HIGH signals get Slack, Jira P2, and email to the safety officer. MONITORING signals accumulate into a weekly digest. A 2-hour escalation timeout re-alerts the VP of Safety if a CRITICAL signal goes unacknowledged.

The handoffs between agents all run through Elastic workflows — nine workflows total covering agent-to-agent coordination, nightly FAERS ingestion on a cron schedule, Slack/Jira/email dispatch, audit logging, and the escalation timeout.

I made a deliberate choice to keep PRR computation inside ES|QL rather than pulling data into Python. Going in, I assumed I'd need pandas for the statistical work. I was wrong.

The full WHO PRR formula, counting, ratio math, thresholds, temporal bucketing all runs as ES|QL queries. The agents call ES|QL tools, reason over the results, and write back — no pandas, no external compute, and no data transfer bottleneck. The stats scale with the cluster.

ES|QL is less flexible than pandas for arbitrary analysis. But for the WHO formula and weekly BUCKET aggregations, it handles the work cleanly. Cutting out that intermediate Python layer simplified the architecture more than I expected — the agents just query and reason, and there's one fewer place for things to break.

PHAROS runs on four Elasticsearch Serverless indices, and the main one, pharos-adverse-events, is where I spent the most design time.

It has a custom clinical_text_analyzer with snowball stemming for narrative search, a drug_name_analyzer on keyword tokenizer for exact drug matching, dense_vector fields (1,536 dimensions) for narrative embeddings, geo_point for geographic clustering, and nested mappings for reactions. Every query the agents need, fuzzy narrative search, exact drug lookup, geographic aggregation, semantic similarity is supported by the index design. The other three indices are more straightforward: pharos-signals stores detected signals with PRR scores and the analyst's reasoning chain, pharos-regulatory-reports holds generated documents, and pharos-audit-log timestamps every agent action.

Getting LLMs to return structured JSON reliably was the fight I didn't anticipate.

You ask an LLM for JSON, you get JSON wrapped in three paragraphs of explanation, or JSON inside markdown code fences, or a conversational preamble followed by JSON followed by a helpful summary. The agents hand structured data to each other, so every response needs to parse cleanly. It doesn't matter how good your signal detection is if the ANALYST's output can't be reliably read by SCRIBE.

I spent a lot of time tuning system prompts and ended up writing a JSON extraction function that handles raw JSON, markdown code fences, and JSON buried inside natural language. It's not interesting work, but it's the kind of thing that determines whether a multi-agent pipeline actually runs or just demos well.

The PRR calculation is currently a point estimate. A production pharmacovigilance system needs chi-squared confidence bounds and Bayesian IC scoring. The data model already has an ic_score field wired up — it's using an approximation instead of the proper Bayesian calculation. That's the first thing I'd change with more time.

The system also treats "blurred vision" and "vision loss" as separate events. The immediate next step is MedDRA ontology-aware reaction grouping so that the system can catch signals across related terms instead of treating each string as independent. After that, I would pull in EudraVigilance data alongside FAERS for cross-continental correlation.

2 million adverse event reports land on someone's desk every year, and the current answer is more analysts running more manual reviews. PHAROS is an argument that the answer is agents that run the WHO statistics, generate the paperwork, and escalate to the right person — all before the analyst has opened their laptop.

PHAROS is open source under MIT. If you work in pharmacovigilance or regulatory affairs and want to run this against real data, I'd like to hear from you.