Arca Found 58 of 60 Records in a Biomedical Search Benchmark
· Updated

We gave five search APIs the same 60 regulatory and life-science questions: Arca, Parallel, Perplexity, Exa and GXL Paperclip. Each got one search per question. Arca returned the right document for 58 of 60. Parallel returned it for 44, Perplexity for 40, Exa for 39 and GXL Paperclip for 24. An agent answering regulatory questions needs search that finds the record itself, so that is what we counted.
This is the search-only companion to LifeSci Diamond, which tests whole regulatory tasks. For literature questions rather than record lookups, see our ResearchQA comparison. We built Arca and graded the results ourselves, so read the limits below.
The 60 questions mix fixed lookups with records drawn at random
The first ten are lookups adapted from LifeSci Diamond tasks and public examples. They include an original US label, a section of 21 CFR and an EMA procedure record.
Twenty more were drawn at random, with a fixed seed, before any tool ran. They cover 8 FDA complete response letters asked by company and month, and 6 Phase 3 trials asked by name for the registration and primary outcome time frame. The remaining 6 are bioRxiv preprints, which a model described from the abstract without giving the title.
The last thirty follow the kinds of questions people ask Arca's search in production. We chose the kinds from recent query text alone, then drew the records at random from FDA's indexes. That gives 8 final guidance documents asked by title, 6 Class I recalls asked by company, product and month, 6 510(k) clearances asked by device, company and month, and 6 current drug labels asked by brand name. The set ends with 4 device event reports asked by MDR number. Each draw came from a limited pool, such as the first 1,000 matching records. Because we drew those 50 records before running any tool, no tool's strengths shaped which questions it faced.
Only the record itself counted as found
A question counted as found when the results held the specific record, such as the letter for that application on that date. For the second batch we also required the record's ID on an official FDA or NIH page. The right topic in the wrong version, or a copy on another site, counted as near.
For 7 of the 8 letters, Exa returned a page about the right letter on a third-party site, mostly Assyro. Those count as near. If copies counted as found, Exa would score 46 of 60 and Parallel 49.
Arca led on complete response letters and recalls
| Kind of question | ![]() | ||||
|---|---|---|---|---|---|
| FDA complete response letters (8) | 8 | 3 | 1 | 0 | 0 |
| Trial registrations (6) | 6 | 5 | 4 | 6 | 5 |
| Preprints (6) | 5 | 5 | 5 | 5 | 3 |
| First ten, mixed (10) | 9 | 7 | 7 | 5 | 4 |
| FDA guidance (8) | 8 | 8 | 8 | 8 | 2 |
| Class I recalls (6) | 6 | 4 | 5 | 4 | 0 |
| 510(k) clearances (6) | 6 | 6 | 6 | 6 | 6 |
| Current drug labels (6) | 6 | 6 | 4 | 5 | 0 |
| Device event reports (4) | 4 | 0 | 0 | 0 | 4 |
| Total (60) | 58 | 44 | 40 | 39 | 24 |
![]() | |||||
|---|---|---|---|---|---|
| Median search time | 17 s | 2.9 s | 0.3 s | 1.1 s | 2.9 s |
Perplexity and Exa were much the fastest. Arca was the slowest, because a lookup for one specific record runs an extra retrieval step that checks candidate documents before it answers. An agent that runs many quick searches over the open web, and can accept a summary of a record, may be better served by Perplexity or Exa.
GXL Paperclip's misses come mostly from what its index holds. Its FDA index has no recall records and thin coverage of guidance, and it returns text without URLs, so we could not confirm a label's source page and its six labels count as near.
The complete response letters show the gap most clearly. Asked for FDA's letter to Minerva Neurosciences from February 2024, Arca's first result was the letter itself, from FDA's published dataset, for NDA 217002, dated February 26, 2024. Exa's first result was a third-party summary of the same letter. Parallel found FDA's letter for 3 of the eight and Perplexity for 1. GXL Paperclip returned FDA review packages for other applications, since its FDA index holds review documents and no published letters. An agent that must cite the letter itself needs the letter.
Where Arca fell short
Arca answered two questions only partly. For the Eylea HD complete response letter it returned FDA's later 2024 action letter for the same application, not the June 2023 one. It also missed the page for one of the six sampled preprints.
The test also leaves Arca some advantages. In the second batch Arca returned 17 to 36 results per question and the others 10, which gave Arca more chances to include the record. Sixty questions is a small set, and we did not test ChatGPT or Claude.
Each tool ran each question once on November 18, 2026, with no retries, asking for 10 results. We used Exa's search API, Parallel's search API in advanced mode, Perplexity's Search API, GXL Paperclip's free plan (FDA, EU, trials and papers sources) and Arca's public research API. One person on the Arca team graded every result with the rule above. No regulatory affairs expert has checked the answer keys, and we have not published the raw results. Search times are from our network. With these limits, the margins are rough, but a gap as wide as 58 against 44 shows that general web search misses many FDA records an agent needs to cite.



