Details
- Perplexity open sources WANDR, an internal benchmark designed to build and evaluate wide-and-deep research capabilities in Perplexity Computer.
- The benchmark focuses on two core agent skills: searching broadly enough to find all qualifying entities and investigating deeply enough to back every claim with cited evidence.
- WANDR encodes these requirements as hierarchical, independently verifiable records across 500 research tasks and 170,495 source-backed records spanning three difficulty tiers.
- It tests how well agents discover large sets of entities and verify specific facts about each one, providing a dense, interpretable evaluation signal that pinpoints where and why an agent fails.
- The WANDR pipeline doubles as a semi-automated factory for training data, turning evaluation runs into structured, source-grounded datasets.
- Tasks are constructed from de-identified production use cases, mirroring real day-to-day research workloads such as competitive analysis, due diligence, literature review, market analysis, product comparison, and talent sourcing.
- Instead of grading against a fixed gold solution, WANDR re-fetches every cited page and checks each claim against the underlying evidence, allowing tasks to include time-varying facts.
- The scoring system includes soft scores that grant partial credit, and hard scores that require every component for a member to be complete and correct before it is marked right.
- Perplexity has released both the benchmark tasks and the evaluation harness as an open-source resource for testing and improving research agents.
Impact
By open-sourcing WANDR, Perplexity introduces a rigorous, evidence-first benchmark that shifts evaluation of AI research agents from static answer keys to live web verification. This raises the bar for competitors building agentic systems, pushes the field toward realistic, production-shaped workloads, and provides a shared framework for measuring broad discovery and deep fact-checking performance over time.
