NeuroAI NEUROAINEUROAI.SITE
ESC

Fudan's OneSearch-VL makes multimodal research agents cite their sources — and beats Qwen3-VL by 20 points

A Fudan-led team released OneSearch-VL, an 8B multimodal research agent that anchors every claim to a visible part of an image or video, beating a tool-using rival by double digits on new benchmarks.

2026-10-15 · 802 words · NeuroAI
Fudan's OneSearch-VL makes multimodal research agents cite their sources — and beats Qwen3-VL by 20 points

A research agent that looks at a photo and writes a confident answer is easy to build. One that can point to the exact pixel that proves the claim is rarer — and far more useful.

In October 2026, researchers from Fudan University and partner institutions released OneSearch-VL, an 8-billion-parameter multimodal deep-research agent. Its pitch: stop hallucinating citations by making the evidence graph part of the model's reasoning, not an afterthought.

The problem: agents that see but can't prove

Multimodal deep research — comparing medical scans, tracking change across satellite images, finding a fact inside a two-hour lecture video — needs the agent to do three things: ground a claim in a specific visual region, fetch supporting facts from the web, and compose an answer that traces back to both.

Most vision-language models break the chain. The perception step and the search step drift apart. By the final sentence, the model has "seen" something it cannot locate, and cited a source it cannot defend.

How the evidence graph works

OneSearch-VL is built around a Visually Grounded Evidence Graph (VGEG). Instead of jumping from image to answer, it encodes the workflow as a directed graph:

  • localized visual anchors (the bounding box that matters),
  • retrieved entity records,
  • verified source facts,
  • the synthesis steps that produce the answer.

The same graph structures the training data and the reward. A second piece, the Evidence-aware Visual-Grounded Rubric (EVGR), replaces a simple right-or-wrong score with a step-level rubric that rewards accurate visual grounding, real citations, and faithful reasoning.

The data engine assembled OneSearch-VL-SFT-110K for supervised tuning and OneSearch-VL-RL-10K for reinforcement learning.

The numbers that matter

On two new benchmarks the team built — OneSearch-MI-Bench for multi-image research and OneSearch-Video-Bench for long video — OneSearch-VL-8B outperformed a tool-augmented Qwen3-VL-8B by:

  • 20.2 percentage points on the multi-image bench,
  • 17.6 percentage points on the video bench.

In blind evaluation, 91.8% of its external citations lined up with a real visual anchor, cutting "evidence drift" — citing something the image does not support — by 43% over the baseline. It also set new results across seven established image benchmarks and the VideoDR video-retrieval suite.

These figures come from the paper's own abstract (arXiv:2610.12419) and have not been independently reproduced at scale yet.

Why an 8B model beats a tool-using rival

The surprising part is not the lead — it is that OneSearch-VL wins without needing more parameters. It is built on the Qwen3-VL-8B base. The gain comes from training the model to keep its evidence graph intact, not from a bigger brain.

That matters for deployment. An 8B agent can run on a single workstation-class GPU, while still doing the cross-image and cross-frame reasoning that usually demands far larger systems plus a hand-wired search pipeline.

The code, weights, and both datasets are on GitHub (appletea233/OneSearch-VL) under Apache 2.0.

One caveat on scope: OneSearch-VL is a research agent, not a drop-in chatbot. It assumes a retrieval backend and a GPU at inference time, and its value shows only when the task demands provenance — pointing an answer back to a frame or a region. For casual "what is in this picture" questions, a plain vision-language model is faster and simpler. The graph overhead earns its keep only when a wrong citation is costly.

Why this matters beyond China

Search is becoming the backbone of agentic AI. A Shanghai team at Fudan University (复旦大学) built a multimodal research agent on top of an open large model (大模型), showing that citation-grounded answers can come from a modest 8-billion-parameter base rather than only the largest closed systems. For students and researchers, that lowers the cost of trustworthy AI research assistance.

Honest limitations

  • The 20.2 and 17.6 point leads are on benchmarks the authors introduced; strong on their own tests, but not yet validated by third-party runs.
  • The 91.8% citation-precision and 43% drift-reduction figures are from the paper's blind evaluations, not an independent audit.
  • OneSearch-VL is an 8B research agent, not a general chatbot. It needs retrieval tooling and a GPU at inference time.
  • Video understanding is bounded by the reference clips you feed it; it is not a substitute for full human review of sensitive footage.

What readers can do now

  • Clone appletea233/OneSearch-VL from GitHub and start from the SFT-110K / RL-10K datasets if you build multimodal agents.
  • Use it where answers must point back to a region of an image or a frame of video — compliance, medical triage, satellite analysis.
  • If you only need casual image Q&A, a plain vision-language model is simpler; the graph overhead pays off only when traceability matters.
  • Read the arXiv abstract (2610.12419) before citing the benchmark leads, and wait for community reproduction before treating 20+ points as settled.
  • Pin the Apache 2.0 weights rather than tracking the live repo, which has seen only a single commit so far.

Related coverage

More in “Foundation Models” → · Back to home · Markdown version