WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agent

Today's paper introduces WebWatcher, a multimodal AI agent designed for deep research tasks that handles both visual and textual understanding. While existing web agents excel at text-based research, they struggle with real-world scenarios that involve visual information like scientific diagrams, charts, or visually rich web interfaces.

Method Overview

WebWatcher operates through a comprehensive pipeline that combines advanced multimodal reasoning with sophisticated tool integration. The overall approach involves training the agent on high-quality synthetic data that requires deep reasoning across both visual and textual modalities, then equipping it with multiple tools for web interaction and information gathering.

The data preparation process begins by generating challenging question-answer pairs through a novel QA-to-VQA conversion pipeline. The method first creates complex textual questions by performing random walks across authoritative web sources like Wikipedia, arXiv, and GitHub, then uses language models to synthesize multi-hop reasoning questions from the collected content. To increase difficulty, the approach creates two levels of questions: Level 1 questions require multi-step reasoning with explicit entities, while Level 2 questions intentionally obscure key entities and replace specific terms with vague descriptions, forcing the agent to infer relationships from context.

The conversion from text-based QA to visual QA involves grounding these complex questions in relevant visual content. For each question, the method retrieves authentic web images related to the target entities and constructs multimodal examples that require both visual understanding and external information gathering. This ensures that perception alone is insufficient - agents must use external tools to gather and integrate evidence.

For tool integration, WebWatcher incorporates multiple capabilities including web image search, web text search, webpage browsing, code interpretation, and optical character recognition (OCR). The method addresses the challenge of coordinating these diverse tools by developing an automated pipeline that constructs high-quality reasoning trajectories from actual tool-use behavior, rather than relying on templated responses. The agent is then fine-tuned using these synthesized trajectories and further optimized through reinforcement learning.

Results

WebWatcher demonstrates significant performance improvements across four challenging benchmarks. On the newly introduced BrowseComp-VL benchmark, WebWatcher achieves 27.0% accuracy compared to 13.4% for GPT-4o, 13.0% for Gemini 2.5-flash, and 11.5% for Qwen2.5-VL-72B. The agent also outperforms existing systems on Humanity's Last Exam VL (13.6% vs 9.8% for GPT-4o), LiveVQA (58.7% vs 41.3% for Gemini 2.5-flash), and MMSearch (55.3% vs 43.9% for Gemini 2.5-flash). These results demonstrate WebWatcher's ability to handle complex multimodal reasoning tasks that require both visual understanding and sophisticated information gathering capabilities.

Conclusion

The paper presents WebWatcher as a significant advancement in multimodal AI agents for deep research tasks. By combining high-quality synthetic training data, sophisticated tool integration, and advanced reasoning capabilities, WebWatcher successfully bridges the gap between text-based and vision-language research agents. For more information please consult the full paper.

Congrats to the authors for their work!

Geng, Xinyu, et al. "WebWatcher: Breaking New Frontiers of Vision-Language Deep Research Agent." arXiv preprint arXiv:2508.05748, 2025.