AI Post Transformers podcast

Paper2Agent: Turning Research Papers into Interactive MCP Agents

0:00
NaN:NaN:NaN
Retroceder 15 segundos
Avanzar 15 segundos
This episode explores Paper2Agent, a Stanford system that converts a research paper and its released code into a Model Context Protocol server, so a chat agent can run the paper's actual methods through pre-tested tools instead of writing fresh code for every question. It explains MCP's three object types (tools, resources and prompts) and how the ReAct pattern lets the agent plan and call locked, paper-specific functions. This is meant to reduce hallucinated API calls, with AlphaGenome as the running example of a hard-to-use genomics API. The discussion places the work among earlier efforts such as executable papers, Papers with Code, AI co-scientists, and benchmarks like PaperBench and CORE-Bench. It stays skeptical of the authors' suggestion that "how easily a paper becomes an agent" could measure reproducibility, since the evidence is only three case studies and wrapping the authors' code also reproduces their bugs. Listeners get a clear, critical look at whether pre-built tool servers make research code trustworthy and usable by non-programmers. Sources: 1. Paper2Agent: Reimagining Research Papers As Interactive and Reliable AI Agents — Jiacheng Miao, Joe R. Davis, Yaohui Zhang, Jonathan K. Pritchard, James Zou, 2025 http://arxiv.org/abs/2509.06917 2. AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation — Qingyun Wu, Gagan Bansal, Jieyu Zhang, et al. (Microsoft Research), 2023 https://scholar.google.com/scholar?q=AutoGen%3A+Enabling+Next-Gen+LLM+Applications+via+Multi-Agent+Conversation 3. SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering — John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, Ofir Press, 2024 https://scholar.google.com/scholar?q=SWE-agent%3A+Agent-Computer+Interfaces+Enable+Automated+Software+Engineering 4. ReAct: Synergizing Reasoning and Acting in Language Models — Shunyu Yao, Jeffrey Zhao, Dian Yu, et al., 2022 https://scholar.google.com/scholar?q=ReAct%3A+Synergizing+Reasoning+and+Acting+in+Language+Models 5. Model Context Protocol (specification and announcement: 'Introducing the Model Context Protocol') — Anthropic (David Soria Parra, Justin Spahr-Summers and colleagues), 2024 https://scholar.google.com/scholar?q=Model+Context+Protocol+%28specification+and+announcement%3A+%27Introducing+the+Model+Context+Protocol%27%29 6. PaperBench: Evaluating AI's Ability to Replicate AI Research — Giulio Starace, Oliver Jaffe, Dane Sherburn, James Aung, Chan Jun Shern, Neil Chowdhury, et al. (OpenAI), 2025 https://scholar.google.com/scholar?q=PaperBench%3A+Evaluating+AI%27s+Ability+to+Replicate+AI+Research 7. CORE-Bench: Fostering the Credibility of Published Research Through a Computational Reproducibility Agent Benchmark — Zachary S. Siegel, Sayash Kapoor, Nitya Nagdir, Benedikt Stroebl, Arvind Narayanan, 2024 https://scholar.google.com/scholar?q=CORE-Bench%3A+Fostering+the+Credibility+of+Published+Research+Through+a+Computational+Reproducibility+Agent+Benchmark 8. Deep Reinforcement Learning that Matters — Peter Henderson, Riashat Islam, Philip Bachman, Joelle Pineau, Doina Precup, David Meger, 2018 https://scholar.google.com/scholar?q=Deep+Reinforcement+Learning+that+Matters 9. Hidden Technical Debt in Machine Learning Systems — D. Sculley, Gary Holt, Daniel Golovin, Eugene Davydov, Todd Phillips, et al. (Google), 2015 https://scholar.google.com/scholar?q=Hidden+Technical+Debt+in+Machine+Learning+Systems 10. SciReplicate-Bench: Benchmarking LLMs in Agent-driven Algorithmic Reproduction from Research Papers — Yanzheng Xiang et al., 2025 https://scholar.google.com/scholar?q=SciReplicate-Bench%3A+Benchmarking+LLMs+in+Agent-driven+Algorithmic+Reproduction+from+Research+Papers 11. Paper2Code: Automating Code Generation from Scientific Papers in Machine Learning — Minju Seo, Jinheon Baek, Seongyun Lee, Sung Ju Hwang, 2025 https://scholar.google.com/scholar?q=Paper2Code%3A+Automating+Code+Generation+from+Scientific+Papers+in+Machine+Learning 12. Biomni: A General-Purpose Biomedical AI Agent — Kexin Huang, Serena Zhang, Hanchen Wang, et al., 2025 https://scholar.google.com/scholar?q=Biomni%3A+A+General-Purpose+Biomedical+AI+Agent 13. Agentless: Demystifying LLM-based Software Engineering Agents — Chunqiu Steven Xia, Yinlin Deng, Soren Dunn, Lingming Zhang, 2024 https://scholar.google.com/scholar?q=Agentless%3A+Demystifying+LLM-based+Software+Engineering+Agents 14. Reflexion: Language Agents with Verbal Reinforcement Learning — Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, Shunyu Yao, 2023 https://scholar.google.com/scholar?q=Reflexion%3A+Language+Agents+with+Verbal+Reinforcement+Learning 15. Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directions — Xinyi Hou, Yanjie Zhao, Shenao Wang, Haoyu Wang, 2025 https://scholar.google.com/scholar?q=Model+Context+Protocol+%28MCP%29%3A+Landscape%2C+Security+Threats%2C+and+Future+Research+Directions 16. Towards an AI co-scientist — Juraj Gottweis, Wei-Hung Weng, et al. (Google), 2025 https://scholar.google.com/scholar?q=Towards+an+AI+co-scientist 17. A large-scale study on research code quality and execution — Ana Trisovic, Matthew K. Lau, Thomas Pasquier, Mercè Crosas, 2022 https://scholar.google.com/scholar?q=A+large-scale+study+on+research+code+quality+and+execution 18. Opportunities and challenges for transcriptome-wide association studies — Michael Wainberg et al., 2019 https://scholar.google.com/scholar?q=Opportunities+and+challenges+for+transcriptome-wide+association+studies Interactive Visualization: Paper2Agent: Turning Research Papers into Interactive MCP Agents

Otros episodios de "AI Post Transformers"