Skip to content
agentusehub
Sign in+ Submit
Research & KnowledgeAny

Try Devin on research tasks (mostly failed)

Answer.AI had Devin fetch an author's recent papers and research transcript summarization; it got stuck parsing HTML and returned generic results, rated a failure.

Read the experience
Time saved
Not reported
Setup time
Not reported
Running cost
Not reported
Cost range
Not reported

Use case overview

Answer.AI had Devin fetch an author's recent papers and research transcript summarization; it got stuck parsing HTML and returned generic results, rated a failure.

Read the original report ↗

Reported experience

  1. What they did

    Answer.AI gave Devin, which works through Slack in its own Docker environment with a browser, editor and shell, two tasks. One was grabbing an author's 25 most recent Google Scholar papers with Playwright, skipping paywalls. The other was researching transcript summarization with accurate timestamps. They then rated the results.

  2. How it went

    Both failed. On the scraping task Devin got stuck parsing HTML and went to sleep. On the research task it returned related but unhelpful material and code that missed the core problem.

  3. Worth knowing

    Across 20 tasks Devin had 14 failures and 3 successes. One research success, a Discord bot, worked because the goal was loose pseudocode-level guidance.

Example prompts

Try with Devin
Research [author or topic] by finding their [number] most recent papers and writing a one-paragraph summary of each, with links to the sources. Then give me a short overview of the main themes across them. If a page can't be read, tell me which one and why instead of guessing, and stop once you have a cited summary I can check.

Related use cases