Gemini Robotics ER 2 drives a Boston Dynamics Spot, watching video to confirm tasks like tying a trash bag are done before moving on.
Gemini Robotics ER 2 drives a Boston Dynamics Spot, watching video to confirm tasks like tying a trash bag are done before moving on.
Developers declare low-level controls, such as Boston Dynamics Spot's navigation and manipulator APIs, as tools. They stream video, audio or text into Gemini Robotics ER 2 through the Gemini Live API. The model sequences the steps, watches the video to track progress and confirm a task is finished, then moves on. The published Spot demo fetches objects, such as a popcorn snack, on a spoken command.
On progress classification, which sorts each video frame into five progress bands, it reached 57.4% accuracy. On moment-finding it reached 91.3% accuracy with a 0.96s mean absolute distance. The post gives no Spot success rate for tying a trash bag.
The Spot demo code is on GitHub, and the model is available as a preview through the Gemini API and Google AI Studio.
Write a Python program that breaks the task [multi-step task, e.g. tidy the workbench] into steps for a robot or simulator I have called [robot or simulator name]. After each step, have it check a camera frame or video clip from [source] to confirm the step is done before moving to the next. Show me the plan and code before running anything on real hardware. Stop when the program completes all steps in a simulated run.
Make pulls contracts from Gmail, Gemini extracts the term and notice period, and a Sheets register emails the owner 60, 30 and 7 days before each deadline.
A self-hosted n8n job reads your calendar, unread email and Notion tasks at 7am and sends one short brief of what matters today.
An n8n and Gemini agent receives patient chat messages, keeps context, uses clinic info from Google Sheets and responds to enquiries.