Gemini detects objects and writes robot API code, and with 10 in-context demos an ALOHA robot packed boxes and folded a dress; voice commands became robot plans.
Gemini detects objects and writes robot API code, and with 10 in-context demos an ALOHA robot packed boxes and folded a dress; voice commands became robot plans.
Gemini 2.5 was given a robot control API (detect object, get grasp pose, move/open/close gripper) and wrote Python plans zero-shot, such as putting a banana in a bowl. For ALOHA box packing and dress folding, 10 demonstrations of interleaved reasoning and actions were added to context. For voice, the same API calls were defined as Live API function-calling tools, with the camera feed streamed in.
Gemini produced two feasible banana plans, one of which re-planned around the right arm's reach limit. ALOHA packed boxes and folded a dress from the in-context demos. The post gives no success rates.
The ALOHA setup is in an open-source Colab with example demonstrations. The Live API voice console is on GitHub, and Gemini Robotics-ER access requires a trusted tester waitlist.
Help me build a simulated robot arm controller in [simulator or language]. Use a vision model to detect objects in [image or camera feed], then write code that calls my robot API [API description] to [task, e.g. pack items into a box]. Run it in simulation only, and show me the generated code before anything is run on real hardware. Stop when the task succeeds in simulation three times in a row.
Make pulls contracts from Gmail, Gemini extracts the term and notice period, and a Sheets register emails the owner 60, 30 and 7 days before each deadline.
A self-hosted n8n job reads your calendar, unread email and Notion tasks at 7am and sends one short brief of what matters today.
An n8n and Gemini agent receives patient chat messages, keeps context, uses clinic info from Google Sheets and responds to enquiries.