Affiliate links on Android Authority may earn us a commission. Learn more.
Gemini gets one step closer to controlling other apps on your Android phone
- Code from the latest Google app beta sheds light on Gemini’s upcoming agentic features on Android.
- Gemini could help you place orders online or hail rides without you needing to actively touch the phone.
- While these may be the initial applications, we could see more, as previewed by Google in its Project Astra demo last year.
- Google could refer to the functionality as “screen automation.”
AI agents are the hottest trend in tech, and the increasing momentum leaves no room for doubting that your mobile devices will soon be controlled by them. Back at I/O 2025, Google demonstrated how it plans to put Gemini in charge of controlling your phone for you. This was dubbed Project Astra and showed Gemini’s capabilities at not just viewing the text and media on your phone, but also scrolling and tapping when needed, basically displaying a spectrum of agentic force in action to help you with simple tasks. While Project Astra is still under development, we’re seeing evidence detailing some of its aspects.
Recently, we’ve discovered more cues that elaborate on how Gemini would be able to control our Android devices for us. In the Google app, version 17.4.66 beta, on Android, we’ve encountered new strings related to the functionality, starting with what it’s likely to be called.
<string name="assistant_robin_bonobo_fre_title">Get tasks done with Gemini</string>
<string name="assistant_robin_bonobo_fre_summary"><a href="https://support.google.com/gemini?p=tasks_in_apps">Gemini can help with tasks</a>, like placing orders or booking rides, using screen automation on certain apps on your device.</string>In addition to revealing the feature’s codename “bonobo,” these strings also indicate it could be called “screen automation” on Android devices.
Further, as per the description, screen automation can be deployed for placing orders online or booking rides (on apps such as Uber or Lyft). The embedded link currently directs to a generic Gemini support page, suggesting it’s a mere placeholder.
Don’t want to miss the best from Android Authority?
- Set us as a favorite source in Google Discover to never miss our latest exclusive reports, expert analysis, and much more.
- You can also set us as a preferred source in Google Search by clicking the button below.
Google has also added appropriate warnings to users when they first see the feature on their Android devices. The following code points to those:
<string name="assistant_robin_bonobo_fre_youre_in_control_full_text">Gemini can make mistakes. You’re responsible for what it does on your behalf, so supervise it closely. You can stop it, view its progress, and manually take control.</string>
<string name="assistant_robin_bonobo_fre_youre_in_control_partial_text">Gemini can make mistakes. You’re responsible for what it does on your behalf, so supervise it closely.</string>In these strings, Google advises users to supervise what the agent does on their phones. The description states that users must closely monitor what Gemini is doing on their phones, which seems counterproductive, especially since the assistance feature is designed to handle routine tasks with minimal human intervention or supervision. However, since “Gemini can make mistakes,” users will ultimately be held responsible for anything it does.
Google has also shared some notes on privacy, and here they are:
<string name="assistant_robin_bonobo_fre_keep_in_mind_full_text">When Gemini interacts with an app, screenshots are reviewed by trained reviewers and used to improve Google services if Keep Activity is on. Don’t enter login or payment information into Gemini chats. Avoid using screen automation for emergencies or tasks involving sensitive information. If screen automation is off, you can still use Gemini with Connected Apps.</string>
<string name="assistant_robin_bonobo_fre_keep_in_mind_partial_text">When Gemini interacts with an app, screenshots are reviewed by trained reviewers and used to improve Google services if Keep Activity is on. Don’t enter login or payment information into Gemini chats.</string>Here, Google notes that “trained reviewers” will see screenshots from your device “if Keep Activity is on.” That’s why you are also advised not to enter any sensitive information or payment details. It’s unclear how Google plans to handle sensitive information — whether users would be required to pause Gemini manually or if it will step away automatically.
Finally, since Gemini will help place orders, we might see them within the app on Android. That’s where a section, such as “Purchases” under Gemini’s My Stuff, could be useful. We’re also seeing a “My orders” section in the Google account switching interface, but it currently doesn’t lead anywhere.
While these strings lend us some clarity on the screen automation feature, we hope to get more insights when we can use it ourselves. We’re unsure which apps could be controlled using Gemini, though ride-hailing, food ordering, or e-commerce seem most likely, based on the description above. Meanwhile, Google is also testing features such as Characters, where you can create a 3D avatar of yourself, and the option to add context from Maps.
Thank you for being part of our community. Read our Comment Policy before posting.