Skip to content

About

A talking phone assistant for Android: Claude or your own model, with screen control that asks before anything risky

Topics

Resources

Contributing

Security policy

Stars

0 stars

Watchers

1 watching

Forks

Repository files navigation

Sprocket

Beta testers wanted

Sprocket is used every day on the phone it was built on, but it's one person's project and hasn't been tried on many other phones yet. Expect rough edges. If you have an Android phone (8 or newer; 12 or newer for background work) and want to help:

  1. Install it (see "Install") and use it for a few days for whatever you'd ask a phone assistant.
  2. When something breaks, open an issue with the bug form: your phone's make and model, its Android version, the brain you use, what you asked, and what happened.

Reports from phones that aren't Samsung help most. There's nothing to sign up for: just install it.

A talking phone assistant for Android. You talk or type; Sprocket answers out loud and does things on your phone. It starts out friendly and plain, and its personality is yours to write in Settings > Personality, which comes with an example to start from: a loud little robot who is sure he was built for war and now sets your timers.

See it

The demo (1 minute 24 seconds, sound on):

sprocket-demo.mp4
Setup, one feature at a time Answers out loud Texts wait for your yes Lists, sorted by aisle
The setup walkthrough asking about texts and calls Sprocket answering a weather question A Send card for a text, with Send and Cancel A grocery list grouped by aisle

Why I built it

I wanted a phone assistant that didn't send everything I say to a company's servers. Sprocket started out on Claude, and I moved it to a model running on my own computer at home, so my questions, texts and notes stay private. You can run it the same way, or use Claude if you'd rather. On your own model, nothing leaves your own machines except what goes to the services you choose to connect (web search, the weather, Notion, GitHub).

Status: a personal project, made public as-is. Tested on one Samsung Android phone, with Claude and with Qwen through the included go-between. Other models may work (see "Brains"). Not on the Play Store: download it from this page instead (see "Install").

What it does

  • Everyday phone jobs: timers, alarms, reminders, calendar, contacts, music, maps, the flashlight, volume, brightness, Do Not Disturb, the camera and photos.
  • Texts and calls: reads your texts to you, writes replies, places calls. Nothing is sent or called until you tap (or say) yes on a card that shows exactly what will happen.
  • Screen control: for anything else, Sprocket reads the screen and taps through other apps like a person. Buttons like pay, buy, delete, install or sign out wait for your yes; banking, wallet and password apps are off limits. After a task that worked, it remembers the way (say, Gmail's spam folder) and next time goes there by itself in a few seconds, stopping and handing back if the app looks different. The saved ways stay on your phone, under ⋯ > Memory to delete. (Ideas from Mika's per-app knowledge files and the TAG-Nav paper.) A soft orange glow around the edge of the screen shows while Sprocket is in control.
  • Work in the background (Android 12 or newer): Sprocket can do its tapping on a hidden screen while you keep using your phone, then put the result on your screen when it's done. See "Work in the background" below.
  • Food orders: say what you want and it fills the cart in your delivery app or a restaurant's own app (DoorDash, McDonald's, Taco Bell and others), checks every item, then stops on the cart for your tap. It never places the order. Give it a budget ("I've got $20 for lunch") and it counts delivery, fees, tax and tip, and suggests what fits.
  • Packages: "where's my package?" looks in Shop, then Amazon, Walmart or Target, then your shipping emails, and opens the tracking link to tell you where it is and when it's due.
  • Call announcer: with the phone on vibrate, it says who is calling and listens for "answer", "decline" or "text". Works over the lock screen. You set the hours.
  • Lists: groceries sorted by aisle, to-dos with due dates, lists from a photo of a recipe, and (optionally) kept in Notion so you can edit them on a computer.
  • Morning brief at 6 am: weather, calendar, what's due, and news on topics you pick.
  • Memory: tell it to remember something and it keeps a note on your phone (up to 2,000). Notes go along with your question when its words fit; "always remember" notes go with every message. Search, pin or delete them under ⋯ > Memory.
  • Also: web search, research write-ups, GitHub threads and repositories (read only).

Every feature can be switched off in Settings.

Brains

Pick one on Sprocket's first screen; you can set up the others later in Settings.

  • Claude: paste your own Claude API key. To get one: sign in at console.anthropic.com, add a little credit under Billing (and a monthly limit under Limits, if you like), then API keys > Create key. A Claude.ai subscription doesn't include this: the API is paid separately, for what you use. Sprocket's first screen has these steps and a button to the Console.
  • DeepSeek: paste your own DeepSeek API key, nothing else to set up. To get one: sign in at platform.deepseek.com, add a little credit, then create a key under API keys. Web search works through DeepSeek's own search. What you ask goes to DeepSeek, a company in China.
  • Your own model: point Settings at any server that speaks the Messages API. The included go-between (a small Python server) translates for any OpenAI-compatible server (llama.cpp, vLLM, Ollama and others) and adds web search through your own Brave Search key. Tested with Qwen.
  • More than one, as backups: set up any of the others in Settings. If the one you picked doesn't answer (your server is off, or a service is overloaded), Sprocket asks on a card whether the next one may answer that question: your own server first, then DeepSeek, then Claude. Nothing switches without your yes, and a question it had already started acting on is never handed over.

What it costs

Claude and DeepSeek charge for what you use; your own model costs nothing per question. Sprocket shows the cost under each answer. Some real examples with DeepSeek (November 2026 prices; half price outside China's working hours and on weekends):

  • A simple question ("what is 5 plus 5"): about $0.0002, up to $0.002 for the first question of a new chat.
  • A food order, filling the cart in a delivery app (about a dozen steps): about $0.004.
  • Sprocket's whole practice set of 62 phone tasks (food orders, settings, texts, packages, searches): $0.12.

The same practice set cost $0.87 on OpenAI's comparable model (not built into Sprocket yet). Prices change: check each provider's own pricing page.

Voices (optional)

  • Speaking: the phone's own voice by default. Or run a voice server that takes POST /speak {"text", "mood"} and returns audio; gateway/voice_server.py is the one used here, shared as an example to adapt (it needs IndexTTS and a voice recording of your own: see gateway/README.md, which also has IndexTTS's licence).
  • Listening: the phone's own recognizer by default. Or any OpenAI-style /v1/audio/transcriptions server (whisper.cpp's server works).

Install

  1. On your phone, open the latest release and download sprocket-<version>.apk (Android 8 or newer).
  2. Open the downloaded file and tap Install. Android asks once to let your browser install apps: allow it. Google Play Protect may warn about an app from outside the Play Store; that's expected here.
  3. Open Sprocket: pick its brain on the first screen, and it walks you through the rest in the chat, asking about each feature (you can ask it questions along the way). Using your own model? See "Running it on your own model" below.

Updates install the same way, over the top, keeping your chat and lists. A copy you build yourself is signed with your own key instead, so it can't update a downloaded one (or the other way round) without uninstalling first.

Build it yourself

You need the Android SDK (platform 35, build-tools 35.0.0) and a JDK 17 or newer. No Gradle.

ANDROID_HOME=/path/to/sdk android/build.sh      # makes android/build/sprocket.apk (and a signing key, once)
adb install android/build/sprocket.apk
scripts/test.sh                                  # the plain-Java tests, on your computer

Then open Sprocket as in step 3 above.

Using a coding agent? Claude Code, Codex or Gemini CLI can do all of this for you: open this folder in it and say "set up Sprocket on my phone". The steps it follows are in AGENTS.md.

Make it your phone's assistant

Then holding the side button (or the home button) opens Sprocket over any app, even on the lock screen. Setup asks about this last; to do it yourself:

  • Most phones: Settings > Apps > Default apps > Digital assistant app > Sprocket.
  • Samsung: Settings > Apps > Choose default apps > Digital assistant app > Sprocket. The side button still opens Bixby until you also change Settings > Advanced features > Side button > Press and hold > Digital assistant.

Optional: let Sprocket switch screen control back on by itself

Android sometimes switches an app's screen control off on its own, and turning it back on means digging through Settings (and can hit a "restricted setting" block). If you have a computer with adb and USB or wireless debugging set up, one command lets Sprocket switch its own screen control back on, both when you ask and when Android turned it off (never after you switched it off yourself):

adb shell pm grant dev.sprocket.app android.permission.WRITE_SECURE_SETTINGS

Sprocket uses that permission for nothing else, but it is a powerful one: an app holding it can change hidden system settings. To take it back: adb shell pm revoke dev.sprocket.app android.permission.WRITE_SECURE_SETTINGS, or uninstall Sprocket. Without it, asking Sprocket to turn screen control on opens the Accessibility page for you.

Work in the background (optional)

On Android 12 or newer, Sprocket can work on a hidden second screen, so you can keep using your phone while it fills a cart or changes a setting. To set it up:

  1. Turn on Developer options, once: Settings > About phone (on Samsung, then Software information) > tap Build number seven times.
  2. In Sprocket, tap ⋯ > Background work (hidden screen). Sprocket opens Developer options > Wireless debugging > "Pair device with pairing code" for you (the orange glow shows while it works), reads the code through screen control, and pairs with your phone's own wireless debugging. If Android asks to allow wireless debugging on this network, tap Allow.
  3. That's it. From now on apps open on the hidden screen. When you ask to see something, or an order needs your tap, Sprocket moves it onto your screen.

Good to know:

  • It needs Wi-Fi, because wireless debugging only runs on Wi-Fi. Away from Wi-Fi, Sprocket simply uses screen control on your own screen, as before, and the hidden screen comes back by itself when you're on Wi-Fi again.
  • After the phone restarts, Sprocket starts it again the next time you open the app.
  • Pairing gives Sprocket the same access a computer with adb would have. It uses that only to start its hidden screen and to give itself the permission described above under "let Sprocket switch screen control back on" (so it can turn wireless debugging back on after a restart).
  • To undo it: tap the menu item again to switch it off, and under Wireless debugging > Paired devices, forget sprocket@ followed by your phone's model.
  • Tested on a Samsung phone and an Android 14 tablet, including with the screen locked. Other phones may differ: reports help.

Updating

  • Hearing about new versions: Sprocket doesn't check for updates itself. To be told, sign in to GitHub and, at the top of this page, click Watch > Custom, tick Releases, then Apply. GitHub then emails you (or shows a notification) when a new version comes out. The changes in each version are listed on the Releases page.
  • The app: download the new sprocket-<version>.apk from the latest release and install it over the top, as the first time. Your chat, lists, notes and settings stay, and so does screen control.
  • The go-between (only if you run your own model): download the new ZIP and copy the files from its go-between folder over your old go-between folder, replacing them. Don't unzip into a fresh folder: the old folder holds your access code (the file token), and a fresh one makes a new code, which your phone won't know. Then stop the go-between (Ctrl+C in its window, or systemctl --user restart go-between if you set it up as a service) and start it again.

Running it on your own model

  1. Start a model server on your computer. New to this? Install Ollama and ollama pull a model that supports tools, as big as your graphics card holds.
  2. Download this repository (on GitHub: Code > Download ZIP, then unzip it) and run python3 go-between/start.py (Python 3, nothing else to install). It finds your model server, makes an access code, and prints exactly what to type into Sprocket.
  3. In Sprocket, tap Your own server, type what it printed, and tap Start. Sprocket tests the connection first and says plainly what's wrong if it can't get through.

The phone has to reach that computer: start.py prints the Tailscale command if you have Tailscale; otherwise see below.

Reaching your own server

Only needed for your own model (or the optional voice servers): with Claude, the phone talks to Anthropic directly. Your phone has to be able to reach the computer running the go-between. Three ways, easiest first:

1. Tailscale (recommended: works anywhere, encrypted, free for personal use)

  1. Install Tailscale on the computer (tailscale.com/download) and the Tailscale app on your phone (Google Play). Sign in to the same account on both.
  2. On the computer, give the go-between a secure address on your Tailscale network: tailscale serve --bg --https=8448 http://127.0.0.1:8048 (the first time, Tailscale may ask you to turn on HTTPS certificates in its admin page: say yes).
  3. In Sprocket, the address is https://<computer-name>.<your-tailnet>.ts.net:8448. Both names are in the Tailscale app, under the computer.

2. A tunnel (an https address on the internet, no Tailscale app on the phone) Cloudflare's free cloudflared is one: install it on the computer, then cloudflared tunnel --url http://localhost:8048 prints an https://....trycloudflare.com address to use in Sprocket. That quick address changes every time it restarts; a permanent one needs a free Cloudflare account and a domain of your own (see Cloudflare's "Tunnel" guide). A tunnel puts the go-between on the open internet, so its access code is the only lock: make it long and random.

3. Your home network (simplest, but only at home) Start the go-between with python3 go-between/start.py --home, which prints the computer's home address (for example http://192.168.1.50:8048) to use in Sprocket. It stops working when the phone leaves your Wi-Fi, and it isn't encrypted: anyone else on that network could read what you ask, and the access code.

Sprocket accepts https:// addresses anywhere, and plain http:// only for Tailscale and home-network addresses.

Questions, ideas and the roadmap

It's a personal project, so answers may take a while.

Privacy

  • Your keys stay on your phone. Your chat, memory and lists stay on your phone (and in your own Notion, if you connect it).
  • What you ask goes to the brain you chose: Anthropic for Claude, DeepSeek for DeepSeek, or your own server.
  • Background work talks only to your own phone's wireless debugging (on the phone itself); its pairing code is read from the screen and never saved.
  • Weather comes from Open-Meteo, GitHub from GitHub's public API, lists from your Notion when connected. Web search goes through Anthropic's search (Claude) or the go-between's Brave Search (your own model).
  • Nothing is collected by anyone else. There are no analytics.

Be careful

Screen control and auto-answering calls are powerful. Sprocket asks before anything that sends, calls, buys or deletes, but it is an AI and can misread a screen. Keep an eye on it the first few times, and switch off what you don't want.

Credits

  • Weather data by Open-Meteo.com (CC BY 4.0); place names from GeoNames.
  • Background work's wireless-debugging pairing client is ported from Mika (Apache 2.0), as is the idea of a hidden screen; see android/src/dev/sprocket/app/adb/NOTICE.md.
  • Sprocket is not affiliated with Anthropic, Notion, GitHub, Brave or any model maker.
  • Written with Claude (Anthropic's AI) as the programmer, directed and tested by its owner.

Licence

MIT (see LICENSE), except the code ported from Mika in android/src/dev/sprocket/app/adb/, which keeps its Apache 2.0 licence (see the NOTICE.md and LICENSE-Mika.txt there). The optional voice server in gateway/ uses IndexTTS, which has its own licence: see gateway/README.md.

About

A talking phone assistant for Android: Claude or your own model, with screen control that asks before anything risky

Topics

Resources

Contributing

Security policy

Stars

0 stars

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages