The handheld for Pashupati patrol robots: drive them, watch through their camera, teach routes and run their AI inspection, from a Retroid Pocket 4 or any Android device in landscape.
Features · Install · Robot side · Controls · Develop
The app talks the UDP teleop protocol v1 straight to the robot's teleop_udp_node
(PROTOCOL.md),
and takes the video over WebRTC. There is no ROS on the handheld.
Drive
- Physical sticks only: left stick drives and strafes, right stick turns. The touch screen is left free for the map.
- AUTO / REMOTE switch, three speed levels, the robot's own actions (stand up, sit down, gaits, ...) as icons. Anything that makes the robot move on its own asks first.
- STOP is the software e-stop. It is released only by holding the button for a second.
See
- Two views, camera and map: one big and one small, swapped with a tap.
- Camera as H.264 over WebRTC, about 0.1 s behind live. When WebRTC is unavailable or fails, the app falls back to JPEG frames over the teleop link.
- AprilTags and detected people are boxed on the video as the robot sees them.
- The map shows the route with its waypoints and stop points, the robot's trail, the lidar and the robot. Pinch to zoom, drag to move, twist to rotate, or let the map turn with the robot.
Patrol
- Record a route, mark stop points, record AprilTags, localize, pick a route, then start, pause, resume or end the patrol, all from icons on the main screen.
- The robot can join a route from beside it, and a route loaded again starts over.
AI inspection (Inspection page)
- Turn the VLM inspection on or off, and choose the model running on the robot through Ollama.
- Three paces. Continuous inspects frames back to back. Interval takes a frame every 0.1 s to 5 min into a queue that is processed first in, first out. Stop points only inspects when the robot stops on its route.
- Own finding types: add a type with a description of what the AI should look for, and choose whether it is reported to NETRA or only shown on the handheld. Types can also be edited or removed.
- The last result is shown on the main screen, e.g. Clean · 3 s ago or Trash · 12 s ago. The queue length is shown there too. A haptic toast pops up on every finding.
- Person detection on or off, with the people count. Patrol speed is set with a slider.
Monitor
- Battery, link quality, protective field, health and the localization source (identity, tag or pose).
- The robot's CPU, GPU and RAM use and its temperature, at a glance, for development.
- The robot's own internet (a ping through its modem, every 5 s): Internet 38 ms, yellow
when pings get lost or slow, Internet offline in red. Needs Pashupati with
state.net. - Trip meter under the glance panel: distance and moving time, speed now, average and top speed
since the last reset. The robot counts (also with no handheld connected) and keeps it across
restarts; hold the panel to reset it to zero. Needs Pashupati with
odometry/trip.
Everything else
- Finds robots on the LAN by itself, and connects with a token.
- Indonesian and English.
From a release (no Flutter needed). Open
Releases, download
pashupati-controller-<version>.apk on the handheld and open it (allow installing from the
browser or file manager once), or from a computer:
adb install -r pashupati-controller-2.8.0.apk # -r: update in place, settings keptReleases are signed with one release key, so a newer release installs over an older one.
From CI. Every push to main also builds a release APK.
- Open Actions and pick the latest green run.
- Download the
pashupati-controller-apkartifact. - Unzip it and install the APK:
adb uninstall com.darmawanrobotics.flutter_teleop # only when switching from a locally built APK
adb install app-release.apkFrom source. You need Flutter 3.35 or newer, and the handheld on USB showing in adb devices.
git clone git@github.com:DarmawanRobotics/PashupatiController.git && cd PashupatiController
flutter pub get
flutter build apk --release
adb install -r build/app/outputs/flutter-apk/app-release.apk # or: flutter run --releaseAPKs built locally are signed with your debug key, releases and CI ones with the release key. Android only updates an app in place with the same key, so when you switch between the two, uninstall first. Uninstalling clears the app's settings.
First connection. On the start screen, tap the version three times to open the connection
settings. Enter the robot address (default 192.168.234.234:9870) and its token. Robots on the
same network also show up in the list by themselves.
The token and the action icons come from the robot's /etc/pashupati/site.yaml:
teleop_udp_node:
ros__parameters:
token: "<secret>" # the same in the app
robot_commands: [stand_up, sit_down, balance_stand_mode, move_mode, slow_speed, normal_speed,
fast_speed, emergency_stop]What the app can do depends on what the robot runs, set in /etc/pashupati/pashupati.env:
| Feature | On the robot | In the app |
|---|---|---|
| WebRTC video | WEBRTC=true, script/setup.sh install webrtc (MediaMTX) |
smooth video; JPEG without it |
| AI inspection | INSPECTION=true, script/setup.sh install ollama |
Inspection page, results on the main screen |
| Person detection | PEOPLE=true, script/setup.sh install people_model |
people count and boxes on the video |
| Fleet | CC_URL, CC_TOKEN, ROBOT_ID |
findings and telemetry reach NETRA |
A robot running an older stack simply offers fewer commands, and the app hides or explains the features it lacks.
| Gamepad | Touch | |
|---|---|---|
| Drive and strafe | left stick | |
| Turn | right stick | |
| STOP (software e-stop) | B, on every page | red button |
| AUTO / REMOTE | Start | switch on top |
| Swap camera and map | Select | tap the small view |
| Speed level (30 / 60 / 100 % of the robot's limits) | D-pad up / down | Settings, Speed |
| Robot actions | L1 + A stand up, L1 + Y sit down | icons under STOP |
| Map | pinch, drag, twist; double tap follows the robot |
The robot only takes the joystick in REMOTE; in AUTO, navigation drives. After a STOP, nothing but zero reaches the robot until Hold to release is held for a second.
Teach and patrol.
- Switch to REMOTE and tap Record.
- Drive the route. Tap Point wherever the robot should stop and inspect.
- Tap Finish.
- Open Routes, pick the route, switch to AUTO and tap Start.
Settings (gear) apply at once: speed level, R1 to drive, video, lidar on the map, robot load on screen, language, and how long the robot inspects at a stop point. A gamepad test is there too.
lib/
├── protocol/ messages.dart (JSON of the teleop protocol), streams.dart (camera / lidar / route)
├── net/ teleop_client.dart (session, commands, joystick), discovery.dart,
│ webrtc_video.dart + whep.dart (camera over WebRTC)
├── input/ gamepad_input.dart (gamepads, normalized)
├── widgets/ hud, camera (tag and person boxes), map, glance (AI, people, load), trip
├── screens/ connect, drive (home), patrol (inspection), routes, settings
├── robot_actions.dart icons and labels of known robot commands
└── l10n.dart Indonesian and English text
flutter analyze
flutter test # protocol, widgets and pages at Retroid Pocket sizeCI runs on every push. It analyzes the code, runs the tests and turns failures into annotations,
reports formatting, and builds the release APK. When a dependency changes, commit the
pubspec.lock that flutter pub get writes.
Bump version in pubspec.yaml (the name and the + build number), merge it to main, then tag
that commit with the same version:
git tag v2.8.0 && git push origin v2.8.0The release workflow tests, builds and publishes pashupati-controller-2.8.0.apk (and its
SHA-256) as the GitHub Release v2.8.0. A tag that differs from pubspec.yaml fails the run.
The release key lives in four repository secrets (Settings → Secrets and variables → Actions):
ANDROID_KEYSTORE_BASE64 (the keystore, base64 -w0 release.jks), ANDROID_KEYSTORE_PASSWORD,
ANDROID_KEY_ALIAS and ANDROID_KEY_PASSWORD. Without them the APK is signed with a debug key
that changes every run. Keep the keystore file and its passwords outside the repository: losing
them means every handheld has to uninstall the app once to take a new key.
Barlow Semi Condensed is under the SIL Open Font License (assets/fonts/OFL.txt). The app is
released under the Apache License 2.0.