The obvious architecture for AI features is the cloud: upload the image, compute on a big server, send the result back. We mostly do it differently. Our models run on the device that holds the camera: in DeepSwing's launch monitor and coach just as much as in the Pevu baby monitor. Here's why, what it takes and when we still go to the cloud.
Latency: physics beats bandwidth
A launch monitor has to measure the ball in the hundredths of a second after impact. Sending 240-frames-per-second video to the cloud, computing there and waiting for the answer takes longer than the whole shot. On an iPhone's Neural Engine, the model computes where the frames are created. The result is ready before the ball lands. How the measurement chain works in detail is in How an iPhone measures ball flight without radar.
Cost: no bill per request
Every cloud call costs money. For an app that analyses every shot, every swing or every second of video, the server bill grows with every user. On the device, the user has already paid for the computing power with their phone. For us that means the product scales without rising infrastructure costs.
Privacy: data that is never sent can't be lost
For Pevu, our baby monitor, this was the central decision. Video and sound travel directly from device to device, encrypted with DTLS and SRTP, without STUN or TURN servers. Noise and motion detection run on the baby unit. There is no server that could see anything and no account that could be hacked.
DeepSwing does the same for analysis: swing videos don't leave the phone for it. Only what someone deliberately shares in the community is uploaded. How the swing analysis works on the device is in How to analyse your golf swing with your phone.
Resilience: works in the basement
Golf ranges often have poor reception, simulators sit in basements, baby monitors run in hotels. On-device AI doesn't need a network for any of that. Pevu even works without internet: a router without internet access is enough, and when travelling one device creates its own network. And the course map in the golf course GPS is drawn from a course package on the phone, so it also works where there's no signal on the course.
What it takes
On-device is not a switch you flip. It takes models that are small enough and still accurate. Our approach:
- Pick the right architecture. Compact YOLO-family detectors for real-time detection, pose models for motion, segmentation for surfaces.
- Optimise for the target hardware. Core ML and the Neural Engine on Apple devices, LiteRT and the NPU on Android, quantisation where it costs no accuracy.
- Measure against a reference. Our launch monitor was compared with a radar unit, with carry within 0 to 2 percent. The course AI was measured on courses it had never seen.
- Test like infrastructure. Pevu has more than 900 automated tests and a manual plan covering twelve device combinations.
| On-device | Cloud | |
|---|---|---|
| Latency | milliseconds, no network needed | depends on connection and server |
| Cost per analysis | no running server costs | grows with every request |
| Privacy | data stays on the device | data leaves the device |
| Model size | has to fit on the device | almost unlimited |
| Updates | with the app update | instantly on the server |
So when is the cloud right?
When the data is huge and the result is small. Our course AI doesn't run on the phone: segmenting aerial imagery for tens of thousands of golf courses runs as a batch on our machines. All that reaches the device is the result, a course package of about 30 to 210 kilobytes. How that works is in Course AI: how we map golf courses from aerial imagery. Training belongs on big hardware anyway.
The rule we decide by: compute where the data is created, unless there is a very good reason not to. More on our edge AI and on-device AI page.
Frequently asked questions
What is on-device AI?
AI that computes directly on the device where the data is created, such as a smartphone with a camera. Images and sound don't have to be sent to a server for it.
Is on-device AI as accurate as cloud AI?
For well-defined tasks like object detection, pose estimation or segmentation, yes, if the models are optimised for the hardware and measured against a reference. Our launch monitor's carry is within 0 to 2 percent of a radar unit.
Which hardware do you use?
Core ML and the Neural Engine on iPhone, iPad and Apple Watch, LiteRT and the NPU on Android, plus edge boards with their own acceleration.
Does Pevu need internet?
No. A Wi-Fi router without internet access is enough, and when travelling one device creates its own network.
If you have a camera, a sensor or a device that needs to recognise something in real time, let's talk.