What's Next // Product Roadmap
Where We're Going.
EmpathAR is a live prototype built for the Creative Hackathon. This roadmap outlines what's shipped, what's in progress, and where the product goes next.
Just Shipped // Jun 2026 // Dog Mode v1.2
Dog Mode is Live.
Point the camera at a dog and a pose skeleton appears — bones, joints, energy state — the same way it does for people. This is the first release of Animal Mode, starting with dogs. We designed a custom 24-keypoint quadruped skeleton from scratch, collected and labelled training data, and fine-tuned a YOLOv8 pose model to learn it. The result is a two-stage pipeline: a lightweight detector finds the dog first, then the pose model runs only on that cropped region — mirroring how MediaPipe works under the hood.
Future updates for other animals — cats and beyond — will live in Thornberri, a standalone app we're building (work in progress). EmpathAR stays focused on people; Thornberri takes the animal side further.
How it works
- A custom 24-keypoint quadruped skeleton, designed and labelled for dog anatomy
- A YOLOv8 pose model fine-tuned on that skeleton — trained from scratch on dog-specific data
- A lightweight COCO detector (3.3 MB) scans the full frame first; the pose model (3.6 MB) runs only on the cropped region
- Both models run in a Web Worker so the UI never blocks
- Everything runs on-device — no frames leave your phone
Limitations & what's next
- Lying-down and top-down poses often miss — the training set is mostly standing dogs. A data gap, not a model failure. Fix: retrain on overhead and resting-pose examples
- Mobile performance not yet on par with human mode — CPU-only on iOS until we swap to the TFLite runtime (expected 2–3× speedup)
- Small or partially occluded dogs may not be detected
Phases
- Real-time pose + face landmarking via MediaPipe
- Social Battery score (posture, expression, movement)
- Five battery states with contextual tip feed
- Multi-person tracking with persistent labels
- Gesture detection: clapping, frowning, yawning, head prop
- On-device only — zero data transmitted
- Mobile layout polish and performance optimisation
- Improved lighting compensation for face landmarking
- Animal social battery detection for dogs
- Species-specific skeleton remapping for quadruped pose estimation
- Tail, ear, and posture signal interpretation
- Sentiment states calibrated for animal body language
- Accuracy improvements across breeds and coat colours
- Per-user baseline calibration for frown and smile sensitivity
- Tip relevance scoring to reduce repeated suggestions
- Session summary: battery trends over time per person
- Group dynamics mode: room-level energy heatmap
- Integrations: Zoom, Meet, Teams overlay via browser extension
- EmpathAR SDK for third-party applications
- Opt-in anonymised research dataset for social signal models
- Audio detection: live mic input layered alongside body language signals
- Voice-to-text transcription of speech detected during active sessions
- Context-aware tips personalised to what was said, not just how someone looked
- Cat support — feline body language signals and species-specific skeleton
- Additional species beyond dogs
- Future updates for other animals will likely live here as a standalone app — Thornberri is a work in progress