I Control My Mac With Hand Gestures (No Mouse, No Touchscreen) | Learn with real Rust app

Apple just launched its first touchscreen MacBook Pro. You don't need one: in this beginner Rust project we build an app that lets you control your Mac with hand gestures through the built-in camera. Point to move the cursor, pinch to click, touch thumb to ring finger to scroll. You learn Rust from scratch as you build it.

Code (free, MIT): https://github.com/qualixar/touchless-mac

0:00 A Mac you control with your hand

1:02 Why Rust, and why now

3:00 Cargo and crates: the app sees the camera

5:05 struct and let mut: the cursor follows your finger

7:43 Option: the cursor waits when your hand leaves

9:46 enum and match: pinch, double pinch and waves

12:45 Result and ?: clear messages instead of crashes

15:11 Make it smooth

16:24 Run it on your Mac, and scroll

18:15 What we build next

No Rust needed to follow along. Every chapter has a git tag in the repo (ch1-cargo to ch6-calibrate), so you can check out the code exactly as it stood at the end of that chapter. The examples folder has one small program per idea that runs without a camera.

What's inside: Google's MediaPipe hand models (palm detection and 21 hand points) running through ONNX Runtime with the ort crate, nokhwa for the camera, enigo for the mouse, and a One Euro filter to keep the cursor steady. Measured on an M5 Pro: about 9 ms to find a hand and 1.5 ms to track it, inside the 33 ms between camera frames.

Credits: the hand models are Google MediaPipe's, in OpenCV's ONNX conversion (Apache 2.0). The Apple logo appears for editorial reference only. Not affiliated with Apple or Google.

I'm Varun Pratap Bhardwaj. Qualixar is my independent research initiative in AI Reliability Engineering: building things and measuring whether they actually do what they claim.

#macbookpro #rustlang #handtracking