Gesture-Controlled 3D-Printed Robotic Arm
A 3D-printed robotic arm that mirrors your hand in real time — move, grab, release, naturally.
- Year
- 2025
- Reading
- 06 min
- Figures
- 27
A three-axis robotic arm, printed in PLA, that copies the operator’s hand through a webcam. MediaPipe tracks 21 landmarks on the hand, a Python script turns position and grip into servo angles, and an Arduino Uno drives the arm over serial. Built by a team of five as a Physical Computing final project.
- Team
- Ben Flora
- Bradli Powell
- Charlotte Chang
- Yiming “Danny” Huang
- Vedant Kapoor
- Disciplines
- Robotics
- Computer Vision
- Physical Computing
- Mechanical Design
- Embedded
- Tools
- Python / OpenCV / MediaPipe Hands / NumPy / pySerial / Arduino Uno / C++ (Arduino) / CAD / PLA 3D printing / Soldering
See, translate, act. A camera tracks the operator’s hand, software turns what it sees into smoothed motion commands, and the arm follows — its claw opening and closing like the hand in front of it.
It can point, rotate and reposition, pinch to grab and release to drop, handling objects up to 1.1 inches wide. Mechanical design, electronic control and software-based motion translation meet in one self-contained build.
From a hand to three servos
OpenCV captures 640×480 video at 30 frames per second. Each frame runs through MediaPipe’s hand-tracking model, which finds 21 landmarks — wrist, fingertips, joints — on a single hand, with a 70% confidence threshold to reject false detections.
Two signals come out of those landmarks. Position: the wrist’s horizontal coordinate sets the base angle and its vertical coordinate sets the elbow, inverted so that raising your hand raises the arm. Grip: extended fingers are counted by comparing each fingertip with its knuckle joint. A closed fist closes the claw; any extended finger keeps it open.
The angles are smoothed, written as a plain command — BASE:90,ELBOW:120,CLAW:OPEN — and sent to the Arduino over USB serial at 115200 baud. The firmware never jumps to a new angle: it steps each servo at most 5° toward its target every 20 ms, so the arm moves smoothly, like a human arm.
21
hand landmarks tracked in every frame
30 fps
webcam capture at 640×480
50 ms
between commands to the Arduino
5°
maximum servo step per 20 ms loop
1.1 in
widest object the claw can grip (28 mm)
Modelled in CAD, printed in PLA
Every structural part was modelled in CAD and 3D-printed in PLA. The claw has two fingers, a claw base, two hinge joints and a pair of interlocking gears — one driven by the servo, one following — so both fingers close together from a single motor.
A forearm and a central axis base carry the pitch servo, with a side cover over the horizontal motor. The main base holds up the arm and doubles as the electronics housing, with a removable top cap and a rear opening for the USB cable. Three servo screws and seven 3 mm flat-head screws hold it all together.
Sketch, print, assemble, test
The arm began as a ballpoint sketch in a notebook, then as rough test prints of gears, fingers and links laid out on a desk before anything was assembled.
Assembly happened at a makerspace bench: loose printed parts were secured with hot glue, the servo wires were soldered to a protoboard, and the code was tested against the arm as it came together.
Fig. 15Low-fi sketch
Fig. 16Prototyping
Fig. 17Assembling — securing loose printed parts with hot glue
Fig. 18Testing code
Engineering decisions
- 01
Two-stage motion smoothing
Python applies an exponential moving average to the base and elbow angles (0.7 × previous + 0.3 × new) to remove hand tremor and landmark noise, and sends at most one command every 50 ms. The Arduino treats each command as a target and slews toward it by at most 5° per 20 ms loop, separating the noisy vision rate from smooth servo motion.
- 02
Gestures without training
MediaPipe returns 21 normalised landmarks per frame. Wrist x and y map linearly to base yaw and elbow pitch (0–180°), and extended fingers are counted with simple geometric tests. A fist grips, anything else opens — no custom model or dataset needed.
- 03
A readable serial protocol
Newline-terminated text commands — BASE:<0–180>, ELBOW:<0–180>, CLAW:OPEN|HALF — that can be combined on one line, plus a STATUS query that reports the current servo positions. The arm can be driven and debugged from any serial monitor, independent of the vision stack.
- 04
A stall-safe gripper
The claw has only two targets, open and half-closed, with no fully closed position, so the servo never pushes hard enough against a gripped object to stall and draw excess current. Every target is clamped to the servo’s valid range before use.
- 05
One motor, two fingers
A servo-driven gear meshes with a follower gear, so both printed fingers, mounted on hinge links, close symmetrically from a single micro servo. The claw grips objects up to 1.1 in (28 mm) wide.
- 06
A self-contained enclosure
The printed base houses the Arduino Uno and a soldered protoboard, with one rear USB port for both power and serial data. All three servos share a common 5 V and ground rail, each on its own PWM pin (9, 10, 11), so every axis can be controlled independently.
Where it could go
Future work could add multi-finger gesture recognition for finer control, automated object identification, and tactile feedback for a more precise grip.
A team project by Ben Flora, Bradli Powell, Charlotte Chang, Yiming “Danny” Huang and Vedant Kapoor.
