Building the interaction into a wearable prototype

A five-stage process connected material exploration, circuit assembly, hardware testing, scenario simulation and software integration.

Microphone moduleSound input
Vibration motorHaptic feedback
Flexible screenVisual prompts
Wearable apron prototype with flexible display and connected electronics
Integrated wearable prototype

From components to a connected circuit

The first two stages established the physical platform for testing.

Electronic components and flexible display prepared on the workbench
01Material preparationFlexible display, breadboards, sensors, controller and wiring
Arduino connected to a breadboard with jumper wires
02Circuit connectionRoute input, feedback and display modules through the controller

Test each interaction before integration

Gesture recognition, sound input and haptic output were checked independently.

Laptop running hand gesture recognition on the apron prototype
03AGesture detection testCamera → landmarks → interface response
Microphone module mounted near the apron strap
03BMicrophone testCapture nearby customer speech
Vibration module attached near the flexible display
03CVibration module testConfirm private haptic alerts
04

Move from bench testing to a real interaction scene

A simulated night-market exchange brought the camera, gesture recognition, wearable display and sound-triggered hardware into one continuous flow.

Observe Recognize Respond
Two people simulating a customer and vendor interaction around the prototype

Two software layers coordinate the prototype

Arduino controls the wearable hardware; Python handles camera-based gesture recognition.

Arduino IDE code controlling the OLED display
Arduino IDEOLED prompts · sensor input · haptic output
Python code for MediaPipe hand gesture recognition
PythonCamera input · MediaPipe landmarks · gesture mapping