INTERACTION FLOW · CUSTOMER × SYSTEM × VENDOR

A two-way conversation without relying on speech

Sound draws attention; visual prompts and gestures carry the order; both sides confirm before moving on.

01

Customer

Initiates and selects

02

System feedback

Detects, prompts and confirms

03

Vendor

Notices, observes and responds

STEP 1Initiate

Approach and make a sound

The customer approaches the stall and speaks or makes a short sound.

Detect voice

The microphone detects nearby speech without storing a recording.

Vibration alert → screen wakes

Receive attention cue

The vendor feels the haptic alert and turns toward the customer.

STEP 2Prompt

Read the visual prompt

The customer sees a simple instruction for the next gesture.

Show one choice at a time

The display presents clear pictograms and large selectable options.

Visual feedback → customer

Observe the interaction

The vendor can follow the same prompt and anticipate the response.

STEP 3Select

Choose with gestures

Select the soup base, spice level and quantity using prompted gestures.

Recognize and advance

The camera recognizes the gesture, records the choice and switches interface.

Selection appears on both sides

Support with gestures

The vendor can repeat or clarify a gesture when the customer hesitates.

STEP 4Confirm

Review the order

The customer checks the visible summary and confirms with a simple gesture.

Display shared confirmation

The screen shows the final order so both people see the same result.

Mismatch → return to selection

Nod or gesture to confirm

The vendor confirms the order and begins preparation.

Sound input → Haptic attention → Gesture selection → Shared confirmation