Using @mediapipe/tasks-vision (running PoseLandmarker with 33 3D body joints + HandLandmarker with 21 hand joints in Wasm/GPU at 60 FPS), I refused to simply map proximity to static noise. Instead, I engineered a full semantic classifier that:
1. Optical Inverse-Square Rangefinder: Estimates physical distance in meters using the Euclidean span between shoulder landmarks 11 and 12 (which immediately hallucinated whenever a child walked into the frame).
2. Harmonic Wrist Zero-Crossing Dance Detector: Analyzes a 1,800 ms rolling window of vertical wrist velocity zero-crossings (1.2 Hz–2.5 Hz) to classify whether the spectator is dancing rhythmically (RHYTHMIC) versus flailing erratically (HIGH_MOTION).
3. Torso HSV Clothing Color Sampler: Samples a 48×48 offscreen canvas across the user's torso, converts RGB to HSV, and classifies their outfit (WARM_RED, COOL_BLUE, DARK_NEUTRAL, NEUTRAL) to tint the YouTube search query mood.
4. Six Antenna Postures & The 1.8s Panic Patch: Mapped 6 specific body postures 1:1 to the 6 CRT screens—originally requiring visitors to hold a pose for 1.8 seconds (1,800 ms), which had to be panic-tuned down to zapHoldDurationMs: 400 because no human being in an art gallery will hold a T-pose for 1.8 seconds in front of strangers without feeling ridiculous.
CRT [01} :: POSE_ANTENNA"Rabbit Ears"
CRT [02] :: POSE_DIAL_TUNER"Yagi Point"
CRT [03] :: POSE_SURPRISE"TV Shock"
CRT [04] :: POSE_WINGSUIT"Horizontal Dipole"
CRT [05] :: POSE_LOOP_HALO"UHF Loop"
CRT [06] :: POSE_SIGNAL_LOCK"Human Capacitor"