Echoes of You

By Mahnoor, Ziheng, Harry

Project Description

In the digital world body language can't be used as we communicate through social media, texting, video games and more. Instead we use gifs or emojis in place of body language to convey the same meaning. However body language is a key part of how we communicate with each other especially when a language barrier is involved. Therefore we wanted to highlight and bring to attention the importance of body language through this puppet. The puppet emphasizes body language as a critical mode of expression, by echoing the participants movements. By incorporating physical gestures in the puppet, we bring attention to the nuanced ways body language fosters connection, emphasizing its power to bridge linguistic divides.

The puppet's face and body are composed primarily of geometric shapes, distinguishing it from traditional puppets, which often feature more detailed facial expressions and body structures. The choice of simplified geometry gives the puppet a clean, futuristic appearance that aligns with our vision. Our work nods to the storytelling tradition in conveying emotion and narrative through simple, silent figures. Beyond its technological functions, the puppet also serves as a companion, quietly reflecting the movements and emotions of the person it mirrors.

The design of the puppet was created by using the laser cutter (cutting, engraving & etching) onto cardboard. We also used a 3D printer to create extra attachments to help secure the servo's securely to the cardboard. The reason we chose cardboard was because it isn't a material that evokes or reminds us of mirrors hence we don't expect to see our movements echoed on it. Additionally we were also intentional about the placement and hiding the components to ensure the main focus was the puppet so that people were not distracted and could feel immersed into the movements.

To create the movement of the puppet we used the body pose code to make the puppet mimic movements. More specifically we used the points elbow, head, waist, shoulder, nose and left eye. Essentially, Echoes of You analyzes the gesture of the participant from a live webcam with p5.js's ml5 library, then controls the puppet to do the same pose using arduino and servos.

System Diagram

The system works by getting participant body pose data from webcam, process it through p5.js's ml5 library, which allows it to calculate rotations for different joints such as head, shoulders and elbows. Then the rotational data is sent to the arduino, which controls 5 servos, each for head, shoulders and elbows. It will rotate each servo to designated angle, which will echo the participant's pose on the puppet.

Circuit Diagram & Description

This circuit uses 5 servos, 1 arduino IoT 33 Nano, 1 USB-C power module. The USB-C power module provides 5V power for all 5 servos, the arduino controls the servos with analog pins 2,3,4,5,6, which controls head, right shoulder, right elbow, left shoulder and left elbow movement.

Code

p5.js Editor: https://editor.p5js.org/zhao840839843/sketches/43d39_Vo-

Photos