the live web collective sonic heart

This semester I’ve been working with Name in the creation of a web-based remote experience for the class The Body Everywhere and Here. We have been working on the top of a WebRTC-Simple-Peer-Example by Lisa Jamhoury to develop the idea of an installation that allows remote users to communicate.

The code allows for two users in different locations to interact on the same p5js canvas using Posenet via ml5. Each user controls this bubble-body minimal avatar with a digital heart positioned in the middle of their chest through body gestures. In the first part of our piece, the user has to answer the question “How is your heart today?” with a color that defines the color of their heart and avatar while inside the experience. Once the edge of the body of one of the users overlays with the other’s edge, their avatars merge into one with the color of both colors mixed.

Goals for Live Web’s midterm project

installation sketch for each of the two users
  • Add a sound layer to the code so when the avatars merge, the users can listen to each other’s microphone and communicate over peer connection;
  • Create a soundscape for the installation while the avatars are not merged;
  • Improve the interface/user experience for choosing the heart’s color and for restarting the experience;
  • Host the website on my ITP-Digital Ocean public server;
  • Experiment with video projection, user-test, and document it.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: