ICM Final

The first sound of the Fire Bird’s Eye

For my final assignment for Introduction to Computational Media, I’ve added a sound layer to my gloves designed for my Introduction to Physical Computing Classes. The gloves are being thought for a performing arts environment, where I’m going to embody this character called Fire Bird to tell a story. They have 2 switches that are activated when touching the index fingers or the thumbs of both hands, producing different LED effects that change according to different combinations of the switches.

Design of the Gloves made with reflective fabric

For the ICM part of it, I’m connecting my Arduino using p5.serialcontrol to create the serial communication with p5.js online editor. The built-in IMU (Inertial Movement Unit composed by an accelerometer and a gyroscope sensor) of the Arduino IOT 33 is collecting three different movement data from the gloves: roll, pitch and heading (see image below to visualize the difference between them), which are the spinning movements that a 3D body can make on the space. And sending them to my p5 sketch with a range of 0 to 360.

Data collected by the accelerometer/gyroscope of the Arduino. Yaw can also be called Heading.

This range is being mapped to change the frequency and the amplitude of the carrier wave in a Frequency Modulation system in its simplest form and generate different sounds in real-time. It’s also moving a 3D object that I designed in p5.js. Initially, it was supposed to look like an eye, but now I feel that it became a spaceship.

For the sound, we basically have two oscillators. One is the carrier, and the other is the modulator. The carrier has a Base Frequency that is modified by the modulator, which operates in a range of amplitude and frequency chosen by me.

Find my code here. To create it, I basically adapted and combined two codes. The first one, used to create serial communication with the Arduino IMU was created by Tom Igoe and can be found here. The second is a sound frequency modulation example found in the p5js examples library. I really feel like studying deeper Frequency Modulation and also understanding better how Matrixes work in coding.

Still of my sketch/animation

The Fire Bird

Body experimentation wearing the interactive gloves. Video by Tundi Szász.

The Fire Bird is a first experiment on building characters exploring interactive wearable technologies. This is a work in progress. I’m imagening here a performing arts environment where the performer will be able to control different elements of the space making different gestures.

To compose the Fire Bird so far, I’ve created gloves that generate different light, sound and animation effects according to the hand gestures and movements of the performer that embodies the Fire Bird to tell their story. Basically, while wearing the gloves, the performer can change the direction of a 3D object on the screen and the frequency and the amplitude of its sound. When he/she make gestures related to the narrative of the Fire Bird (Fire, Bird or Tree), different buttons made with conductive fabric sewed to the gloves are activated changing the color of the LED’s of the gloves and the color and size of the seed (3D object) on the screen. The seed is inside the belly of the Fire Bird, so when someone wear the gloves, he/she is embodying the carachter and flying or dreaming carrying the seed.

The LED effects of the glove change according to gestures

Process of Creation

The work was made during Fall 2019 at ITP, manly in three classes. For more details, click on each part of the following description. In Intro to Fabrication, I fabricated the gloves and the acrylic mask of the Fire Bird. For my Introduction to Computational Media final, I created the p5.js code with the 3D object and the frequency modulation sound system coordinated to the movements of the hands. This is made possible after a serial communication with the Inertial Motion Unit built-in the Arduino IOT of the gloves, which code and circuit were developed during the Introduction to Physical Computing classes.

If you would like to know more about the project, please contact me: fernando.gregorio@nyu.edu

The story of the Fire Bird

The story of the Fire Bird is based on three cells of movements. The idea is to develop it further coreographycally, text-based, and possibly in the format of a game.

Movement 1

The Fire Bird flies away from home because their house was put on fire by someone who couldn’t understand the way the Fire Bird was living.

Movement 2

The Fire Bird flies to the city, alone, and finds a place to sleep.

Movement 3

The Fire Bird dreams for so long, that the invisible seed that was inside of their belly grows and turns into a tree.

Questions to keep moving

What’s the seed that is inside of the belly of the Fire Bird made of?

How does the tree that grows from the belly of the Fire Bird look like?

fabrication – week 6

The fire flower eye motor

For the mounting motor’s assignment, I created a simple sculpture inspired by the character I’ve been developing this semester: The Fire Bird. The initial idea was to work with handshapes that spin creating a fire illusional effect. But it turns out that the DC motor that I used stops very easily when in contact with extra weight, so I simplified the idea and used an acrylic eye that I had instead. It still creates a fire-flower effect when spins, which turned out as an interesting aesthetic experiment for me.

Initial Idea

The hardest part was positioning the hole of the enclosure in a way that it forms a perfect angle with the motor, and does not stop because of the friction and spins without shaking the box.

Movement test

Materials

  • Zip ties;
  • 3mm Shaft / Axle;
  • Black ABS Project box;
  • Euro terminal Strip pin;
  • 9V Battery;
  • Hobby DC motor;
  • Switch;
  • Wires.

Final Result

Final Result

fabrication – week 5

two materials assignment

This week we were tasked to work with two different materials. I’ve been venturing myself in the wearable world, so I took the opportunity to learn how to use the sewing machine and understand how to connect wires with conductive thread or fabrics. The goal was to create gloves with LEDs that change their color effects depending on the gesture you make when wearing them.

Materials:

  • Conductive Fabric;
  • Conductive Thread;
  • Perf board;
  • Silicon wires;
  • Conductive tape;
  • Thread;
  • Reflective Fabric;
  • Stretch Grey Fabric;
  • Neopixels;

Pcomp Final Project

The fire bird light and sound glove

Glove’s palm diagram
Back of the hand diagram and details for the finger’s buttons

storytelling bases for coreography

Movement 1: The fire bird flies to the city.

Movement 2: The fire bird finds a place to die and dies.

Movement 3: The invisible seed that was inside of the belly of the fire bird geminates.

Movement 4: Zoom in the seed and enter it. What is the seed made of?

Movement 5: The seed turns into a tree. How does this tree look like?

Tests/Visuals

Materials

  • Arduino Nano 33 IOT;
  • Portable Battery Pack;
  • Battery for Arduino (Lithium?!);
  • Micro USB – USB Cable;
  • Portable speakers with Bluetooth connection;
  • Gloves;
    • Conductive Fabric;
    • Conductive Thread;
    • Reflective Fabric;
  • Silicon wires;
  • Computer with p5.js;

Next Steps

  • Finish Code;
    • Create each LED effect;
      • Initial;
      • Fire;
      • Bird;
      • Tree;
    • Create sound effects in P5js to produce the sounds connected to accelerometer and gyroscope and in different versions for each LED effect;
  • Build the gloves;
    • Draw the gloves in the fabric;
    • Sew Neopixels to the glove;
    • Build an enclosure to wear the arduino nano on my hand;
  • Work on portable version;
    • Create bluetooth connection;
    • Find the best battery to power it;

animation – after effects

Meet.me

Final Result

Meet.me was created by me together with Wen Chen. We started imagining a future when we’ll be able to built ourselves and design body parts in the real world with advanced augmented reality technologies, before going to a casual date. We also talked about how meeting apps change our behavior in society and modify our subjectivity, especially as gay people, who are a big target audience for this industry.

Process

References

fabrication – week 4

Wearable acrylic enclosure

This week we were tasked to create an enclosure. I made a simple experiment for a wearable using acrylic cut in the laser cutter and standoffs. The idea is to wear the Arduino Nano 33 IOT, which has a built-in accelerometer and gyroscope, to collect movement data from my hands, which are going to be used to create sound and visual effects with a computer.

fabrication – week 3

Laser Cutter hand-eye mask

This week we were tasked to work with the laser cutter. I’m always thinking about the performing arts context, so I designed a mask for the character I’ve been developing this semester: the Fire Bird.

I bought two pieces of acrylic at Canal Plastic orange and teal transparent. The most challenging part of this assignment for me was working with Illustrator for the first time….the laser cutter is a big discovery for myself and I’m pretty happy with the mask design – got amazing feedbacks as well. I want to improve the design – make it more reliable and comfortable for the face and maybe develop a series of masks/glasses.

fabrication – week 2

pyramid puzzle

This week I was tasked to make multiples of a multi-process piece. I wanted to start making puzzles, so after researching for wood puzzles models online, I found one which would be makeable in one week considering my fabrication knowledge level. Then, I found this great instructable guiding the fabrication process, so I basically followed it step-by-step and created five versions of it.

I bought everything I needed at the Home Depot hardware store in Bed-Stuy (a piece of Pinewood, a piece of dowel, and paracord). And I could experience using for the first time all of the machines of the shop introduced during the last class: the milter saw, the drill press, the sander, and the band saw.

The most challenging part was drilling 15 dowels because they are made of bad quality wood and are kind of small for the size of the hole that I needed. After some trials, I found a way of making them stable enough and learned that drilling slowly is better for delicate materials. Then I sanded them to correct some damages to the wood made during this process.

Find below the final result. To solve the puzzle you need to find a way of taking the ring out of the object.