Input/Output: Proof of Concept

Hello! 

I am sharing some of the research and testing I have been conducting for an interactive artwork to be presented in my MADT 304 class. My original concept was to create a dynamic visual projection that responds to users' interactions with conga drums (or bongos) placed in the gallery. When I presented the proposal for this concept during a class feedback session, it was brought to my attention that this concept might not take advantage of the Arduino processes I have learned this semester. When I had planned this idea, I had not realized that the Arduino would not be needed to execute the piece since the sensors could connect directly with a computer.

After realizing the limitations of my initial concept, I felt compelled to create a new concept that would allow me to explore the Arduino. I do not mind switching concepts since I want to take advantage of the opportunity that MADT 304 is giving me to create work with the Arduino. In the future, I can revisit the conga drum idea.

New Concept: Sound Drawers (final name TBD)

This work invites viewers to explore the contents inside wooden drawers. The action of pulling out the drawers triggers various audios which can layer, loop, and fade depending on the participants movements. 

I am developing a concept for this piece relating to the senses, nostalgia, nature, and heritage. I will describe the concept in more detail once the piece is finalized.

I was inspired by the piece Cabinet of Curiousness by the artist duo Janet Cardiff and George Bures Miller. 

Technical Research

I began my research by exploring the various technical approaches I could take in the construction of the piece. I learned about Raspberry Pi, Arduino to HDMI shield, Arduino + the  Adafruit soundboard, Arduino+Processing)

I spoke with Bryan about these paths and was able to get a better understanding of the pros and cons of each path. After the meeting, I did more independent research by asking and answering my own questions in a solo Q&A session to better understand the different approaches.

Q&A Session with Myself Does the Adafruit Sound FX Board include an amp? Yes. If I use just the Arduino and one Adafruit board,  is that more limited (in amount of sounds) than the other options? Is it just 11 sounds because there are 11 pins? Yes, just 11 sounds. One 2MB board allows you to store a “few minutes” of compressed stereo. Can the Adafruit Sound FX Board play multiple sounds at once so that they’re layered? No, they can only play one sound at a time. You might need to buy an entire soundboard for each sound and feed them into a mixer. https://forums.adafruit.com/viewtopic.php?f=19&t=172736 Can I use more than one Adafruit board at a time? Yes, but I am not sure how yet. People in forums are saying that they use mixers. If I were to use Processing, can I have all my sounds playing at once but at 0% volume so that when a drawer is pulled, the sound associated to that drawer is turned to 100% volume? Yes, I think the command for volume would be “amp()” https://processing.org/reference/libraries/sound/SoundFile.html Am I right to think that the Arduino+Adafruit Sound Board combination is not versatile enough for this project? Yes, using this approach would be more complicated and expensive than other methods such as connecting directly to Processing on a computer.

Based on these answers, I made a pros and cons list of each method to help decide my path.

	Arduino to trigger sound in Processing	Arduino +Adafruit Sound FX Board	Raspberry Pi
PROS	Allows for video and audio.  Easy to layer and loop sounds. I can practice my programming skills and write ‘if’ statements.	Fully self-contained, practical, and easy to transport.  I can practice my programming skills in Arduino and write ‘if’ statements.	It’s a computer so I can do anything I want. This microprocessor also allows me to make a self-contained object.
CONS	Needs to have a connected computer that is hidden. The object won’t be fully independent and self-contained.	The sound board has a limit of 11 short sounds. The audio quality is poor. The board can’t layer audios.	It can be more complicated to use because it requires more programming knowledge. I have to buy a new stuff (the board, the SD card, etc.)

Initially, I had wanted to make the work with Arduino + the Adafruit Sound FX Board because I was more familiar with the two devices and because they would 

Creating Code for Arduino & Processing

Below is a video of one of the tests I conducted before incorporating the technology into the physical drawers.

Below is the code I am using. The left side of the screen shows the Arduino code and the right side shows the Processing code.

//ARDUINO CODE int buttonPin1 = 7; // pushbutton 1 pin int buttonPin2 = 4; // pushbutton 2 pin void setup() { Serial.begin(9600); pinMode(buttonPin1, INPUT_PULLUP); // set pushbutton 1 as input with pull-up resistor pinMode(buttonPin2, INPUT_PULLUP); // set pushbutton 2 as input with pull-up resistor } void loop() { // check if pushbutton 1 is pressed if (digitalRead(buttonPin1) == LOW) { Serial.println("Button 1 pressed"); delay(100); // debounce delay } // check if pushbutton 2 is pressed if (digitalRead(buttonPin2) == LOW) { Serial.println("Button 2 pressed"); delay(100); // debounce delay } } //PROCESSING CODE import processing.serial.*; import processing.sound.*; SoundFile sound1, sound2; // two sound files Serial myPort; boolean sound1Playing = false; boolean sound2Playing = false; void setup() { size(400, 400); smooth(); background(255,150,255); sound1 = new SoundFile(this, "owl.mp3"); // load sound file 1 sound2 = new SoundFile(this, "twinkle.mp3"); // load sound file 2 println(Serial.list()); // Print the list of available serial ports myPort = new Serial(this, Serial.list()[0], 9600); println(myPort); // Print the status of the serial port } void draw() { if (myPort.available() > 0) { String buttonState = myPort.readStringUntil('\n'); if (buttonState != null) { if (buttonState.trim().equals("Button 1 pressed") && !sound1Playing) { sound1Playing = true; sound1.play(); // play sound file 1 } else if (buttonState.trim().equals("Button 2 pressed") && !sound2Playing) { sound2Playing = true; sound2.play(); // play sound file 2 } } } // Check if sound1 has finished playing if (sound1Playing && !sound1.isPlaying()) { sound1Playing = false; } // Check if sound2 has finished playing if (sound2Playing && !sound2.isPlaying()) { sound2Playing = false; } }

The Proof of Concept

This is the final video of the first prototype working!

For this prototype, I used plastic organizational drawers since they have fewer drawers for me to worry about. The final version will be wooden, not plastic, and will have multiple drawers which will be fully-functional. For this prototype, only the two top drawers trigger sound when pulled open. The bottom drawer is used to store the Arduino and the breadboard. In the final version, I will make sure that all the drawers are functioning and that the Arduino is fully concealed.