Final Project: The Water Synth

Featured Posts, Physical Computing

A waterfall that functions as a musical instrument. When the user passes their hand under the waterfall, notes will play. The note will be sustained as long as they keep their hand in the same position. It will be possible to play chords or intervals (multiple notes at once) using both hands.

Our first prototype was as low-fi as possible–we created a waterfall by pouring a bucket of water into a tupperware container with a slit in it. As users put their hand under the waterfall, Jarone played notes on a scale from his iPad.


The main questions we wanted to answer in the first round of user testing were, Is this fun? and, Is the system intuitive? The answer to our first question was that people found it fun and entertaining (though there was one user tester that was displeased about getting wet). As for the second question, when we didn’t explain that it was a musical instrument, there were a few people who just sat and stared at the waterfall without putting their hands in it. This can be solved in a number of ways (e.g., a title card with instructions, or using a peg that interrupts the stream so that a note is being played in the “starting position”, or having one of us fool around with it so the audience can see how it works, etc.)

We got really nice feedback over all, with users wanting us to experiment with sensors on the y-axis, include LEDs to light the waterfall, and create individual streams so it’s more obvious where the notes occur.

Laser Testing:
After watching some YouTube videos, Jarone and I were fairly certain we could fire lasers directly through the water stream to get a nice hot spot in the water basin below (the water refracts the light of the laser along its path). However, we didn’t realize that your average laser isn’t powerful enough to travel more than about 6 inches. Since we want our waterfall to be at least 2 feet tall, this presented a problem:


After testing, we realized that we aren’t going to be able to send the laser through the water’s path, so our plan is to plant the laser directly behind it.

System diagram:


Midi Communication:
We hooked up a midi jack and a photo sensor to an Arduino for a dry test:


We’ve succeeded in getting notes to play when the photocell is covered, but we’re still figuring out how to get one sustained note. Our next steps are to fix the code and do a wet test with a laser, water basin, and photo cell below it.

Beginning the waterfall:
We got a 400 gallon-per-hour pump that was too powerful for our initial set up, so the waterfall was shooting toward the wall instead of downward. Then when we added a valve to regulate the flow, it completely killed the water pressure so rather than a sheet we got a sad little stream.

Benedetta thinks the valve was probably rated at a much lower water pressure than the pump allows for. For the time being we’ll use individual streams for each note because producing a waterfall is harder than we expected. At this point, separate streams also makes more conceptual sense because it shows the user where each note occurs. A sheet of water gives the impression of a full range of notes you can slide between, instead of integer values that occur at arbitrary points.

Concurrency Test:
We were able to not just get one sustained note, but multiple notes at the same time! Jarone says the next thing we should do is put all the notes into an array to clean up the code.


The Second Prototype:
When putting together our second prototype, we struggled to find the right sensors. We first tried photocells, but their range of sensitivity was too small. Then we tried both infrared and ultrasonic distance sensors. We ended up using one of each in this second prototype (the IR sensor is the finicky one on the left):


The IR sensor was constantly giving out garbage readings, possibly because it was reflecting off the water in weird ways. The behavior of the ultrasonic sensor didn’t seem to be affected by the water. We decided that the ultrasonic sensor was the better, more reliable option.

Getting Ready for Final Class:
No luck pushing the notes into an array, but at least our wires are labeled.

The third prototype:
For our final physical computing class, we really wanted to get four sensors working. Unfortunately the delays that we used to send and receive ultrasonic pings clogged up the entire system. The current state of our code only allows for two sensors. However, we made progress in terms of getting one sensor to play multiple notes, as seen in this video:


We’re still not sure the best way to “zhush” it up, other than by painting the pvc pipe. We still need to figure out the kind of enclosure we want to use for our circuit, and bring some dignity to the sad plastic basin. We also need to get a working system without using four different arduinos. But the prototype works!


The Enclosure:
We decided to make a wooden enclosure for the PVC pipe and water basin, and seal it with shellac in order to make it water resistant. Our friend Chester was extremely kind and made a model for us, so all Jarone and I had to do was figure out the measurements and assemble it.

Winter Show Highlights:
Jarone and I were fortunate enough to be selected for the ITP’s 2016 Winter Show! We got really amazing feedback. Even though many people needed coaxing to actually roll their sleeves up and get wet, they really enjoyed it once they did. The water synth was a hit with basically every demographic, from toddlers to senior citizens. We had a few professional musicians stop by and they had fun guessing what scale we used and creating quick compositions.

Source Code:

The Slapper’s Dilemma

Physical Computing

For my Physical Computing midterm assignment, my partner Annie (Se Young) Kim and I made a Prisoner’s Dilemma game (working title: The Slapper’s Dilemma). I said that I really wanted to make a slapping machine and Annie was totally down. First we made a prototype with LEDs:

The two green lights meant that both players were choosing to remain silent (and would each receive a light slap). When one player confesses, the other player’s light turns red (meaning that the silent player would get a hard slap, but the confessor doesn’t get slapped at all). When both players choose to confess, both lights turn red, because both players get a hard slap.

Next we made a prototype using servo motors. We also adjusted the code because we wanted players to make their choice and then get slapped, instead of having the button push and slap happen at the same time. We used state change detection and also added a delay. Every 10 seconds, the Arduino would check if each player had chosen to betray or stay silent, and the appropriate slaps would be triggered. Here is what the motor prototype looks like. In the video, each player first receives a quick slap for pressing the silent button. When both players hit the betray button, each receives a long hard slap:

Annie also built a pretty interface in which we wanted to mount the prototype, but every time we tried a different wire came loose and we had to give up:

Takeaways: It would have been nice to use more powerful motors with giant slapping hands for greater effect. The small cardboard hands that Annie laser cut from cardboard were adorable, but kind of pathetic. Also, the next time I try to create any kind of game, I am soldering down every damn wire. Breadboards are not the way!

Observing Jukeboxes

Physical Computing

When I was growing up, there was a Tex Mex joint in my neighborhood called Radio Mexico that was the best place ever. They had a pool table, a pinball machine and a jukebox. My dad would give me a chair to stand on (so I was tall enough to read the song titles) and let me choose whatever music I wanted.

Radio Mexico doesn’t exist anymore, so I decided to visit a few bars on the Lower East Side that reportedly had jukeboxes. My first stop was a bar called Hi Fi:

dsc_0369 dsc_0366


My first thought was, “This layout is hideous!” There’s a pile of seemingly random artists in the middle of the screen, and the gray sidebars are drowning in text.

That said, all the information I’d want to have is available to me. It’s obvious how to search for artists (the cursor is controlled by a trackball below the screen). It’s nice to have the playlist available so I can look to see what’s currently playing, what’s up next, and how long it will be before my song will play (as indicated by the 22:30 in the bottom right corner). Scrolling through albums was also easy once you picked an artist:

The obvious advantage over an old school jukebox is how much more music is available. The first interaction I observed lasted over 10 minutes–there were two friends tag-teaming, searching for their favorite artists. They were enjoying themselves, but eventually it appeared to get tedious scrolling through all the artists (there are many, many artists):

Their conversation devolved into “They have it!” “No they don’t.” “They have to have it!” “They don’t.” Something else that may have contributed to the long interaction: before you buy a song, a popup appears asking you if you’re sure you want to buy the song. I saw them pull up and turn down a bunch of song choices. I guess the popup is nice to prevent accidental purchases, but it also seems kind of wimpy. Like, if you accidentally buy cheesey oreos at the vending machine, you just need to accept that your midday snack is cheesey oreos!

At any rate, the women were happy enough with the selection that they kept refreshing their playlist as the evening wore on. Other than the two women, there was one other guy who went up to the machine and left when he realized he didn’t have any money.

The other bar I visited was B Side:

As you can see, this is a fancy touchscreen jukebox. Oooo. Ahhh. Something that’s immediately off-putting about this device is that not only is it advertising for itself, it’s advertising for random third parties, of which Fanduel’s fantasy sports is just one. Another is scratch off lotto tickets:


Because the opening video made such a big deal about the top plays, I decided to navigate to the “Top Plays” playlist. The jukebox immediately froze. After about a minute, it restarted itself. To see if this was a fluke, I tried to navigate to the playlist later, but it froze yet again. The interaction wasn’t off to a good start. When the machine wasn’t freezing, here’s what it looked like:

This is more visually appealing than the last jukebox, but also a lot more unreliable. And why all the ads? One of the ads was for the Touchtunes mobile app, so I decided to download it and see if it would work. It took a while, but eventually I was able to control the music in the bar with my phone. This is me rocking out to “She’s a Rejecter” by Of Montreal:

The phone app was kind of fun. Because there’s no verification system to prove you’re in the bar when you’re using the app, if you have a grudge against a bartender you can force them to listen to shitty songs during their shift!

Another clever/slimy aspect of the Touchtunes interface is that you can pay more money to have your song jump to the head of the playlist. At B Side, this was no concern,  because no one in the bar had any interest in the jukebox. I was not able to observe any outside interactions with it.

This really made me miss Radio Mexico, and the Radio Mexico jukebox. Their selection might be considered “small” in the digital age, but it was an awesome selection. I loved watching the machine take the compact discs and set them on the player. I loved pressing my sticky little hands against the glass and watching my music play.


A Servo, a Piezo, and the least efficient way to play NBA2K

Physical Computing

This lab did not have fruitful beginnings. To mix things up a bit, I tried to use a temperature sensor instead of a photocell, but I was getting an unworkably small range. I decided to “troubleshoot” by removing a resistor. I shorted the circuit and burnt my finger when I went to test the temperature sensor. Photocells it is!

Here’s what the circuit looked like when I got burnt:


Here is the functioning circuit:


I attached a rubber band to my Servo and wrapped it around a Joystick. Here is the result (Max Chomet with the assist):

This is the set up to get my piezo to play  a couple of notes:


Here is the ditty:

And finally, here is the three-note keyboard. This took forever because it didn’t occur to me that my three photocells would have three completely different ranges!

Digital and Analog Inputs and Outputs

Physical Computing

Digital inputs and outputs:



Because I didn’t make a homemade switch last week I decided to include one this week for fun:


Analog inputs:


Above I’m using a potentiometer as my analog input. In the code, the pot value was divided by four to represent the brightness of the LED, from 0 to 255.

For the last part of the lab, I picked photocells as my variable resistors:


In the video above I’m just checking to see that the values were registering. 3.92 represents the amount of voltage, but I’m still not sure what the “800” means in this context.


The more light the photo cells were exposed to, the dimmer the LEDs. I wonder how I could change the code to make it the opposite (more light = brighter LEDs).

I wish I could have experimented more this time around, but I made a stupid mistake that had me stuck for a long time–I forgot to supply power to the photocells! I am also realizing that I’m going to need to get a pair of wire cutters. The wire situation is starting to get out of hand.


Basic Circuits

Physical Computing

The first thing I did was set up a circuit using a push button. At first I just copied the picture in the Arduino projects book because I wanted to make sure that I could get everything to work without blowing anything up.


Upon examination, I can tell what’s going on. The 220 ohm resistor is connected directly to the power line so I know the LED won’t blow out. The next object in the circuit is the push button. When I push it down, metal leads come into contact, completing the circuit. Until the button gets pushed, it’s an open circuit.

Pushing the button provides power to the LED, and it lights up! The last component is a jumper wire, which connects to ground. (The whole concept of grounding is still confusing to me. But as long as shit is lighting up, my ignorance clearly isn’t holding me back! (I’m going to live to regret these words.))

The next thing I wanted to do was try out the potentiometer. I found some online tutorials and was able to get some sweet potentiometer action. Below you can see the circuit with a single LED, and then two LEDs in series and in parallel:

potentiometer potentiometer-series potentiometer-parallel

Here are two more videos. There was some suspense with the parallel circuit–wasn’t sure the green guy was gonna make it!


What is Interaction?

Physical Computing

As soon as I started reading Chris Crawford’s book called The Art of Interactive Design, I thought about a Facebook post that my friend (and ITP alum!) Sharang Biswas put up a few days ago:

The word “interactive” seems to have become a synonym of “embedded with electronics”… I just saw a post about an “interactive ball”…because obvi balls are never otherwise interactive…

I think Crawford would argue that balls in and of themselves are not interactive (a ball is not an “actor” that listens, thinks, or speaks…but a ball can certainly be the facilitator of interaction). However, Crawford would agree that just because something has an electronic component does not make it “interactive”. The examples he initially cites are not electronic at all–Interactive Rugs! Interactive Shampoo! Interactive Granola Bars! I would pay good money to see an interactive granola bar.

I scoff, but I admit that defining interaction is difficult, and I’m struggling to come up with a definition of my own. I appreciate Crawford’s idea of degrees of interactivity, as opposed to a binary. For example, I interact with a vending machine to get my granola bar, but it’s a fairly low level of interactivity. My granola bar is a zero interactivity object. I can anthropomorphize the thing, create a whole dialogue for it, like, “Please Melissa, I am but a paltry snack, unworthy of your stomach. Spare me!” but all this is for naught. I have no empathy for the bar. I eat it noisily and mirthfully.

The part of Crawford’s definition that trips me up is the “thinking” element. Aren’t musical instruments interactive? But these are unthinking things. Is a guitar only truly interactive if after hearing me play it’s got something to say back? Like, “Melissa, the e-string is out of tune” or “Melissa, the song you’re writing is very derivative of the Pachelbel’s Canon.” Is that better? Do we want to endow our instruments with the insights of our shitty boyfriends?

I strongly agree with Bret Victor that we need to harness human capabilities. Something that strikes me in the “vision of the future” is that there’s no need for humans at all. The girl can look up recipes without her mom’s help, the office dudes can see they need a new pump without a meeting…basically the computers can do all the work, and all we need to do is make little swipey motions. It’s hardly a future I want to live in.