Tag: Exhibitions

This post is about the creation of the project. To see how people reacted and plans for improvements, please check out the post about the Toronto Kids Digital Festival.

Project Details:

Shown at: Toronto Kids Digital Festival

Software: TouchDesigner

Description:

With the use of a mounted webcam, the TouchDesigner app scans a wall and plays a musical note based on how high a sticky note (or any other item with a specific color) was placed on the wall – the higher the sticky note, the higher the musical note. There were four colors – each corresponding with a different musical instrument. Users would create patterns and songs and chose which instruments to hear and watched as it scanned.

The goals were 1) to have participants think of music in another dimension – they put a physical thing higher or lower in an order to create a visual that represents a musical pattern and 2) expose participants to TouchDesigner and the logic behind how the project takes a physical thing, turns it into something digital, and then outputs it as something else.

Random Notes:

I came up with this idea on a high-caffeine and sleep-deprivation night back in November 2018 (I think). It is inspired by ways I learned music when younger, including Nickelodeon’s Director’s Lab.

The biggest issues I ran into were:

• making the app work with the webcam at different distances from the play surface
• blobs in blob detection need space around the blob to detect it
• how to make it understandable to know what to do

To make the app work with both the single foam board I first tested on and a full wall, I created an interface that showed the webcam feed with lines overlayed on it, that I could crop to fit. The app uses this cropped version to avoid reading the entire feed. I also added squares that show what note is being played to help calibrate the wall and show participants what note is being played as it scans.

I found that the program works best when there are spaces between the sticky notes. I wanted participants to automatically add this space and the suggestion test groups gave me were to have a grid on the wall – it makes more sense to fit to a grid if they can see it. I planned on using string because it is only visible up close and it would be awkward to stick a sticky note to the string vs. the wall. I ran out of string, so instead, I made a sign demonstrating the proper space needed to work best. Participants did notice this sign and mentioned it and updated their creation to work better.

As for knowing to take sticky notes and put it on the wall and what the colors mean, I used buckets! I originally pictured two buckets – one with new notes and one with old notes for reuse. At the last minute, I switched to wanting buckets that correspond with each color.

I originally was going to label each with the label on the front, but these buckets would be on the floor and people aren’t going to be able to see that label very well, so I put the label inside each bucket.

I chose buckets because people know what they can do with buckets (reach in, carry) and what they are generally used for (containing and organizing).

Work-in-Progress Gallery:

What a part of a melody should look like for this project. I was adding these post-its to the wall in the GRID Lab when a potential student and his parents were getting a tour. They were great and I think seeing someone doing something random and what looks like could be a fun project made them want to attend OU.

Dec. 6, 2018

Getting closer. Making strides. Thanks @oblowery for your help earlier!

Most of this project has been dealing with physical issues – boards that don’t work, postit notes that don’t stick, short cords, webcams that see too much of the room, dragging old equipment into the living room, can’t power on a 90s musical keyboard after searching through a bin of adapters, having to find other ways to get music, etc. I’m going to the store tomorrow for posterboard, better post its, and D batteries.

Dec. 11, 2018 Instagram Post | See it here

Added sharps, so this is more like a piano than sheet music. It’s also gotten to the point where using sticky notes would be easier than creating the board digitally.

Dec. 11, 2018 Instagram Post | See it here

I’m happy with this test. I think I will mark where notes are and full sticky notes can be used if I have the camera either farther or if I had a full wall. 

Dec. 17, 2018 Instagram Post | See it here

Name that tune before watching the video! I like this test better than the last and it proves that it works even when changing around the space. I also found a way to calibrate it.

Dec. 18, 2018 Instagram Post | See it here

I have decided to use something like thread to create lines to help players. People enjoyed making music with their bodies, but it’s super hard to judge where the note lines are to create songs. I’m also going to have smaller sticky notes so it makes more sense visually.

March 6, 2019 Instagram Post | See it here