CPSC-334/kinetic-sculpture/task2 at main · risxyang/CPSC-334
I was really fascinated by the idea of the brachiograph, after learning about in class— I was interested in its capacity to bridge the digital and the physical, and was wondering in what ways I can challenge its usual applications (1:1 copying of digital images). Perhaps I could use it to represent something that is not initially an image.
In another class, I am working on a project involving musical style transfer, for which I have been looking at MIDI files and how to read/parse them. I thought it might be possible to transform a musical composition into an abstract visual representation. I honestly had no clue what the output would look like — maybe it could be super cool! Maybe it would be completely boring. But at minimum, I was interested in putting the midi files of different compositions into my program and seeing how the results differed between composers' styles and individual pieces. Additionally, I thought it would be cool to have a physical artifact resulting from listening to a composition.
I started by wiring up three servos, as per the instructions here. I will refer to the 3 motors as the shoulder, elbow, and wrist motors/joints — this terminology is used on the brachiograph site, but I also came up with it myself before I saw that. This idea of a correspondence between body parts and machine parts inspired how I programmed the motors to move later on.
As a first step, I had the shoulder move between [0,90] (space limitations), the elbow between [0,120], and the wrist between [0,180]. This actually produced what is probably the most beautiful (imo) image I got out of this device:
This coordinated motion might actually have been more aesthetically pleasing than the Midi-representational motion I got later. At the very least, it looks "organic." I gave the brachiograph a Pentel pocket brush to draw with, which allowed for variation in line width and darkness according to the wrist servo's angle as well as the motor delay.
Emboldened by this test, which I think was successful on the basis of producing an interesting image (which I could not have predicted would be produced), I started programming the motor behavior to change per note of a composition.
I used Music21 to process midi files. The gist of the parsing goes like this:
for each measure:
for each element in measure:
if element is a tempo / metronome mark:
bpm (beats_per_minute) = element
if element is a note:
get shoulder, elbow angles
send angles to ESP32
if element is a chord:
use lowest and highest notes within chord to get shoulder, elbow angles twice
send angles to ESP32, waiting in between
if element is a rest:
set wrist angle to lift pen
send angles to ESP32
before getting the next element, wait for a period of time calculated by the current note's duration in beats multiplied by (60 / bpm
I simplified chord-as-angles representation by just taking the topmost and bottommost notes within each chord (this felt a little disrespectful towards the composers, but I didn't want to overwhelm the motors with too many directions too frequently. I used to play piano, and my teacher would tell me to voice the top and bottom notes of any chord the most— so that is where I got this idea.) Having the brachiograph draw an arc between the topmost and bottommost notes might pass as representing the chord.