Google Glass In The Concert Hall? Cornell Prof Looks To Make Beautiful Music With High-Tech Headgear

Google Glass isn’t just for tech bros -- it could be a new way to experience classical music. A Cornell University professor and undergraduate student are teaming up to move Beethoven, Brahms and all the rest into the digital age.

Cynthia Turner is a conductor, professor of performance at Cornell’s department of music and leader of the university’s winds ensemble. In February, when Google was soliciting applications for its Glass Explorer program, she Tweeted:

Turner was one of more than 4,200 people who got the chance to try Glass out. She’s enlisted the help of an undergraduate student, Tyler Ehrlich, who majors in music but also dabbles in computer programming. Together they’re working to develop programs for Glass that can allow conductors -- and, perhaps someday, musicians -- to exploit Glass’ mostly hands-free setup in the rehearsal room and the concert hall.

One of the more obvious advantages to wearing Glass as a conductor is the potential to remove the music stand. The two main projects Turner and Ehrlich are working on right now are a way to have the musical score scroll along with the performance on the Glass screen, as well as a digital metronome for Glass. With those tools, a conductor would never need to glance down at a sheet of music.

“If you can keep your head up, you’re constantly engaged,” Turner said on Friday at a lunch event with journalists.

However, there isn’t a good way yet to reproduce a music score on the Glass headset. One of the problems is the lack of a good PDF reader for Glass; another is that traditional music notation is a bit unwieldy on such a small screen. Turner and Ehrlich are thinking that they may have to come up with a whole new way of writing musical scores for Glass -- maybe replace traditional notes with lines, and have colors represent directions like ‘forte’ (to be played loudly) or "pizzicato"
(plucking).

Glass could also be a new way for audience members at orchestral performances to experience the music. You could rig it so that the conductor’s Glass beams images up to a screen above the stage.

“Wouldn’t it be interesting for an audience to come in, and instead of seeing my back, they see what I see?” Turner says.

But at present, it’s not feasible to directly stream video of a performance to the audience at an orchestra performance. The problem? Lag times.

When Turner records a video from the conductor’s podium, it shows up on a projected screen delayed by a half-second or more. And while such a delay might not be problematic while watching a baseball game -- where you can hear the crack of a bat connecting with the pitch slightly before it shows up on the Jumbotron -- it would be extremely disorienting in a musical performance.

“Half a second is another bar” in the score, Turner explains.

The problem isn’t really an Internet speed issue, Ehrlich says; it’s more that the video has to be compressed and then decompressed during the transmission process. (Still, it’s pretty impressive that crunching the video, bouncing it off of a server in California back to New York and unpacking it can only take half a second.)

For now, Turner and Ehrlich are side-stepping the issue by working with still photos during live performances. While Turner’s conducting, the Glass will take a picture every second or so, and beam it to a computer where someone will select the nicest photos to be displayed above the ensemble.

Ehrlich is already building the Glass version of a digital metronome by retooling an open-source Android metronome program called Beat Keeper. But when building Glassware, a programmer has to rethink almost every aspect of the user experience. The images on the screen may have to be adjusted, since they’ll be showing up about an inch from the user’s eye. Buttons and drop-down menus are a no-go. Everything has to be about gestures and tapping.

To be truly handsfree, Turner would like the Glass metronome to run on voice commands. But voice commands would exhaust Glass’ battery, which is not particularly robust. Turner would also like to have a zoom option on Glass while taking a photo or video.

One other side effect of taking video at eye level is that the conductor can’t flail about too much. A head-shaking conductor would make for a shaky video (and, perhaps, a sea-sick audience).

Turner jokingly acknowledged that “when making videos, I have to curb my enthusiasm” just a little bit.