01:39 AM

Sound by design


What does a song look like? Students stretch their perceptions through the synesthetic study of ‘Sight Sound Syntax.’

By Cory Phare

Lights overhead dim. A hush falls over the crowd, and the upright-bass player leans in to groove on the brisk tempo counted off.

As an ensemble musical performance, it’s poised to begin much the same as many others.

The only difference? The piece these players are reading isn’t a musical score – it’s graphic design.

This is the MSU Denver course: Sight Sound Syntax, where Communication Design students learn about the interrelation of sonic and visual elements.

“We’re deliberately trying to have students open their heads to look at different approaches to design,” said Martin Mendelsberg, affiliate faculty member in the Communication Design Department and co-instructor of the course.

Students began the project by collecting musical and other field recordings, which they then interpreted, creating a graphic representation using various typographical elements and photography.

So, instead of reading notes on a musical staff, the invited performers used the designer’s abstract notations of music and sound to improvise musical riffs, veering into David Lynch-Angelo Badalamenti territory as an organic free-jazz freak-out. 

“We want to push them out of the ‘this looks like a poster’ kind of thinking into a new way of looking at the world,” said Mendelsberg.

How I wonder what you are…

The experimental form – dubbed a ‘sonigraphic’ by fellow course instructor and affiliate communication design faculty member Scott Surine – had an equally unorthodox beginning: a children’s nursery rhyme.

Running into Surine in the hallway, Mendelsberg described to him how he had students visually represent the A-B-A (or ternary) form that’s the basis for “Twinkle Twinkle Little Star.” They would introduce an idea in the ‘A’ section, elaborate it by contrast in the ‘B’ section, then return to the theme to conclude it.

Surine, also a musician, then had a further thought: What if you could add another layer of sonic improvisation based upon these visuals?

“We hung up these horizontal typographic scores, then asked department of music faculty member Mark Harris to come over and start playing it,” Surine said. “We all started laughing hysterically at what came out – it was unlike anything we’d ever seen or heard before.”

From there, they collectively began designing a curriculum based on the experimental exercise. And what started as a three-day summer workshop eventually morphed into a semester-long class.

Thus, the sonigraphic, as an instructional tool, was born.

“What’s the shape of this sound?”

The sight-sound relationship might not be default thinking, but intuitively makes sense, according to Estevan Ruiz, a classically-trained musician and junior-year communication design student. He detailed the process of developing this truly mixed media throughout the course: “In class, we ask ourselves how we could lay out a sound graphically by using a hierarchy to order things,” he said. “When you think about music, it’s basically notes arranged in time – it’s a design system that correlates to a grid.

A key distinction was to remove the elements of musicality from the visual composition to avoid a preconceived notion of sonic familiarity.

To do this, students captured decidedly non-musical elements in their field recordings. The varied sounds – a car horn, someone running up the stairs, a clanking garbage can or dog barking – were selected for their contrasting abilities to render a distinctive aural palette. Groups then arranged samples together into an industrial-sounding collage that’d feel right at home on the Mr. Robot soundtrack.

The next step? Turning the composition into visual forms.

“While we were sketching out the piece, we’d stop and ask ‘what’s the shape of this sound?’” Ruiz said.

He and his compatriots spliced and laced letterforms into proportional configurations, often focused on ancient mathematical relationships like the Golden Ratio and Fibonacci sequence.

After much trial and error, the project took form: An animation allowed the students to suggest rhythmic structure for musicians (led by Harris) who’d encounter the piece for the first time.

And when paired with the post-performance playing of the initial recordings, the juxtaposition was striking: Removed from its original, the musical improvisation transformed into something new entirely.

For Ruiz, it was a moment of clarity.

“Hearing that all together is when it really came full-circle for me,” he said.

Beautiful solutions are musical

So, that’s cool – but what’s the takeaway?

“The point is to get students outside of their comfort zones and how they naturally interpret their tools – be that Photoshop or a musical instrument,” Surine said.

He noted that when you get musicians to stop strictly following a score or designers to stop looking at type solely as something to read, it takes thinking beyond the predictable expectations – and limitations – of the usual approach.

Innovation arises from the unexpected. As Ruiz detailed, starting with field recordings purposefully created a blank canvas, and one that wasn’t dictated by a Western musical scale.

“I started thinking more about semiotics,” he said. “It’s about how we recognize and extract information from forms to represent something new, differently – that was a real challenge, but the freedom that resulted from it was liberating.”

The intertwined elements of music and visual design prove the forms are more similar than not, found in the very terminology used to talk about both: rhythm, pacing, dynamics.

“The next time you’re creating a poster or brochure, you’ve got the tools to move you in a more exploratory direction,” said Mendelsberg. “Can you actually hear what you’ve designed?”

He added, “Most designers would agree that when you look at beautiful solutions, they’re musical.”

Check out our other stories ...

See how this alumna community advocate is paving the road home.

Take a Coffee Break in our podcast interview with President Stephen Jordan.

This Roadrunner is heading to South Korea and a Fulbright future.