ICS@PSU: Institute for CyberScience

News Archive

Published on July 11, 2017 by Anne Danahy

ICS affiliate to demonstrate July 13 at Arts Festival Booth

While data is often used to form a line on a graph describing a phenomenon — think points on a chart — what happens when it comes to life as a musical score?

For Mark Ballora, a professor of music technology and an ICS affiliate, sonifying data not only makes perfect sense, it also offers a new avenue for experiencing and understanding data. In his case, cosmological phenomena.

“I take the line and turn it into a melody,” Ballora said. “The idea of finding music in nature and science appeals to me on a poetic level.”

Ballora is one of the Penn State faculty members and graduate students who will be offering hands-on demonstrations July 13-15 at The Art of Discovery, Penn State’s booth at the Central Pennsylvania Festival of the Arts. The booth, next to Willard Building on the University Park campus, will feature free workshops for children and adults ranging from seeing 3-D printers in action and experiencing 360-degree viewfinders to watching glass candy get made.

For the full schedule, read more about the booth here. To talk with Ballora and experience data as sound, drop by the booth from 5:30 p.m. to 7:30 p.m. Thursday, July 13.

Among Ballora’s projects is working with former Grateful Dead percussionist and ethnomusicologist Mickey Hart and 2006 Nobel Laureate George Smoot on the DVD project “Rhythms of the Universe.” Ballora created the sonifications for the project, which explores the cosmos and includes visualizations as well as sonifications of astrophysical datasets.

“The dataset becomes like a musical score,” Ballora said.

Ballora — a self-described Deadhead — became involved in the project after he heard Hart and Smoot were working on a movie that would feature sounds of the Big Bang. “I thought, ‘I have to do that,’” Ballora said.

With that and other projects, choosing which instrument goes with which data is subjective. So, Ballora said he gets to ask, What would be a good instrument for a pulsar or an aurora borealis? It’s the same way an artist might choose which shade of blue conveys the nighttime sky.

“That’s the fun of it: designing something that says, ‘Oh, that sounds shimmery.’ Or, the galaxy is like a wind chime in the sky, so let me design something that sounds like a wind chime.”

From there he generates the sound using an audio synthesis software program — mapping the information to sound characteristics like melody and rhythm. He often has to transpose the numbers that describe the electromagnetic frequencies — the pulsars and aurora borealis — to ensure they are suitable for the pitches and that the listener can hear the contours of the lines.

Ballora thinks that while the idea of listening to science — not just looking at it — may be seen as a novelty now, it will be something that grows more common with young people.

“It will be part of the culture of research and science — that you don’t just look at it, you also listen to it. Young people will start finding the things they can discover by listening,” Ballora said. “Because we all respond to music, we dance to it, we tap our feet to it, we get tunes stuck in our heads, and the capabilities of the ear will be put into play. The strengths of the ear are different than the strengths of the eye. The ear detects some things better than the eye does, so discoveries will come about as a natural consequence of people being able to listen to datasets.”

In keeping with that, Ballora’s next projects received seed grants from the National Academies Keck Futures Initiative and the Gulf Research Program.

For one, he is working with marine biologist, underwater acoustician and cellist Heather Spence of Michelle’s Earth Foundation on a project focused on presenting the ocean’s dynamics through sound. Called “Layers of Meaning: How the Ocean’s Natural Acoustics and the Music of its Datasets Can Reveal Hidden Connections,” the project will include educational outreach.

For the other project, “Sonifications of Oxygen and Temperature Data in the Ocean: Creating a ‘Data Stethoscope’ to Detect the Ocean’s Vital Signs,” a “data stethoscope” will be created to translate into sound measurements of temperature, oxygen and biomass collected from the ocean’s depths. Ballora is working with Karen Wishner, professor of oceanography at University of Rhode Island.

This article originally appeared on Penn State News and has been slightly modified to clarify Ballora’s connection to ICS. Read the original here: http://news.psu.edu/story/474332/2017/07/10/impact/grateful-data

Penn State Institute for CyberScience

Help us build the future. Contact ICS at Penn State.

  • email: ics@psu.edu
  • phone: ICS-ACI Support (i-ASK Center): 814-865-4275 | General Inquiries: 814-867-1467

November 2017

  • Nov 7, 201711:00 am - 5:00 pm

    XSEDE HPC Workshop: GPU Programming Using OpenACC

    OpenACC is the accepted standard using compiler directives t…

    More Info
  • Nov 6, 201710:00 am - 3:00 pm

    NIIMBL Teaming Event with Celgene

    Celgene, a biotechnology company, invites researchers to a m…

    More Info
  • Nov 8, 20178:30 am - 4:30 pm

    Amazon Web Services Research Immersion Day

    Come and learn how to do science on the cloud! From scaling …

    More Info
  • Nov 9, 20171:30 pm - 2:30 pm

    SIRO Proposal Development Workshop: Perspectives on Graduate Education

    Dean Evasius, Director, NSF Division of Graduate Education, …

    More Info
  • Nov 2, 201712:00 pm - 1:00 pm

    Introduction to Data Science with R Workshop 8: Managing Multiple Datasets

    In this workshop, we will begin to synthesize what we have l…

    More Info
  • Nov 10, 20172:00 pm - 3:00 pm

    SIRO Proposal Development Workshop: NSF Research Traineeship Program-Limited Submission Opportunity

    The NSF Research Traineeship (NRT) program (NSF 18-507) is d…

    More Info
  • Nov 14, 20179:00 am - 5:00 pm

    GIS Day 2017

    Penn State GIS Day strives to bring together those who are w…

    More Info
  • Nov 9, 201712:00 pm - 1:00 pm

    Introduction to Data Science with R: Working with Text Data in R

    In this workshop, we will cover the basics of manipulating a…

    More Info
  • Nov 14, 20175:00 pm

    Paul Hawken, “Drawdown, the Most Comprehensive Plan Ever Proposed to Reverse Global Warming”

    Environmentalist and entrepreneur Paul Hawken will be the ke…

    More Info
  • Nov 16, 20172:00 pm - 3:30 pm

    Ari Fertig of “The Conversation”

    Penn State will host editor Ari Fertig from the academic med…

    More Info
  • Nov 17, 20174:00 pm - 5:00 pm

    Stochastic Computations Working Group: “Parameter estimation using linear response statistics: Theory and numerical scheme”

    Join ICS Associate John Harlim and Xiantao Li for their bi-w…

    More Info