ICS affiliate to demonstrate July 13 at Arts Festival BoothPosted on July 11, 2017
While data is often used to form a line on a graph describing a phenomenon — think points on a chart — what happens when it comes to life as a musical score?
For Mark Ballora, a professor of music technology and an ICS affiliate, sonifying data not only makes perfect sense, it also offers a new avenue for experiencing and understanding data. In his case, cosmological phenomena.
“I take the line and turn it into a melody,” Ballora said. “The idea of finding music in nature and science appeals to me on a poetic level.”
Ballora is one of the Penn State faculty members and graduate students who will be offering hands-on demonstrations July 13-15 at The Art of Discovery, Penn State’s booth at the Central Pennsylvania Festival of the Arts. The booth, next to Willard Building on the University Park campus, will feature free workshops for children and adults ranging from seeing 3-D printers in action and experiencing 360-degree viewfinders to watching glass candy get made.
For the full schedule, read more about the booth here. To talk with Ballora and experience data as sound, drop by the booth from 5:30 p.m. to 7:30 p.m. Thursday, July 13.
Among Ballora’s projects is working with former Grateful Dead percussionist and ethnomusicologist Mickey Hart and 2006 Nobel Laureate George Smoot on the DVD project “Rhythms of the Universe.” Ballora created the sonifications for the project, which explores the cosmos and includes visualizations as well as sonifications of astrophysical datasets.
“The dataset becomes like a musical score,” Ballora said.
Ballora — a self-described Deadhead — became involved in the project after he heard Hart and Smoot were working on a movie that would feature sounds of the Big Bang. “I thought, ‘I have to do that,’” Ballora said.
With that and other projects, choosing which instrument goes with which data is subjective. So, Ballora said he gets to ask, What would be a good instrument for a pulsar or an aurora borealis? It’s the same way an artist might choose which shade of blue conveys the nighttime sky.
“That’s the fun of it: designing something that says, ‘Oh, that sounds shimmery.’ Or, the galaxy is like a wind chime in the sky, so let me design something that sounds like a wind chime.”
From there he generates the sound using an audio synthesis software program — mapping the information to sound characteristics like melody and rhythm. He often has to transpose the numbers that describe the electromagnetic frequencies — the pulsars and aurora borealis — to ensure they are suitable for the pitches and that the listener can hear the contours of the lines.
Ballora thinks that while the idea of listening to science — not just looking at it — may be seen as a novelty now, it will be something that grows more common with young people.
“It will be part of the culture of research and science — that you don’t just look at it, you also listen to it. Young people will start finding the things they can discover by listening,” Ballora said. “Because we all respond to music, we dance to it, we tap our feet to it, we get tunes stuck in our heads, and the capabilities of the ear will be put into play. The strengths of the ear are different than the strengths of the eye. The ear detects some things better than the eye does, so discoveries will come about as a natural consequence of people being able to listen to datasets.”
In keeping with that, Ballora’s next projects received seed grants from the National Academies Keck Futures Initiative and the Gulf Research Program.
For one, he is working with marine biologist, underwater acoustician and cellist Heather Spence of Michelle’s Earth Foundation on a project focused on presenting the ocean’s dynamics through sound. Called “Layers of Meaning: How the Ocean’s Natural Acoustics and the Music of its Datasets Can Reveal Hidden Connections,” the project will include educational outreach.
For the other project, “Sonifications of Oxygen and Temperature Data in the Ocean: Creating a ‘Data Stethoscope’ to Detect the Ocean’s Vital Signs,” a “data stethoscope” will be created to translate into sound measurements of temperature, oxygen and biomass collected from the ocean’s depths. Ballora is working with Karen Wishner, professor of oceanography at University of Rhode Island.
This article originally appeared on Penn State News and has been slightly modified to clarify Ballora’s connection to ICS. Read the original here: http://news.psu.edu/story/474332/2017/07/10/impact/grateful-data
- IBM donates system to Penn State to advance AI research
- ICS Seed Grants to power projects that use AI, machine learning for common good
- Penn State Berks team advances to MVP Phase of Nittany AI Challenge
- Creepy computers or people partners? Working to make AI that enhances humanity
- Sky is clearing for using AI to probe weather variability
- ‘AI will see you now’: Panel to discuss the AI revolution in health and medicine
- Privacy law scholars must address potential for nasty satellite data surprises
- Researchers take aim at hackers trying to attack high-value AI models
- Girls, economically disadvantaged less likely to get parental urging to study computers
- Seed grants awarded to projects using Twitter data
- Researchers find features that shape mechanical force during protein synthesis
- A peek at living room decor suggests how decorations vary around the world
- Interactive websites may cause antismoking messages to backfire
- Changing how government assesses risk may ease fallout from extreme financial events
- Science of Connection: Researcher to discuss GIS and connected communities
- Hack event gives students a chance to battle back against simulated cyberattack
- ICS associate thinks ‘people will notice’ Net Neutrality Day of Action
- Losing Net Neutrality: ‘People Will Notice’
- ICS offers graduate student travel awards