Rohan Jayasekera

Writer & Researcher

Fundraiser & Project Manager

Tech Consultant

Rohan Jayasekera

Writer & Researcher

Fundraiser & Project Manager

Tech Consultant

My Writing

Artists coding emotional impact into data sets

June 15, 2017 Concepts
Artists coding emotional impact into data sets

Should data scientists & artists ‘design in’ engineered capacity to induce human emotional responses to otherwise impenetrable data sets? Is there a plug-in for a dataset that could fix that for them? Or should they just leave the generation of emotional responses to artists?

In his book Emotional Design: Why We Love (Or Hate) Everyday Things, the former Apple exec and design critic Don Norman discusses three levels of emotional responses to product design — “visceral, behavioural and reflective”.

As Wired Magazine’s review noted, for years Norman had done nothing but rail against hard-to-use products. Yet in his 2004 book, he admitted that emotional attachments often trump practicality. Creative design can trigger a personalised emotional response to even the most hard-to-use, superficially impractical objects.

As a means of communicating information, bulk data sets are probably as hard-to-use and superficially impractical as it gets, even data on a subject as essential as climate change. In fact you don’t need to be a rocket or a data scientist – and Donald Trump is neither – to recognise the limitations of climate change data as a means of generating engagement with the issue.

An alternative approach: Climate Symphony turns hard data on climate change into a symphony, telling the story of what climate change means through sound. It uses data sonification, a technique designed to make statistical analysis more accessible to the mind via sound and the ear, just as traditional data visualisations like graphs make data accessible through imagery and the eye.

“Research shows that sound touches us in inexplicable ways,” Leah Borromeo and Katharine Round of Disobedient Film Company, co-creators of the work with composer Jamie Perera. “By using music,” they told Anita Makri of SciDevNet, “the hope is to create an emotional response to something that for many might look meaningless on a page.”

Norman, of course, was talking about emotional response in the context of product design. How we relate to a visual element, and how to create something that is more appealing, effective or well received via design, by design. But his perspective, coming from a former Apple exec, is instructive in a digital media era when the case for challenging climate change is suddenly less appealing, effective and well received in some crucial quarters.

I’m writing here, gathering thoughts ahead of participating in an all-day workshop on data sonification and its uses for creative advocacy. But as a journalist and media technologist, looking at the premise of Disobedient’s unchallengeably attractive and principled work set me thinking about the whole idea of generating emotional responses in a creative environment, for the purposes of creative advocacy in the public interest.

Politics, media, communications and advocacy for change have been transformed by the spread of media techniques led by emotional design – the capacity to create an emotional response to a packaged information-based argument. Shockingly, we’ve discovered that the technique works even better – and even more profitably – where the package is light on information and argument to start with.

Whether manifested as politically motivated disinformation, ‘fake news for-fun-and-profit’, or deep state subversive ‘information operations’ – the results are proving as catastrophic for the global information environment as human intervention and climate change has for our planetary space.

The “visceral level” is the first impression, a purely instinctive emotional response to information. Emotional designers set their ‘information delivery mechanisms’ to trigger a visceral emotional response – to leave users feeling something. But what kind of emotional response is the emotional designer seeking? Anger? Fear? Grief? Compassion? Empathy?

We need to ask which one, and furthermore, why that one in particular? How does the emotional design of the delivery mechanism secure the desired response at the behavioural level? A successful visceral behavioural response leaves you wanting to interact with the information. And a successful information delivery system should leave you wanting more of the same to act on again and again. Should we use these techniques more cautiously?

Major charities and media organisations have aggressively monetized the generation of visceral behavioural response. Now they must constantly ramp up their trigger mechanisms to sustain repeat business with ever more desperate images of starving African children and ever more improbable exposes of paedophile rings in DC pizza parlours.

What does this say about using cultural practice, such as the creation of symphonies from data sets, to trigger emotional behavioural response? Norman’s emotional design is especially relevant to information tech at behavioural level when it comes to “the UX”, the ‘user experience’ – what users physically need to do to successfully interact with information. And latency. How obstructive to this successful interaction is the information delivery system function’s intrinsic design? Should we ask what is good UX and acceptable latency in a symphony?

This kind of behavioural level context is key to effective emotional design. At a basic level, regardless of motive, in creating something that is more appealing, effective or well received. “At the simplest level, the design of a business card implies a behaviour,” says Norman. “To take it and file the information away for later.” What information can tonight’s audience “take away and file for later”? How will its UX perform? How will its latency stand up to critical review?

This is also a test of Norman’s third level – the “Reflective” – in his view, the “highest” level of the emotional-visual thought process in visual design. It’s the interpretation and understanding of creatively expressed information combined with feelings about it. At this stage of thought, a person determines and creates a lasting impression of something. Is it memorable? Does it leave a lasting impression? Will he or she reference it later?

Memorable, yes. Tonight’s performance in London includes live illustrated performances by leading sound artists Kate Carr and Lee Patterson, who use location recordings captured in the field in the composition of their work.

The performance includes an audio-visual exploration of Ólafsfjörður, a small fishing town in northern Iceland. The set explores the way weather patterns are enmeshed in the experience of home and place. It combines field recordings of iced fences, snow, ice melting, wind, boats and blizzards with recordings of traditional Icelandic sea songs performed by the town’s choir.

Referrable? We’ll see. The idea behind ‘Climate Symphony’ is to translate hard data on climate change into a musical composition that engages the public — encouraging people to question their feelings and the stories behind the data, and create a conversation. “In a world where we’re saturated with hearing the same messages,” say Borromeo and Round, “any way to engage people with a subject [as] important [as] climate change is worthwhile.”

Disobedient & Forma Arts: Climate Symphony Performance #2degreesfestival. 7.30pm, Saturday 17 June, at Toynbee Studios, 28 Commercial Street, London E1 6AB. Supported by Arts Council England and Sculpt the Future Foundation.