Thinking Sound with Generative AI: A Creative Practice Workshop

The rise of Generative AI prompts a rethinking about the language we use to designate concepts, the structures we use to develop narrative, and the strategies we use for creative collaboration across media. While visual media dominates discourse around new technologies, tuning attention to the sonic opens different pathways for critical thinking and creative practice. This workshop hosted by the Literature Catalyst for the Scottish Graduate School for Arts and Humanities offers participants methods for creative and critical responses to narrative, metaphor, improvisation and beyond in the context of Generative AI, as well as an opportunity to engage these methods in practice.

In the afternoon, from 12-4pm, Jonathan Impett and Juan Parra Cancino will present strategies developed both as part of their Three States of Wax project and within the larger framework of the Music, Thought, and Technology research group at the Orpheus Institute in Ghent, Belgium. These strategies aim to structure improvisation by use of technologically informed tropes. Jung In Jung (Abertay) and Lynda Clark (Edinburgh Futures Institute, University of Edinburgh) will consider how the concept of ‘hallucination’ in Generative AI can act as a creative tool, using their collaborative work Traumgraz as a starting point. Finally, Johanna Linsley (Dundee) and Martin Zeillinger (Abertay) will facilitate a conversation with participants about how their own projects and research concerns can draw on the ideas and provocations explored in the workshop.

In the evening from 7-9pm, participants are invited to a public event, Wet Tech, where the workshop leaders will perform the works that informed the afternoon session. Wet Tech takes place at the Chaplaincy on the University of Dundee campus

This workshop is open to all PhD students based at a university in Scotland, whether or not you are funded by SGSAH, and travel costs up to £50 will be reimbursed. Please contact Dr Johanna Linsley at [email protected] if you have question about travel.

FURTHER DETAILS

Three States of Wax is a project of the ‘Music, Thought and Technology’ research group at the Orpheus Institute, Ghent, Belgium (MTT).

The late philosopher of science Michel Serres drew many of his metaphors from music or from the natural world. In L’interférence (1972) he sought to identify the objects of study of modern science, and looked back to Descartes’ example of a piece of wax. Early modern science would have described it in terms of its shape, colour and texture, says Serres – geometrical description. In the later nineteenth and early twentieth century they would have referred to its properties of transmission of heat, of deformation and transformation – physical description. Now, he says, we would more appropriately find an informational description. We should understand it as a carrier and embodiment of knowledge: the feeding habits of the bees, the conditions of its formation, the process of its separation, the entire history of its subsequent use, handling, and environments. And in doing so, we add to that history; every interrogation and knowledge-use becomes part of the material.

Three States of Wax considers improvisation in that light: how can the human and computational responses to material generated in improvisation be based not only on figural or gestural interpretations, but on an understanding of that material as embodying a broader history and environment of its emergence? In Serres’ terms, as the product of a time-based, situated modulation or interference pattern of heterogeneous sources of information?

//

Traumgraz is inspired by the anthropomorphised term ‘hallucination’ used for artificial intelligence’s confident response with unjustified false information. An interactive story is written between a large language model (Bard AI) and writer/researcher Lynda Clark based on pictures sent from Graz by artist/researcher Jung In Jung during her Styrian Artist in Residency (St.A.i.R) and is set in a building with escalating levels of weirdness. Jung performs live with the randomly generated story by choosing paths along with sound materials recorded in Graz and generated by AI. First, the same pictures sent to Lynda were fed to vit-gpt2-image-captioning and then the generated word tokens were used in MusicGen to generate audio. Jagdeep Ahluwalia supported finetuning the result of audio with Jung’s four past sound compositions to apply the moods classified by Jung for each fragment of her compositions

Know what’s Happening Next — before everyone else does.

Get App

Host or Publisher School of Education and Social Work, University of Dundee