Enhancing Creative Musical Practices through Accessible AI Tools
RNCM PRiSM Researchers Dr Hongshuo Fan and Dr Bofan Ma join forces with regional and international partners to lead on building critical and strategic dialogues around AI research that matters to musical creatives today.
This initiative started with three Knowledge-Exchange Case Studies around the development and roll-out of several AI-assisted technological tools emerging out of PRiSM’s substantial research. They focussed upon collective reflections and documentation of practical engagement of these technologies from artists, scholars, and ensembles who are closely connected to Manchester and North West England.
The PRiSM SampleRNN Case Study
Ma & Sargen at the International Conference on AI and Musical Creativity (AIMC), Wolfson College Oxford, September 2024.
The PRiSM SampleRNN Case Study looked into the stories behind PRiSM’s flagship machine-learning software toolkit, PRiSM SampleRNN, and the diverse artistic utilisation of it since 2020. The study involved many artists based in the North West, especially those connected to the ML4M Unsupervised working group over the past few years.
These artists had been a crucial part of the emergence of PRiSM’s innovative approach to foster artist-technologist collaborations.
This timely study, co-led by Ma and PRiSM Doctoral Researcher 2019-24 Ellen Sargen, identifies a novel model of Learning to Learn within a specialist musical conservatoire that is driven by the dynamic working relationships between artists, technologists, and bespoke AI technologies.
PRiSM Musical Audio Gesture Recognition (AGR)
PRiSM Musical Audio Gesture Recognition (AGR, or previously Musical Gesture Recognition) research emerged out of an earlier artistic residency of composer Professor George Lewis, whose RNCM PRiSM commission, Forager, saw him work closely with RNCM staff and student performers in 2022.
The extension of the work involved developing and releasing a fully standalone, open-source and open-access AGR toolkit, which was driven by PRiSM-associated artists collaborating closely with Fan and the International Contemporary Ensemble (ICE) based in New York.
The finalised software tool was launched officially through a headline premiere concert on 16 May 2024 at the acclaimed Roulette Intermedium in Brooklyn (and livestreamed to an international audience).
It was also featured in George Lewis’ recent work – The Reincarnation of Blind Tom (2024) – for SWR Symphonieorchester and Donaueschinger Musiktage that was premiered in Donaueschingen, Germany in October 2024.
PLAY | Music for Video Game Controllers
PLAY team performing with the House of Bedlam ensemble.
PLAY was conducted in partnership with Dr Robert Laidlow (PRiSM Associate and Career Development Fellow in Music at Jesus College Oxford), the Oxford Research Centre in the Humanities and Oxford Cultural Programme, and the Manchester-based ensemble The House of Bedlam.
The project explores how video game controllers (VGCs) can be transformed into accessible musical Instruments.
The multi-stranded project involves building a new lightweight and fully open-access software interface that enables creative utilisation and customisation of incoming data received from any mainstream VGCs (Sony PlayStation 5 DualSense / Access, Nintendo Switch, Xbox):
PLAY | Open-source software tool on Github
It also centres upon new compositional and performance methods using this tool, developing creative strategies for making music with affordable gaming technologies.
Following a successful launch in Oxford in May 2024 (featuring newly composed or devised music by Robert Laidlow, Bofan Ma, Hongshuo Fan, Ellen Sargen, Cassie Kinoshi, and Sarah Hennies), the project returned to Manchester in November 2024, with a further tour around North England being planned for 2025-26.
