AUME (MUMEWE)

description

Exhibited at the MUMEWE as part of ISEA 2013

Sydney, Australia.

Soundscape composition is the creative practice of processing and combining sound recordings to evoke auditory associations and memories within a listener. We present Audio Metaphor, an autonomous system for creating novel soundscape compositions. Audio Metaphor blends concepts from natural language queries derived from Twitter with semantically linked sound recordings from online user-contributed audio databases. We used natural language processing to create audio file search queries. An audio file segmentation algorithm based on general soundscape composition categories is used to cut up the results. A multi-agent composition engine processes and combines cut up audio files for representations of natural language queries. 

This implementation of Audio Metaphor is a 4ch sound installation with the real-time projection of the natural language processing.

Additional System Details:

Audio Metaphor creates unique soundscape compositions that represent the words in an NLP using a process pipeline as follows:

* Transforms a natural language request into a audio file search queries using a simple word feature extraction algorithm.

* Search online for audio file recommendations from the Freesound database.

* Segment audio file recommendations into soundscape regions using a SVM classifier and heuristic for joining  adjacent segments similarly labelled.

* Process and combine audio segments for soundscape composition using a multi-agent approach. Each agent is responsible for one segmented audio file, and uses a heuristic modelled after production notes from Canadian composer Barry Truax to process the different regions.