university of southern california viterbi school of engineering

music computation and cognition laboratory

     

MIMI - multi-modal interaction for musical improvisation

project co-leaders: profs. alex françois, elaine chew
collaborators: dennis thurmond, isaac schankler
undergraduate: katherine desousa (wise ugrp, 2007)

introduction

Mimi in Berlin

Mimi's international debut at the Berlin Musical Instrument Museum, on May 19, 2007, with the Seiler Grand Piano, the first acoustic grand piano with full MIDI capabilities produced in the 1980s. Photo by E. Chew.

Mimi is a multi-modal interactive musical improvisation system that explores the impact of visual feedback in performer-machine interaction. The Mimi system enables the performer to experiment with a unique blend of improvisation-like on-the-fly invention, composition-like planning and choreography, and expressive performance. Mimi's improvisations are created through a factor oracle. The visual interface gives the performer and the audience instantaneous and continuous information on the state of the oracle, its recombination strategy, the music to come, and that recently played. The performer controls when the system starts, stops, and learns, the playback volume, and the recombination rate. Mimi is not only an effective improvisation partner, it also provides a platform through which to interrogate the mental models necessary for successful improvisation.

The Mimi system is designed and implemented using the SAI framework invented by Alexandre François, with design input from Dennis Thurmond, Isaac Schankler, and Elaine Chew. Mimi is a recent example of factor oracle based improvisation software in the OMax family.

concerts

Mimi and Isaac Schankler in concert February 4, 2011, at the Ussachevsky Memorial Festival, Lyman Hall, Thatcher Music Building, Pomona College, Claremont, California. Video: [ HTML ]

Mimi's concert debut with Isaac Schankler at the June 5, 2010, People Inside Electronics concert, Boston Court Performing Arts Center, Pasadena, California. Press release: [ PDF ] Poster: [ PDF ] Videos: [ HTML ]

videos

Videos of an early Mimi with Dennis Thurmond, and of Mimi4x (an installation) with Isaac Schankler.

scientific presentations & demos

François, A. R. J. and E. Chew, Jul 23, 2010, Mimi4x: An Interactive Audio-Visual Installation for High-Level Structural Improvisation. Presented at the ICME 2010 workshop on Interactive Multimedia Installations and Digital Art (IMIDA), Singapore, July, 2010. In Proceedings of the IEEE International Conference on Multimedia and Expo, 1618 - 1623. [ PUB ]

François, A. R. J. and E. Chew, Jun 5, 2010, MIMI Science Minutes, People Inside Electronics concert, Boston Court Performing Arts Center, Pasadena, CA. [ ANN ]

Chew, E. and A. R. J. François, Aug 2, 2007, MIMI in the Context of Music and Computing, MIT Alumni Association, Club of Southern California, Pasadena Lunch, Guest Speakers, The Beckham Grill, Pasadena, CA. [ ANN ]

Chew, E. and A. R. J. François, Jun 19, 2007, MIMI - multi-modal interaction for musical improvisation, International Joint Workshop on Computational Creativity (IJWCC'07), Show and Tell Session, Goldsmiths, University of London (Jun 19) [ ABS ]

François, A. R. J., Jun 11, 2007, Time and Perception in Music and Computation, Colloque sur les Nouveaux Paradigmes pour l'Informatique Musicale, IRCAM, Paris, France. [ ANN, AUDIO ]

François, A. R. J., and E. Chew, Jun 7, 2007, Visual Feedback in Performer-Machine Interaction for Musical Improvisation,, International Conference on New Instruments for Musical Expression (NIME2007), New York University, NYC [ ABS ]

François, A. R. J., and E. Chew, May 19, 2007, Lecture & Demonstration: The Seiler Grand Piano and Computer-Interactive Improvisation with MIMI, International Conference on Mathematics and Computation in Music (MCM2007), Musical Instrument Museum, Berlin, Germany.

publications

François, A. R. J., I. Schankler and E. Chew (to appear). Mimi4x: An Interactive Audio-Visual Installation for High-Level Structural Improvisation. International Journal of Art and Technoloty Special Issue for revised and expanded proceedings from the ICME-IMIDA workshop.

François, A. R. J., E. Chew and D. Thurmond (to appear). Performer Centered Visual Feedback in Human-Machine Improvisation. ACM Computers in Entertainment.

Schankler, I., A. R. J. François and E. Chew (2011). Preparing for the Unpredictable: Identifying Successful Performance Strategies in Human-Machine Improvisation. In Proceedings of the International Symposium on Performance Science, Toronto, Canada, August 24-27.

Schankler, I., J. B. L. Smith, A. R. J. François and E. Chew (2011). Emergent Formal Structures of Factor Oracle-Driven Musical Improvisations. In Proceedings of the International Conference on Mathematics, Computation and Music, Paris, France, June 15-18.

François, A. R. J., I. Schankler, E. Chew (2010). Mimi4x: An Interactive Audio-Visual Installation for High-Level Structural Improvisation. In Proceedings of the IEEE International Conference on Multimedia & Expo (ICME 2010), Singapore, July, 2010.

François, A. R. J. (2009). Time and Perception in Music and Computation. In G. Assayag and A. Gerzso (Eds.): New Computational Paradigms for Computer Music, Editions Delatour France / IRCAM. [ PUB ]

François, A.R.J., E. Chew and D. Thurmond (2007). MIMI - A Musical Improvisation System That Provides Visual Feedback to the Performer. USC Computer Science Department Technical Report No.07-889. [ PUB ]

François, A. R. J., E. Chew, and C. D. Thurmond (2007). Visual Feedback in Performer-Machine Interaction for Musical Improvisation. In Proceedings of International Conference on New Instruments for Musical Expression (NIME2007), New York, June, 2007. [ ABS | PDF | REF ]

updated April 17, 2011.