MacMix:Mixing Music with a Mouse

©1986, 1996 Adrian Freed. All Rights Reserved.

This is a slightly revised version of a paper presented at the ICMC in the Hague in 1986.

MacMix is a tool for browsing, viewing, mixing and editing sounds. Its strengths are the result of an unsurprising user interface, implemented on an Apple Macintosh computer, and Robert Gross's efficient sound file system and mixing program.

Architecture

IRCAM, like many large computer music facilities, has chosen to build a centralised store for sound files. This is the most economical way of providing access to the Gigabytes of file storage required for hours of high-quality sound. MacMix gives a musician a familiar personal work-station environment for processing sounds shared amongst a community of users on a large time-sharing host computer.

The Macintosh computer is connected to the host using a serial communication line. This architecture was inspired by the Blit [Pike, Locanthi and Reiser, 1985].

A custom-designed, robust, stateless communication protocol is used between host and workstation.

User Interface

Three types of window are provided.

Browser. The browser is used to explore the contents of the sound file system. A window of file names in the current sound file directory may be scrolled. Descriptive written information is available using the Get Info. button. The Play button is used to listen to the selected file. The samples in a file may be viewed using the the Open button.

View WIndows. View windows contain a visual representation of a sound. Most of the time, this representation is an amplitude envelope. If sufficient resolution is available, the samples themselves are displayed.

Great care is taken to visually represent sound in an unambiguous way. Lines are not drawn between sample values as this would introduces visual information with no counterpart in the signal. Each pixel in the amplitude envelope display represents the largest absolute sample value in an integer number of samples. This definition of envelope has the advantages of being easy to interpret, cheap to compute and it facilitates identification of clicks or unusually large amplitude excursions.

View windows provide a simple editing facility. A selected portion of a sound file can be extracted and written to a new file. An amplitude envelope can be sketched on top of a selected sound file fragment and will be superimposed on the signal during the extraction process.

Selected parts of sound files are sent to mix windows to form the ingredients of a mix.

Mix Windows. The mix window is a graphical cue-sheet editor. Slices of sound are stacked vertically like lines of text. A gain slider is available for each slice. The position of ingredient sounds in the final mix is specified by dragging the polygonal icons along the horizontal time axis. Fade-in and fade-out times are changed by dragging the upper corners of the polygonal icons. Slices can be edited with the cut, paste and copy operations, much like lines of text are manipulated in a mouse-based text editor.

Multiple views of a single object are maintained and several mixes can be worked on simultaneously. It is thus possible to work on a mix in small manageable pieces in one window, whilst maintaining a global view in another window.

Mixing operations can be applied to anything from a few samples to many hours of sound. This means that MacMix may be applied to large final mixes or to micro-surgery of sound.

Gestures

The user interface described above is not like the gestural controls of a standard mixing console or the standard keyboard and screen of text based mixing programs. Little is lost when taking away the typing and syntax problems of text based mixing programs. However, the loss of the tight coupling between audition and gesture, provided by standard mixing consoles, is too great.

The problem of synchronising the audition of sound on a time-sharing system with gestures on the Macintosh is not as hard to solve as it first may appear. The play and record programs at IRCAM can be invoked with a request that conversion be delayed until an external synchronisation cue is provided. Any source of sound may be used to provide this cue. The main application of this scheme is to synchronise transfers between computer and tape recorder using click tracks. By using the sound output of the Macintosh computer as a synchronisation source, clicks of the mouse button can be accurately synchronised to sounds played on the host computer.

These gestures are logged on the bottom line of the view and mix windows. They may then be used as reference points for future mixing.

Extensions

Uncoupling the user interface from the signal processing and sound storage is extra work for the implementor, but this extra work pays off in many ways.

Implementation and Testing Once the protocol between the Macintosh and host computer was defined and documented, it was possible for the two components of the system to be built simultaneously and independently. The user interface was tested by simulating host responses on the Macintosh. The host programs were tested by typing commands on a conventional terminal, simulating those expected from the Macintosh.

New Workstations. At IRCAM, Dan Timis has developed a complete reimplementation of the MacMix user interface for the Sun Workstation called SunMix. The host software is exactly the same for both the Sun and the Macintosh user interfaces.

It is interesting to compare the two user interfaces. The most striking difference is that SunMix takes advantage of the large screen of the Sun by using tiled windows and permenant menus rather than overlapping windows and pull-down menus.

Rather than building a single user interface and porting it to very different graphics systems, a new user interface is built using the tools and techniques most easily available on each system. People already familiar with Macintosh software find it very easy to use MacMix because the user interface follows many of the conventions of popular word-processing, painting and drafting programs.

New Hosts. The speed with which mixes can be described and changed with MacMix highlights the need for real-time mixing. Robert Gross at Sprocket Systems, a division of Lucasfilm Ltd., has implemented the host software experimentally on a small signal processing and sound file storage system.

Reliability. Small workstations are noticably more reliable than large time-sharing computer systems. For this reason, the MacMix protocol is based on the idea that all critical information about a Mix is stored in the workstation, not in the host. The only data the host is responsible for is the sound file system. The protocol is set up so that if the host computer malfunctions, users just wait for it to be repaired and can continue from exactly where they were before the malfunction.

Extensibility. New commands have been added to the MacMix protocol which extend the system beyond mixing, editing and browsing files. Dan Timis has added commands to access IRCAM's Chant-Formes system, catalysing the Pacha user interface developed by Fabrice Guedy. Robert Rowe has added a command to transfer FFT data to provide spectral views.

Future Work

Reverberation... Many signal processing programs would benefit from graphical user interfaces and integration in an environment such as MacMix. Reverberation is a good example. It would be convenient to describe the shape, size and characteristics of a hall to be simulated by drawing sketches.

Perceptual Controls. The gain sliders and envelope functions control signal amplitude. We would really like to control perceived loudness.

Spatial Mixing. Mixing, as commonly done, is a 2-dimensional process. We would like to be able to mix sounds by placing and moving them in a simulated space.

Acknowledgements

References

PIKE, R. & LOCANTHI, B. & REISER, J. (1985), Hardware/software trade-offs for bitmap graphics on the blit. Software-Practice and Experience , 15, 131-152 (1985).