Reactive Control and Synthesis of Sound

Adrian Freed

HTM

In 1992 I developed the HTM (Hear the Music) software synthesis system. HTM is:
- portable,
- reactive (real-time), and
- has an efficient unit generator library implemented carefully in ANSI C code for efficienty on RISC processors
Further information may be gleaned from these papers:

Fast Additive Synthesis

In 1993 Xavier Rodet suggested and guided my implementation of additive synthesis by inverse FFT, a method he patented with Phillippe Depalle:
- 10-30 times more efficient than frequency and amplitude interpolated oscillators
- efficient noise synthesis
- good potential for VLSI implementation
- Real-time control of hundreds of partials at 44.1kHz on a moderately priced workstation ~$5000
- Over a thousand partials plus noise on high-end workstation ~$20,000
- Scalable from $2000 PC's to $300,000 multiprocessor without custom hardware.
- Tight integration of control structure computations
- Compares well with previous methods:
```
Who                        What                  When       Partials    Fs         Freq. Control       Amp. Control                
Thaddeus Cahill            Telharmonium          1897                   ?          Fixed                                           
DiGiugno                   4A                    1976       256         8kHz       Discrete            linear (4kHz)               
Baudot & Freed             Reson8                1988       350         22kHz      Fixed               Exponential Decay           
Freed & Rodet              FFT-1 on SGI          1995       400         44.1kHz    Variable            Linear breakpoint           
```
  This implementation of additive synthesis was written up in an 1993 ICSPAT paper. A more detailed performance comparison may be found in a 1993 ICMC paper.

BYO: Control by Spectral Description

Benefits of timbre control by spectral description include:
- Perceptually relevant parameterizations
- Sample rate and hardware independent timbre description (difficult with FM and physical modelling)
- Extra dimensions for synthesis resource optimization and allocation:
  - sample rate (bandwidth)
  - temporal and frequency masking
  - partial pruning (timbral precision)
  - adjustable frame rate (temporal precision)
  - partial group assignment accross processors
- arithmetic stability (non-recursive)
The requirement for a flexible way to efficiently describe control over spectral descriptions of sounds lead to BYO.
- BYO control structures are to spectral sound descriptions what unit generators are for time domain signals.
- BYO is a layer between high level programming tools (such as IRCAM/Opcode's MAX [Puckette & Zicaralli 90], Matlab, and the Apple Newton) and interpolated additive synthesis
- BYO controls must be:
  - efficient, because they are executed every frame for every partial
  - easy to write, because musicians will always need control functions that are not already available
Further information on BYO may be obtained from:
- ICMC95 paper
- ICMC 95 slides
Control structures for additive synthesis of interest include:
- Signal modelling:
  - Ph. Depalle, X. Rodet, Th. Galas and G. Eckel,"Generalized Diphone Control", Proc. ICMC 1993, CMA, San Francisco.
- Statistical techniques:
  - Ph. Depalle, G. García and X. Rodet, "Analysis of Sound for Additive Synthesis: Tracking of Partials Using Hidden markov Models", Proc. ICMC 1993, CMA, San Francisco.
  - G. J. Sandell & W. L. Martens , 1992, "Prototyping and Interpolation for Multiple Musical Timbres Using Principal Component-Based Synthesis", Proceedings of the ICMC, 1992, CMA, San Francisco, CA
- Connections models:
  - M. Lee & D. Wessel, "Connectionist Models for Real-Time Control of Synthesis and Compositional Algorithms" , Proceedings of ICMC, 1992.
  - M. Lee & D. Wessel, "Real-Time Neuro-Fuzzy Systems for Adaptive Control of Musical Processes", Proceedings of ICMC 1993.
Current work is focussed on integrating HTM, FFT-1 and BYO into a real-time language environment for computer music