Codevelopment of User Interface, Control and DSP with the HTM
Environment
Adrian Freed
Adrian Freed
Introduction
Collaborative development of audio, musical and multimedia projects
requires:
- Interaction with complete and reactive system
prototypes running in real-time at the application sample rate,
- Open tools environment that enables designers to use their
favorite tools on any computing platform.
Real-Time Signal Processing Performance
Platform Alternatives
Custom VLSI
- Difficult development environment
- Obsoleted quickly, not programmable
DSP
- Not general purpose: optimized for linear operations on fixed
coefficient systems
- Hard limitations, e.g. address space, stack
- Inflexible I/O
RISC workstations, e.g. SGI Indy
- High quality development tools
- Scalable architectures
- Large, growing customer base
- Flagship products for chip vendors
- (higher clock rates than DSP chips)
SGI Indy: a Computer Music Platform that scales well
- 16-bit Stereo Analog I/O
- Mic. preamp. + Headphone amp.
- Digital audio I/O (SPDIF)
- DAT and CD audio over SCSI
- Apple compatible MIDI Serial Ports
- Ethernet, ISDN & * RS422
- SCSI II
- Analog and Digital Video Support
Efficient development tools
Fast prototyping and rapid turnaround requires:
- higher level programming than assembly code
- efficient floating point arithmetic
- strong development tools: debuggers, code profilers, e.g. ANSI C
Efficient arithmetic performance requires:
- clear programming style that leverages code optimizers
- careful use of storage allocators and system facilities
- vector arithmetic
Address standard difficulties of vector arithmetic
- Scheduling of calculations to the precision of a single sample is
critical in some real-time signal processing applications =>
- Dynamically splitting large vectors into smaller ones as required
- Short delays: Recursive structures such as IIR filters, tapped delay lines
cannot be built by composing elemental multiply, add and delay vector functions=>
- Provide a library of commonly used high level functions or "unit
generators".
- Fixing parameter values at the start of each vector computation which often
creates audible artifacts=>
- Unit generators interpolate (usually linearly) all control parameters.
Encourage reuse of efficient, debugged code
Unit generators include:
- oscillators
- filters
- filter banks
- band-limited pulse generators
- neural networks
- frequency modulated oscillators and noise sources.
Unit generators are combined hierarchically.
Signal flows are managed in small memory arrays that can be thought of as "wires".
Reactivity
Hard real-time Goals
- Low latency <10mS
- Low jitter <1mS
- Guarantees
- High reliability
- Synchronicity
Processor scheduling
HTM user-level scheduler based on standard scheduling features of
UNIX
- non-degrading process priority
- locked memory
- grouped I/O semaphores, i.e. the select(2) system call
The HTM scheduler directs traffic between:
- control structure parameter updates and requests
- unit generators
- A/D and D/A converters
Scheduler provides:
- low latency service to parameter updates and requests
- constrains the latency between unit generator computations and workstation
converters between specified high and low water marks.
Good performance of these techniques on SGI workstations depends on this
vendor's fast networking implementation and well designed sound driver.
These support the HTM scheduler's use of a single select(2) system call to
monitor I/O status, thus obviating excessive overhead due to process context
switches.
Memory Allocator
- Fast and deterministic
- Labelled memory for performance measurements, optimization and debugging,
e.g.large arrays that are infrequently accessed can be placed in non-cached
pages of memory.
- Supports dynamic memory probes such as real-time signal and spectrum
displays and on signal recording to a file.
Integration with other tools
Client/Server Architecture
- Scales smoothly from individuals on a single machine to geographically
dispersed collaborators on an internet
-
- User interfaces can be developed independently of signal processing layer
-
- Individual strengths of designers, their tools and machines can be
combined: e.g. Opcode Max on Macintosh
- Signal processing on Indigo
- Concurrency and loose coupling promote rapid prototyping
UDP Protocol from the TCP/IP Suite
- Widely available
- Low latency for small packet sizes
- Reliably delivered in practice
- Routed through gateways for WAN
- Implemented over
- Ethernet & Token Ring
- Appletalk
- ISDN
- Frame Relay
- Serial Ports (SLIP, PPP)
- FDDI and CDDI
Example applications
- Singing voice synthesis: formant filter, lattice filter, &
additive model (Macaulay and Quatieri)
- Resonance (subtactive) synthesis
- Additive synthesis by Inverse FFT
- Non-linear wave equation simulation
- Non-linear oscillators for modeling excitations of musical instruments
- Behavior and control of sound synthesized from nonlinear oscillators and
the Chua circuit
- Virtual reality environment for auditory interpretation of scientific data.
- Real-time sound editing and mixing
Future directions
- New platforms, e.g. PowerPC, Pentium
- Higher level language binding
- Lower latency
- Real-time LAN
Acknowledgments
- Robin Bargar
- Insook Choi
- Mark Goldstein
- Mike Lee
- Dana Massie
- Roger Powell
- Ami Radunskaya
- Xavier Rodet
- David Wessel
- Matt Wright
- Emu, Gibson, SGI