Codevelopment of User Interface, Control and DSP with the HTM Environment

Adrian Freed

Adrian Freed

Introduction

Collaborative development of audio, musical and multimedia projects requires:

• Interaction with complete and reactive system prototypes running in real-time at the application sample rate,
• Open tools environment that enables designers to use their favorite tools on any computing platform.

Real-Time Signal Processing Performance

Platform Alternatives

Custom VLSI

• Difficult development environment
• Obsoleted quickly, not programmable

DSP

• Not general purpose: optimized for linear operations on fixed coefficient systems
• Hard limitations, e.g. address space, stack
• Inflexible I/O

RISC workstations, e.g. SGI Indy

• High quality development tools
• Scalable architectures
• Large, growing customer base
• Flagship products for chip vendors
• (higher clock rates than DSP chips)

SGI Indy: a Computer Music Platform that scales well

• 16-bit Stereo Analog I/O
• Mic. preamp. + Headphone amp.
• Digital audio I/O (SPDIF)
• DAT and CD audio over SCSI
• Apple compatible MIDI Serial Ports
• Ethernet, ISDN & * RS422
• SCSI II
• Analog and Digital Video Support

Efficient development tools

Fast prototyping and rapid turnaround requires:

• higher level programming than assembly code
• efficient floating point arithmetic
• strong development tools: debuggers, code profilers, e.g. ANSI C

Efficient arithmetic performance requires:

• clear programming style that leverages code optimizers
• careful use of storage allocators and system facilities
• vector arithmetic

Address standard difficulties of vector arithmetic

• Scheduling of calculations to the precision of a single sample is critical in some real-time signal processing applications =>
  • Dynamically splitting large vectors into smaller ones as required
• Short delays: Recursive structures such as IIR filters, tapped delay lines cannot be built by composing elemental multiply, add and delay vector functions=>
  • Provide a library of commonly used high level functions or "unit generators".
• Fixing parameter values at the start of each vector computation which often creates audible artifacts=>
  • Unit generators interpolate (usually linearly) all control parameters.

Encourage reuse of efficient, debugged code

Unit generators include:

• oscillators
• filters
• filter banks
• band-limited pulse generators
• neural networks
• frequency modulated oscillators and noise sources.

Unit generators are combined hierarchically.

Signal flows are managed in small memory arrays that can be thought of as "wires".

Reactivity

Hard real-time Goals

• Low latency <10mS
• Low jitter <1mS
• Guarantees
• High reliability
• Synchronicity

Processor scheduling

HTM user-level scheduler based on standard scheduling features of UNIX

• non-degrading process priority
• locked memory
• grouped I/O semaphores, i.e. the select(2) system call

The HTM scheduler directs traffic between:

• control structure parameter updates and requests
• unit generators
• A/D and D/A converters

Scheduler provides:

• low latency service to parameter updates and requests
• constrains the latency between unit generator computations and workstation converters between specified high and low water marks.

Good performance of these techniques on SGI workstations depends on this vendor's fast networking implementation and well designed sound driver.

These support the HTM scheduler's use of a single select(2) system call to monitor I/O status, thus obviating excessive overhead due to process context switches.

Memory Allocator

• Fast and deterministic
• Labelled memory for performance measurements, optimization and debugging, e.g.large arrays that are infrequently accessed can be placed in non-cached pages of memory.
• Supports dynamic memory probes such as real-time signal and spectrum displays and on signal recording to a file.

Integration with other tools

Client/Server Architecture

• Scales smoothly from individuals on a single machine to geographically dispersed collaborators on an internet
• User interfaces can be developed independently of signal processing layer
• Individual strengths of designers, their tools and machines can be combined: e.g. Opcode Max on Macintosh
• Signal processing on Indigo
• Concurrency and loose coupling promote rapid prototyping

UDP Protocol from the TCP/IP Suite

• Widely available
• Low latency for small packet sizes
• Reliably delivered in practice
• Routed through gateways for WAN
• Implemented over
  • Ethernet & Token Ring
  • Appletalk
  • ISDN
  • Frame Relay
  • Serial Ports (SLIP, PPP)
  • FDDI and CDDI

Example applications

• Singing voice synthesis: formant filter, lattice filter, & additive model (Macaulay and Quatieri)
• Resonance (subtactive) synthesis
• Additive synthesis by Inverse FFT
• Non-linear wave equation simulation
• Non-linear oscillators for modeling excitations of musical instruments
• Behavior and control of sound synthesized from nonlinear oscillators and the Chua circuit
• Virtual reality environment for auditory interpretation of scientific data.
• Real-time sound editing and mixing

Future directions

• New platforms, e.g. PowerPC, Pentium
• Higher level language binding
• Lower latency
• Real-time LAN

Acknowledgments

• Robin Bargar
• Insook Choi
• Mark Goldstein
• Mike Lee
• Dana Massie
• Roger Powell
• Ami Radunskaya
• Xavier Rodet
• David Wessel
• Matt Wright
• Emu, Gibson, SGI