This overview describes work at the Center for New Music and Audio Technology (CNMAT) on application of hypermedia tools to musical subjects.
Context: The audience interacts with material stored in HyperCard through nine inch Macintosh computer screen by touching it or by clicking a mouse button. The sound quality of what they hear is that of CD's. The metaphor for this presentation is a book format.
Description: The navigation and play buttons on the lower left of every page form a control panel similar to the transport control of a tape recorder. Since this work is about music, it is possible to hear music at any time and the sound and visual material mutually support each other. Sound is only played when explicitly requested.
The musical "illustration" for the cover page of the "book" (above) is a representative and recognizable song, Crossroads, by Robert Johnson. Clicking the mouse over the guitar causes the first guitar break of the song to be played. Clicking the mouse over the bluesman's head plays a verse starting with Robert Johnson's distinctive falsetto howl. These hidden sources of sound reward the user's explorations of and interactions with the material.
The next chapter in this multimedia "book" is on Blues artists. Two different artists are covered in this prototype: Skip James and Robert Johnson.
(The illustration for the Robert Johnson page was created before photographs of him were "discovered").
Clicking the play button in this case results in a segue to a page in the next chapter which is a collection of songs of these artists.
Prose is traditionally used to explain how the railroad theme figures musically and lyrically in blues music. Connecting the touch of an image of a railroad (above) to the appropriate song materials is a concise, effective way of communicating the same idea without assumption of the user's command of a particular written language.
The dagger + is used in the text above to flag links between the text and the rest of the work. These are the links for which the term hypertext was invented and then - by extension to richer media - the term hypermedia is now in use. Each line of lyrics is indexed to the corresponding sound recording of the song. Touching or clicking the mouse on any line of the song will cause that line to be played. Pushing the play button for the page plays the whole song. Clicking "2 step rhythm" causes a fragment of sound to be repeated while the mouse button is held down. The fragment has been chosen to emphasize the rhythmic basis of the song. The music and tablature notations for this rhythmically representative fragment is included on the same page.
A particular feature of the guitar playing (octave slides) and the whole guitar break can be rapidly auditioned. The first recording of the song is available by clicking on the date, in this case 1931. The user never has to wait more than a second to audition a sound fragment and in most cases the delay between his gesture and sound is barely perceptible.
On the page of Robert Johnson's Crossroads links are provided to a page describing Cream's version of this song, which can be auditioned. These kinds of comparisons can be added as new materials and connections become available by the original author, the user or teacher. The ability to rapidly move through these representations of the space and time of this music and culture has been the most compelling feature to people using this project for the first time.
Context: The audience interacts with materials stored on the computer in a program called SuperCard through a color Macintosh display. Considerably more musical material is available than in the first project through use of a computer controlled jukebox of CD's containing all of Skip James released recordings.
Description: With a larger color display, I have addressed the weakness in the still images in the first prototype. This allows the display of legible maps, photographs and musical scores.
A map of Mississippi is used to illustrate the artist's birthplace - it could also show his travels and places referenced in his songs:
Since considerable interview material is available I decided to let the artist speak for his own music. I indexed three interviews. Blue highlights can be touched or clicked on to take you to a song or fragment referred to by Skip James in these interviews. Since the tapes may still be available for some of these interviews, the possibility exists that this sound material could be made available with the text, increasing comprehension of the oral presentation.
Context: Interaction is with material stored in a program called Max through a large 19 inch color display which allows superposition of computer images (Rasterops video) with live video from a vcr, video disk or computer stored digital video. This is in addition to the CD quality sound material.
Description: This work in progress explores the use of moving images using computer controlled digital video. The idea is to not only expose the audience to the music and history, but to use moving images to also communicate the experience of the performance. Stock footage of blues artists is available with synchronized musical score page turning, bouncing ball and lyrics. Close-up shots of the left and right hands of a guitarist playing the songs is synchronized with the original sounds. The computer may be used to slow material down to make it easier to learn. An animated tapping foot helps communicate the rhythm.
Freed, A. "New Media for Musicological Research and Education -- The Country Blues in HyperMedia", Proceedings of the 15th International Computer Music Conference, Ohio State University, 1989, Computer Music Association.
Freed, A, Wessel, D., Zicarelli, D., "MAX Objects for Media Integration", Proceedings of the 17th International Computer Music Conference, Montreal, 1991, Computer Music Association.
Lee, M., Freed, A., Wessel, D. "Real-Time Neural Network Processing of Gestural and Acoustic Signals", Proceedings of the 17th International Computer Music Conference, Montreal, 1991, Computer Music Association.