iventurelab.com    
 
 

> ENTERTAINMENT MAIN PAGE
> ENTERTAINMENT SPEAKERS
> ENTERTAINMENT FEATURES

> PANEL HIGHLIGHTS

> BROADBAND/OPTICAL
> MICROELECTROMECHANICS
> WIRELESS
> ROBOTICS
> FUTURE TECH/SOCIAL IMPACT
> SECURITY
> CONSUMERS & TECHNOLOGY
> HEALTHCARE & TECHNOLOGY
> NEW MEDIA/ENTERTAINMENT

 

 > 
TECH HEADLINES



 > 
THE JAM-O-DRUM

Goal-driven Audiovisual Experiences
Entertainment Technology Center
Carnegie Mellon University

Project Team: Donnie Antonini, Cliff Forlines, Randy Hsiao, Ning Hu, Ian McCullough
Advisor: Tina Blaine

Introduction
Phases I and II of Jam-O-Drum development occurred at Interval Research ranging from July 1998 to January 1999. The original goals of those phases were centered on the desire to change the context of music making into a casual social setting so that novice players could have the experience of musical interaction in an ensemble.

Building upon the earlier development goals, Phase III sought to incorporate additional game elements so that the musical experiences might be more structured and goal-oriented. To that end, over the course of a six-week period, we developed two new experiences for the Jam-O-Drum. Those two experiences were dubbed "Rhythm Breaker" and "Circle Maze" for development purposes.

View Rhythm Breaker movie (5.4 MB) | view Circle Maze movie (2.8 MB)

Rhythm Breaker

Overview
Rhythm Breaker is a competitive musical game. The players are expected to play the drum pads matching the rhythm of a backing track. If a player is playing well, waves, or ripples, are emitted from his/her pad. The waves push a moving object, referred to as the puck, away from the player. The objective of the game is to push the puck off the edge of the table, and if the puck is pushed off the table on a player’s side, s/he loses.

There are three main parts to Rhythm Breaker: the graphics and animation, the rhythm checking, and the backing track and rhythm. Rhythm Breaker is an experience integrating visual and audio effects, hence the animations and music. Rhythm checking evaluates the players’ performance in terms of how well they match the rhythm while they are playing.

Original Development Plan
Rhythm Breaker was developed using an Apple PowerMacintoshG4. (See Appendix 2 for a full hardware diagram.) We decided to use Cycling 74’s Max as a key element in our programmatic infrastructure. Max is an object-oriented application-programming environment that allows a programmer to drag objects onto a window and connect the objects with lines to create applications. It is designed specifically for developing music applications, since it has extensive capabilities dealing with MIDI messages. Max also allows a programmer to develop external objects that can be used in Max applications. Rhythm Breaker’s external objects were written in C.

Two programmers were assigned to take responsibility for Rhythm Breaker. One focused on the backing track and rhythm composition, while the other handled the graphics and animation coding.

Graphics and Interaction
The original intent was to use Max for the MIDI interface, the graphics, and the rhythm checking. After encountering certain limitations, the graphics and animation were reimplemented with Apple’s QuickDraw API, and the rhythm-checking algorithm was transferred entirely to C code.

During early user testing, a critical issue was that the players did not understand the purpose of the application without explanation when they saw it for the first time. To increase the program’s intuitiveness, the proposed solution was to combine Rhythm Breaker with Conductor — another concept that emerged from brainstorming. Conductor was also based on rhythm matching. It differs from Rhythm Breaker in that there is a stationary "conductor" at the center of the table, that sends out visual cues to the pads to signal the players when to hit. Since it takes time for the visual cues to move from the center to a pad, the players have time to prepare before they hit the pads.

Further user testing revealed that the solution had a positive effect. Nearly all of the players understood that they were supposed to hit the pads when the visual cues reached the pads, and many realized the cues were timed to match the background rhythm.

The next step was to enrich the overall visual experience, and to implement more performance feedback for the players. A series of textual cues were put in place to keep players how they were doing. If the puck leaves the table on a player’s side, a message is displayed on his/her pad reading, "Come on! You can do better!" The intent of this is to encourage rhythmically challenged players to keep trying — hence a message of encouragement as opposed to a reprimand.

A scoring system based on the players’ rhythmic accuracy was also added so that an ultimate victor could be declared at the end of the game. The players are able to see their scores during the game and text messages are also given as visual feedbacks to the players based on how well they are keeping with the beat. For example, if a player hits a pad and matches a beat perfectly, an image of the word "PERFECT!" scrolls upward from his/her pad.

There was also some experimentation as to how the waves should behave. One idea was that the waves would go all the way across the table, but the strength of the wave would be determined by how well the player was matching the rhythm — that is, the more accurate the hit, the thicker the wave. The other idea was that the waves would only move a distance proportional to how well the player is doing — so if a player was doing well, his/her waves will travel farther. Based on user test results, the former idea was implemented, since the latter was confusing for some players.

At this point, QuickDraw began to reveal it’s technological limitations, so the graphics and animation were reimplimented again with Apple’s QuickTime. The use of QuickTime permitted the opportunity for more bitmap images, as opposed to simple sprites, to be used in the application. In effect, this allowed the creation of customized visuals — or "skins" — for the application. By choosing appropriate background music, the Rhythm Breaker game could now be themed differently depending on the environment in which it is place. We created three skins for the game: a standard skin, a Halloween skin, and a skin based on the arcade classic "Pac-Man."

Finally, beginning and ending states for the game were incorporated. During the game-start state, the pads will flash with the message "hit the pad to start". The application would passively wait for a player to hit a pad to begin, which is ideal if the Jam-O-Drum is a stand-alone console because if no one is playing the Jam-O-Drum, the application will simply sit there idly, without needing someone to monitor the process. The game is over and a victor is declared when the song ends.

Music and Audio
Composing for Rhythm Breaker’s backing and rhythm tracks proved especially challenging. We maintained the original project parameter of building experiences that are enjoyable for novice and skilled musicians alike.  The original scheme had different pieces of the background rhythm delegated to different players in order to try and make the entire rhythm a combined product of all of the players’ efforts. User testing revealed that most players perceived that there were three completely different rhythms that didn’t make much sense together. The composition was simplified so that all three players were given a single, unified rhythm to match.

Ultimately, we created two rhythm tracks. One rhythm was used for the standard and Halloween versions, and another for the Pac-Man version. Both rhythms have the characteristics that they are very simple and easy to follow initially, and the same rhythm is repeated several times before switching to a more challenging one.

Audiovisual Integration
A major challenge was striking an appropriate balance between the visual and audio elements. After the "conductor" component was integrated into the experience, most players focused their attention almost exclusively on the visual cues. A significant number felt that if they had diverted their attention to elsewhere on the table, they would not be able to follow the rhythm. Simplifying the musical tracks, starting simple, and pacing the changes started to help, but continued tuning is necessary. The key here is finding a rhythm that is very simple at first, repeat it for a long time, and then gradually increase its difficulty. This would make the players focus on the visual cues initially, learn the rhythm quickly, explore what else is going on, and then, if necessary, have the visual cues available as a crutch when the rhythm becomes more difficult to follow.

Future Directions
Rhythm Breaker does require additional development. It ultimately should be a stand-alone application that does not require an external application such as Max to run. Also, currently, when the game ends, an attendant needs to quit the application and restart it for the next game. Some mechanism should be built in to have it automatically restart, just like any other arcade game. Rhythm Breaker should also be made more customizable in that one could choose what kind of skin or theme to use at the beginning of the game. The biggest challenge with Rhythm Breaker, however, is tuning the interaction so that the players will be more aware of the fact that they are trying to match the rhythm of the backing track rather than just matching the incoming visual cues.

CircleMaze

Overview
Our high-level goal for this experience was to design a game that furthered communication and collaboration among its players. The task for the players is to collect all of the CircleMaze’s pieces in the middle of the game board. Because the maze is divided into three concentric rings, the pieces must pass through each ring in turn; thus, all three players must work together to achieve this goal. Movement of the pieces and rings produce changes in the musical score; therefore, players of the CircleMaze are involved not only in a game to collect all of the pieces, but also in a collaborative music making experience.

Each of the three rings is filled with maze-like paths, some providing the pieces a route from one ring to another, some looping the pieces back into the ring from which they came. Each of the three players is in control of the rotation of one ring. Players turn their ring through turning a wooden disk attached to the side of the Jam-O-Drum. The rings affect one of three musical tracks through their absolute rotation — as a player turns their ring, they mix their track from among several complementary loops.

At the beginning of the experience, a number of pieces are distributed throughout the outer ring of the maze. Each of the many pieces moves blindly through the maze, pulsing to the rhythm of the music. If the paths from one ring to another align, a piece will cross into the new ring; however, if a piece hits a wall, it will simply reverse its direction. When pieces enter the center circle, they cease moving but continue to pulse to the beat.

The action of turning the circles back and forth clearly lends itself to the scratching of a DJ’s record. The game’s music fits with this metaphor. Each player is in control one of three complementary tracks of music — the bass, the beat, or the melody. We chose Techno music for this experience because it lends itself to looping and to the blending of multiple tracks. Its high energy and fast pace fit in nicely with several qualities of the experience. It has the additional quality of being somewhat international, belonging to no particular ethnic group.

The only requirement of the players is that they are able to work together as a team. No musical experience is required as any movement of the rings produces a complementary musical effect. Hopefully, many individuals who might not approach an instrument because of their fears of performance will approach the CircleMaze because of its game-like appearance. Only after playing through the maze might one realize they have been playing an unusual instrument.

User Testing and Revision
Testing out the CircleMaze on people was an important part of the development process. Through this testing, we were able to identify some of the problems with the initial design and make improvements to the experience. On four separate occasions, we brought in a wide range of individuals to play through the game. Below are descriptions of several of the issues and opportunities that arose.

Interaction Confusion: In early rounds of testing, our players did not recognize the wooden disks as interaction devices. Without the ability to turn the maze’s rings, they quickly became frustrated with the game. This confusion was only amplified when we ran these players through another experience first, one in which they interacted with the table’s surface directly.

Rather than allowing the animated images on the surface of the Jam-O-Drum to continue to compete for the player’s attention, we turned off the projector at the beginning of the game until the players made their way to the now brightly painted, black-lit disks in front of them. After making these changes, few individuals had trouble finding the interaction location.

Rotation Confusion: Several players expressed a misunderstanding of their influence on their ring. Instead of seeing rotation around the center of the board, they saw the maze as "reconfiguring" itself as their turned their disk. Instead of working to align the paths between the rings by rotating them into position, they worked at finding the "right" combination of rotations, describing the game as a "combination lock."

We addressed this issue by animating the rotations, a change which improved players understanding of the interaction greatly. Furthermore, we would like to replace the solid color backgrounds of the rings with non-uniform imagery — a change that should further help players understand the movement of the rings.

Musical Control: In early rounds of testing, we had not implemented the player’s influence over the background music. Interestingly, many players incorrectly assumed that they did have control. One player commented, "I loved how when I turned the dial, the music changed!" when in fact there was no change at all.

As we intended to implement this functionality from the beginning, we felt that our players were primed to understand this interaction; however, when we did add this feature our players stopped responding to it. Our best guess is that as the development of the game progressed, the CircleMaze attained a higher level of visual richness that overwhelmed the players’ audio sensitivity. Finding a balance between the visuals and the audio of such an experience remains a topic for further development and study.

Goal of the Game: In the early development of the CircleMaze, we were very worried that players might not understand the goal of the game. We were so worried that we spent a good amount of time brainstorming on how we might teach people to gather all of the board’s pieces in the center of the maze.

Through testing, we found that 53 out of 54 players correctly identified the goal of the game, without any elaborate help from us.

Reference Points: From the beginning of the development, we were intent on creating an experience that forced people to work with one another to achieve a goal. This interdependence is the most important facet of the game; it requires a high level of communication among the players. We quickly found that while we were requiring a lot of talking, we were not providing a means to reference a particular game piece or location on the board. We heard many exchanges among the players of the type, "Lets work on that one. No! That one. No! That one!" Interestingly, while the entire surface of the board was well within reach of all of the players, many individuals did not seem to want to reach into the projection space to point out a particular piece.

Our partial solution was to give all of the pieces a unique color. Players quickly began to focus on "the red one," or "the yellow one." Furthermore, we would like to replace the solid background color of the rings with images that contain visual landmarks, so that players can talk about "the one on the bridge" or "the one by the tower."

Switching Ring Control: In early testing, several players expressed regret that they had the "worst ring." Interestingly, depending on whom we asked, the "worst ring" was sometimes the middle one, sometimes the outer one, and sometimes the inner one.

Whatever the case, we now switch the position of a player’s ring after each round. The individual, who felt cheated having been given the middle ring in round 1, should feel better after controlling the outer ring in round 2, and the inner ring in round 3.

Controlled Exposure: Some of the first words out of several players’ mouths upon seeing the CircleMaze were "Yikes! This is madness!" and "This is so overwhelming!" Initially we were starting them out on a new experience and asking them to not only learn a new interaction technique on a new device, but also asking them to handle the stress of leading 30 pieces into the middle of the board.

The solution we found is to limit their exposure to the facets of the game. Presently, a group of players is given a single piece at the beginning of the experience, so that they can more easily learn the rules of the game. Once they have some practice working together and controlling their rings, they move on to 2 pieces, then 4, and so on.

The End: A problem that we were thrilled to find was that several groups we tested did not want to leave at the end of session. With other groups, interest dwindled slowly but surely over time. To address both of these issues, we added a pair of high-level game elements.

First, we implemented a game clock. Shown at the center of the maze, the game clock counts down the remaining time the players have to clear the board. As pieces enter the center, the players are rewarded with additional time. This game element heightened the level of urgency felt by our players. If the players run out of time, they restart the current level.

Secondly, we limited the number of levels to 5 so that there is now a fixed end to the experience. Instead of continuing onto infinity, a successful group of players are given the total time they took for all five rounds so that they might play again to beat that combined time. We feel that this end will encourage players to replay through the experience, instead of overplaying it in their first encounter.

Future Directions
We would like to continue the development of this experience through further user testing and revision. Like the Rhythm Breaker, the CircleMaze should ultimately become a stand-alone device, requiring little of an attendant’s attention. One avenue of exploration that we were unable to pursue during the course of this project was visual skins. We feel that replacing the solid color background of each of the rings with some form of imagery might allow for the customization of the experience for a particular group. Furthermore, such images would aid players in the communication process — they would be better able to reference particular points on the board if they could talk about their unique characteristics. Further work is required in the development of the game’s control devices. The current hardware implementation fails in that it cannot register a disk’s rotation until the player has already turned it; thus, any animation we add to the movement of the rings adds to the visual lag. A more accurate input device is needed.


For more information:
http://www.etc.cmu.edu/projects/jamodrum/index.htm


Gold Sponsors
iventurelab.com
diamondcluster
Marconi
morganlewis.com
Silver Sponsors

REGISTRATION FEES

Business Professional $150
CMU Alumni $ 75
Students / Faculty / Staff $ 30

PVCA Luncheon (for non-members):
Business Professsional $ 35
Students $ 15

Contents © 2000 & 2001 Carnegie Mellon University
Contact: mailto:info@cmuinterface.com