Kinect Drums - Sketches and Storyboard

Transcription

1 Kinect Drums - Sketches and Storyboard Team members and primary contributions: Nathan Bakirci: 3/12 sketches, interface sketches introduction, one refined sketch Brandon DePalo: 3/12 sketches, storyboard sketches, one refined sketch, formatting Erik Lui: 3/12 sketches Bob Zimmerman: 3/12 sketches, one refined sketch, note taker for design critique, learnings from design critique section, text for task description/storyboard section, formatting, proofreading Abstract With the prevalence of modern video game systems and the Kinect inclusive Xbox One being released soon, users will likely have more ready access to a Kinect than a large, expensive drum set. Previous digital air drums have tracked when motion breaches a particular part of the monitored area, and played a sound accordingly. But with the human body tracking capabilities of the Xbox Kinect, the system could be used to more accurately map the movements of a user to analogous drum like motions. A successful implementation of this idea would be one that does not require expert drum knowledge to be able to use, but which would provide a suitably realistic and engaging simulated drum experience for a novice user. This application would be of the most use to inexperienced drummers who are interested in trying their hand at drumming, but do not want to spend the extra money on buying an entire drum set. Introduction As with many instruments, learning to play the drums requires access to a set on which to practice. The cost of this, especially for an instrument as large as the drums, can be prohibitive to many prospective players. Additionally, the drum set is one of the largest of popular instruments, and requires a great deal of floor space to house. For these reasons, we have chosen to create an application based on the use of Microsoft s Kinect motion sensing technology, in order to simulate a realistic drumming experience without the need for the physical equipment. While many programmers have created prototypes of simulated drum sets with a similar idea in mind, there are few released to the public, and fewer still based on user gestures rather than monitoring designated areas of a screen. Our goal with this project is to create a virtual, customizable drumming simulator that will be comparable in experience to playing an actual drum set. There are of course, some limitations: there will be no tactile feedback, nor will the player be able to make use of their physical contact with the drums (say to create a drum roll, mute a cymbal, or perform a rimshot on the snare); however, we feel that we can deliver a rudimentary, yet satisfactory analogy to an actual drumming experience. The success of this will be confirmed by testing with novice users comparing their experience with what they anticipate actually playing would feel like.

2 Task descriptions and storyboards For our storyboard, we decided to take our fictional user through a few tasks that would utilize many of the core functionalities of our application. We brainstormed ideas on a google doc that were functions related to simulated drumming experience, and then picked what seemed to us to be the most obviously important ones to the application. The user would certainly have to play the drums a bit, and in order to do that, they would have to start and calibrate the software. Once that was completed, we wanted to emphasize the customizability and interactivity of the program, and have the user customize the drum setup and record their own beat on the new layout. Task 1: Start and calibrate the software Calibration Interface In the calibration interface, users perform basic Kinect functions so that the application can determine where they are relative the to the device so that we can correctly scale the interface. Since we don t know how far the player is going to be from the screen, we need calibration so that the drums won t be too small or too large, and so that the user will be able to reach the edges of the screen in order to drag elements where they want while customizing. Scoryboard scene 1: initial user location/primary usage area

3 Scoryboard scene 2: title/intro screen Scoryboard scene 3: basic Kinect interaction to make sure the Kinect is present/functioning

4 Scoryboard scene 4: user calibration Task 2: Customizing and playing the drumset Main interface (playing the drums) The main interface is where most of the interaction with the application will take place. Here, users can play whatever beats they can come up with either on one of several pre designed drum setups based on various musical styles, or using their own custom setup. The main screen will have a live feed of the user playing, overlaid with semi transparent images of the drum pieces on the screen, which will become opaque when struck to provide the user with feedback. There will also be overlays atop the user s hands and feet to illustrate the Kinect tracking and how the user is intended to interact with the drums. Customization interface The customization interface uses drag and drop interactions to allow the user to place however many of a drum piece they want wherever they want on the screen. Once they are finished, they simply grab the edge of the customization bar and push it back into the side of the screen where they got it from, and the application will save their layout for future use and they can continue playing. The customization interface will also allow users to interact with the drums and cymbals they have already placed, to make sure they have the sounds they want where they want them.

5 Scoryboard scene 5: display of/user introduction to initial drum kit setup Scoryboard scene 6: primary user interaction with application

6 Storyboard scene 7: opening the customization interface Storyboard scene 8: using the customization interface

7 Task 3: Recording a beat Recording Interface The recording interface is similar to the customization interface, it will be able to be dragged from the side of the screen. This will allow users to select one of the four buttons by holding their hand on top of it and making a fist. While the user is recording, the record button will flash. When the user is done recording, they will select the stop button to stop recording and then can play back what they just recorded using the play and pause buttons. Storyboard scene 9: switching from the customization screen to the recording screen

8 Storyboard scene 10: user interacts with recording interface Storyboard scene 11: user recording a sequence of beats

9 Storyboard scene 12: user ends and saves recording Storyboard scene 13: user shuts off application and exits scene

10 Initial interface sketches The main goal of our primary interface is to offer a fun and intuitive way to play the virtual drumset. Feedback is a major concern that we have for the primary screen, so that users know exactly what drum they are hitting and how hard. We also want the experience to be consistent and as close to a real drumset as possible so we explored several options, like whether the user sees a video feed of himself or some other kind of representation, whether the drums are actually pictures of real drums or just colored areas, and how the user can customize their drumsets. For the secondary screens, we wanted ways for the user to calibrate the area and range of motion for the kinect, and also an effective customization screen. We want both to be consistent with the playing experience and intuitive to use. Our approach for the sketches was to split up and individually do three sketches each so we could come up with our own ideas first without being influenced by each other. We felt this was best instead of trying to do them all together because it seemed more likely to produce more unique interfaces. Everyone was tasked with doing at least one primary interface and one secondary interface sketch and then we divided the rest based on what was left. After looking at all the sketches we had created, we made a few extra drafts to help diversify the collection since some of our ideas had naturally overlapped. From this final collection of initial sketches, we commented on each others ideas to add and move some features, and made some notes about what we should change for the refinement process. Main interface sketches (playing the drums)

11 Main sketch #1 This is a main interface that is designed to use hand motions only to actually play the drums, while a controller or mouse would be used for other interactions. Feedback is given by the drum flashing and vibrating as it is hit. This interface focuses on recording and playing back recorded music. Main sketch #2 This interface is primarily used to show users what notes they have played, with the possibility of showing what notes should be hit next to play along with a selected song. There would ideally be different modes for the top portion of the screen, which would be selected using the view tab. There is a live video feed that shows where a user is hitting on the virtual drumkit which is represented by resizable and movable squares. The top arrow that has no caption is meant to indicate an empty circle is a missed note. There is also a bottom caption that says that the person will be a live video feed of the person from the kinect.

12 Main sketch #3 In this interface, drum pieces are semi transparent until struck by the user, at which point they become opaque to provide feedback. Pieces are also centered on the user s torso to keep them all within reach, but stable. Note: the top left corner got cut off in scanning, but says: 2. foot motion actually tracked with the knees