Practice is a vital part of a musician's life. Advanced musicians can spend over 4 hours a day practicing alone. This time is spent on refining their technique for a specific piece. Additionally, they coordinate with their group members to find a common time for practicing their performance, and play in sync.
How can we make it convenient for music students to practice a group composition, in spite of their conflicting schedules? Guided by this question, our design process and iterations led to Musi: a foot-controlled recording device, that makes it easy to reflect on, and refine their technique during solo practice, and also synchronize their segments with other member's recordings.
My Role
I was part of a 4 member team for this semester long project as part of the Pervasive Interaction Design course. I was involved in all stages of the process, but I am particularly proud of my inputs in designing the cultural probe, refining the form factor of Musi, and building the working prototype.
Fall 2017, 12 weeks
UX Research
Product Design
Electronic Prototyping
Full-stack Development
Wei Cai
Chelsea Miller
Shu Zhou
Observations and Online Research
We started off by brainstorming possible audiences for the course project, and agreed on two possibilities: musicians, and people working out at the gym. Then, we staked out a the gym and the music school at different times, and observed our audience. Additionally, we looked at online articles and videos for more information about our audience.
When we reconvened as a team and shared our notes, we found a few commanalities between musicians and gym goers: importance of form and technique, lots of repetition, and injuries. We decided to narrow our scope to musicians based on group interests.
To anchor our findings, we brainstormed and sketched preliminary product concepts, and selected the 3 options we found more interesting: An instrument that invites contribution from audience, smart clothing that enhances the expressions of the performer, and a music stand that helps with form and feedback to a musician that is practicing alone.
Cultural Probe and Interviews
Next, we worked on getting a better understanding of our users, their background, their choices, and their vision for the future of music. Broadly, we wanted to know how can we help musicians through technology?
We created diaries with prompts that asked participants to dream big about the future of their music—but what we learned was that classical musicians aren’t looking for radical transformations of their existing routines. Musicians value their practice spaces, their instruments, their teachers and mentors, and their friends in their ensembles—and in their cultural probe responses, dreamed up subtle ways to strengthen their bonds with their musical community.
We also found that, when asked to dream up their ideal instrument, our participants similarly did not want to make drastic changes to their primary instrument; They did, however, express a desire to be able to play a secondary instrument as accompaniment during practice. This insight led to inspiration for the concept that would eventually grow into Musi: a foot-operated mat that musicians could use to record and play accompanying music.
We also conducted two interviews, one of which was with Prof. Wakefield, an expert in music technology. His insights directly affected our final concept. In particular, he asked us to tread carefully around the use of technology to teach music, because it would require highly accurate sensing as well as sensing everything that a human teacher would observe. A teacher should also be able to guide the student in setting the context of the musical piece and emotional expression. This level of research and design cannot be done in one semester. He suggested that we should focus on easing practice.
One interesting approach would be to focus on something that plays accompanying instruments, eases my practice
Based on the insights from this study, we sketched new concepts which were much more grounded in terms of acceptability, scope, and impact.
User Enactment Study and Final Concept Proposal
Based on the feedback from our peers and discussions among team members, we decided to focus on the concept of a musical mat that would support classical musicians practicing alone by connecting them musically with their instructors and peers.
To test our concept, we created open-ended low fidelity prototypes of the musical mat using paper and cardboard. Then, we recruited participants to enact various practice-related scenarios, try the prototype, and share their thoughts. We also invited them to rearrange the mat’s shape and button layout while acting out the scenarios. These participatory prototypes helped us identify that different types of instruments would have very different design requirements and constraints, and learn about additional opportunities for our concept to increase motivation in the practice room.
Altogether, our formative study led to these 5 key insights:
- Practice requires significant repetition as well as meaningful reflection
- Group rehearsal is more difficult to coordinate, but is equally important to solo practice
- Existing live remote collaboration tools for musicians suffer from network latency
- Musicians playing instruments with their hands want to make use of their feet for secondary tasks
- Musicians playing instruments with their hands want to make use of their feet for secondary tasks
Final Concept Design
Musi is designed to help musicians make the most of their practice time. Some of its key features include:
- Record and playback practice sessions
- Set and adjust tempo for metronome and recordings automatically with foot- tapping
- Automatically upload recordings to the cloud
- Companion app to load companion music, share recordings with others, and comment on recordings
Musi is designed to be portable. Musicians are busy and already have to carry around their instruments; Musi removes the hustle of carrying all those speakers, microphones, laptops, and other devices currently used for recording. It can be folded, carried, and worn on your body like a bag. The mat and prism can be separated and placed at different places to adapt to the positioning of various instruments. Imagine: while playing the cello, you could put the mat on the floor while the prism sits on top of a table near you, without getting blocked by the large body of the instrument.
Prototype
With our working prototype, users can experience several of Musi's key features:
- Playback: Users can play preloaded or recently recorded tracks by tapping the "Play" button, second from the right. They can stop the track by pressing the button again.
- Recording: Users press the center button to start recording. If they've preloaded or recently recorded a track, it will play in the background while the user records the second part overtop it. They can stop the recording by pressing the same button again.
- Metronome: Users press the button to the left of the recording button to start the metronome. They can stop the metronome by pressing the same button again.
- Track navigation: Users can press the left-most button to rewind the track, or the right-most button to fast forward.
We invited musicians passing in the Earl V. Moore practice wing to try out Musi. In the video, see a student using Musi to record and playback a duet.
Reflection
After building our prototype, we evaluated our design with stakeholders in two contexts: at the Design Expo at the School of Engineering, where we were able to observe visiting high school students use the system; and at the practice room of the Earl V. Moore building, where we were able to observe an undergraduate music major use the system. From these evaluations, we were able to identify areas to drive future design work and help us realize an ideal concept.
Flexibility
Throughout our design process, we had considered how we could create a physical device flexible enough to meet the needs of differently sized and shaped instruments. We were pleased to find that whether recording with voice or a large instrument like a cello, Musi was flexible enough for users to adapt it to their ideal positioning without sacrificing their technique and posture.
Construction and Maintenance
We created our prototype out of wood and vinyl to evoke the materials and aesthetic commonly found in classical music instruments, cases, and performance areas. While the prototype held up after repeated use, we found - unsurprisingly- that it got dirty very easily. Both the practice room floors as well as the users' shoes presented Musi with a significant amount of mud, dirt, and dust. As a foot-operated device that will typically be used on the floor, Musi will need to be constructed out of material that repels dust and is easy to clean for effective long-term maintenance. This would also benefit Musi when it is carried through wet weather conditions - the current wooden build is unlikely to safely protect Musi's inner components from rain and snow.
Sound quality
While we do not intend for Musi to compete in the high quality audio recording equipment market - it serves as a more casual recording tool that mainly supports practice - it is still important for Musi to record and playback tracks clearly. We used a laptop's internal microphone and external Bluetooth speakers to demonstrate Musi's recording and playback features. We found that while this setup recorded one track cleanly, when participants recorded a second track overtop, they easily maxed out the gain in the recordings, leading to unwanted distortion. Higher quality speakers and microphones would benefit Musi, but even more important is the opportunity to automatically adjust sound input settings. Additionally, the current prototype currently records the metronome if it is active when the users is recording. However, participants indicated that they would not want the sound of the metronome audible when playing back their tracks. Recording with headphones would be one way around this issue, though it would be worthwhile to explore additional design alternatives.
Visual Feedback
Musi's buttons provide an audible click when pressed. Still, our current system interface does not provide visual feedback on the system status: e.g. when the system has started or stopped recording. Additionally, while participants were able to quickly learn how buttons were mapped on the device, the device lacks icons or text indicating the function of each button. Adding some visual signifiers would aid both novice users as well as more advanced users who are focused on their music, not memorizing the arrangement of buttons.
Moving forward, Musi would benefit from iterating the design of Musi with these insights in mind, and undergoing a more extensive usability evaluation.
Ideal System Concept
Our ideal Musi of the future would build upon our existing design by incorporating:
- Higher quality microphones and speakers for improved recording and playback quality
- Automatic adjustment of gain and sound levels based on volume and other attributes of the recording instrument
- Weatherproof construction and sturdy, washable materials
- Streamlined integration of Musi prism with companion app
We hope that with additional evaluation and iteration, Musi can enter the practice room of musicians around the world to permit individuals to practice with their ensemble anytime, anywhere—at the tap of a foot.