Sound Meter

The Sound Meter is designed for deaf individuals, to help them identify and provide detailed information about various sounds and the direction they are coming from. The product is versatile, usable in various settings, and customizable through different modes for specific needs.

What I did?

Conducted interviews and co-design sessions. I designed storyboards, wireframes, and mid-fidelity prototypes.

Role

UX Researcher, UX UI Designer

Duration

3 months

Tools

Figma, Photoshop, Whimsical

Team

3 UX Reserachers and Designers

1 Codesigner (Deaf and Mute)

Background

Deaf people are unable to hear important sounds can be a significant safety concern. Not being able to hear a fire alarm, for example, can be life-threatening. In addition to safety concerns, there is a sense of isolation and disconnect from the world, as sounds are an integral part of human communication.

Although there are existing products in the market that can detect and identify the sound, none of them can provide an indication of the directions. Some products can give the location of the sound, but because they rely on the Internet of Things (IoT), they can be less flexible for changing environments. Without the information to locate the sound source, it can be frustrating and potentially dangerous for deaf people.

Process

I have worked closely with our participant throughout the design process, from problem identification and ideation to prototyping and testing. These collaborations were crucial for ensuring that our design was on the right track and for making necessary pivots along the way.

Problem identification:

1-hour Zoom interview

Ideation

1-hour in-person co-design session

Prototyping

45-min Zoom design feedback session

Meet Emily

Occupation: ASL teacher
Age: 30 years old

Emily has been deaf and mute since birth and currently lives alone. She relies on video calls with her friend to help her detect any unusual sounds in her home. Due to her inability to hear, Emily is deeply concerned about her safety while at home.

I wish I can notice these important sounds like any hearing people can.

Initial Research

We conducted an 1-hour long interview with our participant in order to gain deeper insight into her daily experiences and challenges as a deaf person.

Assumptions made before session, She might have difficulties

Communicating with people who are unfamiliar with American Sign Language (ASL)

Communicating online on video calling tools like zoom and meets

Interacting with voice-based home devices

Brainstroming

During the interview, our participant was very open to share her experiences as a deaf person, which largely confirmed our assumptions. Together, we identified two areas where we could work collaboratively to address the challenges she faces:

How can we help Emily communicate effectively with people who don’t know ASL in public spaces such as grocery stores?

How can we help Emily to recognize important sounds within the home environment, such as a doorbell ringing, dog barking, or a washer/dryer?

Based on the two challenges we identified, we researched the existing products and brainstormed 10 ideas, from smart dog collars, voice-sensed camera to generative text-to-speech tool. Through internal discussion, we fleshed out two ideas that we plan to share with our participant.

AI Communication Assistant

Hearing spoken language and generating text-based responses for users to select.
Autofilling the remainder of a sentence based on user input keywords, reducing the time and effort required for the user to type every single word.
Speaking out the response for a more natural way of communicating that is easier for the listener.

Accessible Smart Doorbell

Translating spoken words into captions on the user’s phone
Generating responses to assist in the conversation
Speaking out the response to the visitor

Pivot

Our participant expressed that while she liked our ideas, she believed a tool to help her recognize important sounds would be more beneficial.

As a design team, we prioritize the participant’s needs, so we took a step back and quickly shifted our focus to address her specific concerns.

Painpoints

We highlighted several pain points together in terms of recognizing important sounds

Not knowing when/where important sounds are happening

Real-time sound detection

Including sound intensity, type, and the origin’s direction to locate the sound

Not being able to track past sounds

History tracking

allowing users to monitor past events

Needs a tool that can be used in environments such as the classroom and office

Customization

enabling users to tailor sound detection modes to specific situations

How might we help DHH (Deaf and Hard of Hearing) individuals in detecting, identifying, and locating sounds in their surroundings?

To address our user’s challenges, we decided to create a digital product that can be used on both phones and smartwatchs. This product will detect and recognize a wide range of sounds, helping DHH (deaf and hard of hearing) individuals in identifying significant sounds.

Wireframes

Final Design

Real-time sound detection

Identify real-time sound and where it is coming from. Locator directs to the source of the sound and the graph shows the visual representation of sound for a deaf person to visualize sound. view the location of sound on the floor plan as well

Track History

Track past events and sounds and locate them.

Noise Threshold

Receive a notification when the sound level reaches this threshold. They can also preview what common sounds that can be detected in this threshold.

Customize Mode

Customize the mode based on surroundings and requirements. Add and adjust modes and sounds as you like.

Quickly switch mode on smartwatch

When entering a different environment, users can quickly switch the mode on their watches, as well as preview the detail of each mode.

Real-time monitor on watch face

Without going into the app, users can monitor sound information directly on their watch face. This includes details such as sound intensity, type, and direction, as well as various levels of severity that are indicated by different colors.

After designing the mi-fi prototype, we created a story book (link to the story book) and shared it with DHH users in two feedback sessions: one with our participant and another as a public showcase at the University of Washington.

Reflections

Importance of inclusive and accessible design

At every step of the way, I was confronted with the reality of biases that exist. Whether it was knocking on a door or calling participants on the phone, everything seemed to be designed with the hearing person in mind. This experience taught me to identify my own biases as well as the biases that exist in the world. It emphasized the crucial importance of designing with inclusivity in mind. While it is often said that no product can be completely accessible or inclusive, we must nevertheless strive towards this goal.

Going extra mile

Recruiting a participant proved to be a challenging task. Despite our initial attempts to reach out to potential candidates via cold emails, we were unable to find a suitable participant. We were on the verge of switching to a different plan when I decided to take the initiative to visit the office of a potential participant in person. The personal touch turned out to be the key to success, and it marked the beginning of our project.

Have a plan but be flexible

During our second meeting, we presented our ideas to the participant. While she liked our concepts, she suggested another area that she felt was more important. Our team had already agreed to prioritize the participant's input, so we promptly pivoted and delved into her idea. By asking her to elaborate, we learned the value of flexibility and responsiveness in co-design. This mindset allows us to deliver outcomes that truly meet the needs of our participants.

Future Considerations

Ensuring the product has an easy on/off feature
Considering scenarios where multiple sounds may occur simultaneously
Addressing the issue of filtering out strong background noise
Including suggested actions, such as a call-to-action button to call 911 in case of an emergency.