Overview

This project is focused on the experience of a hard-of-hearing individual, specifically on their needs and preferences for assistive technologies. Using participatory design methods, we are collaborating with our participant to gain an understanding of her challenges and goals. Through this process, we’ll identify opportunities to co-create an assistive technology that prioritizes autonomy and customization.

 My Role

Product Designer

Time Frame

10 Weeks

Participant

1 hard-of-hearing person

Tools

Figma, Zoom, Procreate, Miro

Initial Design Discussion

Our session began with efforts to put the participant at ease in a room that met her requirements. The structured session started with team introductions, team session roles, session goals, and a reminder to let us know if she needed any changes to improve communication. An icebreaker activity was used to dispel expectations of polished responses.

Session Goals:

  1. Gather information and understanding of the participants' lived experience

  2. Explore how participant uses assistive technologies to communicate

  3. Define a design problem/opportunity domain. Next was the semi-structured interview with pre-written questions designed to explore how hearing loss affects Suzanne’s interactions with people and events, what technologies she uses, what helpful features, what pain points and workarounds she experiences, and venue challenges.

“I like, I wish, I wonder”

Ambient Noises in Loud Spaces

Improving Audio Communication

Initial Discussion Themes

Theme: Being autonomous in the world

Overview: Suzanne highly values her independence and the freedom to travel through spaces on her own terms. Recognizing that not all environments are fully accessible, she prioritizes self-sufficiency and the ability to adapt her assistive technologies to suit her specific needs, regardless of where she is.

Insight: We see an opportunity to design a solution that empowers Suzanne to move through the world on her own terms, rather than waiting or asking for accommodations to be made for her. That means this design solution could be universally compatible with other assistive technologies or have customizable settings for various scenarios.

Theme: Connectedness & Interdependence

Overview: As much as Suzanne values her independence, she also prioritizes maintaining close connections with friends and family. Suzanne has a desire to feel connected to others without her hearing loss being at the center. She worries about imposing on others with clunky AT tools and often feels her lived experience is misunderstood by hearing people. Suzanne wants to navigate social interactions with comfort and ease.

Insight: We see an opportunity to utilize interdependence as a framework 1 for designing an assistive solution that will help Suzanne comfortably and seamlessly navigate social interactions to feel at ease in social situations and more connected to those around her.

Theme: Options & Customization

Overview: Suzanne shared that people with hearing loss have a range of needs, preferences and accessibility features. To accommodate this, assistive technology should be customizable, multimodal, and employ universal connectivity. Additionally, a service design methodology should be applied to consider the whole experience of assistive technology application.

Insight: Inclusion is increased when users can tailor their assistive technologies to their needs in a given situation. We see an opportunity to accomplish this by shifting to individual control, allowing customization, and providing multiple assistive technology options.

Theme: Inequitable Access

Overview: The lack of accommodations for people with disabilities in public spaces creates barriers to equitable access and participation. This norm is perpetuated by previously installed oppressive systems; navigating through “normal” spaces is inherently exhausting and isolating for folk who aren’t cookie-cutter examples of the people who the space is originally intended for. In the context of our research and participant, this has come to fruition in many different instances.

Insight: We see an opportunity to empower users by equipping them with the right tools to navigate spaces that weren’t originally intended for them.

Project opportunity selection
To close the session, we asked Suzanne to select a project focus area. She chose to improve audio communication in live presentation settings (like theaters and movies). Based on insights from the interview, we refined our focus to enhancing audio-communication assistive technology as an opportunity to increase autonomy for users with hearing loss in settings providing both audio and visual information.

Competitive analysis

Each of these assistive technologies has their own strengths and weaknesses. Some devices rely on physical technology, like the Phonak Roger Select. Others are wireless, like the Bluetooth Auracast. And others can be managed from the user’s phone, like the EasyLine Remote. Some of these devices also have extra features, like troubleshooting support or a built-in rechargeable battery. These technologies also depend on the user’s preferences and use cases (e.g. price, environment they’ll use it in, or whether they prefer not to wear physical assistive devices.) From these existing products, we can infer that attributes like flexible customization settings, troubleshooting, and battery life are important for everyday use.

Co-Design
Key Tasks

1. Getting connected

Getting Connected - You've found your seat and are getting settled 10 minutes before showtime.  Imagine your phone is the hub controlling your assistive technologies. You need to get your device set up and ready to go before the movie starts.

  • Discover how the participant prefers to connect their device to the theater. What sort of signals (e.g. icons, QR codes) would they expect to see indicating that audio and captioning is connected to the right screen.

2. Watching the Movie (lights dimmed)

Watching the Movie - The movie is starting. Time to sit, back, relax and enjoy the show. Let's think about how to make your experience as comfortable and easy as possible. What would you expect to control for an audio and captioning experience?

  • Discover what customization options the user wants for audio: volume, boost dialogue, SFX

  • Discover what customization options the user wants for captioning display: text size, text color, location and positioning of caption display screen

  • Discuss device charging needs

3. Getting Help  (lights off)

Oh no! You're 30 minutes into the movie and suddenly the captions on your device freeze. Let's figure out how you'd solve this problem without a major disruption.

  • Discover troubleshooting steps, trigger for customer service intervention, communication modes and preferences for customer service, desired solutions, and concerns with using screen technology in the theater.



 

1. Getting Connected (Findings)

  • Participant wants one smartphone app to control audio in hearing aids and captioning on a separate screen display.

  • Participant wants an alert confirming that the device is connected to sound and captioning services, theater number, movie title and perhaps start time.

2. Watching the Movie (Findings)

  • Participant wants an app to control audio to stream directly to hearing aids.

  • Volume control should be provided for each hearing aid independently. Slide control is a good UI (like the EasyLine app participant uses now and demonstrated, see Appendix).

Captioning

  • Captioning data is equally important to audio.

  • Captioning should be displayed on a separate device from the smartphone, located at eye level, either on the seat back or a gooseneck connected to the seat.

  • Display multiple lines of text so she can read back if necessary (not a single line of text that disappears).

  • Caption text should be located below the movie.

  • Allow customization of text and background color, text height and number of lines displayed.

Seating

  • Include at-seat phone charging.

3. Getting Help (Findings)

  • The app should allow the user to easily notify customer service that there is a problem.

  • There should be an app chat feature with customer service.

  • A visual signifier should provide feedback that customer service has been notified and that help is on the way (e.g. "we're on the way to seat 34, theater 5 to provide assistance").

  • App to provide visual feedback signifiers for a message received, instructions for the user, and what customer service will do.

  • Participant wants a quick, readily discoverable software interaction to reset the system in case of cation freeze. App should provide feedback for system reset. Participant would try this before contacting customer service.

Final Storyboard Solution

Below our storyboard describes 3 key tasks users would go through to employ our solution at the movie theatre. 


Task 1 - Getting Connected: You've found your seat and are getting settled 10 minutes before showtime.  Imagine your phone is the hub controlling your assistive technologies. You need to get your device set up and ready to go before the movie starts.

Nicole is happy to see that her local theater has a new assistive technology available for all screenings. She books a 12 o'clock matinee to see Wicked and goes to the theater.


Note: Currently captions are provided on the movie screen and showtimes providing captioning are limited. Individual captioning and audio streaming allows all showtimes to be accessible for these users.

When she gets to the theater, she scans a QR code to download the new app.

Nicole gets a confirmation that her device is connected to theater 5.

Task 2 - Watching the Movie: The movie is starting. Time to sit, back, relax and enjoy the show. Let's think about how to make your experience as comfortable and easy as possible. What would you expect to control for an audio and captioning experience?

While she's waiting for the movie to start, she's able to customize her captioning settings for the best experience.

The movie has started. Nicole reads the captions from the seat back screen in front of her.


Note: Our participant went back and forth on preferring a seat back screen vs. reading the captions via screen in a gooseneck holder. Further testing with more participants would allow us to gain more insights and confidently choose a direction.

Nicole notices the volume on her hearing aids doesn't feel right. She adjusts the volume on her app.


Note: During our sessions we learned hearing aids can be adjusted independently. Our participant preferred a slide bar UI but other UI should be tested with a wider participant pool.

Task 3 - Getting Help: Oh no! You're 30 minutes into the movie and suddenly the captions on your device freeze. Let's figure out how you'd solve this problem without a major disruption.

Oh no! The captions suddenly freeze.

Nicole tries to solve the problem on her own by restarting the device.

Restarting the app didn't work. So, Nicole clicks a help button and is notified that help is on the way.

 

The employee pushes an update that fixes the screen. They offer her a snack coupon for her next visit.

Nicole enjoys the rest of her movie.

 

Participant Feedback

Method

In order to meet Suzanne’s communication needs, we chose to email her our storyboard and follow-up questions. This allowed her to spend more time with our content in a way that was more technologically accessible to her. 

Participant Feedback 

  1. Suzanne rated the final design solution a “5” due to how accurately and sensitively we captured her challenges.

    • In her email, she described feeling “heard and understood” while working with us. The most valuable experience was role playing the experience of being in a movie theater. Our questions and suggestions for different approaches to assistive technologies helped her define her desired solution. However, she would have liked more time to work collaboratively toward a solution.

  2. She couldn’t think of anything to change in our final design solution or in her time working with us. 

Questions

  1. On a scale of 1-5 (1 being “Not at All” and 5  being “Very Well”), how well do you think our final design addresses your need for more accessible movie-going experiences?

    1. Please explain why you gave that rating; what worked well, what did not?

  2. If you had a magic wand and could change anything about our design, what would it be? Why?



Next Steps

Our session revealed many features of physical space design that we would need to research, prototype and test to further develop this solution. Given that we have been working with only one participant, any market application would undergo wider user testing.

1. Prototype different form factors for screen placement.

2. Research lower-cost solutions for theaters that don't require the purchase of new seating with in-seat captioning screens and chargers.

3. Prototype and test accessible seating areas with chargers and seatback screens.

4. Prototype and test advanced volume controls (e.g. dialog boost) capability to deliver more consistent speech intelligibility vs relying solely on captions.