Making short-form videos accessible for the Blind and Low Vision Viewers
Short-form videos on TikTok, Instagram Reels, and YouTube Shorts remain inaccessible to millions of blind and low vision (BLV) users. I joined the Human-Computer Interaction Lab at the University of Texas at Austin to help address this challenge.
Challenge
Making short-form videos accessible for the Blind and Low Vision (BLV) Viewers
Today, 1+ billion users watch short-form videos on TikTok, Instagram Reels, and YouTube Shorts. However, based on the literature review, they remain inaccessible to millions of blind and low vision (BLV) viewers due to their rapid visual changes, on-screen text, and music or meme-audio overlays.
Research
Short-form videos on TikTok rated only 3.2/7 in accessibility. I uncovered key insights through 7 in-depth user interviews to understand why.
To uncover the extent, contexts, and causes of inaccessibility in short-form videos, I analyzed data from seven 1.5-hour interviews with BLV participants, each featuring two video-watching tasks. The average rating of 3.21 shows these videos are less than halfway accessible.
Insight 1 — Unlike sighted users, BLV users can’t quickly determine a video's content and decide whether to continue watching it.
Insight 2 — BLV users face difficulties comprehending various short-form video formats due to inconsistent or inadequate metadata and descriptions.
Insight 3 — BLV Users relying on VoiceOver encounter significant accessibility barriers due to poorly labeled video control buttons and frequent changes in the interface layout.
Design
Providing video descriptions emerged as a straightforward and effective design solution given their compatibility with screen readers
I designed a short description of video content so that BLV users can quickly access essential information and make decisions. -> Insight 1
I implemented consistent, multi-level descriptions compatible with screen readers to ensure all users can easily understand and engage with the content. -> insight 2
I standardized button labels, ensuring compatibility with screen readers, and minimizing unnecessary layout change. I also redesign the feature layouts for quicker access and more intuitive navigation. -> Insight 3
Final Solution
An interface that allows users to access video descriptions at four levels of detail based on interest
I developed ShortScribe, which features a video pane and a description pane.
The video pane (built upon existing short-form video platforms) includes
A short description feature.
A redesigned button and information layout tailored to BLV users' preferences.
A new button to access the description pane feature.
The description pane (unique to our interface) provides
Hierarchical video summaries at three levels of detail - long description, on-screen text, and shot-by-shot description.
Video Showcase
User Evaluation
Short-form videos became measurably more accessible for BLV users with our system
After pipeline evaluation, I conducted a within-subjects study where participants compared our system, ShortScribe, to a baseline interface for watching short-form videos.
All of ShortScribe's description features—long, short, shot-by-shot, and on-screen text—were rated significantly more useful than the baseline information.
For video selection, users rated our system 2.2x more useful than the current one.
For video comprehension, participants rated our system 2.7x more useful than the current one.
Participants praised the accessibility of explicit video controls in our interface.
Design improvements
I made 3 improvements to the system, based on the user testing
Feedback 1
Despite high accuracy in pipeline evaluation, some errors in the descriptions persisted, often unaware by users.
Solution 1
Add a setting option to display accuracy scores next to descriptions, helping users gauge their reliability.
Feedback 2
The design inevitably created some redundancy in descriptions, both with the original video audio and across different summaries. However, our usability test revealed some users preferred having more information rather than less.
Solution 2
Add a customization icon at the top of the description panel, allowing users to tailor and save their preferences for all future descriptions.
Feedback 3
Participants wanted to know video-length information to help them make informed choices when selecting videos.
Solution 3
Added a setting option to display video length on the Video Pane.
