Voices Help Correlate Signs and Words
Analyzing Deaf and Hard-of-Hearing (DHH) TikTokers’ Content, Practices, and Pitfalls
Analyzing Deaf and Hard-of-Hearing (DHH) TikTokers’ Content, Practices, and Pitfalls
Video-sharing platforms such as TikTok have offered new opportunities for d/Deaf and hard-of-hearing (DHH) people to create public-facing content using sign language – an integral part of DHH culture. Besides sign language, DHH creators deal with a variety of modalities when creating videos, such as captions and audio. However, hardly any work has comprehensively addressed DHH creators’ multimodal practices with the lay public’s reactions taken into account. In this paper, we systematically analyzed 308 DHH-authored TikTok videos using a mixed-methods approach, focusing on DHH TikTokers’ content, practices, pitfalls, and viewer engagement. Our findings highlight that while voice features such as synchronous voices are scant and challenging for DHH TikTokers, they may help promote viewer engagement. Other empirical findings, including the distributions of topics, practices, pitfalls, and their correlations with viewer engagement, further lead to actionable suggestions for DHH TikTokers and video-sharing platforms.
The increasing popularity of video-sharing platforms such as TikTok in recent years has greatly enriched social media users’ and creators’ communication modalities, including for d/Deaf and hard-of-hearing (DHH) people who regard sign language as an integral part of their culture. With the support of multimodal communication on video-sharing platforms, DHH creators can now create content using sign language more freely, enhancing communication with both DHH and hearing viewers. However, prior work suggests that while most DHH users prefer sign language communication on social media, they still share primarily using written language, indicating a gap between their preferences and practices for communication modalities. It is therefore crucial to further understand the reasons behind such a gap by looking into the practices and challenges of DHH people on video-sharing platforms.
Research investigating the practices and challenges of DHH people on video-sharing platforms has centered chiefly around DHH users as content consumers rather than creators. New issues and needs arise when DHH people take the role of creators, such as captioning for hearing viewers, online trolls, content demonetization, etc. Existing literature on DHH creators’ practices and challenges on video-sharing platforms remains in a nascent stage and thus has only examined a narrow range of video topics, practices, and modalities. Moreover, most prior work on DHH content creation has only approached creation challenges from DHH perspectives only, without taking into account wider audiences at play.
Since prevailing video-sharing platforms like TikTok serve as a channel for enhancing mutual communication between DHH and the
lay public, how general audiences react to DHH creators’ content is critical for designing an inclusive social environment on
public-oriented video-sharing platforms. To address the gaps, we seek to answer three research questions (RQs):
● RQ1: What are the topics of videos created by DHH TikTokers (i.e., TikTok video creators)?
● RQ2: What are the common production practices and pitfalls in videos created by DHH TikTokers?
● RQ3: How do viewers react to different topics, practices, and pitfalls of DHH TikTokers’ content creation?
To answer RQ1 which explores the types and distributions of topics created by DHH TikTokers, we qualitatively coded the topics of 308 publicly available videos created by DHH TikTokers. We then coded DHH TikTokers’ practices and pitfalls (RQ2). To answer RQ3 regarding how viewers react to these DHH-authored videos, we analyzed the correlation between video practices and metadata that represented viewer engagement (e.g., likeCount, playCount, and shareCount). We also qualitatively analyzed top comments especially where creator-viewer interaction existed.
Our findings highlight the following points for the research questions:
(1) RQ1:
● DHH TikTokers mainly covered the topics of disclosure of disability challenges (26.0%), sign
language content (24.0%), raising deaf awareness (21.4%), entertainment (20.1%), and interactions with social ties (10.4%).
(1) RQ2:
● Among a series of practices examined, the most surprising finding is that while most DHH-authored videos were
captioned (86.04%), most videos did not contain rich voice features, including synchronous voices (without: 61.69%) and
dubbing (without: 65.26%).
● Although most DHH-authored videos were captioned, low-quality and absent captions were
still the most common pitfall (51.61%). Meanwhile, audiovisual mismatches (32.26%) and low-quality sound editing (29.03%)
were also common pitfalls.
(1) RQ3:
● Amongst the five topics, raising deaf awareness tended to be less competitive in receiving
like and share counts.
● While the use of sign language tended to lead to better viewer engagement of any kind, dubbing,
background music, and lip-syncing tended to lead to worse viewer engagement.
● Intriguingly, some sound-editing pitfalls increased the authenticity of DHH TikTokers.
These empirical findings further lead to actionable suggestions for DHH TikTokers, video-sharing platforms, and future work. For instance, we suggest that DHH TikTokers whose content is mainly tailored to the hearing public has more room for improvement in their audio quality (e.g., synchronous voices) than visual quality (e.g., captions). Our design implications for video-sharing platforms revolve around captioning-support tools, filming-support tools, and other inclusive features. For more details, please refer to our full paper.
Cao, J., Peng, X., Liang, F., & Tong, X. (2024).
In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI ’24).
