I-Corps: Real-Time Crowd Captioning

实时字幕在5秒内提供相当于听觉演讲的文本，使(I)聋人和重听者能够观看现场活动(例如主流讲座)，(Ii)法庭上的即时文字记录，以及(Iii)联邦政府授权的电视直播节目。目前，唯一可靠的解决方案是聘请专业速记员。这个项目调查了一种技术和相关算法的使用，以提供实时字幕。它使3-5个非专业人员的表现可以与专业速记员相媲美。由于这项技术能够为用户提供更低的价格点和更灵活的劳动力，它可能允许人们以以前无法使用的方式使用实时字幕，例如只在必要的时候按需为他们生活中的秘密事件提供字幕。这项拟议的研究可以通过提供偶然的机会来改善聋人和重听学生的教育，例如课后对话或没有安排翻译或字幕解说员的最后一刻的客座讲座。实时字幕也可用于其他设置，例如政治事件、学校节目、艺术表演，并为组织提供为现场事件提供高质量字幕的更具成本效益的手段。年长的重听成年人通常更喜欢字幕，这代表了一个相当大的且不断增长的人口。听力方面的人可能会受益，因为字幕是听觉语音自动翻译的第一步。此外，由于Scribe允许主要根据领域专业知识而不是打字能力来选择字幕人员，因此可以在具有复杂术语的技术领域收集更准确的字幕，这可能有助于减少STEM领域中聋哑生的代表性不足。

Real-time captioning provides a text equivalent to aural speech within 5 seconds, providing (i) access to live events for deaf and hard of hearing people (e.g. mainstream lectures), (ii) an immediate transcript in the courtroom, and (iii) federally mandated access to live television programs. Currently, the only reliable solution is to engage professional stenographers. This project investigates the use of a technology and associated algorithm to provide real-time captioning. It enables 3-5 non-expert individuals to match the performance of an expert stenographer. Due to the technology's ability to provide users with a combination of a lower price point and a more elastic work force, it may allow people to use real-time captioning in ways they previously could not, e.g. to caption surreptitious events on-demand in their lives for only as long as necessary. The proposed research can improve education for deaf and hard of hearing students by enabling access to serendipitous opportunities, such as conversations after class or last-minute guest lectures for which no interpreter or captionist was arranged. Real-time captioning may also be used in other settings as well, e.g. political events, school programs, artistic performances, and provide organizations with a much more cost effective means of providing high-quality captions for live events. Older hard of hearing adults usually prefer captioning, and represent a sizable and growing population. Hearing people may benefit because captioning is a first step in automatic translation of aural speech. Furthermore, because Scribe allows captionists to be selected primarily based on domain expertise instead of typing ability, more accurate captions can be collected in technical domains with complex jargon, potentially helping to reduce the underrepresentation of deaf and hard of hearing students in STEM fields.