US 12,482,458 B2
Semiautomated relay method and apparatus
Robert M. Engelke, Madison, WI (US); Kevin R. Colwell, Middleton, WI (US); and Christopher R. Engelke, Verona, WI (US)
Assigned to Ultratec, Inc., Madison, WI (US)
Filed by Ultratec, Inc., Madison, WI (US)
Filed on May 24, 2019, as Appl. No. 16/422,662.
Application 16/422,662 is a continuation in part of application No. 15/982,239, filed on May 17, 2018, abandoned.
Application 15/982,239 is a continuation in part of application No. 15/729,069, filed on Oct. 10, 2017, abandoned.
Application 15/729,069 is a continuation in part of application No. 15/171,720, filed on Jun. 2, 2016, granted, now 10,748,523.
Application 15/171,720 is a continuation in part of application No. 14/953,631, filed on Nov. 30, 2015, granted, now 10,878,721.
Application 14/953,631 is a continuation in part of application No. 14/632,257, filed on Feb. 26, 2015, granted, now 10,389,876.
Claims priority of provisional application 61/946,072, filed on Feb. 28, 2014.
Prior Publication US 2019/0312973 A1, Oct. 10, 2019
Int. Cl. G10L 15/183 (2013.01); G10L 15/26 (2006.01); G10L 25/60 (2013.01); H04M 1/247 (2021.01); H04M 1/72433 (2021.01); H04M 1/72478 (2021.01); H04M 3/42 (2006.01)
CPC G10L 15/183 (2013.01) [G10L 15/26 (2013.01); G10L 25/60 (2013.01); H04M 1/2475 (2013.01); H04M 1/72433 (2021.01); H04M 1/72478 (2021.01); H04M 3/42391 (2013.01); H04M 2201/40 (2013.01); H04M 2201/60 (2013.01)] 15 Claims
OG exemplary drawing
 
1. A captioning system for presenting captions to an assisted user (AU) during communication with a hearing user (HU) where the hearing user uses a hearing user's device to facilitate the communication, the system comprising:
a captioned device including a display screen, a speaker, and a microphone, the microphone for receiving the AU voice signal for transmission to the HU's device, the display screen for presenting captions corresponding to an HU voice signal, and the speaker for broadcasting the hearing user's voice signals;
a processor specially programmed to receive voice signal during an ongoing call between the AU and the HU and, during the ongoing call;
(i) run an automatic speech recognition (ASR) engine to generate initial ASR captions associated with the voice signal;
(ii) automatically assessing at least one caption quality factor associated with prior initial ASR captions generated during the ongoing call;
(iii) delaying broadcast of HU voice signal to the AU via the speaker; and
(iv) providing the ASR captions to the AU device for display on the display screen;
(v) based on the at least one caption quality factor, automatically adjusting a duration of the HU voice signal broadcast delay to better temporally align the HU voice signal broadcast via the speaker with the captions presented via the display screen.