US 12,444,415 B2
Interactive system using speech recognition and digital media
Anthony Tomasic, Forth Worth, TX (US); Erik Thiessen, Pittsburgh, PA (US); Cassondra Eng, Pittsburgh, PA (US); and Amy Ogan, Pittsburgh, PA (US)
Assigned to Carnegie Mellon University, Pittsburgh, PA (US)
Filed by Carnegie Mellon University, Pittsburgh, PA (US)
Filed on Oct. 26, 2022, as Appl. No. 17/974,272.
Claims priority of provisional application 63/408,397, filed on Sep. 20, 2022.
Claims priority of provisional application 63/271,736, filed on Oct. 26, 2021.
Prior Publication US 2023/0131393 A1, Apr. 27, 2023
Int. Cl. G10L 15/22 (2006.01); G10L 15/08 (2006.01); H04N 21/422 (2011.01)
CPC G10L 15/22 (2013.01) [G10L 15/08 (2013.01); H04N 21/42203 (2013.01); G10L 2015/088 (2013.01); G10L 2015/223 (2013.01)] 21 Claims
OG exemplary drawing
 
1. A server system, comprising:
a data store configured to store one or more data items each representing at least a portion of a given media item;
an interface in communication with the data store, the interface configured to enable a user to input one or more instructions specifying a playback configuration for the given media item;
a controller in communication with the interface and the data store, the controller configured to perform operations comprising:
receiving at least one instruction specifying a portion of a media item and at least one event associated with the portion of the media item, the at least one event representing a solicitation of a user input to a playback device during playback of the portion of the media item by the playback device when the playback of the media portion is associated with a first state, the user input specifying one or more objects, concepts, or both objects and concepts related to the portion of the media item;
generating, based on the at least one instruction, executable code representing a playback configuration of the portion of the media item, the playback configuration comprising:
generation of additional media based on the user input, the additional media representing at least one of the objects, concepts, or both objects and concepts specified in the user input when the playback of the portion of the media item is associated with the first state;
incorporation of the additional media into the portion of the media item, the additional media representing at least one of the objects, concepts, or both objects and concepts specified in the user input when the playback of the portion of the media item is associated with the first state, the incorporation of the additional media representing a constructive media experience relative to one or more prior states; and
in a second state associated with the playback of the media item, causing a playback of the portion of the media item including the additional media incorporated into the portion of the media item, the playback of the portion of the media item in the second state being based on which of the at least one of the objects, concepts, or both objects and concepts are specified in the user input.