US 12,217,743 B1
System and method for speech recognition using deep recurrent neural networks
Alexander B. Graves, London (GB)
Assigned to Google LLC, Mountain View, CA (US)
Filed by Google LLC, Mountain View, CA (US)
Filed on Jul. 28, 2023, as Appl. No. 18/227,769.
Application 18/227,769 is a continuation of application No. 17/700,234, filed on Mar. 21, 2022, granted, now 11,756,535.
Application 17/700,234 is a continuation of application No. 17/013,276, filed on Sep. 4, 2020, granted, now 11,282,506, issued on Mar. 22, 2022.
Application 17/013,276 is a continuation of application No. 16/658,697, filed on Oct. 21, 2019, granted, now 10,770,064, issued on Sep. 8, 2020.
Application 16/658,697 is a continuation of application No. 16/267,078, filed on Feb. 4, 2019, granted, now 10,453,446, issued on Oct. 22, 2019.
Application 16/267,078 is a continuation of application No. 15/043,341, filed on Feb. 12, 2016, granted, now 10,199,038, issued on Feb. 5, 2019.
Application 15/043,341 is a continuation of application No. 14/090,761, filed on Nov. 26, 2013, granted, now 9,263,036, issued on Feb. 16, 2016.
Claims priority of provisional application 61/731,047, filed on Nov. 29, 2012.
Int. Cl. G10L 15/16 (2006.01); G06N 3/044 (2023.01); G06N 3/08 (2023.01)
CPC G10L 15/16 (2013.01) [G06N 3/044 (2023.01); G06N 3/08 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A method performed by one or more computers and for training a speech recognition neural network system comprising one or more neural networks, the method comprising:
pre-training one or more of the neural networks in the speech recognition neural network system on a corpus comprising text data, wherein pre-training the one or more of the neural networks in the speech recognition neural network system on a corpus comprising text data comprises:
pre-training one or more of the neural networks in the speech recognition neural network system on the corpus comprising text data to perform next-step prediction; and
after the pre-training, training the one or more neural networks in the speech recognition neural network system on training data that maps audio to text transcriptions.