US 12,329,517 B2
Cognitive state vehicle navigation based on image processing and modes
Maha Amr Mohamed Fouad, New Cairo (EG); Rana el Kaliouby, Milton, MA (US); Abdelrahman N. Mahmoud, Somerville, MA (US); and Panu James Turcot, Pacifica, CA (US)
Assigned to Affectiva, Inc., Boston, MA (US)
Filed by Affectiva, Inc., Boston, MA (US)
Filed on Jul. 19, 2021, as Appl. No. 17/378,817.
Application 17/378,817 is a continuation in part of application No. 16/261,905, filed on Jan. 30, 2019, granted, now 11,067,405.
Application 16/261,905 is a continuation in part of application No. 15/875,644, filed on Jan. 19, 2018, granted, now 10,627,817.
Application 15/875,644 is a continuation in part of application No. 15/273,765, filed on Sep. 23, 2016, abandoned.
Application 15/273,765 is a continuation in part of application No. 14/796,419, filed on Jul. 10, 2015, abandoned.
Application 14/796,419 is a continuation in part of application No. 14/460,915, filed on Aug. 15, 2014, abandoned.
Application 14/460,915 is a continuation in part of application No. 13/153,745, filed on Jun. 6, 2011, abandoned.
Claims priority of provisional application 63/083,136, filed on Sep. 25, 2020.
Claims priority of provisional application 63/071,401, filed on Aug. 28, 2020.
Claims priority of provisional application 62/679,825, filed on Jun. 3, 2018.
Claims priority of provisional application 62/637,567, filed on Mar. 2, 2018.
Claims priority of provisional application 62/625,274, filed on Feb. 1, 2018.
Claims priority of provisional application 62/611,780, filed on Dec. 29, 2017.
Claims priority of provisional application 62/593,440, filed on Dec. 1, 2017.
Claims priority of provisional application 62/593,449, filed on Dec. 1, 2017.
Claims priority of provisional application 62/557,460, filed on Sep. 12, 2017.
Claims priority of provisional application 62/541,847, filed on Aug. 7, 2017.
Claims priority of provisional application 62/524,606, filed on Jun. 25, 2017.
Claims priority of provisional application 62/503,485, filed on May 9, 2017.
Claims priority of provisional application 62/469,591, filed on Mar. 10, 2017.
Claims priority of provisional application 62/448,448, filed on Jan. 20, 2017.
Claims priority of provisional application 62/370,421, filed on Aug. 3, 2016.
Claims priority of provisional application 62/301,558, filed on Feb. 29, 2016.
Claims priority of provisional application 62/273,896, filed on Dec. 31, 2015.
Claims priority of provisional application 62/265,937, filed on Dec. 10, 2015.
Claims priority of provisional application 62/222,518, filed on Sep. 23, 2015.
Claims priority of provisional application 62/128,974, filed on Mar. 5, 2015.
Claims priority of provisional application 62/082,579, filed on Nov. 20, 2014.
Claims priority of provisional application 62/047,508, filed on Sep. 8, 2014.
Claims priority of provisional application 62/023,800, filed on Jul. 11, 2014.
Claims priority of provisional application 61/972,314, filed on Mar. 30, 2014.
Claims priority of provisional application 61/953,878, filed on Mar. 16, 2014.
Claims priority of provisional application 61/927,481, filed on Jan. 15, 2014.
Claims priority of provisional application 61/924,252, filed on Jan. 7, 2014.
Claims priority of provisional application 61/916,190, filed on Dec. 14, 2013.
Claims priority of provisional application 61/867,007, filed on Aug. 16, 2013.
Claims priority of provisional application 61/467,209, filed on Mar. 24, 2011.
Claims priority of provisional application 61/447,464, filed on Feb. 28, 2011.
Claims priority of provisional application 61/447,089, filed on Feb. 27, 2011.
Claims priority of provisional application 61/439,913, filed on Feb. 6, 2011.
Claims priority of provisional application 61/414,451, filed on Nov. 17, 2010.
Claims priority of provisional application 61/388,002, filed on Sep. 30, 2010.
Claims priority of provisional application 61/352,166, filed on Jun. 7, 2010.
Prior Publication US 2021/0339759 A1, Nov. 4, 2021
Int. Cl. G05D 1/00 (2024.01); A61B 5/1171 (2016.01); B60W 50/00 (2006.01); G06V 10/764 (2022.01); G06V 20/56 (2022.01); G06V 20/59 (2022.01); G06V 40/16 (2022.01); G06V 40/20 (2022.01); G07C 5/02 (2006.01); G06V 10/44 (2022.01)
CPC A61B 5/1176 (2013.01) [B60W 50/0098 (2013.01); G06V 10/764 (2022.01); G06V 20/56 (2022.01); G06V 20/597 (2022.01); G06V 40/161 (2022.01); G06V 40/176 (2022.01); G06V 40/20 (2022.01); G07C 5/02 (2013.01); B60W 2420/403 (2013.01); B60W 2420/408 (2024.01); B60W 2540/10 (2013.01); B60W 2540/12 (2013.01); B60W 2540/229 (2020.02); B60W 2540/30 (2013.01); B60W 2554/4042 (2020.02); B60W 2554/4043 (2020.02); B60W 2555/20 (2020.02); B60W 2556/10 (2020.02); B60W 2556/45 (2020.02); G06V 10/454 (2022.01)] 26 Claims
OG exemplary drawing
 
1. A computer-implemented method for vehicle navigation comprising:
obtaining one or more images of a vehicle occupant using a first imaging device within a vehicle, wherein the one or more images include facial data of the vehicle occupant;
analyzing, using a first computing device, the one or more images to determine cognitive state data for the vehicle occupant, wherein the analyzing uses a machine learning technique that utilizes a deep convolutional neural network;
mapping the cognitive state data to location data along a vehicle travel route;
weighting mode data based on real-time analysis of cognitive state data obtained for one or more occupants of the vehicle, wherein the weighting of the mode data is further based on travel route noise;
updating information about the vehicle travel route based on the weighting of the mode data, wherein the weighting of the mode data is based on the travel route noise, the cognitive state data, and the mode data for the vehicle; and
providing the information that was updated for vehicle control.