US 11,688,203 B2
Systems and methods for providing visual allocation management
Andres Mauricio Muñoz Delgado, Leamington Spa (GB); Bryan L. Reimer, Newton, MA (US); Joonbum Lee, Seattle, WA (US); Linda Sala Angell, Grosse Pointe Farms, MI (US); Bobbie Danielle Seppelt, Brookline, MA (US); Bruce L. Mehler, Jamaica Plain, MA (US); and Joseph F. Coughlin, Sudbury, MA (US)
Assigned to MASSACHUSETTS INSTITUTE OF TECHNOLOGY, Cambridge, MA (US)
Filed by Massachusetts Institute of Technology, Cambridge, MA (US)
Filed on Dec. 11, 2020, as Appl. No. 17/120,009.
Application 17/120,009 is a continuation of application No. 15/682,108, filed on Aug. 21, 2017, granted, now 10,902,331.
Claims priority of provisional application 62/377,213, filed on Aug. 19, 2016.
Prior Publication US 2021/0150390 A1, May 20, 2021
This patent is subject to a terminal disclaimer.
Int. Cl. G06N 5/04 (2006.01); G06V 40/19 (2022.01); G06N 20/00 (2019.01); G06V 20/59 (2022.01); G06F 18/20 (2023.01); G06N 7/01 (2023.01); G06V 10/84 (2022.01)
CPC G06V 40/19 (2022.01) [G06F 18/295 (2023.01); G06N 5/04 (2013.01); G06N 7/01 (2023.01); G06N 20/00 (2019.01); G06V 10/85 (2022.01); G06V 20/597 (2022.01)] 35 Claims
OG exemplary drawing
 
1. A system for managing visual allocation, comprising:
at least one memory operable to store one or more models, the one or more models including at least one of:
(1) a model corresponding to a plurality of candidate states; and
(2) a plurality of models in which each model corresponds to one of the plurality of candidate states; and at least one processor communicatively coupled to the at least one memory, the at least one processor being operable to:
receive visual data corresponding to a person engaging in an activity during a continuous period of time;
identify a sequence of glances from the visual data and, for each glance in the sequence of glances, identify corresponding glance information, the glance information including a glance direction;
classify each of the glances in the sequence of glances into a spatial region from among a set of predetermined spatial regions based on their respective glance information;
identify, based on the one or more models, the presence of one or more states of the person while engaging in the activity during the continuous period of time by inputting into each of the stored one or more models model input data including one or more of (1) the classification of the glances in the sequence of glances and (2) the glance information of the glances in the sequence of glances; and
output feedback based on the identified one or more states, wherein the one or more models correspond to one or more candidate states of the person that are not at least one of observable or detectable from the received visual data, and
wherein each candidate state and identified one or more states corresponds to at least one of an attention, awareness, emotion, a mental state, or a physical state of a person.