US 12,135,943 B1
Mood- and mental state-aware interaction with multimodal large language models
Michel François Valstar, Nottingham (GB); Anthony Brown, Nottingham (GB); and Mani Kumar Tellamekala, Nottingham (GB)
Assigned to Blueskeye AI Ltd, Nottingham (GB)
Filed by Blueskeye AI Ltd, Nottingham (GB)
Filed on Apr. 16, 2024, as Appl. No. 18/636,434.
Claims priority of provisional application 63/496,725, filed on Apr. 18, 2023.
Int. Cl. G06F 40/30 (2020.01); A61B 5/024 (2006.01); G06V 40/16 (2022.01)
CPC G06F 40/30 (2020.01) [A61B 5/024 (2013.01); G06V 40/174 (2022.01)] 8 Claims
OG exemplary drawing
 
1. A system comprising:
a first module set comprising:
1) A first input prompt module having a first input prompt module output;
2) A first recognizing mood module using valence, arousal, and dominance to review the first input prompt module output, and create a first recognizing mood module output;
3) A first customizing prompt module to customize the first recognizing mood module output, and create a first customizing prompt module output;
4) A first computing large language model (LLM) response module using valence, arousal, dominance and the first customizing prompt module output to create a first computing LLM response output;
5) A first capturing module to determine an expressive reaction of a user to the first computing LLM response output;
a mental state estimator operating on the first computing LLM response output to create a mental state estimate output;
a second module set comprising:
1) A second input prompt module having a second input prompt module output;
2) A second recognizing mood module using valence, arousal, and dominance to review the second input prompt module output, and create a second recognizing mood module output;
3) A second customizing prompt module to customize the second recognizing mood module output based at least in part on the mental state estimate output, and to create a second customizing prompt module output.