US 12,444,505 B2
Multimodal metrics for remote monitoring of condition progression
Hardik Kothare, Burlingame, CA (US); Michael Neumann, Winterbach (DE); and Vikram Ramanarayanan, San Francisco, CA (US)
Assigned to Modality.AI, Inc., San Francisco, CA (US)
Filed by Modality.AI, Inc., San Francisco, CA (US)
Filed on Aug. 20, 2024, as Appl. No. 18/810,489.
Claims priority of provisional application 63/551,060, filed on Feb. 7, 2024.
Prior Publication US 2025/0253056 A1, Aug. 7, 2025
Int. Cl. G16H 50/50 (2018.01); G10L 25/60 (2013.01); G10L 25/66 (2013.01); G16H 40/67 (2018.01); G16H 50/70 (2018.01)
CPC G16H 50/50 (2018.01) [G10L 25/60 (2013.01); G10L 25/66 (2013.01); G16H 40/67 (2018.01); G16H 50/70 (2018.01)] 20 Claims
OG exemplary drawing
 
1. A system for identifying efficacious markers to improve predictive modelling of condition progression, comprising:
memory storing remotely collected multimodal digital markers from a first cohort experiencing a sign onset in the condition progression and a second cohort experiencing a non-sign onset in the condition progression;
feature selection logic configured to identify a subset of the multimodal digital markers that are best at capturing differences between the sign onset and the non-sign onset;
responsiveness determination logic configured to determine a responsiveness parameter that specifies how a rate of change in the identified subset of the multimodal digital markers differs between the sign onset and the non-sign onset;
time to detect change determination logic configured to determine a time to detect change parameter that specifies a time period required to detect a meaningful change in the identified subset of the multimodal digital markers from condition onset in the first cohort and the second cohort;
sample size effect determination logic configured to determine how the responsiveness parameter and the time to detect change parameter change depending on a sample size of the first cohort and the second cohort; and
sensitivity determination logic configured to determine a sensitivity parameter that specifies whether the identified subset of the multimodal digital markers detect condition deterioration during intervals of time when no changes are reported in an external gold standard;
wherein the multimodal digital markers are collected by the first and/or second cohort engaging in a conversation with a virtual guide to perform a task or tasks, and audio and video streams of the first and/or second cohort performing the task or tasks are uploaded to the cloud in real-time for downstream analysis;
wherein the conversation comprises the virtual guide asking the first and/or second cohort to do the task or tasks, and the virtual guide providing feedback to the first and/or second cohort comprising demonstrations of how a task or tasks should be performed.