US 11,853,950 B2
Multimodal data framework for digital system representation
Martin Wezowski, Berlin (DE); Hans-Martin Will, Marina Del Ray, CA (US); Rohit Jalagadugula, Bengaluru (IN); Kavitha Krishnan, Bangalore (IN); Sai Hareesh Anamandra, Bangalore (IN); Vinay George Roy, Kozhikode (IN); Parthasarathy Menon, Kozhikode (IN); and Alexander Schaefer, Sunnyvale, CA (US)
Assigned to SAP SE, Walldorf (DE)
Filed by SAP SE, Walldorf (DE)
Filed on Sep. 27, 2021, as Appl. No. 17/486,126.
Prior Publication US 2023/0096720 A1, Mar. 30, 2023
Int. Cl. G06Q 10/0639 (2023.01); G06Q 10/067 (2023.01); G06Q 10/0633 (2023.01); H04L 67/50 (2022.01)
CPC G06Q 10/06398 (2013.01) [G06Q 10/067 (2013.01); G06Q 10/0633 (2013.01); H04L 67/535 (2022.05)] 16 Claims
OG exemplary drawing
 
1. A system, comprising:
at least one data processor; and
at least one memory storing instructions, which when executed by the at least one data processor, result in operations comprising:
retrieving, from a plurality of data sources, multimodal data, the plurality of data sources comprising a wearable device worn by a user and a camera recording an action of the user, the multimodal data comprising a browser history data, an application usage data, time-series data generated by any of the wearable device and the camera, and a coding pattern associated with one or more collaborative platforms, the time-series data corresponding to one or more of a heart rate, a sleep pattern, and an activity level of the user, the coding pattern comprising one or more of a type of programming language, a first quantity of time spent on each commit, a second quantity of time spent resolving each issue, a size of each commit, and a coding frequency;
categorizing the browser history data based on a type of website to determine a website interaction quantity for the respective type of website;
categorizing the application usage data based on a type of application to determine a screen time quantity associated with the respective type of application;
generating enriched multimodal data by updating the multimodal data to comprise the website interaction quantity and the screen time quantity;
generating, by a data controller, a digital profile of the user comprising a plurality of dimensions, and each dimension of the plurality of dimensions being generated based on the enriched multimodal data associated with the user, the plurality of dimensions comprising a technical dimension associated with the coding pattern and a wellbeing dimension associated with the website interaction quantity and the screen time quantity; and
providing a recommendation to be displayed, the recommendation being based on an analysis of a change in the technical dimension relative to the wellbeing dimension of the digital profile of the user.