US 11,720,604 B2
Automated rearrangement of digital whiteboard content
Raghavendra Bhagavatha, San Jose, CA (US); Alicia Kepner, San Francisco, CA (US); Stephen Newton, San Jose, CA (US); Ihor Shadko, San Francisco, CA (US); Jeffrey William Smith, Milpitas, CA (US); and Jordan Thiel, Leander, TX (US)
Assigned to Zoom Video Communications, Inc., San Jose, CA (US)
Filed by Zoom Video Communications, Inc., San Jose, CA (US)
Filed on Oct. 31, 2021, as Appl. No. 17/515,536.
Prior Publication US 2023/0140747 A1, May 4, 2023
Int. Cl. G06F 16/28 (2019.01); G06N 20/00 (2019.01)
CPC G06F 16/287 (2019.01) [G06N 20/00 (2019.01)] 19 Claims
OG exemplary drawing
 
1. A method, comprising:
training, at one or more server devices using a training data set corresponding to past digital whiteboard content item rearrangements, a machine learning model for text detection and recognition using natural language processing to enable the trained machine learning model to determine metadata associated with digital whiteboard content items;
instantiating, at the one or more server devices during a video conference implemented using conferencing software of a unified communications as a service software platform and to which one or more user devices are connected, a digital whiteboard using digital whiteboard software of the unified communications as a service software platform;
generating, at the one or more server devices according to input obtained from the one or more user devices during the video conference, content items within the digital whiteboard at initial locations relative to one another;
for each of the content items, processing, at the one or more server devices during the video conference, the content item using the trained machine learning model to determine first metadata corresponding to text or non-text content visually represented within the content item and second metadata corresponding to data not visually represented within the content item;
determining, at the one or more server devices during the video conference, a first rearrangement of the content items based on the first metadata determined for each of the content items, wherein locations of the content items relative to one another in the first rearrangement are different from the initial locations;
determining, at the one or more server devices during the video conference, a second rearrangement of the content items based on the second metadata determined for each of the content items, wherein locations of the content items relative to one another in the second rearrangement are different from the initial locations and from the locations of the content items in the first rearrangement;
responsive to determining the first and second rearrangements, automatically outputting, from the one or more server devices for display at each of the one or more user devices, the first rearrangement to a first layer of the digital whiteboard software and the second rearrangement to a second layer of the digital whiteboard software to enable the one or more user devices to navigably access, within the digital whiteboard, either of the first rearrangement and the second rearrangement at a given time; and
further training, at the one or more server devices, the trained machine learning model according to rearrangement information obtained from a user device of the one or more user devices after the automatic outputting of the first rearrangement to the first layer and the second rearrangement to the second layer, the rearrangement information indicating a new location for a content item different from a rearranged location of the first rearrangement or the second rearrangement.