US 12,461,616 B1
Real-time synchronization between a physical entity and a mixed reality environment
Stephen Rudofski, West Bloomfield, MI (US); Danni Zhang, Auburn Hills, MI (US); Ramtin Parvizi, Windsor (CA); Robert Ondrus, Northville, MI (US); and Luciano L. Oetting, Washington Township, MI (US)
Assigned to GM Global Technology Operations LLC, Detroit, MI (US)
Filed by GM GLOBAL TECHNOLOGY OPERATIONS LLC, Detroit, MI (US)
Filed on Oct. 28, 2024, as Appl. No. 18/928,326.
Int. Cl. G06F 3/038 (2013.01); G06F 3/0346 (2013.01); G06T 11/00 (2006.01)
CPC G06F 3/038 (2013.01) [G06F 3/0346 (2013.01); G06T 11/00 (2013.01); G06T 2200/24 (2013.01)] 20 Claims
OG exemplary drawing
 
20. A vehicle comprising:
a physical entity and a plurality of sensors operatively connected to the physical entity;
a microcontroller integrated with the physical entity and having a first processor and a first tangible, non-transitory memory, the microcontroller being configured to map a respective baseline position of the plurality of sensors;
a virtual reality (VR) controller in communication with the microcontroller and having a second processor and a second tangible, non-transitory memory, the VR controller being adapted to create a digital twin of the physical entity;
wherein the microcontroller is configured to:
receive sensor data from the plurality of sensors, the sensor data being converted into a respective position value;
determine, in a predefined priority order, if the respective position value has deviated from the respective baseline position of an item and create a respective data packet when the respective position value has deviated, the respective data packet including a unique letter code encoding an axis of movement of the item and an identity of the item, the respective data packet including a numeric position coordinate of the item;
wherein the VR controller is adapted to:
receive incoming position data from the microcontroller and route the incoming position data based in part on the unique letter code;
generate an updated digital twin by updating a respective virtual position value in the digital twin based on the unique letter code; and
transmit the updated digital twin to a virtual reality medium or a mixed reality medium and control operation of the physical entity based in part on the updated digital twin in real-time.