US 12,272,267 B2
System for remotely accessing real and/or virtual instruments
Mariam Mnatsakanyan, New South Wales (AU)
Assigned to Mariam Mnatsakanyan, New South Wales (AU)
Appl. No. 17/618,011
Filed by Mariam Mnatsakanyan, New South Wales (AU)
PCT Filed Jun. 17, 2020, PCT No. PCT/AU2020/050611
§ 371(c)(1), (2) Date Dec. 10, 2021,
PCT Pub. No. WO2020/252526, PCT Pub. Date Dec. 24, 2020.
Claims priority of application No. 2019902108 (AU), filed on Jun. 18, 2019.
Prior Publication US 2022/0246054 A1, Aug. 4, 2022
Int. Cl. G09B 9/00 (2006.01); G06Q 50/20 (2012.01); G06T 19/00 (2011.01); G06N 20/00 (2019.01)
CPC G09B 9/00 (2013.01) [G06Q 50/205 (2013.01); G06T 19/003 (2013.01); G06T 19/006 (2013.01); G06N 20/00 (2019.01)] 18 Claims
OG exemplary drawing
 
1. A system for remotely accessing a real instrument which has an associated digital twin instrument, the system including:
a user terminal for virtually connecting with the real instrument, the user terminal including:
a user input device configured to receive user input data;
a display device configured to receive display data and, in response, project images visible to the user on a display of the display device; and
an internet enabled computer in communication with the display device and user input device, the internet enabled computer configured for communicating with an instrument server to send the user input data and receive the display data;
an instrument server configured to host software associated with the real instrument, the software including:
virtual instrument software configured to virtually represent the associated digital twin instrument by a static model, wherein the static model includes at least a three-dimensional model of the associated digital twin instrument, including the same inputs and outputs and software to run the real instrument; and
remote instrument access software configured to allow the user to remotely access the real instrument through one or more sensors and/or actuators embedded within or mounted to the real instrument;
wherein the instrument server is further configured to:
remotely control the real instrument based on the user input data;
in response to the user input data, generate instrument data indicative of a current state of the real instrument as modified by the user input data;
generate display data associated with the instrument data;
send the display data to the internet enabled computer to allow the user to view data indicative of the real instrument in real-time thereby to simulate the user using the real instrument in real-time; and
execute a machine learning process to iteratively learn patterns and relationships between the inputs and outputs to the real instrument to learn how the real instrument operates and to build a dynamic model of the associated digital twin instrument by updating the virtual instrument software of the associated digital twin instrument based on received instrument data from the real instrument, by updating or adding rules or functions to the static model of the virtual instrument.