US 11,861,511 B2
Systems and methods for ensuring safe, norm-conforming and ethical behavior of intelligent systems
Matthias J. Scheutz, Boston, MA (US); and Thomas H. Arnold, Andover, MA (US)
Assigned to Trustees of Tufts College, Medford, MA (US)
Appl. No. 16/652,866
Filed by Trustees of Tufts College, Medford, MA (US)
PCT Filed Oct. 3, 2018, PCT No. PCT/US2018/054073
§ 371(c)(1), (2) Date Apr. 1, 2020,
PCT Pub. No. WO2019/070790, PCT Pub. Date Apr. 11, 2019.
Claims priority of provisional application 62/567,816, filed on Oct. 4, 2017.
Prior Publication US 2020/0302311 A1, Sep. 24, 2020
Int. Cl. G06N 5/04 (2023.01); G06N 20/00 (2019.01); G06F 11/26 (2006.01); G06F 11/34 (2006.01); G06F 9/455 (2018.01)
CPC G06N 5/04 (2013.01) [G06F 9/455 (2013.01); G06F 11/261 (2013.01); G06F 11/3457 (2013.01); G06N 20/00 (2019.01)] 18 Claims
OG exemplary drawing
 
1. A method comprising:
generating a clone of an intelligent system, the intelligent system configured of operating in a real-world environment;
generating, within a simulation environment, a simulation of at least a portion of the real-world environment;
connecting the clone to the simulation included in the simulation environment;
testing, within the simulation environment, operation of the simulation by the clone; and
determining whether the clone passes or fails the testing,
wherein at least a portion of the intelligent system runs on a virtual machine of at least one data processing device that includes a physical hardware layer that interfaces to the real-world environment, and
further wherein the generating of the clone, the generating of the simulation, the connecting, the testing and the determining are performed within a hardware/software layer of at least one data processing device that is disposed between the physical hardware layer and the virtual machine running the at least a portion of the intelligent system;
wherein:
if the clone passes the testing, permitting the intelligent system to continue operating in the real-world environment; and
if the clone fails the testing, overriding the intelligent system in the real-world environment, wherein the overriding is performed within the hardware/software layer disposed between the physical hardware layer and the virtual machine running the at least a portion of the intelligent system.