US 12,124,353 B2
Operation logs acquiring device, operation logs acquiring method, and operation logs acquiring program
Kimio Tsuchikawa, Musashino (JP); Takeshi Masuda, Musashino (JP); Fumihiro Yokose, Musashino (JP); Sayaka Yagi, Musashino (JP); and Yuki Urabe, Musashino (JP)
Assigned to Nippon Telegraph and Telephone Corporation, Tokyo (JP)
Appl. No. 17/617,175
Filed by Nippon Telegraph and Telephone Corporation, Tokyo (JP)
PCT Filed Jun. 11, 2019, PCT No. PCT/JP2019/023167
§ 371(c)(1), (2) Date Dec. 7, 2021,
PCT Pub. No. WO2020/250320, PCT Pub. Date Dec. 17, 2020.
Prior Publication US 2022/0342794 A1, Oct. 27, 2022
Int. Cl. G06F 11/34 (2006.01)
CPC G06F 11/3476 (2013.01) [G06F 2201/86 (2013.01)] 4 Claims
OG exemplary drawing
 
1. A device for logging user operations, the device comprising:
processing circuitry configured to:
store a determination image that is an image of a characteristic area of an operation screen that is used to perform a predetermined operation, an event occurrence area in which an event may occur when the predetermined operation is performed, the event occurrence areas are different with each other for each operation type, operation type specifying information that specifies the operation type, and an acquisition image area that is to be acquired when the predetermined operation is performed, in association with each other;
monitor user input events for user inputs via one or more input devices;
upon detecting an occurrence of a user input event, acquire an occurrence time of the user input event, an occurrence position of the user input event on the operation screen, and a captured image of the operation screen; and
determine whether or not the captured image acquired includes a matching area that the determination image matches, acquire event occurrence areas of the determination image corresponding to each operation type, respectively, if the captured image includes the matching area, determine whether the occurrence position of the user input event is included in the event occurrence area, acquire an operation image that is a captured image of the acquisition image area if the occurrence position of the user input event is included in any of the acquired event occurrence areas, and record operation type specifying information that corresponds to the determination image, the occurrence time of the user input event, and the operation image, in association with each other as an operation log.