US 12,249,150 B2
Instrument monitoring device and instrument monitoring method
Yoshiyuki Sankai, Ibaraki (JP)
Assigned to CYBERDYNE Inc., Ibaraki (JP)
Appl. No. 17/776,957
Filed by CYBERDYNE Inc., Ibaraki (JP)
PCT Filed Nov. 24, 2020, PCT No. PCT/JP2020/043627
§ 371(c)(1), (2) Date May 13, 2022,
PCT Pub. No. WO2021/106859, PCT Pub. Date Jun. 3, 2021.
Claims priority of application No. 2019-212107 (JP), filed on Nov. 25, 2019.
Prior Publication US 2022/0392227 A1, Dec. 8, 2022
Int. Cl. G06V 20/52 (2022.01); G06T 7/20 (2017.01); G06T 7/70 (2017.01); G06V 20/60 (2022.01); G10L 15/22 (2006.01)
CPC G06V 20/52 (2022.01) [G06T 7/20 (2013.01); G06T 7/70 (2017.01); G06V 20/60 (2022.01); G10L 15/22 (2013.01); G06T 2207/30004 (2013.01); G06T 2207/30232 (2013.01); G06V 2201/034 (2022.01)] 12 Claims
OG exemplary drawing
 
1. A tool monitoring apparatus for monitoring each of a plurality of tools when a worker performs specified work on an object in a workroom and while each tool is sequentially extracted from a tool holder, in which the plurality of tools are retained, until each tool is housed in a tool counter,
the tool monitoring apparatus comprising:
a plurality of image capturing units that capture images centered on an area around hands of the worker and the object inside the workroom from multiple directions;
an image recognition unit that recognizes each of the tools, which moves within an image capture range of the plurality of the image capturing units, in the images with respect to each of the image capturing units while assigning an identifier to each of the tools in an order of extraction from the tool holder;
a movement locus storage unit that stores trace data indicating a movement locus of each tool by associating the trace data with the identifier corresponding thereto, while tracking a moving position of each tool for each of the image capturing units with respect to each tool recognized in the images by the image recognition unit;
a loss judgment unit that judges that the tool is lost, at a second time point after an elapse of a certain amount of time or longer after a first time point when it is judged that the tool is not recognized at all in the images by the image recognition unit, with respect to each of the tools regarding which the trace data is stored by the movement locus storage unit; and
a movement locus display unit that displays the movement locus of the tool based on the trace data on a screen, with respect to the tool which is judged by the loss judgment unit to have been lost, by reading the trace data corresponding to heads-up time, which is from immediately before the first time point to immediately after the first time point, from the movement locus storage unit with reference to the corresponding identifier.