CPC G06F 16/951 (2019.01) [H04L 63/1425 (2013.01)] | 6 Claims |
1. A method for collecting data in a data collection device, comprising:
a step A of collecting data using a distributed crawler from a dark web site belonging to a network where channels are established by randomly connecting at least one or more network nodes that perform network routing functions, the dark web being not accessible with a general web browser and being assessable with preset specific software; and
a step B of standardizing the collected data in a preset format and generating metadata for the collected data,
wherein the step A includes
collecting domain information of the network;
identifying whether collected domains have been changed, and preferentially allocating a domain which is identified as being most recently registered to the distributed crawler; and
operating a plurality of network nodes that perform the routing function and collecting data from the dark web corresponding to an arbitrary domain by processing a request of the distributed crawler in the network nodes.
|