| CPC G06F 16/2237 (2019.01) [G06F 16/278 (2019.01); G06F 16/287 (2019.01)] | 20 Claims |
|
1. A method for performing vector similarity search (VSS), the method comprising:
performing, by a computing device based on a set of predetermined thresholds, a sparse partitioning of n-dimensional hyperspace into a plurality of grid bins to generate a hyperspace partition map, wherein n is a number, wherein only grid bins of the plurality of grid bins to which vectors mapped to are managed by the hyperspace partition map, wherein each grid bin of the plurality of grid bins points to a unique storage location in a storage device;
ingesting, by the computing device, a vector of the vectors;
using, by the computing device based on a result of a determination made in real-time as to whether the vector is related to incorrect information, the hyperspace partition map to identify a grid bin of the plurality of grid bins for the ingested vector and to identify coordinates of the grid bin;
mapping, by the computing device, the ingested vector to the grid bin;
hashing, by the computing device, the coordinates to generate a storage location key associated with a second storage location, wherein the ingested vector is stored to the second storage location using the storage location key;
after the ingested vector is stored to the second storage location:
receiving a request from a user via a device;
upon the receiving the request, performing the VSS to query a second vector related to the request;
using the hyperspace partition map to:
look up a second grid bin to which the second vector should be mapped, wherein the looking up includes making a determination that vector query latency is greater than a user-defined service level agreement;
identify second coordinates of the second grid bin,
identify a neighboring grid bin of the second grid bin, and
identify third coordinates of the neighboring grid bin;
hashing, by the computing device, the second coordinates of the second grid bin to generate a second storage location key that is different from the storage location key;
hashing, by the computing device, the third coordinates of the neighboring grid bin to generate a third storage location key that is different from the storage location key and the second storage location key;
while performing the VSS, finding, by the computing device via the second storage location key and the third storage location key, a set of vectors stored in storage locations pointed by the second grid bin and the neighboring grid bin;
computing, by the computing device, a distance between the second vector and each vector of the set of vectors;
identifying, by the computing device based on the computing the distance, a third vector that is closer to the second vector; and
initiating, via a graphical user interface of the computing device, displaying of data associated with the third vector in as a response to the request; and
prior to the receiving the request from the user via the device: updating, after deleting the vector from the storage device, the hyperspace partition map stored in the storage device while performing a second VSS on a computing device, the second VSS is different from the VSS.
|