US 12,443,574 B1
Systems and methods for dynamic evaluation of metadata consistency and data reliability
Shitij Kulshreshtha, Glen Allen, VA (US); Rama Mohan Boppana, Glen Allen, VA (US); Sean Murray, McLean, VA (US); Jared W. McDaniel, McLean, VA (US); Rahul Babu Mummaneni, McLean, VA (US); Andrew Seaton, McLean, VA (US); Rachel Rogers, McLean, VA (US); and Sudeesh Madhusudanan, McLean, VA (US)
Assigned to Capital One Services, LLC, McLean, VA (US)
Filed by Capital One Services, LLC, McLean, VA (US)
Filed on Apr. 11, 2024, as Appl. No. 18/633,389.
Int. Cl. G06F 16/215 (2019.01); G06F 16/11 (2019.01); G06F 16/23 (2019.01)
CPC G06F 16/215 (2019.01) [G06F 16/125 (2019.01); G06F 16/2379 (2019.01)] 20 Claims
OG exemplary drawing
 
1. A system for efficiently minimizing excess data retention in data management systems while reducing computer resource utilization involved in data retention decisions involving security protocols using dynamic evaluation of metadata consistency and data quality, the system comprising:
one or more processors; and
one or more non-transitory, computer-readable media storing instructions that, when executed by the one or more processors, cause operations comprising:
retrieving, via a database associated with a data management system, (i) first metadata associated with a first set of retained data and (ii) second metadata associated with a second set of retained data, wherein the first set of retained data comprises a first set of records and the second set of retained data comprises a second set of records;
retrieving a metadata ruleset indicating retention criteria to retain data;
generating (i) a first metadata consistency metric indicating a first measure of consistency of the first metadata with the metadata ruleset and (ii) a second metadata consistency metric indicating a second measure of consistency of the second metadata with the metadata ruleset;
in response to determining that the first metadata consistency metric is greater than a threshold consistency metric, determining to process each record of the first set of records as a batch in lieu of processing each record of the first set of records independently, wherein determining to process each record of the first set of records as the batch comprises generating a first quality metric corresponding to an entirety of the first metadata;
retaining the first set of retained data based on the first quality metric satisfying a threshold quality metric to adhere to security protocols associated with the first set of retained data;
in response to determining that the second metadata consistency metric is not greater than the threshold consistency metric, determining to process each record of the second set of records independently in lieu of processing each record of the second set of records as a batch, wherein determining to process each record of the second set of records independently comprises generating a respective quality metric for each record of the second set of records; and
retaining records of the second set of records of the second set of retained data based on respective quality metrics for of the second set of records satisfying the threshold quality metric to adhere to security protocols associated with the second set of retained data.