US 11,755,612 B2
Techniques and architectures for importation of large data load
Brian Jaffery Tajuddin, Seattle, WA (US); Samarpan Jain, Fremont, CA (US); Reuben Cornel, San Francisco, CA (US); Siddhi Mehta, Milpitas, CA (US); Brian Esserlieu, Walnut Creek, CA (US); Jan Asita Fernando, San Francisco, CA (US); Jonathan Mark Bruce, San Rafael, CA (US); and Jay Hurst, San Francisco, CA (US)
Assigned to Salesforce, Inc., San Francisco, CA (US)
Filed by Salesforce, Inc., San Francisco, CA (US)
Filed on Feb. 1, 2021, as Appl. No. 17/164,731.
Application 17/164,731 is a continuation of application No. 15/374,787, filed on Dec. 9, 2016, granted, now 10,908,947.
Prior Publication US 2021/0311776 A1, Oct. 7, 2021
Int. Cl. G06F 16/25 (2019.01); G06F 11/07 (2006.01); G06F 9/48 (2006.01); G06F 16/23 (2019.01)
CPC G06F 16/258 (2019.01) [G06F 9/4881 (2013.01); G06F 11/0766 (2013.01); G06F 16/2386 (2019.01)] 18 Claims
OG exemplary drawing
 
1. A method for ingesting data, the method comprising:
receiving a request to import data into a database environment;
analyzing a characteristic of importing the data to determine whether the request is associated with a first type of data load or a second type of data load;
when the request is associated with the first type of data load, invoking a first handler for the first type of data load for performing:
performing validation operations on the data;
importing records of the data that is validated into the database environment;
writing validation data to an error file, wherein the validation data is deduplicated while writing so that identical errors are written to the error file once;
during validation, tracking a number of records that have validation errors based on the validation operations; and
terminating the importing of the data and generating a user notification when the number of records having validation errors being tracked meets a threshold and has reached a selected percentage of total records failed; and
when the request is associated with a second type of data load, importing the data into the database environment using a second handler for the second type of data load without performing the validation operations on the data.