US 11,734,584 B2
Multi-modal construction of deep learning networks
Rahul A R, Bangalore (IN); Neelamadhav Gantayat, Bangalore (IN); Shreya Khare, Bangalore (IN); Senthil K K Mani, Bangalore (IN); Naveen Panwar, Bangalore (IN); and Anush Sankaran, Bangalore (IN)
Assigned to International Business Machines Corporation, Armonk, NY (US)
Filed by International Business Machines Corporation, Armonk, NY (US)
Filed on Apr. 19, 2017, as Appl. No. 15/491,162.
Prior Publication US 2018/0307978 A1, Oct. 25, 2018
Int. Cl. G06N 5/04 (2023.01); G06N 3/10 (2006.01); G06N 5/022 (2023.01); G06F 30/20 (2020.01); G06F 111/10 (2020.01)
CPC G06N 5/04 (2013.01) [G06F 30/20 (2020.01); G06N 3/105 (2013.01); G06N 5/022 (2013.01); G06F 2111/10 (2020.01)] 20 Claims
OG exemplary drawing
 
1. A computer-implemented method for generating source code for a deep learning network, comprising:
extracting, from user-provided multi-modal inputs, one or more items related to generating a deep learning network model, wherein the one or more extracted items comprise an image of a deep learning network design and a classification task to be performed by the deep learning network model;
processing the image to extract information comprising at least one of: text from the image using an optical character recognition process; and edges and nodes from the image using an edge detection process;
retrieving a similar pre-existing deep learning network model, created using deep learning source code corresponding to a first library language, from a repository based at least in part on a comparison of: the information extracted from the image of the deep learning network design; and multiple deep learning network models stored in the repository;
adapting the retrieved pre-existing deep learning network model to the one or more multi-model inputs to generate the deep learning network model, wherein said adapting comprises: changing a last layer in the pre-existing deep learning network model to have a number of nodes corresponding to a number of classes associated with the classification task to be performed by the deep learning network model; and re-training and updating one or more parameters of the retrieved pre-existing deep learning network model;
creating an intermediate representation of the deep learning network model, wherein the intermediate representation comprises: one or more items of data pertaining to the deep learning network model; and one or more design details attributed to the deep learning network model;
automatically converting the intermediate representation into source code corresponding to a second library language that is different than the first library language;
automatically performing a static validation of the deep learning source code to determine whether one or more specified network layers are present in the generated deep learning network model; and
outputting the deep learning source code corresponding to the second library language to at least one user;
wherein the steps are carried out by at least one computing device.