US 12,112,112 B2
Method for co-design of hardware and neural network architectures using coarse-to-fine search, two-phased block distillation and neural hardware predictor
Niv Zehngut, Tel-Mond (IL); Amir Ben-Dror, Ness Ziona (IL); Evgeny Artyomov, Rehovot (IL); Michael Dinerstein, Rehovot (IL); and Roy Jevnisek, Givatayim (IL)
Assigned to SAMSUNG ELECTRONICS CO., LTD., Suwon-si (KR)
Filed by SAMSUNG ELECTRONICS CO., LTD., Suwon-si (KR)
Filed on Nov. 12, 2020, as Appl. No. 17/095,937.
Prior Publication US 2022/0147680 A1, May 12, 2022
Int. Cl. G06F 30/39 (2020.01); G06F 117/04 (2020.01); G06F 117/08 (2020.01); G06F 119/06 (2020.01); G06N 3/04 (2023.01); G06N 3/082 (2023.01); G06N 3/086 (2023.01)
CPC G06F 30/39 (2020.01) [G06N 3/04 (2013.01); G06N 3/082 (2013.01); G06N 3/086 (2013.01); G06F 2117/04 (2020.01); G06F 2117/08 (2020.01); G06F 2119/06 (2020.01)] 20 Claims
OG exemplary drawing
 
1. A method comprising:
identifying a plurality of dimensions of a neural architecture search into a first group of dimensions including a coarse dimension and a second group of dimensions including a fine dimension;
selecting a value of the coarse dimension during a coarse search phase of a neural architecture search using a controller based search algorithm, wherein the controller based search algorithm for the coarse search phase includes iteratively altering the value of the coarse dimension using a policy function;
identifying a search space corresponding to the fine dimension from the second group of dimensions for a fine search phase of the neural architecture search based at least in part on the value of the coarse dimension selected during the coarse search phase of the neural architecture search;
selecting a value for the fine dimension during the fine search phase of the neural architecture search using a differential search algorithm, wherein the differential search algorithm is based on a loss function; and
generating a neural network architecture based on the value of the coarse dimension and the value of the fine dimension.