US 12,462,547 B2
Robotic kitchen assistant for preparing food items in a commercial kitchen and related methods
David Zito, Pasadena, CA (US); Sean Olson, Pacific Palisades, CA (US); Ryan Sinnet, Pasadena, CA (US); Robert Anderson, Pasadena, CA (US); Benjamin Pelletier, Pasadena, CA (US); Grant Stafford, Pasadena, CA (US); Zachary Vinegar, Los Angeles, CA (US); and William Werst, Pasadena, CA (US)
Assigned to Miso Robotics, Inc., Pasadena, CA (US)
Filed by Miso Robotics, Inc., Pasadena, CA (US)
Filed on Jun. 27, 2023, as Appl. No. 18/215,123.
Application 18/215,123 is a division of application No. 16/490,534, abandoned, previously published as PCT/US2018/021066, filed on Mar. 6, 2018.
Claims priority of provisional application 62/592,130, filed on Nov. 29, 2017.
Claims priority of provisional application 62/467,743, filed on Mar. 6, 2017.
Claims priority of provisional application 62/467,735, filed on Mar. 6, 2017.
Prior Publication US 2023/0351740 A1, Nov. 2, 2023
Int. Cl. G06V 10/82 (2022.01); A23L 5/10 (2016.01); A47J 36/32 (2006.01); A47J 37/06 (2006.01); A47J 37/12 (2006.01); B25J 9/00 (2006.01); G02B 27/01 (2006.01); G05B 19/42 (2006.01); G06F 18/2413 (2023.01); G06Q 10/0631 (2023.01); G06Q 20/20 (2012.01); G06Q 50/12 (2012.01); G06T 7/70 (2017.01); G06T 7/73 (2017.01); G06V 10/764 (2022.01); G06V 20/52 (2022.01); G06V 20/68 (2022.01)
CPC G06V 10/82 (2022.01) [A23L 5/10 (2016.08); A23L 5/15 (2016.08); A47J 36/32 (2013.01); A47J 37/06 (2013.01); A47J 37/12 (2013.01); B25J 9/0081 (2013.01); G02B 27/017 (2013.01); G05B 19/42 (2013.01); G06F 18/2414 (2023.01); G06Q 10/06311 (2013.01); G06Q 20/202 (2013.01); G06Q 50/12 (2013.01); G06T 7/70 (2017.01); G06T 7/73 (2017.01); G06V 10/764 (2022.01); G06V 20/52 (2022.01); A23V 2002/00 (2013.01); G05B 2219/36184 (2013.01); G05B 2219/40391 (2013.01); G06T 2207/10024 (2013.01); G06T 2207/10028 (2013.01); G06T 2207/10048 (2013.01); G06T 2207/20084 (2013.01); G06V 20/68 (2022.01)] 20 Claims
OG exemplary drawing
 
1. A robotic kitchen assistant system for preparing a plurality of food items in a kitchen environment, the system comprising:
a base and support frame housing;
a plurality of wheels arranged along the base and operable to allow the system to be rolled across a floor and adjacent kitchen equipment;
a foot pedal extending from the base and including an interlocking feature adapted to lock and release to a mating mounting fixture attachable to the floor;
a robotic arm extending from the housing, and comprising an end;
a tool changer coupled to the housing;
a plurality of tools held by the tool changer, each tool being detachably mountable to the end of the robotic arm, and at least one tool being actuatable by a motion driver in the housing, and wherein one of the plurality of tools is a weight sensitive tool operable to determine the weight of the food item to be cooked when the food item is being manipulated by the weight sensitive tool;
a sensor assembly fixed above the housing by an elongate vertically-extending frame element, and comprising a plurality of cameras aimed at a kitchen workspace including the robotic arm for preparing the plurality of food items, wherein at least one of the plurality of cameras is an IR camera operable to generate IR data of the food items;
a computer arranged within the housing and comprising a processor operable to:
compute a volume of the food item based on the weight calculated from the weight sensitive tool;
determine an estimated cooking time based on the volume;
determine a target surface temperature of the food item based on the weight calculated from the weight sensitive tool;
compute a measured surface temperature of the food item based on the IR data,
determine a remaining cooking time based on the IR data and the target surface temperature arising from the weight sensitive tool;
track system state and compute food preparation instructions based on the system state, and data from the at least one camera, order and recipe information, wherein the food preparation instructions comprise food preparation steps for the robotic arm, and optionally the human kitchen worker, to complete;
command the robotic arm to perform a food preparation step on the plurality of food items in the kitchen workspace based on the compute step and while constantly monitoring for the presence of a human kitchen worker in the path of the robotic arm and signal an immediate stop, different path, or to slow down, if a human kitchen worker is detected; and
a display coupled to the housing and operable to show to the human kitchen worker (a) status of the orders in real-time, and (b) food items being cooked in real-time, and optionally augmenting each food item being displayed with said remaining cooking time.