US 12,462,615 B2
System and method for computer vision assisted parking space monitoring with automatic fee payments
Anil Merchant, Dublin, CA (US); Harris Graves Lummis, Houston, TX (US); Ganesh Sankar Harsha Vardhan Vanama, Eastvale, CA (US); Sai Prajwal Kotamraju, Los Angeles, CA (US); Raghak Radhakrishnan, Los Angeles, CA (US); Ashwin Mohan, Pasadena, CA (US); and Michelle L'Estrange, Danville, CA (US)
Filed by Automotus Inc., Los Angeles, CA (US)
Filed on May 15, 2024, as Appl. No. 18/665,031.
Application 18/665,031 is a continuation in part of application No. 17/962,169, filed on Oct. 7, 2022, abandoned.
Prior Publication US 2024/0296700 A1, Sep. 5, 2024
Int. Cl. G07B 15/02 (2011.01); G06Q 30/0283 (2023.01); G06Q 30/04 (2012.01); G06T 7/20 (2017.01); G06T 7/70 (2017.01); G06V 10/26 (2022.01); G06V 20/52 (2022.01)
CPC G07B 15/02 (2013.01) [G06Q 30/0284 (2013.01); G06Q 30/04 (2013.01); G06T 7/20 (2013.01); G06T 7/70 (2017.01); G06V 10/26 (2022.01); G06V 20/52 (2022.01); G06T 2207/30232 (2013.01); G06T 2207/30252 (2013.01); G06V 2201/08 (2022.01)] 20 Claims
OG exemplary drawing
 
1. A system for automated parking space monitoring, automated fee charging, and automated payment for parking comprising:
a single parking and traffic monitoring device mounted to monitor at least one parking space and having:
a housing;
at least two feed sensors;
at least one IR sensor;
at least one illumination element;
a power unit;
one or more processors comprising a single processing unit;
a computer vision pipeline;
wherein the computer vision pipeline includes the feed sensors operably coupled to the single processing unit;
wherein a first feed sensor is configured to capture a first image having a field-of-view that is sufficient to cover multiple parking spaces along a street and the first image is at least used to detect at least one object, and a second feed sensor is configured to capture a second image having a narrower field-of-view with higher-resolution that is suitable for detecting and identifying the at least one object present in the first image and the second image;
wherein the single processing unit uses an image processing pipeline to multiplex the first image from the first feed sensor and the second image from the second feed sensor;
wherein the multiplexed images are analyzed by an object inference program to detect and identify all identifiable objects within each image;
wherein the identified objects are tracked through a plurality of images by a multiple object tracking program;
wherein the computer vision pipeline, via edge processing, generates at least one event message for each multiplexed set of images containing the at least one object detected, identified, and tracked as a vehicle that caused at least one observed event related to any of the vehicle's: (1) entrance into any feed sensor's field-of-view, (2) entrance into one of the monitored parking spaces, (3) exit from one of the monitored parking spaces, and (4) exit from any feed sensor's field-of-view;
wherein images are temporarily stored on local memory for each image that contains the vehicle that caused the at least one observed event to occur and generate the at least one event message for that observed event;
wherein the single parking and traffic monitoring device causes a unique identifier to be generated and assigned linking the at least one event message to each image from each feed sensor containing the vehicle;
wherein each vehicle in each image having a unique identifier is tracked throughout each image using the multiple object tracking program;
wherein all of the foregoing operations are accomplished via edge processing on the single parking and traffic monitoring device;
a communication unit capable of communicating via a network;
an image uploader operably connected to the communication unit;
wherein the computer vision pipeline is operably connected to an event ingestion service via the network to receive the at least one event message;
wherein the image uploader is operably connected to an image store via the network to receive the at least one image per feed sensor; and
wherein the single parking and traffic monitoring device sends the event message to the event ingestion service and the at least one image per feed sensor to the image store;
an image indexing service on the network which receives the at least one image from the image store and sorts the at least one image, having the unique identifier, based on at least one sort criteria for the vehicle within the at least one image that caused the observed event;
a data store on the network which receives the at least one event message via the event ingestion service and the at least one image per feed sensor via the image store, matches the at least one event message and the at least one image per feed sensor via matching the unique identifier assigned to the vehicle in each image and each event message, and stores the matching images and event messages;
a transaction creation service on the network that assesses whether a transactionable event occurred based more than one event messages, which are linked by the unique identifier assigned to the vehicle that caused the event messages to be generated, using a policy API to determine a fee for a user based on the event messages;
an application including a user API that allows the user to operably connect to the system via the network;
wherein a driver who is the user may, prior to the at least one observed event, register an account including information related to the driver and provide to the system (1) information related to at least one payment method, (2) features of the driver's vehicle, and (3) pre-authorization for payment for at least one fee associated with the driver parking the driver's vehicle in the parking space monitored by the single parking and traffic monitoring device generating at least one transactionable event, wherein the driver's vehicle is the object associated with unique identifier subject to the fee for the transactionable event; and
wherein the system automatically charges the user's payment method for the user's transactionable event without further interaction by the user due to the information provided in the registered user account, active payment method, and pre-authorization for payment for the user's parking subject to fees assessed under the policy API and charged to the user by the system.