| CPC G06V 20/17 (2022.01) [G06F 16/583 (2019.01); G06F 18/214 (2023.01); G06N 20/00 (2019.01); G06V 10/751 (2022.01); G06V 20/10 (2022.01); G06V 20/64 (2022.01)] | 20 Claims |
|
1. A system comprising:
a first aerial vehicle comprising:
a camera; and
an inertial measurement unit comprising at least one gyroscope and at least one accelerometer, at least one magnetometer and at least one position sensor; and
at least one computer system in communication with the first aerial vehicle, wherein the at least one computer system is programmed with one or more sets of instructions that, when executed by the at least one computer system, cause the at least one computer system to perform a method comprising:
receiving a first image from the first aerial vehicle, wherein the first image was captured by the camera at a first time;
identifying a first position and a first orientation of the camera at the first time, wherein the first position and the first orientation are identified based at least in part on data captured by the inertial measurement unit at the first time;
receiving a second image from the first aerial vehicle, wherein the second image was captured by the camera at a second time;
identifying a second position and a second orientation of the camera at the second time, wherein the second position and the second orientation are identified based at least in part on data captured by the inertial measurement unit at the second time;
generating a trajectory having a plurality of positions in three-dimensional space, wherein at least a first position of the trajectory is in a field of view of the camera at the first time and at least a second position of the trajectory is in the field of view of the camera at the second time;
determining a first location within an image plane of the camera at the first time, wherein the first location corresponds to the first position of the trajectory at the first time;
determining a second location within the image plane of the camera at the second time, wherein the second location corresponds to the second position of the trajectory at the second time;
estimating a homography between the image plane of the camera at the first time and the image plane of the camera at the second time based at least in part on the first image and the second image;
adjusting the second location within the image plane of the camera at the second time based at least in part on the homography;
rendering a third image, wherein the third image depicts a second aerial vehicle in the first location of the image plane at the first time;
rendering a fourth image, wherein the fourth image depicts the second aerial vehicle in the adjusted second location of the image plane at the second time;
generating a fifth image, wherein the fifth image is generated by compositing the first image and the third image; and
generating a sixth image, wherein the sixth image is generated by compositing the second image and the fourth image.
|
|
6. A method comprising:
capturing, by a camera provided aboard a first aerial vehicle, a first image at a first time;
determining, based at least in part on first data captured by an inertial measurement unit provided aboard the first aerial vehicle, a first position of the camera at the first time and a first orientation of the camera at the first time;
capturing, by the camera, a second image at a second time;
determining, based at least in part on second data captured by the inertial measurement unit, a second position of the camera at the second time and a second orientation of the camera at the second time;
generating a trajectory of a synthetic object, wherein the trajectory comprises a first position of the synthetic object at the first time and a second position of the synthetic object at the second time, wherein the first position of the synthetic object at the first time is within a field of view of the camera at the first time, and wherein the second position of the synthetic object at the second time is within the field of view of the camera at the second time;
estimating a homography between the first image and the second image;
selecting a first location of the synthetic object within the first image based at least on the trajectory;
selecting a second location of the synthetic object within the second image based at least in part on the trajectory and the homography;
generating a first composite image based at least in part on the first image and the first location of the synthetic object within the first image, wherein the first composite image comprises a first rendered image of the synthetic object at the first time located in the first location of the synthetic object within the first image;
generating a second composite image based at least in part on the second image and the second location of the synthetic object within the second image, wherein the second composite image comprises a second rendered image of the synthetic object at the second time in the second location of the synthetic object within the second image; and
storing the first composite image and the second composite image in association with the first location and the second location in at least one data store.
|
|
16. A method comprising:
capturing, by a camera provided aboard an aerial vehicle, a sequence of images at a plurality of times;
identifying data captured by an inertial measurement unit provided aboard the aerial vehicle at the plurality of times;
determining, based at least in part on the data, positions and orientations of the camera at each of the plurality of times;
identifying a trajectory of a synthetic object, wherein the trajectory comprises a plurality of points in three-dimensional space over at least some of the plurality of times;
determining, for each of the plurality of points, a location corresponding to the point within an image plane of the camera at one of the times, wherein determining the location corresponding to the point comprises projecting the point into the image plane of the camera at the one of the plurality of times;
estimating a homography for each pair of consecutive images of the sequence, wherein each of the homographies is a relation between at least one pixel of the image plane of the camera at the one of the times and the at least one pixel of the image plane of the camera at a preceding one of the images of the sequence;
adjusting each of the locations corresponding to the point within the image plane of the camera at the one of the plurality of times based at least in part on a corresponding one of the homographies;
generating, for each of the images of the sequence, a rendered image of the synthetic object based at least in part on the trajectory;
compositing, for each of the images of the sequence, the rendered image of the synthetic object into the one of the images of the sequence for which the rendered image was generated at the adjusted location; and
storing the composited images in association with the adjusted locations in at least one data store.
|