	US 12,236,547 B2
	Systems and methods for 3D registration of curves and surfaces using local differential information
João Pedro de Almeida Barreto, Coimbra (PT); and Carolina dos Santos Raposo, Coimbra (PT)
Assigned to SMITH & NEPHEW, INC., Memphis, TN (US)
Filed by Smith & Nephew, Inc., Memphis, TN (US)
Filed on May 3, 2022, as Appl. No. 17/735,592.
Application 17/735,592 is a continuation of application No. 17/020,043, filed on Sep. 14, 2020, granted, now 11,335,075.
Application 17/020,043 is a continuation of application No. 16/492,760, granted, now 10,796,499, issued on Oct. 6, 2020, previously published as PCT/US2018/022512, filed on Mar. 14, 2018.
Claims priority of provisional application 62/471,339, filed on Mar. 14, 2017.
Prior Publication US 2022/0406025 A1, Dec. 22, 2022
Int. Cl. G06T 19/20 (2011.01); G06T 7/00 (2017.01); G06T 7/33 (2017.01); G06V 10/764 (2022.01); G06V 20/64 (2022.01)

CPC G06T 19/20 (2013.01) [G06T 7/0012 (2013.01); G06T 7/344 (2017.01); G06V 10/764 (2022.01); G06V 20/653 (2022.01); G06T 2207/10024 (2013.01); G06T 2207/10028 (2013.01); G06T 2207/10081 (2013.01); G06T 2207/10088 (2013.01); G06T 2207/30008 (2013.01)]

21 Claims

1. A method comprising:

outputting for presentation to a user an annotated video of a surgical site, the annotated video comprising information overlayed on video of the surgical site, wherein the outputting comprises:

identifying, in the video of the surgical site, a three-dimensional (3D) position of a point depicted in the video of the surgical site, wherein identifying the three-dimensional position of the point comprises applying passive reconstruction to one or more frames of the video of the surgical site;

overlaying information on the video of the surgical site to yield the annotated video, wherein overlaying the information on the video of the surgical site comprises aligning a 3D model of the surgical site on the video of the surgical site at a location based on the 3D position of the point determined by applying the passive reconstruction to the one or more frames of the video of the surgical site and/or based on one or more local differential characteristics of a surface including the point at the 3D position of the point; and

outputting the annotated video of the surgical site for presentation,

wherein the 3D model of the surgical site is a second 3D model of the surgical site, and wherein aligning the 3D model of the surgical site comprises aligning the second 3D model to a first 3D model by:

defining a plurality of first points and first vectors in the first 3D model, wherein the first points correspond to points on a surface in the first 3D model and the first vectors correspond to normals to the first points;

identifying at least one second point and second vector in the second 3D model, wherein the at least one second point corresponds to a point on a surface of the second 3D model and the second vector corresponds to a normal to the at least one second point; and

aligning the second vector in the second 3D model to one of the first vectors in the first 3D model.