Facebook, Inc.

Meta Platforms Technologies, LLC

Daniel Scharstein

Richard S Szeliski

Jan Herling

David James Geraghty

In one embodiment, a method includes the steps of capturing an image from a camera viewpoint, the image depicting a physical keyboard, detecting one or more shape features of the physical keyboard depicted in the image by comparing pixels of the image to a predetermined shape template, the predetermined shape template representing visual characteristics of spaces between keyboard keys, accessing predetermined shape features of a keyboard model associated with the physical keyboard, and determining a pose of the physical keyboard based on comparisons between (1) the detected one or more shape features of the physical keyboard and (2) projections of the predetermined shape features of the keyboard model toward the camera viewpoint.

Physical keyboard tracking

In one embodiment, a method includes accessing an image of a physical world scene, detecting a number of straight-line segments in the accessed image, identifying a first vanishing point and a second vanishing point in the image, where each vanishing point corresponds to a sub-set of the number of straight-line segments, and where the first vanishing point and the second vanishing point are orthogonal, identifying a planar region in the accessed image represented by one or more straight-line segments associated with the first vanishing point and one or more straight-line segments associated with the second vanishing point, transforming a virtual object associated with the planar region based on one or more properties associated with the planar region, and displaying the transformed virtual object over the image.

Growing community of inventors

Weybridge, VT, United States of America

Daniel Scharstein