Capsovision Inc.

General Electric Company

Beijing Jingdong Shangke Information Technology Co., Ltd.

Jd.com American Technologies Corporation

Micron Technology Incorporated

Yi Xu

Kang-Huai Wang

Chenyu Wu

Gordon Cook Wilson

Hui Zhou

Lynn Ann DeRose

Weston Blaine Griffin

Huan Tan

Balajee Kannan

Ying Mao

Yuzhang Wu

Chan H Yoo

Frederick Wilson Wheeler

Steven R Eskildsen

Akshay N Singh

Shanglin Yang

Liana Foster

Bernhard Erich Hermann Claus

Ali Can

Thomas Baby Sebastian

Jilin Tu

Yang Zhao

Ana Isabel Del Amo

Dan Miao

Chenyu Yu

Xianqiao Tong

Lynn Ann Derose

A method and system for generating a three-dimensional (3D) model of a garment. The method includes: providing an image of the garment; generating a 3D garment representation based on the image; registering the 3D garment representation to a 3D model to obtain a preliminary 3D garment model; projecting the preliminary 3D garment model dressed on the 3D model to an 2D projected image; and comparing the 2D projected image with the image of the garment, so as to refine the preliminary 3D garment model to obtain the final 3D model of the garment.

System and method for automatically generating three-dimensional virtual garment model using product description

An integrated image sensor for capturing a mixed structured-light image and regular image using an integrated image sensor are disclosed. The integrated image sensor comprises a pixel array, one or more output circuits, one or more analog-to-digital converters, and one or more timing and control circuits. The timing and control circuits are arranged to perform a set of actions including capturing a regular image and a structured-light image. According to the present invention, the structured-light image captured before or after the regular image is used to derive depth or shape information for the regular image. An endoscope based on the above integrated image sensor is also disclosed. The endoscope may comprises a capsule housing adapted to be swallowed, where the components of integrated image sensor, a structured light source and anon-structured light source are enclosed and sealed in the capsule housing.

Single image sensor for capturing mixed structured-light images and regular images

A method and apparatus of processing and displaying images captured using an in vivo capsule camera are disclosed. One or more overlapped areas between a target image and each image in a neighboring image group are determined, which comprises at least two neighboring images around the target image. Marked pixels in the target image are then determined, where a pixel in the target image is designated as a marked pixel if the pixel is within an overlapped area between the target image and at least one neighboring image. If the total number of the marked pixels in the target image exceeds a threshold and the number of the marked pixels associated with the overlapped area(s) between the target image and any image in the neighboring image group is below the threshold, the target image is excluded from a set of images to be displayed on a display device.

Method of image processing and display for images captured by a capsule camera

A method and apparatus of processing images captured from human gastrointestinal (GI) tract by a capsule camera are disclosed. High frame-rate images captured from human gastrointestinal (GI) tract by a capsule camera are received for processing. The high frame-rate images comprise first images at a first spatial resolution corresponding to a regular frame rate and second images at a second spatial resolution, the first images and the second images are interleaved, and the second spatial resolution is lower than the first spatial resolution. Motion models among the high frame-rate images are derived by applying image registration to the high frame-rate images. The high frame-rate images are stitched according to the motion models to generate stitching outputs comprising stitched images and non-stitched images. The stitching outputs are provided.

Method and apparatus for image stitching of images captured using a capsule camera

An approach relates to manipulation of tools or instruments in the performance of a task by a robot. In accordance with this approach, sensor data is acquired and processed to identify a subset of instruments initially susceptible to manipulation. The instruments are then manipulated in the performance of the task based on the processed sensor data.

Method and robotic system for manipulating instruments

A method and apparatus for imaging a body lumen are disclosed. According to the method, an imaging apparatus is induced into the body lumen. Structured light from the imaging apparatus is projected into the body lumen. The structured light reflected from anatomical features in the body lumen is detected by the imaging apparatus. A first structured light image is generated from the detected structured light by the imaging apparatus. Non-structured light is emitted from the imaging apparatus into the body lumen. The non-structured light reflected from the anatomical features in the body lumen is detected by the imaging apparatus. A non-structured light image is generated from the detected non-structured light by the imaging apparatus. The frame period of the first structured light image is shorter than the frame period of the non-structured light image. In one embodiment, the imaging apparatus corresponds to a capsule endoscope.

The present approach relates to integrated planning of robot navigation and manipulator motion in performing tasks. In particular, as discussed herein sensor information, such as visual information, may be used for the robot to locate a target object when it arrives at a working location. Adjustments may be made based on this information that may include moving the robot and planning a manipulator arm motion.

Architecture and methods for robotic mobile manipulation system

A method and device for improving the accuracy of depth information derived from a structured-light image for a regular image are disclosed. In one example, an additional structured-light image is captured before a first structured-light image or after a regular image. The depth information for the regular image can be derived from the first structured-light image and corrected by incorporating depth information from the additional structured-light image. A model for depth information can be used to predict or interpolate depth information for the regular image. In another example, two regular sub-images may be captured with a structured-light image in between. If substantial frame differences or substantial global motion vector/block motion vectors are detected, the two regular sub-images will not be combined in order to avoid possible motion smear. Instead, one of the two sub-images will be selected and scaled as the output regular image.

A system and method for constructing a multi-scale voxel representation of a three-dimensional (3D) sparse point map. The method includes: providing the 3D sparse point map; receiving coordinates of a 3D point from the 3D sparse point map; converting the coordinates of the 3D point to integer coordinates; and when the integer coordinates are not encompassed by any voxel entries of a hash table: creating a new unit voxel at scale level 0, scale level 0 being a minimum voxel size; calculating a hash value using the integer coordinates; inserting the new unit voxel into the hash table; and merging the new unit voxel with neighboring voxels along at least one axis of the integer coordinates, so as to obtain the multi-scale voxel representation of the 3D sparse point map.

Three-dimensional representation by multi-scale voxel hashing

A method for simultaneous localization and mapping. The method includes the step of detecting two-dimensional (2D) feature points from a current frame captured by a camera; matching the 2D feature points from the current frame directly to three-dimensional (3D) map points in a 3D map, so as to obtain correspondence between the 2D feature points and the 3D map points; and computing a current pose of the camera based on the obtained correspondence. Each of the 2D feature points and the 3D map points has a feature descriptor. The step of matching is performed by comparing the feature descriptors of the 2D feature points and the feature descriptors of the 3D map points.

System and method for monocular simultaneous localization and mapping

The present approach relates to navigation (e.g., route planning and movement) of robots in an indoor environment shared with humans. The present approach includes detecting human activity over time, including but not limited to human motion; modeling human activities using the historical human activity, and using the modeled human activity to plan robotic motion or movement.

System and methods for robotic autonomous motion planning and navigation

Embodiments of the present disclosure disclose a method and an apparatus for determining a planar surface. A specific embodiment of the method comprises: determining, from obtained images of the environment that include a to-be-determined planar surface, feature points belonging to the to-be-determined planar surface; judging whether the number of the feature points belonging to the to-be-determined planar surface is smaller than a preset number threshold; if yes, generating first prompt information for prompting a user to add feature points on the to-be-determined planar surface; obtaining at least two first images including the added feature points; and in response to the number of the feature points belonging to the to-be-determined planar surface in the first images being no less than the preset number threshold, reconstructing the to-be-determined planar surface based on the first images so as to determine a location and an orientation of the to-be-determined planar surface. This embodiment implements reconstruction of a weakly-textured or texture-less planar surface.

Method and apparatus for determining a planar surface

A system and method for realizing augmented reality. The system includes a visual sensor configured to capture images of an environment and a computing device in communication with the visual sensor. The computing device has a processor and a storage device storing computer executable code. The computer executable code, when executed at the processor, is configured to: identify two-dimensional (2D) objects in the captured images; construct a three-dimensional (3D) map of the environment using the captured images; define 3D objects in the 3D map by mapping the 2D objects identified in the captured images to the corresponding points in the 3D map; and render a 3D model on the 3D map based on the 3D objects defined in the 3D map.

System and method for mobile augmented reality

An integrated image sensor for capturing a mixed structured-light image and regular image using an integrated image sensor are disclosed. The integrated image sensor comprises a pixel array, one or more output circuits, one or more analog-to-digital converters, and one or more timing and control circuits. The timing and control circuits are arranged to perform a set of actions including capturing a regular image and a structured-light image. According to the present invention, the structured-light image captured before or after the regular image is used to derive depth or shape information for the regular image. An endoscope based on the above integrated image sensor is also disclosed. The endoscope may comprises a capsule housing adapted to be swallowed, where the components of integrated image sensor, a structured light source and a non-structured light source are enclosed and sealed in the capsule housing.

Growing community of inventors

Menlo Park, CA, United States of America

Yi Xu