Electronic Scripting Products, Inc.

Optical Biopsy Technologies, Inc.

Other

Leland Stanford Junior University

N K Technology Ltd

Screen Holdings Co., Ltd.

Optreal Biopsy Technologies, Inc.

Electronics Scripting Products, Inc.

Opeical Biopsy Technologies, Inc.

Michael J Mandella

Hector H Gonzalez-Banos

Gordon S Kino

Marek Alboszta

Mark H Garrett

Dale Buermann

Stewart R Carl

Abdul Rahim Forouhi

Olav Solgaard

Guanghua G Zhang

Christopher H Contag

Timothy J Hawks

Butrus Thomas Khuri-Yakub

Ning Yeung Chan

Omer Oralkan

Jae Eun Hwang

Joseph Russell Landry

Ryosuke Itoh

Hyejun Ra

Cristina Zavaleta

Paul Cristman

Jonathan T C Liu

Hector H Gonzalez

Zhen Qiu

Jae-Woong Jeong

The present invention discloses systems and methods for both viewing and interacting with a virtual reality (VR), an augmented reality (AR) or a mixed reality (MR). More specifically, the systems and methods allow the user to interact with aspects of such realities including virtual items presented in such realities or within such environments by manipulating a control device that has an inside-out camera mounted on-board. The apparatus or system uses two distinct representations including a reduced representation in determining the pose of the control device and uses these representations to compute an interactive pose portion of the control device to be used for interacting with the virtual item. The reduced representation is consonant with a constrained motion of the control device.

Realistic virtual/augmented/mixed reality viewing and interactions

A system, including a structured illumination stage to provide a spatially modulated imaging field is provided. The system further includes a spatial frequency modulation stage to adjust the frequency of the spatially modulated imaging field, a sample interface stage to direct the spatially modulated imaging field to a sample, and a sensor configured to receive a plurality of fluorescence emission signals from the sample. The system also includes a processor configured to reduce a sample scattering signal and to provide a fluorescence emission signal from a portion of the sample including the spatially modulated imaging field. A method for using the above system to form an image of the sample is also provided.

Structured illumination in inverted light sheet microscopy

The present disclosure describes a Raman topography system, which includes a container with at least one Raman probe positioned within. Also described are methods of imaging which include the Raman topography system.

Raman topography system and methods of imaging

A manipulated object using high optical contrast features and an on-board photodetector that detects light produced, reflected or scattered by the features and outputs data indicative thereof. The manipulated object uses one or more controllers to determine its position and/or orientation, including full pose, in the real three-dimensional environment based on data from the photodetector. Data from one or more auxiliary motion sensing devices, e.g., a relative motion sensor such as an inertial device or other auxiliary motion device relying on acoustics, optics or electromagnetic waves within or outside the visible spectrum, can be used to supplement the position and/or orientation data from the photodetector. The manipulated object can be embodied by any suitable device manipulated by a user, including a tablet computer or a phone.

Computer interface for manipulated objects with an absolute pose detection component

A remotely controlled object (e.g., an aircraft) or a wearable article (e.g., virtual or augmented reality glasses) using high optical contrast features and a photodetector that detects light produced, reflected or scattered by the features and outputs data indicative thereof. The remotely controlled object or wearable article uses one or more controllers to determine its position and/or orientation in the real three-dimensional environment based on data from the photodetector. Data from one or more auxiliary motion sensing devices, e.g., a relative motion sensor such as an inertial device or other auxiliary motion device relying on acoustics, optics or electromagnetic waves within or outside the visible spectrum, can be used to supplement the position and/or orientation data from the photodetector.

Computer interface for remotely controlled objects and wearable articles with absolute pose detection component

Aspects of the present invention are directed to apparatuses, arrangements, systems and methods for collecting information using one or more modalities. As consistent with one or more embodiments, an apparatus includes first and second scanning mirror arrangements having different scanning axes and respectively facing different directions. The first scanning mirror arrangement directs source light and image light in two paths, and the second scanning mirror arrangement directs image light from a target to the first scanning mirror arrangement. The first and second scanning mirror arrangements cooperatively scan source light from the first scanning mirror and via the second scanning mirror to target locations with at least two degrees of freedom, and direct image light from the target locations.

Confocal microscope, system and method therefor

Efficient techniques of recovering the pose of an optical apparatus exploiting structural redundancies due the conditioned motion of an apparatus are disclosed. The techniques are based on determining a reduced homography consonant to the conditioned motion of the optical apparatus. The optical apparatus comprises an optical sensor on which space points are imaged as measured image points. The reduced homography is based on a reduced representation of the space points, obtained by exploiting the structural redundancy in the measured image points due to the conditioned motion. The reduced representation consonant with the conditioned motion is defined by rays in homogeneous coordinates and contained in a projective plane of the optical sensor.

Reduced homography based on structural redundancy of conditioned motion

Various aspects as described herein are directed to apparatuses, methods, and systems including a sandwiched arrangement having a light-access port, a scanning mirror, an optics region, and a spacer. The spacer provides a light-directing optical region that separates the scanning mirror and the optics region, and includes a mirrored surface that reflects light between the light-access port and the optics region. Additionally, the optics region includes a curved-shaped window that provides a field of view by communicating beams of light between a target region. The optics region also includes a curved-shaped mirror having a surface that reflects light between the scanning mirror and the mirrored surface. Light beams, as conveyed between the light-access port and the curved-shaped window, are folded by being reflected off the scanning mirror, the curved-shaped mirror and the mirrored surface.

Arrayed dual axis confocal microscopes

A wearable article, such as glasses for a virtual reality program or glasses for an augmented reality application, using light sources and a photodetector that detects their light and outputs data indicative of the detected light. The wearable article uses one or more controllers to determine its position and/or orientation in the environment based on the data output by the photodetector. Data from one or more auxiliary motion sensing devices, e.g., a relative motion sensor such as an inertial device or other auxiliary motion device relying on acoustics, optics or electromagnetic waves within or outside the visible spectrum, can be used to supplement the position and/or orientation data from the photodetector.

Computer interface employing a wearable article with an absolute pose detection component

The present invention relates to interfaces and methods for producing input for software applications based on the absolute pose of an item manipulated or worn by a user in a three-dimensional environment. Absolute pose in the sense of the present invention means both the position and the orientation of the item as described in a stable frame defined in that three-dimensional environment. The invention describes how to recover the absolute pose with optical hardware and methods, and how to map at least one of the recovered absolute pose parameters to the three translational and three rotational degrees of freedom available to the item to generate useful input. The applications that can most benefit from the interfaces and methods of the invention involve 3D virtual spaces including augmented reality and mixed reality environments.

Deriving input from six degrees of freedom interfaces

A reduced homography H for an optical apparatus to recover pose parameters from imaged space points Pusing an optical sensor. The electromagnetic radiation from the space points Pis recorded on the optical sensor at measured image coordinates. A structural uncertainty introduced in the measured image points is determined and a reduced representation of the measured image points is selected based on the type of structural uncertainty. The reduced representation includes rays {circumflex over (r)}defined in homogeneous coordinates and contained in a projective plane of the optical apparatus. At least one pose parameter of the optical apparatus is then estimated by applying the reduced homography H and by applying a condition on the motion of the optical apparatus, the condition being consonant with the reduced representation employed in the reduced homography H.

Reduced homography for recovery of pose parameters of an optical apparatus producing image data with structural uncertainty

A system with a remote control or wand equipped with a relative motion sensor that outputs data indicative of a change in position. The system has one or more light sources and a photodetector that detects their light and outputs data indicative of the detected light. One or more controllers are used to determine the absolute position of the wand based on the data output by the relative motion sensor and by the photodetector. The wand's absolute pose is determined from the data and includes the absolute position of a reference point on the wand and the wand's absolute orientation. A display is used to show an image defined by two orthogonal axes, e.g., those of world coordinates (X,Y,Z). The display is rendered as a function of absolute position of the wand in or along a third orthogonal axis.

Computer interface employing a manipulated object with absolute pose detection component and a display

A system that has a remote control, e.g., a wand, equipped with a relative motion sensor that outputs data indicative of a change in position of the wand. The system also has one or more light sources and a photodetector that detects their light and outputs data indicative of the detected light. The system uses one or more controllers to determine the absolute position of the wand based on the data output by the relative motion sensor and by the photodetector. The data enables determination of the absolute pose of the wand, which includes the absolute position of a reference point chosen on the wand and the absolute orientation of the wand. To properly express the absolute parameters of position and/or orientation of the wand a reference location is chosen with respect to which the calculations are performed. The system is coupled to a display that shows an image defined by a first and second orthogonal axes such as two axes belonging to world coordinates (X,Y,Z). The one or more controllers are configured to generate signals that are a function of the absolute position of the wand in or along a third axis for rendering the display. To simplify the mapping of a real three-dimensional environment in which the wand is operated to the cyberspace of the application that the system is running, the third axis is preferably the third Cartesian coordinate axis of world coordinates (X,Y,Z).

An apparatus and method for processing pose data derived from a pose of an elongate object such as, for example, a jotting implement, a pointer, a robotic arm or a cane. The elongate object has a tip contacting a plane surface with one or more invariant features. The pose of the elongate object is measured optically from on-board by an optical measuring system with the aid of the invariant feature. The pose is used for preparing a corresponding pose data and a subset of the pose data is identified and transmitted to an application such as a user application, where the subset can serve as command data or input data. Since the elongate object moves while its tip is contacting the surface the pose is measured periodically at sufficiently frequent measurement times tto describe the motion at a desired temporal resolution. The subset can include all or a portion of the orientation data that describe the orientation of the elongate object in space and/or position data of the tip on the surface. The position can be a relative position of the tip with respect to any feature or its previous position, or an absolute position in world coordinates. The subset can also contain a mix of orientation and position data.

Growing community of inventors

Palo Alto, CA, United States of America

Michael J Mandella