Bayspec, Inc.

Salesforce, Inc.

Biospex, Inc.

Shu Zhang

Wei Yang

Ran Xu

Caiming Xiong

Li Chen

Ming Chai

Changqing Wang

Junnan Li

Chetan Ramaiah

Ning Yu

Guocai Shu

William Yang

Juan Carlos Niebles Duque

Xinyi Yang

Le Xue

Silvio Savarese

Chia-Chih Chen

Yihao Feng

Embodiments described herein provide a feedback based instructional image editing framework that employs a diffusion process to follow user instruction for image editing. A diffusion model is fine-tuned using a reward model, which may be trained via human annotation. The training of the reward model may be done by having the image editing model output a number of images, which a human annotator ranks based on their alignment with the original image and a given instruction.

Systems and methods for feedback based instructional visual editing

A method of training a neural network based three-dimensional (3D) encoder is provided. A first plurality of samples of a training dataset are generated using a first 3D model. An image generator with multi-view rendering is used to generate a plurality of two-dimensional (2D) images having different viewpoints of the first 3D model. A first language model is used to generate a plurality of texts corresponding to the plurality of 2D images respectively. A first text for a first image is generated by using one or more text descriptions generated by the first language model. A point cloud is generated by randomly sampling points in the 3D model. The first plurality of samples are generated using the plurality of 2D images, the corresponding plurality of texts, and the point cloud. The neural network based 3D encoder is trained using the training dataset including the first plurality of samples.

Systems and methods for multimodal pretraining for three-dimensional understanding models

Embodiments described herein a CROss-Modal Distribution Alignment (CROMDA) model for vision-language pretraining, which can be used for retrieval downstream tasks. In the CROMDA mode, global cross-modal representations are aligned on each unimodality. Specifically, a uni-modal global similarity between an image/text and the image/text feature queue are computed. A softmax-normalized distribution is then generated based on the computed similarity. The distribution thus takes advantage of property of the global structure of the queue. CROMDA then aligns the two distributions and learns a modal invariant global representation. In this way, CROMDA is able to obtain invariant property in each modality, where images with similar text representations should be similar and vice versa.

Systems and methods for vision-language distribution alignment

The system has a form analysis module that receives an image of a form into which values have been filled for the possible fields of information on the form, such as first name, address, age, and the like. By using a library of form templates, a form analysis module allows both flexibility of form processing and simplicity for the user. That is, the techniques used by the form analysis module allow the processing of any form image for which the library has a form template example. The form image need not precisely match any form template, but rather may be scaled or shifted relative to a corresponding template. Additionally, the user need only provide the form image itself, without providing any additional exemplars, metadata for training, or the like.

Template-based key-value extraction for inferring OCR key values within form images

Methods and systems for spectroscopy are provided. Exemplary methods include: illuminating, with a tunable laser, an analyte with first light; detecting, with a filtered sensor, a first Raman signal; illuminating, with the tunable laser, the analyte using second light; detecting, with the filtered sensor, a second Raman signal, the second Raman signal being shifted from the first Raman signal by a second predetermined increment; illuminating, with the tunable laser, the analyte using third light; detecting, with the filtered sensor, a third Raman signal, the third Raman signal being shifted from the second Raman signal by the second predetermined increment; constructing a Raman spectrum using the first Raman signal, the second Raman signal, and the third Raman signal; and determining at least one molecule of the analyte using the Raman spectrum and a database of predetermined Raman spectra.

Wearable spectroscopy using filtered sensor

Systems for spectroscopy are provided. Exemplary systems include: an enclosure; a tunable laser disposed in the enclosure; an opening on a surface of the enclosure; a beam splitter disposed in the enclosure, the beam splitter being optically coupled to the tunable laser and the opening; a sensor, a sensing surface of the sensor having a filter disposed thereon; and electronics coupled to the tunable laser and the sensor, the electronics including a processor, memory, and a battery.

Wearable spectrometer with filtered sensor

Systems and methods for hybrid time-resolved and time-shifted spectroscopy for measuring biological analytes are disclosed. Exemplary methods include: illuminating an analyte using first light from and excitation source, the first light having a first excitation wavelength; detecting a first spectrum from the analyte illuminated by the first light using a time-resolved spectroscopy technique, the first spectrum including a first Raman signal and fluorescence; illuminating the analyte using second light, the second light having a second excitation wavelength; detecting a second spectrum using a time-resolved spectroscopy technique; illuminating the analyte using third light, the third light having a third excitation wavelength; detecting a third spectrum; recovering the first Raman signal using the first spectrum, the second spectrum, and the third spectrum using an inverse transform; and using the first Raman signal to identify and measure at least one molecule of the analyte using a database of identified Raman signals.

Hybrid time-resolved and time-shifted spectroscopy for measuring biological analytes

Methods and media for compensating for irregular motion in three-dimensional spectroscopy are provided herein. Exemplary methods include: receiving a plurality of spectrographs for a series of respective locations and corresponding images of the respective locations, each spectrograph of the plurality of spectrographs being produced using a spectrographic data set of a plurality of spectrographic data sets, each of the plurality of spectrographic data sets being measured by a spectrometer and each of the corresponding images being captured by a camera at substantially the same time, the spectrometer being coupled to the camera such that the spectrometer and camera move in tandem and at least partially share the same point of view; generating a continuous image using the images; identifying a respective corresponding position in the continuous image for each spectrograph, such that each spectrograph is a measurement of the respective position; and associating each spectrograph with the respective position.

Irregular motion compensation for three-dimensional spectroscopy

Systems for adjusting for irregular movement during spectral imaging are provided herein. Exemplary systems include: a spectrograph measuring a plurality of spectrographic data sets; a camera capturing images, a processor communicatively coupled to the spectrograph and the camera; and a memory coupled to the processor, the memory storing instructions executable by the processor to perform a method comprising: receiving a plurality of spectrographs for a series of respective locations and the images corresponding to the respective locations; generating a continuous image using the images; identifying a respective corresponding position in the continuous image for each spectrograph, such that each spectrograph is a measurement of the respective position; and associating each spectrograph with the respective position.

Adjustment for irregular sensor movement during spectral imaging

A monitoring system provides for monitoring the optical performance of a plurality of different channels of a fiber optic network, each channel having an associated channel wavelength. The system includes: means for generating a calibration signal including a plurality of stable calibration wavelengths; means for combining a tapped communication signal and the calibration signal to generate a combined signal; and a multi-channel monitoring device. The monitoring device includes: an input optical fiber for receiving the combined signal, and for radiating an input light beam including the communication wavelengths and the calibration wavelengths; a detector unit having an array of pixel detector elements each being responsive to light beams incident thereon, and operative to generate an associated data signal; collimating and focusing means for collimating and focusing beams propagating between the fiber end and the detector unit; and a transmissive grating assembly for diffracting and spatially separating beams propagating between the input fiber end and the detector unit. The grating assembly, collimating and focusing means, and pixel detector elements are configured so that the diffracted beams are spatially separated and focused onto associated ones of the detector elements according to their wavelengths. A data processing unit receives the data signals from the detector unit, and is operative to provide information indicative of the intensities and wavelengths associated with each of the channels of the main communication signal. The processing unit uses the calibration wavelengths as standards to calibrate positional relationships between the spatially diffracted beams and the associated pixel detector elements.

Compact optical performance monitor

An optical cross switch device for selectively switching light beams between a plurality of fiber optic elements. The device includes an input fiber mounting assembly for securing at least one input fiber terminating in a fiber end for radiating an associated input light beam; collimating means for collimating the at least one input light beam; focusing means for focusing light beams incident thereon; an output fiber mounting assembly for securing at least one output fiber terminating in an output fiber end for receiving an associated light beam; and at least one mirror unit having at least one reflective element for selectively redirecting an associated one of the collimated input light beams toward an associated selected one of the output fiber ends via the focusing means.

Wavelength selective optical cross switch and optical add/drop multiplexer using volume phase grating and array of micro electro mechanical mirrors

An optical multiplexing and demultiplexing device includes: a fiber mounting means for securing a plurality of optical fibers each terminating in a fiber end, at least one of the fiber ends radiating a corresponding input beam having a polarization state defined by a polarization plane that may vary as a function of time, and at least one of the fiber ends for receiving an output beam; collimating and focusing means having a focal length, and providing collimation of the input beams, and focusing of the output beams; diffraction means including at least one diffractive element providing for dispersion of optical beams passing therethrough, each diffractive element having an associated diffraction efficiency that varies as a function of the polarization state of light beams passing therethrough; and a polarization rotating means for rotating the plane of polarization of a beam passing therethrough by a specified number of degrees. The device provides for a particular input beam having a first polarization state to be diffracted by the diffractive elements in accordance with a first diffraction efficiency based on the first polarization state. The beam is then passed through the polarization rotating means at least once and thereby rotated into a second polarization state. The rotated beam is then diffracted by at least one of the diffractive elements in accordance with a second diffraction efficiency based on the second polarization state, whereby the device provides a spectral resolution that is substantially insensitive to variations in the polarization state of the input beam.

Compact wavelength multiplexer-demultiplexer providing low polarization sensitivity

An optical multiplexing and demultiplexing device including a fiber mounting assembly for aligning optical fibers, a set of collimating and focusing optics, a transmission grating element, and a mirror element. The fiber mounting assembly alignment element aligns a series substantially close-spaced optical fibers with their ends flush and assembled in a substantially straight line. The optical fibers are either multi-mode or single mode communication fibers capable of transmitting optical wavelengths with substantially high transmittance. The set of collimating and focusing optics, preferably a lens (or mirror) system that contains two (2) or more elements, may be made from optical glasses with uniform refractive indices, optical glasses with gradient indices, plastic optical elements, or diffractive lenses. The transmission grating element can be made from holographic techniques utilizing photosensitive media, where the media having sufficient thickness to provide high diffractive efficiency and wide operation waveband. The photosensitive media providing high spatial resolution in order to generate high groove density&mdash;thus high spectral resolution for DWDM applications. The mirror element, having a high reflectance coating for the wavelengths of interest, is arranged at such an angle that the optical radiation will be reflected back to the grating. It will then be diffracted again by the grating then focused to the optical fibers.

Compact double-pass wavelength multiplexer-demultiplexer

A method and apparatus for non-invasively measuring animal/human blood glucose and other metabolites including an excitation laser source, a negative pressure based sampling port which interfaces to a human or other animal tissue sample in vivo, a Raman spectrometer, and data analysis and display devices. The device can be made in a compact size and may be portable, and it can be used in homes, offices or clinics. A negative atmospheric sampling port is made of a vacuum chamber that is connected with an electrically or manually driven vacuum pump which creates a negative air pressure inside the vacuum chamber. Under the negative air pressure, a substantial amount of blood is 'sucked' into a small area of the human finger so that measurement of an enhanced Raman signal can be made.

Method and apparatus of non-invasive measurement of human/animal blood

An optical multiplexing and demultiplexing device includes: a plurality of optical fibers each terminating in a fiber end for radiating and receiving corresponding light beams; a fiber mounting assembly configured to support the optical fibers so that the fiber ends terminate in substantially the same plane, the mounting assembly also being configured to position the fibers in a regular array of N rows and M columns; reflecting means for reflecting beams radiating from the fiber ends; lens means for collimating and focusing beams propagating between the fiber ends and the reflecting means, the lens means having a focal length associated therewith; and diffraction means for diffracting beams propagating between the lens means and the reflecting means. The reflecting means includes a plurality of N reflective elements, each of the N reflective elements being associated with a corresponding one of the N rows. The array is substantially defined by a row axis that is substantially parallel to each of the rows and a column axis that is substantially parallel to each of the columns. Each of the reflective surfaces forms a corresponding first angle of rotation about a first line parallel to the row axis, and a corresponding second angle of rotation about a second line parallel to the column axis. The first angle of rotation of a corresponding one of the reflective surfaces is proportional to a distance measured between the center fiber and the row associated with corresponding reflective surface.

Growing community of inventors

Fremont, CA, United States of America

Shu Zhang