University of California

Other

University of Dayton

Truong Q Nguyen

Ramsin Khoshabeh

Mainak Biswas

Lam Tran

Can Bal

Ho Chan

Harinath Garudadri

Yu-Hwa Lo

Patrick Mercier

Deli Wang

Keigo Hirakawa

Ai-Mei Huang

Joel A Rosiene

Siarhei Vishniakou

Jason Juang

Yeejin Lee

Brian Wellington Lewis

Changyoung Han

Kyoung-Rok Lee

Young Ouk Kim

Yoav Mintz

Wonha Kim

Jack Tzeng

Andrea Molino

Fabrizio Vacca

Arthur Boothroyd

Sanjeev Kumar

Carol Mackersie

Yee Jin Lee

A method for processing image or video data is performed in an image processing pipeline. Color filtered mosaiced raw image or video data is received. A one-level wavelet transform of subbands of the color filtered mosaiced raw image or video data to provide LL, HH, LH and HL subbands. The LH and HL subbands are de-correlated by summing and difference operations to provide decorrelated sum and difference subbands. Additional n-level wavelet transformation on the sum and difference subbands and the LL and HH subbands to provide sparsified subbands for encoding. LL and HH and sum subbands are recombined into standard color images e.g., red, green, and blue color components, which are subsequently processed by color correction, white balance, and gamma correction. The sparsified subbands are encoded.

Sampled image compression methods and image processing pipeline

A hearing assistance and/or noise suppression device leverages computing power of an external device with a digital signal processor, such as a special unit that is configured to communicate with a smart device (e.g., a smart phone, smart watch or smart pendant) or a smart phone with a digital signal processor. Methods include having a hearing transducer communicate with and offload computing tasks to an external device with a digital signal processor. Systems include a hearing transducer with transducer circuitry that receives, amplifies and outputs digital signal processed audio from another device. Methods provide self-adjustment and fitting through a touch screen interface, which can be conducted outside of a clinical setting in a real world environment, and method can include remote data collection and communications with clinicians.

External device leveraged hearing assistance and noise suppression device, method and systems

Systems and devices for recording and reproducing senses. One or more of touch sensations and smell sensations are received to produce at least one of touch input signals and smell input signals. The input signals are stored and processed, and at least one output signal can be produced. One or more of a touch output and a smell output can be generated. The touch input, the smell input, or both can be integrated with one or more of sight input and sound input.

Systems and devices for recording and reproducing senses

An embodiment of the invention is a method that provides common data and enforces multiple view consistency. In a preferred embodiment, a coordinate alignment matrix is generated after initial warped views are formed. The matrix is used efficiently to repair holes/occlusions when synthesizing final views. In another preferred embodiment, a common background layer is generated based upon initial warped views. Holes are repaired on the common background layer, which can then be used for synthesizing final views. The cost of the latter method is sub-linear, meaning that additional views can be generated with lesser cost. Preferred methods of the invention also use a placement matrix to generate initial warped views, which simplifies warping to a look-up operation.

Multi view synthesis method and display devices with spatial and inter-view consistency

A method for real-time disparity estimation of stereo video data receives sequence of frames of stereo video data. Image-based disparity estimation is initially conducted to produce initial disparity estimates, and the disparity estimates are refined in a space-time volume. The algorithm produces disparity via multi-thread process in which an output is independent of the input for each step of the process.

Independent thread video disparity estimation method and codec

A method for processing image or video data receives color filtered mosaiced raw image or video data. The method, performed in an image processing pipeline, de-hazes the color filtered mosaiced raw image or video data, e.g. Bayer image data, to create de-hazed mosaiced image or video data and de-mosaicing the de-hazed mosaiced image or video data to crate de-hazed and de-mosaiced image or video data.

Raw sensor image and video de-hazing and atmospheric light analysis methods and systems

A method for disparity estimation of stereo video data receives a sequence of frames of stereo video data. Image-based disparity estimation is initially conducted on a frame-by-frame basis to produce initial disparity estimates. A plurality of initial disparity estimates is grouped into a space-time volume. Disparity error is reduced in the space-time volume to refine the initial disparity estimates.

Video disparity estimate space-time refinement method and codec

An embodiment of the invention is a method that provides common data and enforces multiple view consistency. In a preferred embodiment, a coordinate alignment matrix is generated after initial warped views are formed. The matrix is used efficiently to repair holes/occlusions when synthesizing final views. In another preferred embodiment, a common background layer is generated based upon initial warped views. Holes are repaired on the common background layer, which can then be used for synthesizing final views. The cost of the latter method is sub-linear, meaning that additional views can be generated with lesser cost. Preferred methods of the invention also use a placement matrix to generate initial warped views, which simplifies warping to a lookup operation.

A preferred method receives a color image and a corresponding raw depth map from a sensor or system. Unreliable regions are determined in the raw depth map by calculating pixel reliabilities for pixels throughout the depth map. Information is collected from the color image, for corresponding pixels in the unreliable regions of the raw depth map, from neighboring pixels outside the unreliable regions. The depth of pixels in the unreliable regions is updated with information collected from the reliable regions. Robust multi-later filtering is conducted on the adjusted depth map to produce an enhanced depth map.

Sampling-based multi-lateral filter method for depth map enhancement and codec

The present Invention relates to a camera system suitable for use in minimally invasive surgery (MIS), among other applications. In at least one embodiment, the camera system includes an autonomous miniature camera, a light source assembly providing features such as steerable illumination and a variable radiation angle, and a control and processing unit for processing images acquired by the camera Io generate improved images having reduced blurring using a deblurring algorithm.

Camera system with autonomous miniature camera and light source assembly and method for image enhancement

A preferred method for hierarchical motion vector processing determines reliability levels of blocks in image data according to residual energy levels. Macroblocks of an image frame are merged according to reliability levels of the motion vectors of blocks. Single motion vectors are selected for merged macroblocks. Motion vectors of blocks merged in the step of merging are iteratively assigned by minimizing the bi-directional prediction difference on successively smaller merged blocks. The reliability levels are preferably determined by measure residual energy of both chrominance and luminance components. In preferred embodiments, motion vector correlation is used to assist the MV reliability classification and the merging and iterative assignment. Refinement and smoothing can be conducted on successively finer block sizes. Additionally, preferred methods account for occlusions by choosing only one of forward or backward prediction for occlusion regions depending upon the class of the occlusion. Results of motion vector classification of the invention can be used in motion compensated frame interpolation and other techniques.

Hierarchical motion vector processing method, software and devices

A video codec of a preferred embodiment is suitable for implementation on a variety of platforms, including platforms where devices require both encoding and decoding engines. Various features of preferred embodiment codecs reduce complexity for operations such as motion determination. Channel resources are allocated in preferred embodiments unequally, with an allocation being determined based upon channel conditions and the error level in a frame being encoded. A decoder of a preferred embodiment, provided with an error frame and motion vector information may conduct decoding with motion compensation. In an example encoder of the invention, a motion estimator identifies motion vectors using a phase plane correlation technique based on phase quantization, where the phase difference between the Fourier transformed version of blocks in a reference frame and a current frame is estimated by dividing the complex plane into sectors and assigning the Fourier domain difference to the sectors.

Video encoding methods and devices

The invention provides methods for global motion estimation, determining a coarse estimation, and refining a coarse estimation. Embodiments of the invention provide a fast and robust global motion estimation algorithm based on two-stage coarse-to-fine refinement strategy, which is capable of measuring large motions. An embodiment of the invention may be applied as a modification of any standard, e.g. MPEG-4 that uses the affine model of motion estimation. Embodiments of the invention may be used in the six parameter affine motion model, and other embodiments of the invention are applicable to the two parameter translation model, the four parameter RST model, and the eight parameter projective model. In a preferred embodiment, a coarse estimation is developed in a translation invariant domain, and then is refined in the spatial domain.

Global motion estimation image coding and processing

A phase plane correlation motion vector determination method identifies images areas in which motion took place. A reference image is shifted by the candidate motion vectors obtained from the phase plane correlation. The shifted reference image is correlated to determine similarity with the current image and to eliminate spurious motion vectors. Candidate motion vectors are validated to determine valid motion vectors and identify problem areas. In a preferred embodiment, a correlation resolution is set to a sub-block size. Sub-blocks of the reference image are shifted by candidate motion vectors for corresponding blocks. Shifted sub-blocks are correlated with sub-blocks of a current image using a sub-block neighborhood to determine valid vector assignments. Where an assignment proves invalid, global motion parameters are determined to assign motion vectors to sub-blocks where correlation fails.

Phase plane correlation motion vector determination method

In an apparatus and method for image/video enhancement, pre-processing and post-processing techniques are employed to effectively modify the transforms used in a fixed, standardized data compression coder. In this manner, alternative transforms, for example overlapping-basis-type transforms, are made to be applicable to, and compatible with, various data compression standards, thereby improving system performance.

Growing community of inventors

San Diego, CA, United States of America

Truong Q Nguyen