Siemens Healthcare Gmbh

Siemens Healthineers Ag

Kaloian Petkov

Daphne Yu

Christoph Vetter

Sandra Sudarsky

Rishabh Shah

Tommaso Mansi

Shun P Miao

Klaus Engel

Dorin Comaniciu

Bogdan Georgescu

Atilla Peter Kiraly

Jin-hyeong Park

Chen Liu

Stefan Assmann

Feng Qiu

Wei Hong

John Paulus, Jr

Ruth J Soenius

Lev Gretskii

Babu Swamydoss

Philipp Treffer

A layout of labels for annotating a rendered image is optimized. A rendered image is obtained, and locations of a plurality of regions of interest in the rendered image are determined. Semantic information associated with the plurality of regions of interest is obtained. Based on the semantic information and the locations of the plurality of regions of interest, and, taking into account a visibility of the labels and a further visibility of the regions of interest in the rendered image, the layout of the labels for annotating the plurality of regions of interest in the rendered image is determined.

Dynamic layout optimization of annotation labels in volume rendering

For real-time rendering of medical images from volumetric data obtained from a medical scanner, one or more optical properties of the received volumetric data are determined. A light volume associated to a spherical light source is constructed. The light volume comprises a series of consecutive spherical slices through which light propagates while determining a fraction of the light propagating from one spherical slice to a neighboring spherical slice depending on the optical properties. The constructed light volume is sampling with a gradient-free shading that depends on the determined optical properties. At least one medical image is rendered in relation to the received volumetric data based on the sampled light volume.

Technique for real-time rendering of medical images using virtual spherical light sources

An exposure value is dynamically adapted for consistent interactive rendering of a volumetric medical dataset. A rendering set of a plurality of rendering types with each rendering type adapted for providing an image dataset derived from the volumetric medical dataset is provided. An image dataset according to a first rendering type using a first exposure value is output. An exposure value for each rendering type within the rendering set is determined based on the first exposure value of the first rendering type and on a consistency requirement for changing rendering types of the image dataset output. Upon receiving a user input for modifying the output of the image dataset, a second rendering type with a second exposure value is selected and output.

Technique for dynamically adapting an exposure value in interactive rendering

Systems and methods for determining rendering parameters based on authored snapshots or templates. In one aspect, clinically relevant snapshots of patient medical data are created by experts to support educational or clinical workflows. Alternatively, the snapshots are created by automation processes from AI-based organ and disease segmentations. In another aspect, clinically relevant templates are generated. Rendering parameters are derived from the snapshots or templates, stored, and then applied for either rendering new data or interactive viewing of existing data.

Systems and methods for automated rendering

An automatic light arrangement for medical visualization includes: providing a medical 3D image (D), providing spatial information about a region of interest (R) in this 3D-image (D) and spatial information about a virtual camera (C), determining a plurality of possible arrangements for light sources (L) by using depth information based on the 3D image (DI) together with the spatial information about the region of interest (R) in this image and the spatial information about the virtual camera (C), wherein valid arrangements are those where shadows (S) on the region of interest (R) are below a predefined threshold, and/or wherein the determination or arrangements is based on a number of predefined perceptual metrics applied specifically to the regions of interest, prioritizing the determined arrangements, and choosing the arrangement with the best prioritization.

System and method for automatic light arrangement for medical visualization

Systems and methods for automatically determining an image quality assessment of a rendered medical image are provided. A rendered medical image is received. One or more measures of interest are extracted from the rendered medical image. An image quality assessment of the rendered medical image is determined using a machine learning based image quality assessment network based on the one or more measures of interest. The image quality assessment of the rendered medical image is output.

Image quality assessment for refinement of imaging rendering parameters for rendering medical images

Distance estimation is optimized in virtual or augmented reality. A distance map of a surgical instrument to a region of interest is determined, at least at the beginning and when a position of the surgical instrument has changed. A render-image is rendered based on a medical 3D image and the position of the surgical instrument, at least at the beginning and when the position of the surgical instrument has changed. At least the region of interest and those parts of the surgical instrument positioned in the volume of the render-image are shown in the render-image. Based on the distance map, at least for a predefined area of the region of interest, visible, acoustic, and/or haptic distance-information is added.

Method and system for optimizing distance estimation

The present embodiment relates to a renderer and an interactive method for image editing of medical 3D anatomical data. The method includes receiving a dataset with volumetric image data, which have been acquired from an image acquisition modality, and providing a signed distance field data structure of the received dataset. Further, editing operations are received from a user interface for editing at least a part of the provided signed distance field data structure. A visualization of the editing operations is calculated and displayed on a display.

Interactive image editing

Computer-implemented methods for rendering a volumetric dataset and a surface embedded in the volumetric dataset are described. One method includes performing a volume rendering process to generate a volume rendering of the volume. Based on the volume rendering process, depths of respective locations in the volume rendering are determined, and the depths stored in association with the respective locations. A surface rendering process generates a surface rendering of a surface using the depths and respective locations. The volume rendering and the surface rendering are combined into a combined rendering of the volume and surface. In another method, depths of the surface at respective locations with respect to the volume are determined and the depths are stored in association with the locations. A volume rendering process includes a shading process which uses the depths of the surface and the respective locations.

Method of rendering a volume and a surface embedded in the volume

For three-dimensional rendering, a machine-learnt model is trained to generate representation vectors for rendered images formed with different rendering parameter settings. The distances between representation vectors of the images to a reference are used to select the rendered image and corresponding rendering parameters that provides a consistency with the reference. In an additional or different embodiment, optimized pseudo-random sequences are used for physically-based rendering. The random number generator seed is selected to improve the convergence speed of the renderer and to provide higher quality images, such as providing images more rapidly for training compared to using non-optimized seed selection.

Consistent 3D rendering in medical imaging

Physically-based volume rendering generates a lightfield. The locations of scattering modeled in physically-based rendering are used to assign depths for the lightfield. The previously assigned depths and previously rendered lightfield are used for lightfield rendering, which may be performed more rapidly than the physically-based volume rendering.

Lightfield rendering based on depths from physically-based volume rendering

An embodiment suggests a method for visualizing an image data set, in particular a medical image data set, wherein the visualized data set displays a three dimensional arrangement having at least a first object and a second object. The method includes assigning a first set of parameter to the first object; assigning a second set of parameters to the second object; dividing the medical image data set into a first sub-region and a second sub-region; and providing a visualisation of the three dimensional arrangement by a volume rendering method, in particular by a ray-casting method or a photorealistic volumetric path tracing, the first set of parameter being applied to the first sub-region for visualizing the first object and the second set of parameter being applied to the second sub-region for visualizing the second object.

In-context photorealistic 3D visualization for surgical decision support

Systems and methods are provided for generating smooth transitions between volume rendering presets when the volume rendering is used as part of an animation system. A windowing-compensated look-up table interpolation is used to interpolate between adjacent keyframes that include user defined rendering presets. A constrained spline interpolation may be used to prevent overshooting.

Volume rendering animations

For interactive rendering in medical imaging, physically-based volume rendering of a volume of a patient may better assist physicians in diagnosis, prognosis, and/or planning. To provide for more rapid interaction, direct volume rendering is used during interaction. The rendering then transitions to physically-based rendering when there is no interaction. For smoothing the transition and/or preserving cohesive perceptual details, images from the different types of rendering may be blended in the transition and/or during interaction.

Hybrid interactive mode for rendering medical images with ray tracing

A 2D medical image is colorized. In one approach, a deep-learnt classifier is trained to colorize from color 2D medical images. The color 2D medical images for training are cinematically rendered from slabs to add color. In another approach, a deep machine-learnt generator creates slices as if adjacent to the 2D medical image. The slices and 2D medical image form a slab, which is cinematically rendered to add color. The result is a colorized 2D medical image.

Growing community of inventors

Lawrenceville, NJ, United States of America

Kaloian Petkov