Siemens Corporate Research, Inc.

Edda Technology, Inc.

Siemens Medical Systems, Inc.

Jianzhong Qian

Guo-Qing Wei

Li Fan

Xiaolan Zeng

Cheng-Chung Liang

Yuanfeng Mao

Carol L Novak

Zhenyu Wu

Feng Ma

Helmuth Schramm

Firdous Saleheen

John C Engdahl

Xiaonan Zang

Sreerama K Murthy

Peggy C Hawman

Ming Fang

Hong Shen

Ping Hua

Jin-Shin Chou

Yash Evalekar

Sreerama K V Murthy

Beilei Xu

Li Zhang

Ming-Yee Chiu

Hong Chen

Manfred Bromba

Raymond P DeVito

Xin Dou

Jiahong Dong

Jon D Treffert

Benjamin Odry

Aaron Bobick

Helmut Schramm

Eric Q Chen

Ma Feng

Zeng Xiaolan

A method, implemented on a machine having at least one processor, storage, and a communication platform capable of connecting to a network for characterizing a tumor embedded inside a medium, comprising: measuring at least one of a mechanical property and a spectral property of the tumor.

Spectro-mechanical imaging for characterizing embedded lesions

The present teaching relates to automated trocar/robot base location determination. An input relates to a surgical operation with a 3D model for an organ including cut points thereon forming a surgical trajectory. A surgical instrument is controlled by a robot to reach the cut points to carry out the surgical operation. Candidate combinations of insertion location for inserting the surgical instrument and base location for deploying the robot are generated. One of the candidate combinations is identified based on evaluation vectors associated therewith. Each evaluation vector provides assessment information on characteristics and spatial relationships of the respective candidate locations. The selected combination of locations is used for the surgical operation.

System and method for automated simultaneous trocar and robot base location determination

The present teaching relates to surgical position marking. A 3D model for an organ includes cut points forming a surgical trajectory. Each cut point has a 3D coordinate and a surface norm in the model space. When projected into a workspace, a mapped cut point is created with a mapped 3D coordinate and a mapped surface norm in the workspace. With a surgical instrument with a tip and a force sensor attached thereto, some mapped cut points are marked along a direction determined based on a reaction force sensed by the force sensor when the tip touches the cut point.

System and method for automated surgical position marking in robot-assisted surgery

The present teaching relates to method, system, medium, and implementations for region boundary identification and overlay display. An input associated with a 3D object is received. A 3D volumetric model is obtained for the 3D object with multiple regions, each of which includes multiple labeled voxels. A mesh representation is generated for the surface of the 3D object. Region boundaries on surface of the 3D object are identified based on the 3D volumetric model and the mesh representation. The 3D object is rendered, and the region boundaries are overlaid on the rendered 3D object.

System and method for region boundary guidance overlay for organ

The present teaching relates to a method and system for path planning. A target is tracked via one or more sensors. Information of a desired pose of an end-effector with respect to the target and a current pose of the end-effector is obtained. Also, a minimum distance permitted between an arm including the end-effector and each of at least one obstacle identified between the current pose of the end-effector and the target is obtained. A weighting factor previously learned is retrieved and a cost based on a cost function is computed in accordance with a weighted smallest distance between the arm including the end-effector and the at least one obstacle, wherein the smallest distance is weighted by the weighting factor. A trajectory is computed from the current pose to the desired pose by minimizing the cost function.

Deterministic robot path planning method for obstacle avoidance

The present teaching relates to automated trocar/robot base location determination. An input related to a surgical operation includes a three-dimensional (3D) model for an organ with cut points on the organ forming a surgical trajectory. A surgical instrument is controlled by a robot to reach the cut points to carry out the operation. An insertion location for inserting the surgical instrument is identified. Based on the identified insertion location, a robot base location is determined with respect to the identified insertion location to deploy the robot for controlling the surgical instrument to reach the cut points. Signals for configuring the surgical setting are then generated based on the insertion location for the surgical instrument and the base location for the robot.

System and method for automated trocar and robot base location determination

A method, system, medium, and implementations for computer-aided preoperative surgical planning are described. Input data acquired with respect to a part of a patient is received by the system. The part corresponds to an organ, e.g., lung, of the patient to be operated on and includes one or more lesions to be removed during an operation. Then, an anatomic 3D model of the part of the patient is generated. Based on the generated anatomic 3D model, a preoperative plan for linear-cutting stapler resection of the one or more lesions from the organ to be carried out during the operation is obtained. The stapler cartridge size and the staple length are estimated based on the preoperative plan. Further, the resection based on the preoperative plan is visualized.

System and method for planning surgical resection of lesions by a linear cutting stapler

The present teaching relates to method, system, medium, and implementations for fusing a 3D virtual model with a 2D image associated with an organ of a patient. A key-pose is determined as an approximate position and orientation of a medical instrument with respect to the patient's organ. Based on the key-pose, an overlay is generated on a 2D image of the patient's organ, acquired by the medical instrument, by projecting the 3D virtual model on to the 2D image. A pair of feature points includes a 2D feature point from the 2D image and a corresponding 3D feature point from the 3D virtual model. The 3D coordinate of the 3D feature point is determined based on the 2D coordinate of the 2D feature point. The depth of the 3D coordinate is on a line of sight of the 2D feature point and is determined so that the projection of the 3D virtual model from the depth creates an overlay approximately matching the organ observed in the 2D image.

Method and system of model fusion for laparoscopic surgical guidance

The present teaching relates to a method and system for path planning. Information of a current pose of a robotic arm having a plurality of operable segments is obtained. The information includes a plurality of values, each of which corresponds to an angle formed between consecutive operable segments of the robotic arm. A desired pose where the robotic arm needs to reach is also obtained. An angle step-value is computed for the current pose of the robotic arm based on a function of a distance between the current pose and the desired pose, wherein the angle step value is to be used to determine a plurality of candidate next poses of the plurality of operable segments. One or more of candidate next poses is selected based on at least one criterion, and a trajectory is determined from the current pose to the desired pose based on the selected next poses.

Fast method for robot path planning with obstacle avoidance

The present teaching relates to method, system, medium, and implementations for estimating 3D coordinate of a 3D virtual model. Two pairs of feature points are obtained. Each of the pairs includes a respective 2D feature point on an organ observed in a 2D image, acquired during a medical procedure, and a respective corresponding 3D feature point from a 3D virtual model, constructed for the organ prior to the procedure based on a plurality of images of the organ. The depths of the first and the second 3D feature points are substantially the same. A first 3D coordinate of the first 3D feature point and a second 3D coordinate of the second 3D feature point are automatically determined based on the pairs of feature points so that a first distance between the determined first 3D coordinate and the determined second 3D coordinate equals to a second distance between a first actual 3D coordinate of the first 3D feature point and a second actual 3D coordinate of the second 3D feature point in the 3D virtual model.

Method and system of depth determination with closed form solution in model fusion for laparoscopic surgical guidance

The present teaching relates to method, system, medium, and implementations for robot path planning. Depth data of obstacles, acquired by depth sensors deployed in a 3D robot workspace and represented with respect to a sensor coordinate system, is transformed into depth data with respect to a robot coordinate system. The 3D robot workspace is discretized to generate 3D grid points representing a discretized 3D robot workspace. Based on the depth data with respect to the robot coordinate system, binarized values are assigned to at least some of 3D grid points to generate a binarized representation for the obstacles present in the 3D robot workspace. With respect to one or more sensing points associated with a part of a robot, it is determined whether the part is to collide with any obstacle. Based on the determining, a path is planned for the robot to move along while avoiding any obstacle.

Method and system for obstacle avoidance in robot path planning using depth sensors

The present disclosure relates to robot path planning. Depth information of a plurality of obstacles in an environment of a robot are obtained at a first time instance. A static distance map is generated based on the depth information. A path is computed for the robot based on the static distance map. At a second time instant, depth information of one or more obstacles is obtained. A dynamic distance map is generated based on the one or more obstacles, wherein for each obstacle that satisfies a condition: a vibration range of the obstacle is computed based on a position of the obstacle and the static distance map, and the obstacle is classified as a dynamic obstacle or a static obstacle based on a criterion associated with the vibration range. A repulsive speed of the robot is computed based on the dynamic distance map to avoid the dynamic obstacles.

Robot path planning method with static and dynamic collision avoidance in an uncertain environment

The present teaching relates to surgical procedure planning. In one example, at least one 3D object contained in a 3D volume is rendered on a display screen. The at least one 3D object includes a 3D object corresponding to an organ. First information related to a 3D pose of a surgical instrument positioned with respect to the at least one 3D object is received from a user. A 3D representation of the surgical instrument is rendered in the 3D volume based on the first information. Second information related to a setting of the surgical instrument is received from the user. A 3D treatment zone in the 3D volume with respect to the at least one 3D object is estimated based on the first and second information. The 3D treatment zone in the 3D volume is visualized on the display screen. Controls associated with the 3D representation of the surgical instrument and/or the 3D treatment zone are provided to facilitate the user to dynamically adjust the 3D treatment zone via the controls.

Method and system for adjusting interactive 3D treatment zone for percutaneous treatment

Systems and methods relating to placement of an ultrasound transducer for a procedure. In a non-limiting embodiment, a 3D environment including images of a body region of a patient, as well as images including a first virtual representation of an ablation needle at a first location and a second virtual representation of an ultrasound transducer at a second location is rendered on a display device. A determination may be made as to whether the first and second virtual representations collide at a first collision point. If so, at least one parameter associated with an orientation and/or position of the second virtual representation may be adjusted. A determination may then be made as to whether or not the first and second virtual representations still collide and, in response to determining that there is no collision, position data indicating the location of the second virtual representation after the adjustment, is stored.

Growing community of inventors

Princeton Junction, NJ, United States of America

Jianzhong Qian