Procedure guidance and training apparatus, methods and systems

Abstract

Procedure training apparatus, methods and systems which in one embodiment allow surgical trainees to receive expert clinical guidance during live surgery or to practice surgical skills on anatomical models in a realistic manner with an augmented reality headset which is in audio and visual communication with a remotely located trainer having a similar AR headset. A unique feature of the system includes providing live or static imaging of the surgical site to the AR headsets of the trainee and trainer wherein the imaging is provided from medical or other devices (e.g., ultrasound) through a device(s) media hub. The images are displayed in the field of view of the surgeon so that they may see their hands, the patient (or model) and the imaging simultaneously. Controls allow the imaging on the respective AR headsets to be switched on command typically by the trainer which allows the trainer to deliver the trainer's surgical site image (e.g., on their own model) to the AR headset of the trainee to illustrate correct procedure, for example. Once the trainee views the trainer's imaging of correct procedure, the imaging source may be switched back to the trainee's surgical site so that they may continue the procedure. Instruction or live clinical support during surgical procedures allow procedures to be recorded and recordings segregated by machine learning model and class, allowing for semi-supervised and/or unsupervised learning. After machine learning models are created, automated guidance and/or alerts may be given during the surgical procedure.