Cognex Corporation

Cognex Technology and Investment Corporation

Shipin Systems Inc.

Other

Michael, David J.

David J Michael

Aaron S Wallack

Juha Tapio Koljonen

Hongwei Zhu

Gang Liu

Simon Barker

Adam Wagman

Nitin M Vaidya

Andrew Hoelscher

Moran Cohen

Ilan Naslavsky

Osher Perry

Arman M Garakani

Ruibing Wang

Nathaniel R Bogan

Daniel Moreno

Ben R Carey

Nickolas James Mullan

Gilbert Chiang

Yukang Liu

Cyril C Marrion

Lifeng Liu

Robert A Wolff

Yasunari Tosa

Paul Dutta-Choudhury

Ali Moghaddam Zadeh

John W Schwab

Hongjun Jia

Laurens Nunnink

William M Silver

Sanjay J Nichani

William H Equitz

Lowell D Jacobson

David Y Li

Xiangyun Ye

Lei Wang

Igor Reyzin

Jason Davis

Philip B Romanik

Guruprasad Shivaram

John P Petry, Iii

Fariborz Rostami

Peter Roberts

Tian Gan

Han Xiao

Matthew R Reome

James Clark

John B Boatner

Martin Karnacewicz

Nelson Tarr

David J Wilson

James Harald Clark

Aditya Gupte

Benjamin D Klass

This invention provides a system and method for detecting events related to a main power failure/blackout conditions on a commercial maritime vessel by visual detection of changes in the environment of one or more onboard areas that are visually affected by blackouts. One or more camera(s) acquire image(s) of the area to detect changes in ambient lighting that provide evidence of a blackout. Such changes in lighting can be trained as part of the visual detection process using training images comprising exemplary blackout conditions versus normal operation. Such changes in acquired image data are logged with an appropriate timestamp/condition. This data can be aggregated into statistics that can allow for determination of the root causes of the blackout and provide predictive and remedial actions to occur. This data can also be used for risk assessment relative to the vessel and fleet, assisting in risk management and insurance assessment.

System and method for automatic detection of maritime blackout events

This invention provides a system and method that uses a hybrid model for transportation-based (e.g. maritime) visual event detection of events. In operation, video data is reduced by detecting change and exclusively transmitting images to the deep learning model when changes are detected, or alternatively, based upon a timer that samples at selected intervals. Relatively straightforward deep learning models are used, which operate on sparse individual frames, instead of employing complex deep learning models that operate on multiple frames/videos. This approach reduces the need for specialized models. Independent, rule-based classifiers are used, based on the output of the deep learning model into visual events that, in turn, allows highly specialized events to be constructed. For example, multiple detections can be combined into higher-level single events, and thus, the existence maintenance procedures, cargo activities, and/or inspection rounds can be derived from combining multiple events or multiple detections.

System and method for automatic detection of visual events in transportation environments

The techniques described herein relate to methods, apparatus, and computer readable media configured to identify a surface feature of a portion of a three-dimensional (3D) point cloud. Data indicative of a path along a 3D point cloud is received, wherein the 3D point cloud comprises a plurality of 3D data points. A plurality of lists of 3D data points are generated, wherein: each list of 3D data points extends across the 3D point cloud at a location that intersects the received path; and each list of 3D data points intersects the received path at different locations. A characteristic associated with a surface feature is identified in at least some of the plurality of lists of 3D data points. The identified characteristics are grouped based on one or more properties of the identified characteristics. The surface feature is identified based on the grouped characteristics.

Methods and apparatus for identifying surface features in three-dimensional images

A system and method assessing maritime vessel risk in response to automatically detected maritime visual events is provided. At least one maritime visual event is detected by at least one camera aboard a vessel that provides image data of the visual event to a processor. The visual event can be associated with at least one of, safety, security, maintenance, crew behavior, and cargo. A risk assessment score is produced in response to the detected visual event, and that risk assessment score is provided to a user in a desired format. Production of the risk assessment score can entail comparing the visual event to data of complying or non-complying model visual events from a data storage. Risk assessment scores can be aggregated from plurality of events and/or a fleet of vessels to generate overall scores for the vessel and fleet.

System and method for maritime vessel risk assessment in response to maritime visual events

This invention provides a system and method that employs reduction in bandwidth and the amount of data stored, along with queuing of data, so that the significant and/or relevant shipboard visual information is detected and communicated continuously from the ship (or other remote vehicle/location) to shore without loss of useful/high-priority information and within the available bandwidth of that typical, available satellite link. The system and method supports remote configuration and management from shore to the ship over the same communications channel but in the reverse direction. The system and method further facilitates the logical summarization of continuous visibility into shipboard activities, shipboard behavior, and shipboard status so that a land/shore-based manager/reviewer/auditor can review, comprehend and synthesize such information, at a glance, using a land-based user interface/dashboard, related to events that have recently transpired on the sailing commercial merchant vessel without the need to review hours/days/weeks of multiple channels of video.

System and method for bandwidth reduction and communication of visual events

The techniques described herein relate to methods, apparatus, and computer readable media configured to determining a two-dimensional (2D) profile of a portion of a three-dimensional (3D) point cloud. A 3D region of interest is determined that includes a width along a first axis, a height along a second axis, and a depth along a third axis. The 3D points within the 3D region of interest are represented as a set of 2D points based on coordinate values of the first and second axes. The 2D points are grouped into a plurality of 2D bins arranged along the first axis. For each 2D bin, a representative 2D position is determined based on the associated set of 2D points. Each of the representative 2D positions are connected to neighboring representative 2D positions to generate the 2D profile.

Methods and apparatus for extracting profiles from three-dimensional images

This invention applies dynamic weighting between a point-to-plane and point-to-edge metric on a per-edge basis in an acquired image using a vision system. This allows an applied ICP technique to be significantly more robust to a variety of object geometries and/or occlusions. A system and method herein provides an energy function that is minimized to generate candidate 3D poses for use in alignment of runtime 3D image data of an object with model 3D image data. Since normals are much more accurate than edges, the use of normal is desirable when possible. However, in some use cases, such as a plane, edges provide information in relative directions the normals do not. Hence the system and method defines a 'normal information matrix', which represents the directions in which sufficient information is present. Performing (e.g.) a principal component analysis (PCA) on this matrix provides a basis for the available information.

System and method for simultaneous consideration of edges and normals in image features by a vision system

The techniques described herein relate to methods, apparatus, and computer readable media configured to generate point cloud histograms. A one-dimensional histogram can be generated by determining a distance to a reference for each 3D point of a 3D point cloud. A one-dimensional histogram is generated by adding, for each histogram entry, distances that are within the entry's range of distances. A two-dimensional histogram can be determined by generating a set of orientations by determining, for each 3D point, an orientation with at least a first value for a first component and a second value for a second component. A two-dimensional histogram can be generated based on the set of orientations. Each bin can be associated with ranges of values for the first and second components. Orientations can be added for each bin that have first and second values within the first and second ranges of values, respectively, of the bin.

Methods and apparatus for generating point cloud histograms

A system and method for estimating dimensions of an approximately cuboidal object from a 3D image of the object acquired by an image sensor of the vision system processor is provided. An identification module, associated with the vision system processor, automatically identifies a 3D region in the 3D image that contains the cuboidal object. A selection module, associated with the vision system processor, automatically selects 3D image data from the 3D image that corresponds to approximate faces or boundaries of the cuboidal object. An analysis module statistically analyzes, and generates statistics for, the selected 3D image data that correspond to approximate cuboidal object faces or boundaries. A refinement module chooses statistics that correspond to improved cuboidal dimensions from among cuboidal object length, width and height. The improved cuboidal dimensions are provided as dimensions for the object. A user interface displays a plurality of interface screens for setup and runtime operation.

System and method for refining dimensions of a generally cuboidal 3D object imaged by 3D vision system and controls for the same

This invention provides a system and method for runtime determination (self-diagnosis) of camera miscalibration (accuracy), typically related to camera extrinsics, based on historical statistics of runtime alignment scores for objects acquired in the scene, which are defined based on matching of observed and expected image data of trained object models. This arrangement avoids a need to cease runtime operation of the vision system and/or stop the production line that is served by the vision system to diagnose if the system's camera(s) remain calibrated. Under the assumption that objects or features inspected by the vision system over time are substantially the same, the vision system accumulates statistics of part alignment results and stores intermediate results to be used as indicator of current system accuracy. For multi-camera vision systems, cross validation is illustratively employed to identify individual problematic cameras. The system and method allows for faster, less-expensive and more-straightforward diagnosis of vision system failures related to deteriorating camera calibration.

System and method for runtime determination of camera miscalibration

This invention provides an easy-to-manufacture, easy-to-analyze calibration object which combines measurable and repeatable, but not necessarily accurate, 3D features&#x2014;such as a two-sided calibration object/target in (e.g.) the form of a frustum, with a pair of accurate and measurable features, more particularly parallel faces separated by a precise specified thickness, so as to provide for simple field calibration of opposite-facing DS sensors. Illustratively, a composite calibration object can be constructed, which includes the two-sided frustum that has been sandblasted and anodized (to provide measurable, repeatable features), with a flange whose above/below parallel surfaces have been ground to a precise specified thickness. The 3D corner positions of the two-sided frustum are used to calibrate the two sensors in X and Y, but cannot establish absolute Z without accurate information about the thickness of the two-sided frustum; the flange provides the absolute Z information.

System and method for calibrating a plurality of 3D sensors with respect to a motion conveyance

This invention provides a system and method for fleet management, including handling visual events in association with an activity. One or more visual event detector(s) is/are provided, having at least one camera that acquires images of the activity, and a processor that receives the images. A visual event detection process operates to process the images. A queuing process identifies visual events from the visual event detector and stores the visual events. A communications process transmits the stored visual events in a queued format to a remote location. A display that is accessed by users to view visual events, and a collaboration process displays information related to collaboration between permitted users based upon a context of each of the visual events&#x2014;which can comprise fleet management activities related to at least one of bridge operations, cargo operations, maintenance operations, safety operations, and vetting and security with respect to a vessel.

System and method for fleet management and communication of visual events and collaboration with the same

Growing community of inventors

Wayland, MA, United States of America

David J Michael