Digimarc Corporation

Other

William Y Conwell

Tony F Rodriguez

Geoffrey B Rhoads

Bruce L Davis

Douglas B Evans

Ravi K Sharma

Osama M Alattar

Hugh L Brunk

Brett Alan Bradley

Tomas Filler

John Douglas Lord

Vojtech Holub

Brian T MacIntosh

Sean Calhoon

Joel R Meyer

Alastair M Reed

Kenneth Lee Levy

Adnan M Alattar

Ajith Mulki Kamath

Eliot Rogers

Nicole Rhoads

Robert Craig Brandis

Emma C Sinclair

Jerrine K Owen

Edward Bruce Knudson

John F Stach

J Scott Carr

Shankar Thagadur Shivappa

Tyler J McKinley

Marc D Miller

Matthew M Weaver

Vahid Sedighianaraki

Joshua V Aller

Utkarsh Deshmukh

Gilbert B Shaw

David A Cushman

Richard F Gibson

Brian T Macintosh

Aspects of the detailed technologies concern training and use of neural networks for fine-grained classification of large numbers of items, e.g., as may be encountered in a supermarket. Mitigating false positive errors is an exemplary area of emphasis. Novel network topologies are also detailed—some employing recognition technologies in addition to neural networks. A great number of other features and arrangements are also detailed.

Image processing arrangements

A sequence of images depicting an object is captured, e.g., by a camera at a point-of-sale terminal in a retail store. The object is identified, such as by a barcode or watermark that is detected from one or more of the images. Once the object's identity is known, such information is used in training a classifier (e.g., a machine learning system) to recognize the object from others of the captured images, including images that may be degraded by blur, inferior lighting, etc. In another arrangement, such degraded images are processed to identify feature points useful in fingerprint-based identification of the object. Feature points extracted from such degraded imagery aid in fingerprint-based recognition of objects under real life circumstances, as contrasted with feature points extracted from pristine imagery (e.g., digital files containing label artwork for such objects). A great variety of other features and arrangements—some involving designing classifiers so as to combat classifier copying—are also detailed.

Learning systems and methods

In a next-generation retail store, the presence of a shopper at a particular shelf location, at about the time that an item was removed from that shelf location, gives rise to a hypothesis that the item may have been picked for purchase by that shopper. An ensemble of different such hypotheses, about different items removed by different shoppers, is evaluated jointly to determine which hypotheses are most likely. Other factors can influence the determinations, such as a physical distance between picked products, together with a time interval between their picking. If cameras are used to detect removal of items from shelves, hand depictions captured by such cameras are further factors that may be considered, e.g., helping associate certain item removals with a common shopper. By generating purchase lists for each shopper with such methods, shoppers can be relieved of the need of having each item scanned at a checkout. Some embodiments are privacy-preserving—neither employing nor capturing personally-identifying information. Other embodiments don't require tracking movements of shoppers through the store. A great number of other features and arrangements are also detailed.

Sensor-based maximum-likelihood estimation of item assignments

A plastic item, such as a beverage bottle, conveys two distinct digital watermarks, encoded using two distinct signaling protocols. A first, printed label watermark conveys a retailing payload, including a Global Trade Item Number (GTIN) used by a point-of-sale scanner in a retail store to identify and price the item when presented for checkout. A second, plastic texture watermark conveys a recycling payload, including data identifying the composition of the plastic. The use of two different signaling protocols assures that a point-of-sale scanner will not spend its limited time and computational resources working to decode the recycling watermark, which lacks the data needed for retail checkout. In some embodiments, a recycling apparatus makes advantageous use of both types of watermarks to identify the plastic composition of the item (e.g., relating GTIN to plastic type using an associated database), thereby increasing the fraction of items that are correctly identified for sorting and recycling. A great number of other features and arrangements are also detailed.

Recycling methods and systems, and related plastic containers

Items are identified in a waste stream for purposes of recycling, using deterministic and/or probabilistic techniques. Imagery of the waste stream from multiple viewpoints permit creation of a 3D depth draped image representation, from which one or more 2D planes can be synthesized. Phase-coherent patches of recoverable encoded data can be identified from among soiled and crumpled object surfaces, and used in combination to recover object identification information. Recognition of certain items can trigger further image processing that is specific to such items. (Detection of a catsup bottle, for example, can trigger image analysis to discern the presence of catsup residue.) Information about recognized objects can be provided to external data customers, e.g., to track grey market diversion of particular products into unlicensed territories. These and other features and advantages, which can be used alone or in combination, are detailed herein.

Information-client server built on a rapid material identification platform

A decade from now, a visit to the supermarket will be a very different experience than the familiar experiences of decades past. Product packaging will come alive with interactivity—each object a portal into a rich tapestry of experiences, with contributions authored by the product brand, by the store selling the product, and by other shoppers. The present technology concerns arrangements for authoring and delivering such experiences. A great variety of other features and technologies are also detailed.

Imagery and annotations

A plastic item, such as a beverage bottle, can convey two distinct digital watermarks, encoded using two distinct signaling protocols. A first, printed label watermark conveys a retailing payload, including a Global Trade Item Number (GTIN) used by a point-of-sale scanner in a retail store to identify and price the item when presented for checkout. A second, plastic texture watermark may convey a recycling payload, including data identifying the composition of the plastic. The use of two different signaling protocols assures that a point-of-sale scanner will not spend its limited time and computational resources working to decode the recycling watermark, which may lack data needed for retail checkout. In some embodiments, a recycling apparatus makes advantageous use of both types of watermarks to identify the plastic composition of the item (e.g., relating GTIN to plastic type using an associated database), thereby increasing the fraction of items that are correctly identified for sorting and recycling. In other embodiments the plastic item (or a label thereon) bears only a single watermark. A great number of other features and arrangements are also detailed.

Methods and arrangements for sorting items, useful in recycling

A store includes first and second racks of shelving that are respectively viewed by first and second cameras. A computer system, including one or more processors and memory, defines a first convolutional neural network (CNN) trained to recognize a first set of items stocked on the first rack of shelving, and a second CNN trained to recognize a second set of items stocked on the second rack of shelving. (The first camera provides imagery for recognition by the first CNN, and the second camera provides imagery for recognition by the second CNN.) In such arrangement, the second neural network is trained to recognize a particular item that the first neural network is not trained to recognize, and the first neural network is trained to recognize a certain item that the second neural network is not trained to recognize. Many other features and arrangements are also detailed.

Aspects of the detailed technologies concern training and use of neural networks for fine-grained classification of large numbers of items, e.g., as may be encountered in a supermarket. Mitigating false positive errors is an exemplary area of emphasis. Novel network topologies are also detailed&#x2014;some employing recognition technologies in addition to neural networks. A great number of other features and arrangements are also detailed.

Imagery captured by an autonomous robot is analyzed to discern digital watermark patterns. In some embodiments, identical but geometrically-inconsistent digital watermark patterns are discerned in an image frame, to aid in distinguishing multiple depicted instances of a particular item. In other embodiments, actions of the robot are controlled or altered in accordance with image processing performed by the robot on a digital watermark pattern. The technology is particularly described in the context of retail stores in which the watermark patterns are encoded, e.g., on product packaging, shelving, and shelf labels. A great variety of other features and arrangements are also detailed.

Image processing methods and arrangements useful in automated store shelf inspections

A decade from now, a visit to the supermarket will be a very different experience than the familiar experiences of decades past. Product packaging will come alive with interactivity&#x2014;each object a portal into a rich tapestry of experiences, with contributions authored by the product brand, by the store selling the product, and by other shoppers. The present technology concerns arrangements for authoring and delivering such experiences. A great variety of other features and technologies are also detailed.

In one aspect, assembly of multi-part food packaging is checked by reference to payloads of steganographically-encoded digital watermarks printed across the packaging components. Marking all surfaces of the packaging components allows arbitrary orientation of feed stock in assembly equipment, and wide latitude in placement of inspection cameras along the packaging line. In another aspect, a scanner at a retail checkout station is alert to any gap detected in steganographic encoding on retail product packaging and, if found, alerts an operator to possible presence of an adhesive label with a misleading barcode. A great variety of others features and arrangements are also detailed.

Digital watermarking applications

Mobile phones and other portable devices are equipped with a variety of technologies by which existing functionality can be improved, and new functionality can be provided. Some aspects relate to visual search capabilities, and determining appropriate actions responsive to different image inputs. Others relate to processing of image data. Still others concern metadata generation, processing, and representation. Yet others concern user interface improvements. Other aspects relate to imaging architectures, in which a mobile phone's image sensor is one in a chain of stages that successively act on packetized instructions/data, to capture and later process imagery. Still other aspects relate to distribution of processing tasks between the mobile device and remote resources ('the cloud'). Elemental image processing (e.g., simple filtering and edge detection) can be performed on the mobile phone, while other operations can be referred out to remote service providers. The remote service providers can be selected using techniques such as reverse auctions, through which they compete for processing tasks. A great number of other features and arrangements are also detailed.

Methods and systems for content processing

A sequence of images depicting an object is captured, e.g., by a camera at a point-of-sale terminal in a retail store. The object is identified, such as by a barcode or watermark that is detected from one or more of the images. Once the object's identity is known, such information is used in training a classifier (e.g., a machine learning system) to recognize the object from others of the captured images, including images that may be degraded by blur, inferior lighting, etc. In another arrangement, such degraded images are processed to identify feature points useful in fingerprint-based identification of the object. Feature points extracted from such degraded imagery aid in fingerprint-based recognition of objects under real life circumstances, as contrasted with feature points extracted from pristine imagery (e.g., digital files containing label artwork for such objects). A great variety of other features and arrangements&#x2014;some involving designing classifiers so as to combat classifier copying&#x2014;are also detailed.

Growing community of inventors

Portland, OR, United States of America

William Y Conwell