Digimarc Corporation

United Video Properties, Inc.

Rovi Guides, Inc.

L-1 Secure Credentialing, Inc.

Other

Digimore Corporation

Bruce L Davis

Tony F Rodriguez

Geoffrey B Rhoads

Edward Bruce Knudson

Michael D Ellis

William L Thomas

William Y Conwell

J Scott Carr

John F Stach

Brian T MacIntosh

Alastair M Reed

Shankar Thagadur Shivappa

Eric D Evans

Douglas B Evans

Rebecca L Gerlach

Eliot Rogers

William C Hein, Iii

Robert Craig Brandis

Kenneth Lee Levy

Brett Alan Bradley

Tomas Filler

Adnan M Alattar

Burt W Perry

Donald Haaga

Ravi K Sharma

Robert Alan Knee

Larry J Miller

John Douglas Lord

Yang Bai

Michael D Morris

Phillip Andrew Seder

Joshua V Aller

Eoin C Sinclair

Jerrine K Owen

Gilbert B Shaw

Emma C Sinclair

Roger Youman

Jerry Alten

Anthony R Favia

Joshua A Rosenthol

Colin P Cornaby

Marc G Footen

Richard F Gibson

Brian T Macintosh

James Gutman

Michael Heydt

Paul Darata

Michele S Schwartz

Debbie Collins

John Harry Dasch

Jonathan Seth Zucker

Mitchell S Gorman

Michael Dean Ellis

Michael Morris

A decade from now, a visit to the supermarket will be a very different experience than the familiar experiences of decades past. Product packaging will come alive with interactivity—each object a portal into a rich tapestry of experiences, with contributions authored by the product brand, by the store selling the product, and by other shoppers. The present technology concerns arrangements for authoring and delivering such experiences. A great variety of other features and technologies are also detailed.

Imagery and annotations

Deterministic identifiers fuel reliable efficient capture of product discovery, purchase and consumption events, which in turn enable more reliable product recommendation, more accurate shopping list generation and in-store navigation. A mobile device, equipped with image and audio detectors, extracts product identifiers from objects, display screens and ambient audio. In conjunction with a cloud-based service, a mobile device application obtains product information and logs product events for extracted identifiers. The cloud service generates recommendations, and mapping for in-store navigation. The detectors also provide reliable and efficient product identification for purchase events, and post shopping product consumption events.

Object identification and device communication through image and audio signals

In some arrangements, product packaging is digitally watermarked over most of its extent to facilitate high-throughput item identification at retail checkouts. Imagery captured by conventional or plenoptic cameras can be processed (e.g., by GPUs) to derive several different perspective-transformed views&#x2014;further minimizing the need to manually reposition items for identification. Crinkles and other deformations in product packaging can be optically sensed, allowing such surfaces to be virtually flattened to aid identification. Piles of items can be 3D-modelled and virtually segmented into geometric primitives to aid identification, and to discover locations of obscured items. Other data (e.g., including data from sensors in aisles, shelves and carts, and gaze tracking for clues about visual saliency) can be used in assessing identification hypotheses about an item. Logos may be identified and used&#x2014;or ignored&#x2014;in product identification. A great variety of other features and arrangements are also detailed.

Methods and arrangements for identifying objects

A smart phone senses audio, imagery, and/or other stimulus from a user's environment, and acts autonomously to fulfill inferred or anticipated user desires. In one aspect, the detailed technology concerns phone-based cognition of a scene viewed by the phone's camera. The image processing tasks applied to the scene can be selected from among various alternatives by reference to resource costs, resource constraints, other stimulus information (e.g., audio), task substitutability, etc. The phone can apply more or less resources to an image processing task depending on how successfully the task is proceeding, or based on the user's apparent interest in the task. In some arrangements, data may be referred to the cloud for analysis, or for gleaning. Cognition, and identification of appropriate device response(s), can be aided by collateral information, such as context. A great number of other features and arrangements are also detailed.

Intuitive computing methods and systems

A decade from now, a visit to the supermarket will be a very different experience than the familiar experiences of decades past. Product packaging will come alive with interactivity&#x2014;each object a portal into a rich tapestry of experiences, with contributions authored by the product brand, by the store selling the product, and by other shoppers. The present technology concerns arrangements for authoring and delivering such experiences. A great variety of other features and technologies are also detailed.

In one aspect, a retail store has multiple sensors, including item sensors in a shopping cart for gathering data from a shopper-selected first item. At least certain of the sensor data is provided to a classifier, which was previously-trained (using data including optical data from known items) to identify possible item matches corresponding to data sensed from the first item. An item identification hypothesis that the shopper-selected first item has a particular identity is evaluated based on (a) information from the classifier, and (b) store layout data indicating items associated with a store location visited by the cart or shopper. The item identification hypothesis has a confidence score. If the score meets a criterion, an item of the hypothesized identity is added to a shopping tally. A great number of other features and arrangements are also detailed.

Cart-based shopping arrangements employing probabilistic item identification

In one aspect, a retail store includes a multitude of cameras, including a plurality of 3D cameras, and a plurality of other cameras. Certain of the cameras provide imagery from which a shopper's track through the store is monitored, and certain of the cameras are positioned to detect removal of items from store shelves. The store also includes a computer system that provides a database of information about store layout, indicating stock locations of different items. The computer system receives imagery from the cameras (or information derived from such imagery) and uses this data, together with information from the database and information derived from other sensors in the store, to produce a probabilistic tally of items selected by a store shopper. This tally includes an item bearing a barcode, but is produced without reading the barcode. Each item on the tally is associated with a confidence score that meets a computer system-determined threshold. A great number of other features and arrangements are also detailed.

Retail store with sensor-fusion enhancements

Self-checkout technologies for retail stores are improved by incorporation of enhanced fraud detection arrangements. For example, if a shopper enters an item on a self-checkout tally, while positioned at a location remote from the normal stock location for that item, a responsive action may be taken. Likewise, if a shopper enters the same item on a self-checkout tally twice, at widely separated times, a responsive action may be taken. These responsive actions can include dispatching a store clerk to assist the shopper, or increasing a risk score that is repeatedly re-calculated during the shopper's visit. A great variety of other features and arrangements (e.g., powering arrangements for mobile phones in shopping carts) are also detailed.

Self-checkout arrangements

Arrangements involving portable devices (e.g., smartphones and tablet computers) are disclosed. One arrangement enables a content creator to select software with which that creator's content should be rendered&#x2014;assuring continuity between artistic intention and delivery. Another utilizes a device camera to identify nearby subjects, and take actions based thereon. Others rely on near field chip (RFID) identification of objects, or on identification of audio streams (e.g., music, voice). Some technologies concern improvements to the user interfaces associated with such devices. For example, some arrangements enable discovery of both audio and visual content, without any user requirement to switch modes. Other technologies involve use of these devices in connection with shopping, text entry, and vision-based discovery. Still other improvements are architectural in nature, e.g., relating to evidence-based state machines, and blackboard systems. Yet other technologies concern computational photography. A great variety of other features and arrangements are also detailed.

Smartphone-based methods and systems

An illustrative implementation of the technology includes three primary components: a desktop application, a mobile phone application, and connections to retailer inventory and pricing APIs (e.g., for Walmart and/or Best Buy). The experience begins with the consumer going to an online retailer's website (e.g., Amazon) to search for a product. The desktop application automatically searches for the same product using the APIs of Walmart and/or Best Buy. If matches and near-matches of the product are found, the product name, model, price, and local availability at affiliate locations are shown. With a mobile phone camera-scan of the product page, relevant information is transferred to the consumer's phone. From there, the consumer can interact with the options on the mobile phone to be directed to the nearby brick and mortar store of choice carrying that product at the price they want. Along the way, the retailer can present offers and additional product information directly to the consumer. A great variety of other technologies and arrangements are also detailed.

Methods and arrangements including data migration among computing platforms, e.g. through use of audio encoding

A system senses audio, imagery, and/or other stimulus from a user's environment, and responds to fulfill user desires. In one particular arrangement, a discovery session is launched when the user speaks a cueing expression, which serves to switch the system from a lower activity state to a heightened alert state. The system may recognize that the speech expresses a user request that requires analysis of camera-captured imagery to fulfill. In response the system can apply an operation, such as a recognition operation (e.g., barcode decoding), to the imagery and take an action based on resulting information. Operation of the system can be aided by collateral information, such as context. A great number of other features and arrangements are also detailed.

Arrangements involving portable devices (e.g., smartphones and tablet computers) are disclosed. One arrangement enables a content creator to select software with which that creator's content should be rendered&#x2014;assuring continuity between artistic intention and delivery. Another utilizes a device camera to identify nearby subjects, and take actions based thereon. Others rely on near field chip (RFID) identification of objects, or on identification of audio streams (e.g., music, voice). Some technologies concern improvements to the user interfaces associated with such devices. Others involve use of these devices in connection with shopping, text entry, sign language interpretation, and vision-based discovery. Still other improvements are architectural in nature, e.g., relating to evidence-based state machines, and blackboard systems. Yet other technologies concern use of linked data in portable devices&#x2014;some of which exploit GPU capabilities. Still other technologies concern computational photography. A great variety of other features and arrangements are also detailed.

A variety of technologies having practical application in retail stores are detailed. One is an improved method of identifying items selected by customers. This method includes receiving sensor data from plural sensors, including (a) ceiling-mounted cameras that monitor tracks of customers through aisles of the store, and (b) inventory sensors that are positioned to monitor removal of stock from store shelves. This received sensor data is employed in evaluating plural alternate item identification hypotheses. These hypotheses include a first hypothesis that a customer selected an item having a first identity, and a second hypothesis that the customer selected an item having a second identity. A confidence score is associated with each of the first and second item selection hypotheses. These confidence scores are refined as sensor data is received, e.g., increasing a confidence score of one hypothesis, and reducing a confidence score of another. Such refining continues until one of the hypotheses becomes a winner, due to an associated confidence score fulfilling a predetermined criterion (e.g., reaching a threshold value), at which time the item can be added to a tally for that individual. The winning item identification hypothesis may identify a barcoded item, without that item's barcode ever having been read by a barcode reader. A great number of other features and arrangements are also detailed.

Methods of identifying barcoded items by evaluating multiple identification hypotheses, based on data from sensors including inventory sensors and ceiling-mounted cameras

Growing community of inventors

Lake Oswego, OR, United States of America

Bruce L Davis