Sony Computer Entertainment America LLC

Sony Computer Entertainment Inc.

Sony Interactive Entertainment Inc.

Sony Interactive Entertainment America LLC

Other

Sony Computer Entertainment Europe Limited

Sony Corporation

Analog Devices,inc.

Onix Microsystems, Inc.

Xtellus, Inc.

Sony Interactive Entertainment Europe Limited

Gary M Zalewski

Richard Lee Marks

Albert S Penilla

Xiadong Mao

Glen Van Datta

Riley R Russell

Charles Nicholson

Tomas Gillo

Xiaodong Mao

Ennin Huang

Attila Vass

Anton Mikhailov

Payton R White

Teiyu Goto

Scott Christopher Waugaman

Mitchell Goodwin

Jae Won Paik

Heinrich Shih Ko

James E Marr

Hrishikesh R Deshpande

Hrishikesh Despande

Michael J Daneman

Stephen Cain Detwiler

Gregory Alan Corson

Aaron Brunstetter

Mark Francis Villa

Scott Rohde

Colby B Springer

Crusoe Xiaodong Mao

Ken Kutaragi

Masayuki Chatani

Dominic Saul Mallinson

Boris Kobrin

Eric James Larsen

Kan Ebisawa

Shinichi Okamoto

Timothy E Beerling

Behrang Behin

Chuang-Chia Lin

Scott D Dalton

Keiso Shimakawa

Shanti A Cavanaugh

Phil Harrison

Murali Chaparala

Vinod Tandon

Makoto Kubo

Sean Turner

Oliver R Capio

Yutaka Kagiwada

Andrew House

Mark W Chang

Stephen Panyko

Steven Wagner

Brad Pollard

Mike Haigh

A device is provided that includes a persistent memory, a microcontroller and a power source configured to generate power in response to energy harvesting. The power is transferred to a power storage of the device such that when the power storage has an amount of power the microcontroller performs processing of a part of a task resulting in state data related to the task stored to the persistent memory in response to a first cycle of energy harvesting. In response to one or more additional cycles of energy harvesting, the device continues performance of the task by retrieving the state data for the part of the task from the persistent memory and resulting in assembly of a payload that comprises an update to software running on the device.

Devices that use power harvesting power sources for operation

A batteryless device is provided. The batteryless device includes a logic processor, a power storage cell, a sensor, a persistent memory, a wireless logic and an antenna configured to harvest power in response to exposure to radio frequency (RF) energy. The harvested power is transferred to the power storage cell such that when the power storage cell has a threshold amount of power the logic processor is activated to start or resume execution of code associated with a task of sampling the sensor, processing sensor data from the sensor into an encrypted message, and transmitting the encrypted message. The batteryless device transmits one or more of said encrypted messages using said wireless logic to a local device proximate to a location of the batteryless device. The one or more encrypted messages providing information related a state of a package to which the batteryless device is attached. The local device is configured for sending data related to said one or more encrypted messages to a cloud server for processing metrics related to the location and said state of the package. The local device is one of a plurality of local devices that become proximate to the batteryless device when the package moves proximate to said plurality of local devices at different times. The metrics enable tracking over time of the package and said state of the package.

Energy harvesting package tracking for use in carrier logistics with cloud state monitoring

A method for providing real-time recommendations to user in a shopping environment involves sampling a shopping environment using video cameras to generate video features related to a shopper in connection to an item, the sampling input to a machine learning model to create labels related to a state of a scenario, the scenario including the shopper handling the item. Supplemental information is provided to the shopper in connection with the item. The makeup of the supplemental information may be sourced from online service or device associated with the shopper. The supplemental information may be delivered to the shopper, or to a shopper-aware display in the store, or elsewhere. A processing entity associated with the store detects a scenario to identify the shopper as having finished shopping, and causing a charge of the item to a cashierless shopping cart associated with the shopper. In one example, passive, wireless weight sensors may be used to supplement input features for inferences made by the machine learning model. In another example, wireless, battery-free displays may be coupled to shelves to provide information related to supplemental information relating the item held by the shopper.

Artificial intelligence systems and methods for cashierless tracking and feedback

Method of identifying actions of a shopper to account for taken items by the shopper in a cashierless checkout includes sampling a shopping environment using one or more video cameras to generate video features related a shopper in connection to an item and sampling using one or more supplemental sensors to generate supplemental sensor feature data, receiving output of the sampled video and supplemental sensor features as feature inputs to a deep learning model used for making inferences related to the state of a scenario involving shopper action of taking the item into their possession or held and other actions including moving outside a zone initially associated with the item.

Methods and systems for identifying actions of a shopper to account for taken items in cashier-less transactions

A system and controller are provided. In one example, the system includes a controller for playing a video game. The controller includes input buttons for controlling interaction during play of the video game. The controller includes an inertial sensor for detecting translational or rotational movement of the controller and a vibrational device. A voice input device is associated with the controller. A wireless communication circuit of the controller is provided for communicating data between the controller and a local computing device. The local computing device is connectable to a network. The data includes control responsive to presses of said input buttons or control received from said voice input device.

Voice command controller for communicating with a gaming system

Methods and system for processing shopping activity for a retail store are provided. One example method includes associating a plurality of sensor devices to monitor a shelf of the retail store. The shelf having one or more physical items and each of the physical items is capable of being associated with a state change in response to user interaction. The method includes detecting user interaction by a user in the retail store with a physical item of the one or more physical items. The user interaction causes the state change in a sensor device associated with the shelf, and responsive to the state change transmitting a message related to the state change over a network to an end node. The end node is configured to receive the message causing processing of an action based on the message. The one or more physical items are offered for sale in the store and data regarding the user interaction of the physical item is associated to a user account of the user. The user interaction is one of the physical item being touched, being lifted from the shelf, being moved from the shelf, or being released from the shelf of the retail store.

Methods for monitoring state information of items

A cashierless tracking system uses a biometric sensor disposed at the entry of a store or the entry of a shelf or an entry to a section of a store is configured to capture biometric feature aspects of a user. Cashierless shopping is enabled without requiring a user to supply mobile phone or device to identify themselves. Instead, processing and classification of biometric features occur via a neural network to identify at least one label representing classified features of the user, the label being used to identify a profile for the user. A plurality of sensors in the store produces data to identify a take of an item by a single user or by a group of users tied to a single account. Included tracking embodiments involve overlapping cameras, skeletal tracking, microphone input, feature extraction and/or feature engineering. The take of the item is chargeable to an electronic shopping cart of the user. Items may be charged or accounted for in relation to user activity, store membership, subscription, entitlement, direct debit, etc.

Cashierless system using deep learning processing of hand scan biometric feature data for labeling an identity of a tracked shopper

A computer-implemented methods for generating a mini-game are provided. One method includes recording gameplay of a game on a computer game system and receiving a signal to generate said mini-game, and responsive to said signal suspending play of the game. The method includes identifying a start point and an end point for the mini-game from the recorded game, and using said start point and said end point for generating executable instructions to enable play of the mini-game. The method includes saving said generated executable instructions to a distribution system. The distribution system includes at least one server to enable access to the mini-game to access said generated executable instructions. And, selection of the generated executable instructions enables interactive gameplay of the mini-game.

User-based mini-game generation and distribution

A scene of a portion of a simulated environment within a field of view of a virtual camera from a virtual camera point of view (camera POV) is displayed on a display. The scene displayed is changed during a user's interaction with the simulated environment. A revised virtual camera path is generated that places more of an advertising target within the field of view of the virtual camera. The advertising target is characterized by a field of attraction of the camera POV.

Increasing the number of advertising impressions in an interactive environment

Methods and systems related to tracking activity in a store in association with a cashier-less environment are provided. One example method includes identifying actions in a store. The method includes sampling a shopping environment using one or more sensors that include at least one camera capable of providing depth sensing to produce image data of a scene that shows a shopper in the store and tracking data related to one or more limbs of the shopper in connection to an item. The method includes receiving output of the sampling as feature inputs to one or more machine learning models and deriving one or more label inferences of a behavior state of the shopper in connection with a state of the item. At least one processing entity associated with the store detects the state of the item to change from one as item handled by said shopper to one as item queued for purchase. The sensor data from said one or more sensors is used to identify a scenario that indicates the item chargeable to an electronic shopping cart of the shopper. In some examples, in addition to analyzing output from cameras, the analysis for identifying the item and/or the take of the item is augmented by processing data produced by sensors installed in or around shelfs or locations where the item or items are placed in the store.

Methods and systems for identifying actions of shoppers in stores in relation to items provided for sale in cashier-less transactions

Method and systems are provided for processing actions in a store. One example method includes capturing sensor output from two or more sensors in a shopping environment. One sensor includes a first camera to capture scene data where the scene includes movement of a shopper in the store performing interaction with an item in the store. Another sensor includes a second camera capturing at least part of the scene from a different perspective. The method includes processing, by a processing entity associated with the store, at least one of the camera's output to generate feature data. The feature data is processed by one or more machine learning models to produce engineered feature data. The engineered feature data includes data relating to tracking skeletal movement of shopper. The method includes processing, by a processing entity associated with the store, the feature data including engineered feature data using said one or more machine learning models to produce a prediction that a take of the item has occurred by the shopper. The prediction is based on a characterization that said feature data or engineered feature data is labeled to infer that the movement of the shopper is interaction with the item that is classified as the take as having occurred.

Machine learning methods and systems for tracking shoppers and interactions with items in a cashier-less store

Ancillary information associated with one or more images embedded in a digital broadcast data stream may be accessed by a digital broadcast receiving device. The device may include a receiver unit, a processor, and a memory. The processor may execute program instructions to access the ancillary information. Control data may be associated with one or more visual components of one or more video images. The control data and images may be combined into a digital broadcast data stream. The device may receive the images and control data. The images may be presented on a display device. The receiving device may use the control data to present or store a perceptible indication that ancillary information is associated with the visual components in conjunction with presenting the images. The device may use the control data to present or store the ancillary information in response to an input signal.

Integration of control data into digital broadcast content for access to ancillary information

Systems and methods for verifying an advertisement impression in a digital environment are provided. In some aspects, methods of the subject technology include operations for defining a portion of the digital environment as an impression area, wherein the impression area is associated with a tagged advertisement area, providing a stream of an advertisement to the tagged advertisement area, and updating advertising impression information stored in memory regarding the advertisement, wherein an advertising impression is based on the identification of the character within the impression area and the availability of an unobstructed line-of-sight between the character and the tagged advertisement area. In some aspects, computer readable media are also provided.

Advertising impression determination

A method of tracking a user in a store and tracking takes and puts in regard to items in said store. One method includes tracking a shopper in a physical store using two or more cameras that are overlapping to infer and account for shopping activity performed by the shopper. The method includes providing output of at least one of the two or more cameras to a processing entity to extract skeletal limb features of the shopper. Then, processing the skeletal limb features of the shopper to detect a take of an item from the store into possession of the shopper. The method further includes detecting, based on the tracking, that the shopper has exited the store and processing a charge to an account of shopper for the item based on the detected take.

Using image and voice tracking to contextually respond to a user in a shopping environment

A device is provided that includes a logic processor, a power storage cell, a sensor, a wireless logic and a power source. The power source is configured to harvest power in response to exposure to radio frequency (RF) energy using an antenna element of the device. The harvested power is transferred to the power storage cell such that when the power storage cell has a usable amount of power the logic processor (a) samples data using the sensor, (b) processes the sample data into a message that is encrypted, and (c) transmits the message using the wireless logic to an end node that is configured to receive the message from the device. When usable amount of power is not present in the power storage cell during any one of (a)-(c), data is saved in persistent memory until additional harvested power is present in the power storage cell.

Energy harvesting sensor device with wireless communication

Growing community of inventors

Piedmont, CA, United States of America

Gary M Zalewski