Savant Systems, Inc.

Apple Inc.

Nicholas J Cipollo

Michael C Silva

Robert P Madonna

Arthur A Jacobson

David W Tatzel

Andrew R Hamm

Duarte M Avelar

Kevin C Kicklighter

Bryan S Bonczek

David McKinley

Gerald W Shields

David M Benvenuti

Jose J Ronquillo Rodriguez

Cameron Pulsford

Steve P Jobs

Timothy Ray Locascio

George T Katsiris

Ryan E Kitson

Teodor Lefter

James F Allen

Andrew Hamm

Nathan A Trapp

Erick W Drummond

Christopher Perry

Mark Tubinis

David Mckinley

Timothy R Locascio

Cameron B Pulsford

In one embodiment, a configuration application executing on a tablet computer or smartphone presents a configuration user interface on a touch screen for configuring a home automation system of a structure. A plurality of components of the home automation system that have wireless capabilities are detected. The configuration application receives an indication of a zone of the structure within which each component is located, the indication provided by the user dragging a representation of each component to a location provided in the configuration user interface. Based on the indication of the zone of the structure in which each component is located and the type of each component, automatically generate one or more user interface screens for accessing and controlling the components in each zone, where the automatically generated user interface screens are displayable during use of the home automation system.

Configuration user interface for a home automation system

In one embodiment, a virtual room-based user interface is provided. The interface displays a view of a virtual room that corresponds to a physical room and that includes a representation of a first device present in the physical room and at least one of a portion of structure of the physical room or a portion of furnishings present in the physical room. User input is received via the interface that indicates a change of state in the physical room. In response to the user input, a change is caused to a state provided by the first device, and the interface dynamically updates appearance of the virtual room to change appearance of the representation of the first device and at least one of the portion of the structure of the physical room or the portion of the furnishings present in the physical room to reflect the change to the state.

Small screen virtual room-based user interface

In one embodiment, a technique is utilized to generate a virtual-room of a virtual room-based user interface that controls one or more devices within a physical room of a structure. A user is prompted to select one or more services provided by the one or more devices within the physical room. A plurality of images of the physical room are obtained that each share substantially a same perspective. The plurality of images include an all-off image, in which all of the one or more of the services are in an inactive state, and one or more service-specific images, in which a single service is in an active state and any other services are in the inactive state. The plurality of images are aligned to create an aligned image stack. One or more hit boxes are designated. The virtual room is generated based on the aligned image stack and the designated one or more hit boxes.

User generated virtual room-based user interface

In one embodiment, a virtual room-based user interface suited for use on a 'small screen' is provided that organizes representations of devices in a virtual room into a sequence, allows a user to cycle through the representations of devices in the sequence so only a single one is selected at any given time, and individually control the device corresponding to a selected representation using a designated control region separate from the representation of the selected device.

In one embodiment, a tiled video display is shown on a screen of a display device. The tiled video display includes a plurality of video tiles that each show video content of a different video stream. The video tiles are arranged on the screen according to a tiling layout. A user interface (UI) is shown on a touch sensitive screen of a wireless mobile device separate from the display device. The UI includes a plurality of UI tiles that each correspond to a respective video tile of the tiled video display. The UI tiles are arranged on the virtual display screen according to the tiling layout of the video tiles. In response to a gesture over at least a portion of a particular one of the UI tiles on the touch sensitive screen, a modification is made to both the particular UI tile and a correspond video tile.

Video tiling

Techniques are provided for power optimization of home automation system devices (e.g., battery-powered devices). In one example embodiment, the power optimization is based on service states for a room associated with the device. A service state is determined for each of one or more services available in the room. The device is maintained in a low-power inactive state while there are no active services in the room. In response to at least one service being activated, the device transitions from the inactive state to a full-power in-use state. Such transition is performed preemptively, absent any current attempt to use the device by a user or another device. In response to expiration of a timer since a last use of the device or the transition to the in-use state was made and no actual use occurred, the device exits the in-use state and may eventually return to the inactive state.

Home automation system device power optimization

In one embodiment, a general-purpose wireless mobile device having a touch-sensitive screen and executing a remote control application is used to remotely control a vehicle (e.g., a marine vessel). The general-purpose wireless mobile device communicates via a wireless network with an interface (e.g., a server) that is coupled to an electronic control system of the vehicle (e.g., the vessel). In operation, environmental information and/or system status information is collected through the electronic control system, propagated to the interface (e.g., server), and then sent over the wireless network to the wireless mobile device. Similarly, control input is sent over the wireless network to the interface (e.g., server), which passes the information to the electronic control system, which in turn issues appropriate control signals to the vehicle subsystems (e.g., marine subsystems) to control the motion of the vehicle (e.g., the vessel).

Remote motion control using a wireless mobile device

In one embodiment, graphical programming is used to configure a system of audio, video, lighting, HVAC and/or security components. Representations of audio, video, lighting, heating, HVAC and/or security components are displayed in a graphical user interface. Each representation corresponds to a physical component that is available for inclusion in the system. A zone map that represents one or more zones is displayed in the graphical user interface. A selected representation of an audio, video, lighting, heating, HVAC or security component is dragged and dropping to a selected zone of the zone map to indicate installation of a selected physical component in the corresponding room. One or more realized services that the system is capable of providing are determined based at least in part on the selected zone of the zone map. Thereafter, an output is provided based on the one or more realized services.

Configuring a system of components using graphical programming environment having a zone map

In one embodiment, a virtual room-based user interface includes one or more virtual rooms. Each virtual room is rendered from one or more images captured of a corresponding physical room of a structure, and includes depictions of one or more light fixtures within the physical room, one or more furnishings within the physical room and one or more boundaries of the physical room. A user selects a particular depiction of a particular light fixture within a particular virtual room. In response, a state of the particular light fixture within the corresponding physical room is changed. Also, appearance of the particular virtual room is updated such that the depiction of the particular light fixture shows the particular light fixture with the changed state and the depictions of the one or more boundaries or the one or more furnishings show lighting effects resulting from the changed state.

Generating a virtual-room of a virtual room-based user interface

Techniques are provided for defining and activating scenes in a home automation system. In one embodiment, a user-defined scene is automatically captured based on current states in a home automation system and a last media query. Current states in the home automation system are automatically captured for services to build a set of states, the set of states including states of services provided by multiple different types of devices, the multiple different types including two or more of audio/video (A/V) devices, lighting devices, heating ventilation and cooling (HVAC) devices, security devices, shade control devices, energy conservation devices, or communications devices. A last media query performed by the home automation system is also captured. A user-defined scene is generated from both. The user-defined scene is stored, and when it is determined it should be activated, service requests are sent.

User-defined scenes for home automation

In one embodiment, history-based key phrase suggestions are provided for voice control of a home automation system. A host controller builds a history database that maintains a history of events, each event indicating one or more services activated, one or more scenes activated or one or more types of media content presented by the home automation system. A predictions process on the host controller selects a subset of the events of the history database based on a present condition of the home automation system to produce a predication dataset. A grammar process on a remote control or mobile device selects one or more intentions from a list used by a voice recognition engine, based on the prediction dataset. Each intention includes a key phrase paired with an action. A voice user interface (UI) process suggests key phrases from the one or more selected intentions in a voice control UI on the remote control or mobile device.

History-based key phrase suggestions for voice control of a home automation system

In one embodiment, a technique is provided for automatically generating a custom user interface (UI) of a control device usable to control a home automation system. A configuration database is maintained, for example, on a host controller, of the home automation system. The configuration database stores logical representations that describe a configuration. The configuration database is transferred (e.g., downloaded) to a control device. The control device accesses the configuration database for one or more of the logical representations, and translates returned information related to the one or more of the logical representations to UI element, using predefined mappings. The controll device then displays the UI elements as a custom UI.

Automatic configuration of control device user interface in a home automation system

In one embodiment, a configuration application executing on a tablet computer or smartphone presents a configuration user interface on a touch screen for configuring a home automation system of a structure. A plurality of components of the home automation system that have wireless capabilities are detected. Component profile information for the detected components is obtained. The configuration application prompts a user of the tablet computer or smartphone to indicate a zone of the structure within which each component is located. Based on the component profile information and the indications of the zone of the structure in which each component is located, one or more services available in each zone are determined and one or more user interface screens are automatically generated for accessing and controlling the services, wherein the automatically generated user interface screens are displayable on the tablet computer or smartphone or another tablet computer or smartphone.

Configuration interface for a programmable multimedia controller

In one embodiment, a custom user interface (UI) is provided for a plurality of devices of a home automation system within a structure. A configuration database is constructed that stores logical representations. The logical representations stored in the configuration database to include components, services, rooms, and zones. A mobile application (app) finds and connects to the configuration database. The mobile app then queries the configuration database. Based on the querying, the mobile app presents a custom user interface (UI) to the user on a display screen, wherein the custom UI is usable to control one or more of the plurality of devices of the home automation system.

Growing community of inventors

Boston, MA, United States of America

Nicholas J Cipollo