Vmware, Inc.

Microsoft Technology Licensing, LLC

Salim S AbiEzzi

Kosar Akram Jaff

Jose Fonseca

Arun Ayyagari

Timothy Mark Moore

William Michael Zintel

Christopher J Corbett

Gregory Bollella

Mark Sheldon

Keith Whitwell

Amer A Hassan

Ole Agesen

Warren Vincent Barkley

Greg D Schechter

Jeffrey W Sheldon

Scott Manchester

Benjamin E Nick

Osten Kit Colbert

Colin L Campbell

Todd B Knoblock

Conal M Elliott

Chun-Fu Ricky Yeung

Sandro Moiron

Jeffrey R Glasson

Greg Bollela

Alan Hourihane

The management of internet of things (IoT) objects through a self-describing interoperability framework is described. In one example, a method for self-described object management includes communicating, by an internet of things (IoT) object, a request to register the IoT object, receiving, by the IoT object, an inquiry from an IoT management system, and communicating, by the IoT object, a self-describing declaration to the IoT management system. The self-describing declaration can include an interface parameter schema for the IoT object and an operating parameter schema for the IoT object, among other data structures. The method can also include establishing an interoperability framework between the IoT object and the IoT management system based on the interface parameter schema and the operating parameter schema. Based on self-describing declarations from various IoT objects, a number of different IoT objects can be easily recognized, integrated with, and managed by the IoT device management system.

Internet of things management through self-describing objects

Systems and techniques are described for remoting application user interfaces. One of the techniques includes receiving a first user request to access an application from a first user device, wherein the application comprises a respective user interface code path corresponding to each of a plurality of user device classes, and wherein each user interface code path, when executed, generates a user interface that is specific to user devices of the corresponding user device class; classifying the first user device into a first user device class of the plurality of user device classes; causing the application to generate a user interface by executing a first user interface code path corresponding to the first user device class; and providing the user interface for presentation on the first user device in response to the first user request.

Remoting application user interfaces

Techniques for implementing fine-grained access control in an IoT (Internet of Things) deployment are provided. In one set of embodiments, a gateway of the IoT deployment can create/maintain a device proxy pertaining to an IoT device and a persona in the IoT deployment, where the device proxy includes one or more access methods for accessing the IoT device, and where the one or more access methods reflect access rights that are deemed appropriate for the persona with respect to the IoT device. An application instance of the IoT deployment can receive a request from the persona to access the IoT device. Networking equipment interconnecting the application instance with the gateway can then automatically route, via one or more SDN micro-segmentation rules, the request to the device proxy for processing via the proxy's access methods.

Fine-grained IoT access control via device proxies and SDN-based micro-segmentation

The management of internet of things (IoT) objects through a self-describing interoperability framework is described. In one example, an IoT object can be registered with an IoT device management system. The IoT device management system can then query the IoT object for a self-describing declaration. The self-describing declaration can include a schema that defines certain operating and interface parameters for the IoT object. The IoT device management system can parse the self-describing declaration to determine, for example, both normal or expected and abnormal or pathologic data from the IoT device. The self-describing declaration can also define function call parameters to establish an interoperability framework between the IoT device management system and the IoT object. Based on such self-describing declarations from various IoT objects, a number of different IoT objects can be easily recognized, integrated with, and managed by the IoT device management system.

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for changing virtual machine user interfaces. One of the methods includes receiving a first request from a first client device to initiate a first remote session, detecting, for the first remote session, a first display property of the first client device in response to receiving the first request, configuring, for the first remote session, a virtual display device for the virtual machine to be a display device having the first display property, receiving a second request from a second client device to initiate a second remote session, detecting, for the second remote session, a second display property of the second client device in response to receiving the second request, and configuring, for the second remote session, the virtual display device for the virtual machine to be a display device having the second display property.

System and method for dynamically configuring virtual displays and virtual inputs for different remote sessions that each present content for a virtual machine

Systems and techniques are described for remoting application user interfaces. One of the described techniques includes initiating a remote session with a first user device that allows user interfaces generated by an application to be presented on the first user device and user events associated with the presented user interfaces to be provided as input to the application; and during the remote session with the first user device: receiving a request to initiate a multimedia streaming session between the first user device and a different system through the application; establishing the multimedia streaming session with the different system; and providing the streaming session parameters to the first user device for use by the first user device in establishing a connection between the first user device and the different system and streaming multimedia content over the connection.

Establishing streaming sessions during remote sessions

Methods, techniques, and systems for user interface remoting using video streaming techniques are provided. Example embodiments provide User Interface Remoting and Optimization System ('UIROS'), which enables the efficient remoting of pixel-oriented user interfaces on behalf of their guests using generic video streaming techniques, such as H.264, to send compressed user interface image information in the form of video frame encoded bitstreams. In one embodiment, the UIROS comprises server side support including a UI remoting server, a video encoder, and rendering support and client side support including a UI remoting client, a video decoder, and a display. These components cooperate to implement optimized UI remoting that is bandwidth efficient, low latency and CPU efficient.

User interface remoting through video encoding techniques

Systems and techniques are described for executing an application locally. A described technique includes providing, by a user device and to a remote application system, data identifying user events associated with the application while data communication is available between a user device and a remote application system executing an application. The user device receives from the remote application system user interfaces for the application and presents the user interfaces on a display of the user device. The user events are provided as inputs to a local version of the application executing on the user device. A determination is made that data communication between the user device and the remote application system has become unavailable. In response, a user interface generated by the local version of the application is presented on the display of the user device.

Local operation of remotely executed applications

Systems and techniques are described for remoting application user interfaces. One of the described techniques includes receiving, by a remote application system, a first user request to access an application from a user device; initiating, by the remote application system, a remote session with the user device that allows user interfaces generated by the application to be presented on the user device and user events associated with the presented user interfaces to be provided as input to the application; during the remote session with the user device; obtaining off-screen user interface data; providing the off-screen user interface data for storage on the user device; receiving data identifying a first user event from the user device; determining that the first user event triggers presentation of the off-screen user interface data stored on the user device; and providing instructions to the user device to present the off-screen user interface data.

Using off-screen user interface data during remote sessions

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for changing virtual machine user interfaces. One of the methods includes detecting, for a remote session, a first resolution for a content presentation area in a web browser, generating, using the first resolution, a first stream of pixels representative of a user interface, providing the first stream of pixels for display in the web browser, receiving input indicating a change to a resolution of the content presentation area in the web browser, detecting a second resolution for the content presentation area in the web browser, generating, using the second resolution, a second stream of pixels representative of the user interface, and providing the second stream of pixels for display in the web browser by the client device.

Virtual machine pixel remoting

Frames of a virtual desktop are encoded using a hybrid approach that combines the strength of software encoding by a central processing unit (CPU) and hardware encoding by a graphics processing unit (GPU). A method of encoding frame data of one or more virtual desktops in hardware and in software and transmitting the encoded frame data to one or more client devices, includes the steps of encoding a first portion of the frame data in the GPU to generate a first encoded frame data, encoding a second portion of the frame data in software, i.e., programmed CPU, during encoding of the first portion, to generate a second encoded frame data, and transmitting the first encoded frame data and the second encoded frame data from a host computer of the one or more virtual desktops to the one or more client devices as separate video streams.

Hybrid software and GPU encoding for UI remoting

Systems and techniques are described for multi-user support on set top boxes and game consoles. A described technique includes executing a hypervisor that monitors a plurality of virtual machines that execute a set top box operating system or a game console operating system, providing a selection menu to a first display device, receiving a first selection of a first virtual machine, executing the first virtual machine, providing a first stream of content for a first user interface of the first virtual machine to the first display device, providing the selection menu to a second display device, receiving a second selection of a second virtual machine, executing the second virtual machine, and providing a second stream of content for a second user interface of the second virtual machine to the second display device while providing the first stream to the first display device.

Multi-user support for set top boxes and game consoles

Methods, techniques, and systems for dynamically allocating graphics processing units among virtual machines are provided. Example embodiments provide a dynamic GPU allocation system ('DGAS'), which enables the efficient allocation of physical GPU resources to one or more virtual machines. In one embodiment, the DGAS determines runtime profiles of a plurality of virtual machines. The runtime profiles are determined by determining a runtime assessment of workload indicative of a potential benefit that the virtual machine can receive from using a GPU resource. The DGAS then causes the processing bandwidth of the physical GPU resources to be allocated to some of the plurality of virtual machines based upon the dynamically determined runtime assessments of workload.

Runtime profile determinations of virtual machines for graphics processing unit (GPU) allocation

Methods, techniques, and systems for dynamically allocating graphics processing units among virtual machines are provided. Example embodiments provide a dynamic GPU allocation system ('DGAS'), which enables the efficient allocation of physical GPU resources to one or more virtual machines. In one embodiment, the DGAS comprises a GPU allocation list for use in allocating the physical GPU resources comprising one or more virtual machine entries each containing a designation of a virtual machine, an indication of a GPU benefit factor associated with the designated virtual machine, and an indication of processing bandwidth requirements associated with the designated virtual machine. The entries are ranked based at least upon the GPU benefit factor associated with each designated virtual machine. Available GPU resources are allocated to some subset of these ranked virtual machines as physical GPU capacity is matched with the requirements of the subset.

Growing community of inventors

Kirkland, WA, United States of America

Salim S AbiEzzi