Generating device identifiers and device controls based on hand gestures

Abstract

Techniques of controlling electronic devices using gestures use a wearable device on a user which translates, via a model, user movements into signals that both identify an electronic device to be controlled and a specific action to take with regard to that electronic device. The wearable device includes an inertial measurement unit (IMU) sensor and a photoplethysmography (PPG) sensor and measure six degrees of freedom (6DOF). The model is a convolutional neural network (CNN) that takes x, y, and z-acceleration signals generated by the IMU and PPG and places each acceleration component generated from each sensor in a separate channel. The CNN takes the input from each channel and generates a respective, separate model for each channel. The output at each of the stacked layers are combined in a fully connected layer to produce CNN output identifying an electronic device and a control for the electronic device.