The patent badge is an abbreviated version of the USPTO patent document. The patent badge does contain a link to the full patent document.
The patent badge is an abbreviated version of the USPTO patent document. The patent badge covers the following: Patent number, Date patent was issued, Date patent was filed, Title of the patent, Applicant, Inventor, Assignee, Attorney firm, Primary examiner, Assistant examiner, CPCs, and Abstract. The patent badge does contain a link to the full patent document (in Adobe Acrobat format, aka pdf). To download or print any patent click here.
Patent No.:
Date of Patent:
Dec. 07, 2021
Filed:
Dec. 20, 2019
Samsung Electronics Co., Ltd., Suwon-si, KR;
Alborz Rezazadeh Sereshkeh, Toronto, CA;
Gary Leung, Toronto, CA;
Krish Perumal, Toronto, CA;
Iqbal Mohomed, Toronto, CA;
Afsaneh Fazly, Toronto, CA;
Caleb Phillips, Toronto, CA;
Minfan Zhang, Toronto, CA;
SAMSUNG ELECTRONICS CO., LTD., Suwon-si, KR;
Abstract
An apparatus for vision and language-assisted smartphone task automation, includes a processor configured to execute instructions to identify whether an utterance of a user refers to one among at least one existing automation script, and based on the utterance being identified to not refer to the one among the at least one existing automation script, capture one or more screenshots and one or more coordinates of a display, the one or more screenshots and the one or more coordinates corresponding to one or more interactions of the user with the display while the user is demonstrating a new automation script, perform object detection and optical character recognition on the captured one or more screenshots and the captured one or more coordinates to obtain user interface (UI) elements corresponding to the one or more interactions, and obtain the new automation script, based on the obtained UI elements.