Methods and systems for autonomous task composition of vision pipelines using an algorithm selection framework

Abstract

This disclosure relates generally to systems and methods for autonomous task composition of vision pipelines using an algorithm selection framework. The framework leverages transformer architecture along with deep reinforcement learning techniques to search an algorithmic space for unseen solution templates. In an embodiment, the present disclosure describes a two stage process of identifying the vision pipeline for a particular task. At first stage, a high-level sequence of the vision pipeline is provided by a symbolic planner to create the vision workflow. At second stage, suitable algorithms for each high-level task are selected. This is achieved by performing a graph search using a transformer architecture over an algorithmic space on each component of generated workflow. In order to make the system more robust, weights of embedding, key and query networks of a visual transformer are updated with a Deep Reinforcement Learning framework that uses Proximal Policy Optimization (PPO) as underlying algorithm.