Systems and methods for creating generative ai frameworks on network state telemetry

Abstract

In some implementations, the device may include initiating a stream producer that sends formatted data to a topic. In addition, the device may include ingesting, by a sink connector, the formatted data into an object storage service. The device may include implementing an event-driven serverless compute, where the event-driven serverless compute is triggered automatically when any new data is ingested to the object storage service, and where the event-driven serverless compute reads the JSON data, converts it to transformed data, and writes the transformed data to a distributed data store. Moreover, the device may include creating an ETL job, where the ETL job reads the data, further transforms the data, and writes it back into the distributed data store as ETL transformed data. Also, the device may include sending the ETL transformed data to an LLM API in batches to create inference results, where the batches are queued to manage the rate limits. Further, the device may include storing the inference results in cache storage. In addition, the device may include implementing an API gateway for secure access to inference results.