Company Filing History:
Years Active: 2020-2023
Title: Innovations of Joel Hestness
Introduction
Joel Hestness is an accomplished inventor based in Mountain View, CA (US). He has made significant contributions to the field of deep learning and has been awarded 3 patents for his innovative work. His research focuses on understanding the relationships between training set size, computational scale, and model accuracy improvements.
Latest Patents
One of his latest patents is titled "Predicting Deep Learning Scaling." This patent presents a large-scale empirical study of error and model size growth as training sets increase. It introduces a methodology for measuring these relationships and predicts other metrics, such as compute-related metrics. The findings indicate that power-law can represent deep model relationships, such as error and training data size. Additionally, it shows that model size scales sublinearly with data size, which has significant implications for deep learning research and practice.
Another notable patent is "Embeddings with Classes." This patent describes systems and methods for word embeddings that avoid discarding rare words appearing less than a certain number of times in a corpus. It involves grouping words into clusters or classes and generating multiple copies of the training corpus to replace each word with the appropriate class. A word embedding generating model is then run on these multiple class corpora to create class embeddings. The effectiveness of this approach has been demonstrated through test results.
Career Highlights
Joel Hestness currently works at Baidu USA LLC, where he continues to push the boundaries of deep learning technology. His work has been instrumental in advancing the understanding of deep learning scaling and word embeddings.
Collaborations
He has collaborated with notable colleagues, including Hee Woo Jun and Sercan Omer Arik, contributing to a dynamic research environment that fosters innovation.
Conclusion
Joel Hestness is a prominent figure in the field of deep learning, with a focus on scaling and word embeddings. His patents reflect his commitment to advancing technology and improving methodologies in this rapidly evolving field.