Hirevue, Inc.

Benjamin Taylor

Loren Douglas Larsen

Daniel Bray

Christopher Luman

A computing system includes a processing device to execute instructions to: store a trained modeling function that embodies a correlation between historical ratings and historical evaluation results of recorded responses that were responsive to prompts presented to a set of persons; receive a request to initiate a digital evaluation process for a second person; receive recorded responses of the second person that are responsive to the prompts; generate, by applying the trained modeling function to the recorded responses of the second person, a reviewing sequence by which to view the recorded responses of the second person. The reviewing sequence orders the recorded responses from a most predictive prompt to a least predictive prompt to reduce a time required to efficiently determine an evaluation result. The system further transmits data that causes display of a list of the recorded responses of the second person in the reviewing sequence.

Model-assisted evaluation and intelligent interview feedback

A method for detecting bias in an evaluation process is provided. The method includes operations of receiving evaluation data from a candidate evaluation system. The evaluation data is provided by a set of evaluators based on digital interview data collected from evaluation candidates. The operations of the method further include extracting indicators of characteristics of the evaluation candidates from the digital interview data, classifying the evaluation candidates based on the indicators extracted from the digital interview data, and determining whether the evaluation data indicates a bias of one or more evaluators with respect to a classification of the evaluation candidates.

Model-driven evaluator bias detection

Described are methods and systems for interview competency and question validation and analysis to improve the effectiveness of evaluation campaigns. In one method, processing logic of an interview design program, hosted by a digital evaluation platform, receives a request from a first device to create a digital interview for a position. The interview design program sends a list of competencies associated with the position and receives a selection of a set of desired competencies. The interview design program determines a list of questions that differentiate candidates within the desired competencies, ranks the list of questions by importance, and sends the list of questions and the ranking information to the first device and receives a selection of a set of desired questions. The interview design program creates the digital interview with the set of desired questions. The digital evaluation platform presents the digital interview to a candidate on a second device.

Competency-based question selection for digital evaluation platforms

Technology for training a predictive model is described. A processing device collects digital interview data including features identified from candidate interviews. A model training tool selects a data set of the digital interview data. The data set includes a predicted performance outcome and an actual performance outcome for each of a plurality of candidates. The model training tool determines an error metric for each of the plurality of candidates. The error metric includes a relationship between the predicted performance outcome and the actual performance outcome for each candidate. The model training tool determines a number of candidates whose digital interview data includes a feature corresponding to a protected class. The model training tool normalizes an effect of each candidate on the error metric based on the corresponding protected class and applies the normalized error metric to reduce bias in the predictive model with respect to the protected class.

Performance model adverse impact correction

A method for facilitating the evaluation of candidates in an evaluation process is provided. The method includes operations of selecting a first data set including ratings of recorded responses of a set of evaluated persons and of selecting a second data set including additional recorded responses of a person under evaluation. The recorded responses are responsive to multiple prompts presented to each of the set evaluated persons in a recording sequence. The first data set further includes an evaluation result of each evaluated person in the evaluation. The additional recorded responses are responsive to the plurality of prompts and ordered in a first order corresponding to a recording sequence. The method includes analyzing the first data set to calculate a reviewing sequence and presenting the additional recorded responses in a second order corresponding to the reviewing sequence. Associated systems and other methods are also provided.

Described herein are methods and systems for interview question or prompt recommendation and analysis to improve the quality and efficacy of subsequent evaluation campaigns by combining data sets are described herein. In one method, processing logic selects a first prompt from a first data set of a first candidate evaluation campaign and a second prompt from a second data set of a second candidate evaluation campaign. The processing logic determines whether a degree of similarity between the first prompt and the second prompt exceeds a threshold and combines data from the first data set with data from the second data set to create a combined data set associated with the first prompt and with the second prompt based on the determination.

Automatic interview question recommendation and analysis

Methods and systems for model-driven candidate sorting for evaluating digital evaluations are described. In one embodiment, a sorting tool selects a data set of digital evaluation data for sorting. The data set includes candidate for evaluation candidates. The sorting tool analyzes the candidate data for the respective evaluation candidate to identify digital evaluation cues and applies the digital evaluation cues to a prediction model to predict an achievement index for the respective evaluation candidate. The list of evaluation candidates is sorted according the predicted achievement indices and the sorted list is presented to the reviewer in a user interface.

Model-driven candidate sorting

Methods and systems for model-driven candidate sorting for evaluating digital interviews are described. In one embodiment, a model-driven candidate-sorting tool selects a data set of digital interview data for sorting. The data set includes candidate for interviewing candidates (also referred to herein as interviewees). The model-driven candidate-sorting tool analyzes the candidate data for the respective interviewing candidate to identify digital interviewing cues and applies the digital interview cues to a prediction model to predict an achievement index for the respective interviewing candidate. This is performed without reviewer input at the model-driven candidate-sorting tool. The list of interview candidates is sorted according the predicted achievement indices and the sorted list is presented to the reviewer in a user interface.

Methods and systems for model-driven candidate sorting based on audio cues for evaluating digital interviews are described. In one embodiment, a processing device converts audio data of a candidate in a digital interview into voice-to-text data. An audio cue generator executing on the processing device, generates audio cues of the digital interview based on the voice-to-text data. The audio cues are applied to a prediction model to predict an achievement index for the candidate based on the audio cues. The processing device displays the candidate in a list of candidates based on the achievement index. The list of candidates is sorted according to the candidates' achievement index.

Model-driven candidate sorting based on audio cues

Methods and systems for model-driven candidate sorting based on audio cues for evaluating digital interviews are described. In one embodiment, an audio cue generator identifies utterances in audio data of a digital interview. The utterances each include a group of one or more words spoken by a candidate in the digital interview. The audio cue generator generates audio cues of the digital interview based on the identified utterances. The audio cues are applies to a prediction model to predict an achievement index for the candidate based on the audio cues. The candidate is displayed in a list of candidates based on the achievement index. The list of candidates is sorted according to the candidates' achievement index.

Growing community of inventors

Lehi, UT, United States of America

Benjamin Taylor