The science behind Retorio's behavioral analytics A.I.

Based on science

We work with scientifically validated personality and culture models with significant explanatory power in terms of behavior and performance

Globally applicable

Our models have been trained with a demographically diverse dataset. We have analyzed +100k people from all around the world

Debiased models

Our models are blind to age, gender, or skin color. We ensure that our models remove human biases and discrimination tendencies

Machine learning

We exclusively employ supervised behavioral machine-learning models that analyze actual behavior, rather than self-reported data

We build upon the Big 5 - The gold standard of personality research

Retorio is based on the scientifically validated Big 5 model (McCrae & John, 1992) – probably the only model of personality that has reached academic concensus. The Big 5 model describes a person's personality based on five dimensions: Openness, conscientiousness, extraversion, agreeableness, and neuroticism.

Each of these traits has further descriptive subdimensions. Openness, for example, is based on the subdimensions intellectual curiosity, aesthetic interest, and creative imagination.

Click here to read more about the Big 5

We match personality with organizational culture

Organizational culture reflects behavioral patterns and attitudes lived through the members of an organization (e.g., individuals, groups, teams, etc.). With Retorio we employ one of the most dominant and well researched taxonomies of culture, which relies on seven different facets (cf. O'Reilly III et al., 1991).

Organizational researchers have connected these seven cultural facets with individual-level traits and attributes. Based on this matching, individual-level personality traits can be aggregated towards a higher-level culture model. As a result, we can determine person-culture fit and estimate whether a person is more or less likely to thrive within an organizational culture. Empirical reseach suggests that low scorers on person-culture fit are substantially more likely to leave an organization than high scorers.

Retorio AI Science culture organization fit

We analyze observed behavior rather than self-reports

Retorio describes personality based on observed behavior rather than conventional questionnaire-based self-estimations. Questionnaire-based personality assessments, such as Hogan HPI, NEO-PI-R, etc., are widespread. However, there are (at least) two good reasons to rely on observer ratings instead.

Observer ratings are less sensitive to gaming and manipulation

As part of our past research we often observed participants who overestimated their traits in order to maximize their chances of being hired or promoted. In conventional questionnaires, it just takes a few deliberately misplaced crosses on a sheet of paper to become a completely different person.

In contrast, observer ratings rely on observable behavior and on actual performance. Of course, observer ratings can also be manipulated to some extent. Yet, a person who successfully pretends to be a great communicator can be seen as a great communicator; A hidden introvert who has no problems approaching other people deserves to be "misclassified" as an extrovert.
Observer ratings reliably predict future behavior (and thus performance)

Science supports the use of observer ratings instead of self-ratings. In a workplace context, observer ratings have more predictive validity than self-reporting (Mount et al., 1994).

At the same time, personality can be reliably assessed by strangers. Research shows that strangers can correctly predict extraversion only after 50 milliseconds exposure to a face (Borkenau et al., 2009). For other Big 5 personality dimensions like agreeableness or conscientiousness, similar effects were found after 20-30 seconds (Kogan et al., 2011). Showing people short video clips (i.e., 30 seconds) is sufficient to form reliable judgements, which have predictive power in terms of job performance (Ambady et al., 2006; Ambady & Rosenthal, 1993).

Our machine-learning approach and training data

Using automated video interviews, Retorio's AI analyzes behavioral cues like facial expression, body language, and voice, in order to derive a behavioral personality profile. Moreover, we transcribe spoken words and analyse use of language. Retorio uses proprietary neural networks and NLP models for its analyses. Our approach of combining of different machine learning methods (ensemble models) offers security, transparency, and deeper insights. Currently, we combine the following machine-learning approaches:

Deep learning for visual computing (facial expression, body language, etc.)
Acoustic analysis of voice data
Deep learning for speech-to-text recognition (externally sourced) Natural Language Processing (NLP) for language analysis (identifying langauge patterns, use of language, etc.)
Conventional ML-models for multi-modal interpretation and matching (e.g. OLS-regression analysis, etc.)

Retorio machine-learning approach and training data AI & Science

Deep Learning

The core of Retorio's AI is represented by our cutting-edge deep learning models. Deep learning is a form of machine learning that trains computers to perform human-like tasks, such as recognizing speech, identifying emotions from video data or making predictions in terms of personality or job fit. Instead of structuring data and running it through predefined equations, deep learning sets up basic parameters about the data and trains the computer to learn on its own by recognizing patterns using many layers of processing.

A lot of computational power is needed to solve deep learning problems, such as classifying personality, because of the iterative nature of deep learning algorithms, their complexity as the number of layers increase, and the large volumes of data needed to train the networks. Yet, our approach of using deep learning enables us to do things that were unimaginable just a few years ago.

First, deep learning enables us to improve the accuracy and performance of our neural networks. Through improved algorithms and more computing power, we can add greater depth to our personality and culture fit predictions.

And second, deep learning presents a great opportunity to introduce more dynamic behavior into analytics. With Retorio we can offer greater personalization of customer analytics based on customer-specific data. By providing us information about your team and the people you hire, we can provide our models with a continuously improved understanding of criteria that drive performance within your organization.

Datasets

Any AI is just as good as the data from which it learns. We have trained our AI on a broad data set ― millions of data points ― spanning a wide range of ages, sexes, and ethnicities. Retorio is a so-called "supervised AI" – it only learns under human supervision and from scientifically audited datasets. In order to avoid bias and generate representative results, our datasets represent behaviors and perceptions of people from diverse populations.

We ensure that only factors that are within the control of the applicant are included in the results. For example, in our datasets, we compare mean scores of Big 5 estimations between caucasians and blacks, young and old people, men and women, etc. If we would discover significant mean differences attributable to group membership, we would adjust the mean values and distributions to adjust for discriminatory bias in our training and test sets.

We regularly test our corrected models against large, scientifically sound datasets, such as UCLA's Fairface dataset, which contains approximately 100,000 individuals from different cultures, age groups, etc. We publish our results transparently. Our findings clearly indicate that Retorio evaluates applicants regardless of their skin color, gender, or age. Accordingly, Retorio AI contributes to more fair and objective talent decisions.

Our AI – 90% accurate and growing

Predictive accuracy indicates how far Retorio's predictions are, on average, from the perception that a representative group of people has towards an individual. Currently, Retorio achieves an accuracy of approximately 90%. Thus, in an attempt to predict a person's external perception using the Big 5 taxonomy, we observe a deviation of about 10% on average. This deviation corresponds to the measurement error of our procedure. It should be noted here that our AI was trained for a specific purpose: To evaluate typical job application videos or everyday situations in organizations. All reported accuracy levels refer to this specific purpose.

Prevent unequal treatment in HR processes

Talents are still discriminated against on the basis of their skin color, religion, and culture. For instance, based on a field experiment, a 2018 study by WZB shows that black and white applicants are treated significantly differently, even when they apply with identical CVs.

Retorio ensures that only factors that are within the control of the applicant are included in the results. We actively debias our datasets and models to ensure Retorio does not see skin color, gender, or age.

Moreover, Retorio does not consider a person's dialect, the background of an application video, hair color, etc. For humans, avoiding these factors is substantially more difficult.

Reduce recruiter workload, improve interview quality

Overworked HR departments often have little time to prepare for and follow up on their interviews. In their accelerated workday, decision makers need to rely on their intuition which might be subject to biases.

Whereas in the past only a fraction of all applicants were able to introduce themselves in person, Retorio provides each individual with an opportunity to introduce himself or herself and to show exactly what a resume does not reveal: who he or she is.

Recruiters have access to all video applications and form their own judgment. They only meet with candidates who really stand a chance and have access to meaningful data to prepare for the interview.

Focus on culture-fit to avoid 46% failure rate

46% of new hires fail within 18 months. According to a Leadership IQ study, 89% of these failures result from a bad person-culture fit. 82% of managers reported that in hindsight, interviews with the candidates elicited subtle clues that these talents would be headed for trouble. Yet, typical interviews fixate on experience rather than personality, motivation, and culture fit.

To this day, most hiring decisions are ultimately based on unstructured interviews, although this method of interviewing leads to biased decisions (e.g. Barrick et al. 2009). Researchers point to structured interviews as a better way of recruiting. For instance, Schmidt and Zimmerman (2004) find that THREE to FOUR independent unstructured interviews are required to provide the same level of validity for predicting job performance as ONE structured (and ideally uninterrupted and standardized) interview.

With questions focused on specific attributes and skills, video-based structured interviews are a more effective way of testing a candidate’s culture fit and potential performance on the job.

Meet Retorio's team:

Rooted in AI, psychology, and organizational research

Retorio started off as a research project at the Technical University of Munich.

After years of PhD research in behavioral and differential psychology, we decided to make our insights available, equitable, and scalable to everyone. We employ psychologists, AI specialists, and business experts from more than 10 countries. As a research-first product, we believe in delivering solid, empirically-validated outcomes for our customers and users.

TOUCH

Dr. Florian Abicht Chief Data Officer

Florian studied physics at University of Munich (LMU) and obtained his PhD from Max-Born-Institute at TU Berlin. Before joining Retorio as our Chief Data Officer he worked as a Lead Data Scientist for Compass inc. in San Francisco and as a Group Head of Applied Intelligence for Plan.Net Business Intelligence.

Dr. Florian Abicht Chief Data Officer

TOUCH

Dr. Christoph Hohenberger Founder and Chief Psychologist

Christoph is one of Retorio's co-founders and our Chief Psychologist. After his studies in Business Psychology he obtained his PhD from TU Munich where he explored the intersection between AI and psychology. In between, he became a research scholar at MIT and University of Tokyo.

Dr. Christoph Hohenberger Founder and Chief Psychologist

TOUCH

Dr. Patrick Oehler Founder and Head of AI Communication

Patrick is one of our co-founders and in charge of product communication. He holds a M.Sc. in Strategic Management from LSE and a PhD in Organizational Research from TU Munich. During his PhD, Patrick explored the intersection between behavioral patterns in organizations and performance. He was a visiting research scholar at AY lab (Uni Tokyo).

Dr. Patrick Oehler Founder and Head of AI Communication

The science behind Retorio's behavioral analytics A.I.