The Project

The purpose of this project is to creatively solve a particular “people” problem that is frequently faced by organizations.

It is widely known that soft skills (e.g., teamwork, conflict management) are vital for individual success, team effectiveness and organizational performance. For individual contributors, soft skills are key to productive interactions with their colleagues and team members; for managers and team leaders, soft skills are absolutely essential for establishing smooth functioning and effective teams.

The problem is, soft skills are notoriously difficult to evaluate and measure well. The right tools are simply not yet available for organizations to make the best decisions around selection, promotion and training of employees. And, as in any endeavour, taking important actions without the right decision-making tools can lead to problems, such as poor in-role performance, interpersonal conflicts, low team morale and engagement, failed projects – you know the list!

To address this problem, we have assembled an interdisciplinary team of experts to design and build an accurate and bias-free assessment tool by leveraging our combined knowledge of latest technologies and innovative thinking.

Deficiency of Current Methods

The currently available methods for evaluating soft/interpersonal skills have significant deficiencies.

Self-assessments: Having employees rate their own soft skills is the most common method. However it generally produces the least accurate results, as people tend to either consistently overestimate or underestimate their own skills.

Direct observations: This method includes peer-assessments, supervisory ratings and 360-degree evaluations. It is a step up in terms of accuracy, although subject to bias. It is generally not usable for screening and selection of new hires and can be time-consuming and difficult to administer when used to evaluate current employees.

Situational Judgment Tests (SJTs): This approach involves presenting individuals with scenarios depicting a workplace situation and asking them to pick the best course of action from a list of alternatives. This is one of the most valid and administratively versatile methods (e.g., Christian et al., 2010). On the down side, SJTs tend to reflect the respondent’s procedural knowledge about effective and ineffective behaviours, rather than a demonstration of a skill itself (Lievens & Motowidlo, 2016). Moreover, research suggests that SJTs are not well suited for measuring interpersonal skills of individuals with higher propensity to experience anger, frustration and bitterness, because such people tend to overreact to frustrations in real-life and behave inconsistently across situations (Slaughter et al., 2014).

Our Solution

Our design is based on the principle that it is best to evaluate how people actually behave in various situations, rather than accept what they say they would do in those situations.

We have designed a simulation that recreates common team dynamics through the use of chatbot AI and developed an algorithm that automatically evaluates people’s natural language responses across various situations.

We are currently in the process of completing our first simulation. It is designed to be very intuitive and simple to use. Yet, like most seemingly simple things, it reflects a complex design and sophisticated technology. Later this year, we will be running several studies to empirically investigate its effectiveness.

If you are a researcher and would like to participate in these studies, feel free to email igor.kotlyar@uoit.ca.