SKILL SIMULATOR

Exploring better ways to assess & develop interpersonal skills

Surveys and self-reports capture what people say about themselves. Our research develops AI simulations that assess interpersonal skills from observed behaviour.

Our foundational work centered on the Teamwork Skill Simulator, a rule-based simulation we developed and validated through multiple studies. This tool laid the groundwork for our current LLM-powered assessments.

Some key findings

  • Simulation-based algorithmic scoring can assess teamwork skills with accuracy comparable to human experts
  • Students respond positively when scenarios and virtual teammates feel authentic
  • Well-designed AI feedback can narrow the gap with human feedback in how students perceive and respond to it
  • Cross-cultural differences affect how students engage with simulation assessments

Our ongoing work explores how advanced AI, including LLMs, can make skill assessment more effective and equitable across diverse global contexts.

Some of our publications

  • Kotlyar, I., & Krasman, J. (2025). Student reactions to AI versus human feedback in teamwork skills assessment. International Journal of Educational Technology in Higher Education 22 (57). https://doi.org/10.1186/s41239-025-00555-9
  • Kotlyar, I., Pearse, N. J., & Krasman, J. (2024). Understanding cross-country differences in assessment simulations: insights from South African and Canadian students. Discover Education, 3(1), 1-23. https://doi.org/10.1007/s44217-024-00125-7
  • Kotlyar, I., Sharifi, T. & Fiksenbaum, L. (2023). Assessing Teamwork Skills: Can a Computer Algorithm Match Human Experts?. International Journal of Artificial Intelligence in Education, 33(4), 955-991. https://doi.org/10.1007/s40593-022-00318-x
  • Kotlyar, I., & Krasman, J. (2022). Virtual simulation: New method for assessing teamwork skills. International Journal of Selection and Assessment, 30(3), 344-360. DOI: https://doi.org/10.1111/ijsa.12368
  • Kotlyar, I., Krasman, J., & Fiksenbaum, L. (2021). Virtual high-fidelity simulation assessment of teamwork skills: How do students react?, Journal of Research on Technology in Education, 53:3, 333-352, DOI: 10.1080/15391523.2020.1783401

Our research has been supported by

  • Social Sciences and Humanities Research Council of Canada (SSHRC) Insight Development Grant, “Exploring Methods for Evaluating Interpersonal (Teamwork) Skills”
  • Teaching Innovation Fund (TIF), Ontario Tech University
  • Career Ready Fund, Ontario Tech University