Assessments through extended reality

About this trend

Extended Reality (XR) is simulation-based learning and includes both augmented reality (AR) and virtual reality (VR). It replicates aspects of the real world, requiring learners to react to situations and prompts in an interactive environment, make decisions and take action. Assessments through XR log those reactions, decisions and actions, providing rich data about how a learner performs throughout the simulation. These assessments allow for in-depth and meaningful analysis and reflection of learner performance. Did the learner learn what they needed to learn? Did they respond appropriately to x, y and z? Where did they stumble? What areas need further attention and exploring?

With these assessments, you find out if someone can perform a particular skill or demonstrate a required behaviour and to what standard at a very granular level.

These are the kind of learning analytics learning teams need to gather. Assessment is key to learning, but it’s an area that L&D struggles to get right – it has a history of asking questions such as ‘did they like the learning?’ or ‘did they complete the learning?’ This line of questioning does not yield useful, actionable results and is simply a tick box exercise.

Assessments through XR enable learning teams, line managers and learners to have more authentic assessments. They get feedback on the learning experience and outcome. This is particularly useful and important when the simulations are in situations that are difficult to replicate and perform in the real world – hazardous, high-risk environments such as aviation, healthcare, construction, etc. When used in these scenarios, organisations can test procedural knowledge in complex environments, without real-world risks or logistical constraints. This is very compelling for any organisations operating in these high-risk environments.

Extended realities are also increasingly being used to hone people’s interpersonal skills. A learner could be placed in a simulated world where they face a challenging situation with a customer, testing out their conflict resolution skills. Assessing them through that conversation, the steps they take and the behaviours they exhibit, as well as the outcome, is very useful for organisations and the individual involved. The assessment is highly personalised to the situation and the learner.

The expert view

Myles Runham, Senior Analyst for Digital Learning at Fosway Group

Assessment isn’t usually the primary driver for organisations to use extended reality for workplace learning – it’s usually more driven by learner experience considerations. Assessment also has connotations of formalised marking schemes, of pass and fail metrics. A lot of L&D assessments do lead down that path, but I think the real benefit of XR assessment in a corporate context is the ability to evaluate and monitor performance and understanding.

With simulated activity, assessment is part of the experience – the simulation is, in effect, the test. You can evaluate performance, understanding and behaviour as people work through an extended reality experience without necessarily having to stop and test them. Their behaviour and activity during the learning is the signal of whether they understand and are proficient, safe, skilled…

There is enormous value in this, from the rich behavioural and activity learner data you can gather. Did a person pick up a piece of equipment in the right order or not? How did they react when x happened? All of that can be gathered and recorded so that L&D and managers can know with confidence that the learning has happened, that the learner has the required behaviour understanding. That’s not the sort of performance data that you get from typical e-learning assessments, when you might know from the score that someone hasn’t got the learning, but you can’t see why or where it hasn’t happened.

I have spoken to a number of vendors and corporates who are using XR to check that workers can do what they need to do before they apply it in the workplace – companies operating in high consequence environments, such as oil, gas, pharmaceutical, manufacturing, maritime and construction. For example, a pharma company builds a virtual, high fidelity lab environment because it’s the only way of making the learning available without taking over or building a production facility – doing that would be economically and logistically impractical.

Companies can use the technology to ensure employees are work ready. The assessment tells them that they aren’t going to make any mistakes, that they’re not going to breach health and safety guidelines.

For 32% of respondents to our latest Digital Learning: Realities research, immersive technologies is a high priority. It was 21% the year before, so that’s a significant growth.

It is an authentic and compelling way of assessing people’s understanding, skills, knowledge and behaviour.

Professor Agnes Kukulska-Hulme
Institute of Educational Technology, The Open University

Resources:

• Virtual Reality & Augmented Reality Agency: The 22 best examples of how companies use virtual reality for training: https://www.vrowl.io/the-22-best-examples-of-how-companies-use-virtual-reality-for-training/
• An investigation into the use of virtual reality on assessment: A pilot study carried out at The Sheffield College through the Assessment Innovation Fund in collaboration with NCFE: https://www.ncfe.org.uk/media/nq3afg0n/final-report-the-sheffield-college.pdf
• What does virtual reality and the metaverse mean for training? PwC: https://www.pwc.com/us/en/tech-effect/emerging-tech/virtual-reality-study.html