Month: June 2023

Using the ADDIE Model to Build Better Compliance Training, Part 6: E is for Evaluation

NXLevel Compliance Team , , ,

In the final installment in this series, we look at the E in ADDIE and discuss a less obvious, but effective, way to evaluate the effectiveness of your compliance training.

The last “E” in ADDIE, evaluation, is often the most overlooked. With so many demands on your time, it’s easy to feel that evaluation is a form of looking back, a luxury that you don’t have the time for. The truth is evaluation is all about the future and focusing on how to make your next training program more effective.

Most compliance training includes an assessment at the end that is meant to measure how well the learner achieved the objectives of the training. But as we know, measuring learning objectives immediately after training is completed is not a good indicator of how well the learner will transfer that training to their job. The forgetting curve slope can be fast and steep.

A better way to evaluate the effectiveness of a training program is to assess your learners a few weeks or even a few months after they have completed the training. But most learners view assessments with the same level of joy as a visit to the dentist, and not all assessments provide useful information. So how to assess learning without using an assessment? Play a game!

Recently, NXLevel worked with a client to evaluate the effectiveness of their annual compliance training by using the JEOPARDY! game from our suite of compliance training games.

JEOPARDY! is a proven and popular learning format employees are genuinely excited to play, and our version provides detailed reporting that allows you to drill down to see how employees responded to individual questions. It’s also the only licensed JEOPARDY! game on the market, so it features the same graphics, music, and gameplay as the television version.

Working with the client, we developed a series of role-specific JEOPARDY! games with questions that focused on critical topics pulled from their current compliance training. We configured each game so learners could play it on their own in about 10-15 minutes.

The client identified a representative subset of employees for each role and emailed each of them a link to their game.

No one was required to play, but employees could earn points towards their company’s rewards program for completing the game and more points for being high scorers. Through their company’s rewards program, employees redeem points for merchandise, gift cards, and other items.

The games were a big success! Employees appreciated that they  were short, fun, and engaging. More importantly, the client obtained valuable data on how employees responded to each question, and they are now using that data to help determine how to revise their compliance training. By looking at questions where employees scored well, where employees seemed to struggle, and which incorrect answers were chosen, the client has a clearer picture of their compliance learning needs. Such practices also align well with the DOJ’s increased emphasis on a data-driven approach to compliance.

And with that, we’ve returned to the first step in ADDIE, analysis. We’re ending this blog series on using the ADDIE model where we began, because learning never ends and each step in the ADDIE model informs and interacts with the others.

Thanks for reading. We hope you’ve found the entire series informative and helpful. As always, any and all feedback is welcome!

Using the ADDIE Model to Build Better Compliance Training, Part 5: The “I” Is for Implementation

NXLevel Compliance Team , , , ,

For this installment of our series on using ADDIE to build better compliance training, I interviewed one of NXLevel’s Senior Project Managers, Pam Dorini, to hear her tips and suggestions for successfully implementing a compliance training program. According to Pam, a successful launch is built around one idea: begin with the end in mind.

Here are the highlights of our conversation.

As a project manager for custom and off-the-shelf compliance training projects, when do you start thinking about implementation?

Absolutely from the start of the project. The instructional designers, writers, and clients need to think about what we want the learners to know and be able to do as a result of the training. Then, we need to proactively think about the factors that are going to contribute to the learners doing that.

There are so many components to a successful launch, and so many stakeholders that need to be involved, implementation has to be a critical part of the plan from the start of the project. And how well that implementation goes is ultimately driven by the earlier steps in ADDIE – analysis, design, and development.

What do you see clients overlooking or underestimating in terms of implementation?

We are all pressed for time, so I think where clients go amiss is that they want to get going, get the training done, meet the immediate needs, and move onto the next project. If you don’t take the time to think through all the issues at the start, you’ll run into problems at implementation because that is where it all comes together. The time it takes to implement the training is usually quite short compared to the other stages, but implementation is where all those other stages come together, and it has to be planned as carefully.

Often, the analysis, design, and development stages are driven by the implementation date and that can lead to issues, especially when that development window is compressed, and clients are rushed to meet the deadline.

What is important for clients to consider regarding their LMS?

They need to think about the obvious and the not-so-obvious technical specifications. I will ask the client lead if he or she has released something on their LMS that is similar to what we are developing, and in most cases, we prefer to deliver a protype module for testing.

Time allocation needs to be top of mind. For example, does the LMS team have enough time to complete testing and remediation?

When we are working with a new LMS or authoring tool, testing conducted in tandem with module development is a good practice that has helped us avoid pitfalls or delays late in the process (i.e., implementation). As a company, we have integrated training into just about every LMS imaginable and there are idiosyncrasies from one to another. In some cases, we even uncover things we couldn’t have anticipated otherwise, like a font that doesn’t display correctly in a client’s system.

Are some timeframes better than others for launching compliance training?

They can be. Keeping the needs of learners at the forefront of decision making is critical. Timeframes need to be considered in correlation to the amount of time learners are given to complete the training and what other training is expected of them during that same period. Many of our clients tend to avoid launching big training initiatives over July and August because that’s when employees are often on vacation. Another challenging time can be late Autumn into the end of year as holidays and company shutdowns approach. We seem to get a lot of requests for launches in the Spring or in September and early October.

Can you share an example of a client who you think does a really good job with the implementation of training and talk about why?

I am thinking of one in particular who takes the time to understand all the various organizational goals. We always spend a lot of time planning with her and her team. She understands the capabilities and limitations of her LMS, and she engages with the business to get input on what they feel their learners need. She also understands the need for consistent communication with internal stakeholders and with us. By the time we get to the implementation point, everyone knows what to expect.

What type of changes have you seen in recent years related to implementation?

While the LMS is still the primary modality for compliance training, our clients are delving into other formats to better provide ongoing learning, reinforcement, and performance support. In addition to modules, we are developing infographics, quick reference guides, digital assets, games, and videos, which are made available in various ways. As an example, I’ve seen several clients build a library of compliance micro videos and set them up on their company SharePoint or intranet platform, then announce and link to them via email.

Conclusion

As Pam emphasized throughout our discussion, the implementation stage of compliance training drives all stages of a project. Always considering who will be taking the training, how they will be taking it, and when they will be taking it is key to making your training more effective. To roll out training that reduces risk across your company, begin with the end in mind, and keep it in mind from project start to finish.

Thanks for reading!

Sean Murphy
NXLevel Compliance