Editor’s Note - Building Better Training - part three of three: this 3-part series written by Mark Murrell and Jane Jazrawy, co-founders of CarriersEdge, looks at ways to make training more meaningful, engaging and effective through setting objectives, organizing and presenting content, and using a variety of testing methods to check knowledge. This third installment, written by Jane Jazrawy, talks about creating objective test questions to measure the effectiveness of your training program.
So here's a secret. I can pass a lot of tests on subjects I know nothing about.
In fact, I know a lot of people who can do it. I'm reasonably sure many commercial drivers are passing written tests without much understanding of the material they're supposed to be learning.
How do I do it? I don't “Google” the answers and don't cheat in any way. And I can't do it all the time. But when people write poor test questions, I can often figure out the answer without knowing anything about the content. If the test is full of true/false questions, I know I have a 50 percent chance of getting the answers right, basically without even reading the question.
To be fair, creating test questions is awful. It's like spinach to a 10-year-old. You know you have to eat it for your own good, but you're going to have to grit your teeth and just get through it. For me, a final test is a chore at the end of a lot of creative and satisfying work. It's a difficult task and a thankless one. In 25 years, no one has ever come up to me and told me how much they enjoyed my final test.
So how do you write an effective test? One that I won't be able to pass just by reading your questions and deducing the answer? Let's take a look at how you should structure tests and the kinds of questions you can ask beyond True/False.
Review your learning objectives.
Remember those learning objectives from part one? They're going to be very useful now as you try to think of test questions. Let’s look at our fatigue management learning objectives.
After completing this training session, participants will be able to:
- Define fatigue as it relates to commercial driving
- List five common signs of fatigue
- Describe three ways that fatigue can impair judgement
- Correctly identify the employer and employee responsibilities in managing fatigue
- Explain four ways to manage fatigue
Your objectives will give you a guideline to the questions to ask in a quiz or final test. For example, asking a question about the definition of fatigue is a good place to start.
In addition to your learning objectives, review your content to get some ideas. If you created your content to meet your learning objectives, that content should be useful for measuring those objectives in a final test. Use definitions, symbols (e.g. hazmat or HCS/WHMIS symbols), process steps, comparisons (e.g. driver vs. carrier responsibilities), and scenarios. These are all good places to draw from for test questions.
Creating your test questions.
True/False questions are the easiest to write, but the problem is that they are the also the easiest to guess. If you cannot think of a question for a particular objective other than True/False, it usually means that your content is so straightforward that everyone knows it already, you didn't go into enough detail, or you need to get some sleep and start again in the morning. Sometimes you can't get around using them, but recognize that they are probably just going to be an easy mark.
What else can you try? Here are some examples of question types that you can use:
The bedrock of question types. The nice thing about multiple choice is that you can incorporate images, scenarios, fill-in-the-blanks and other devices into these. Just don't give the answer away by making it much longer or more detailed than the others.
Example: Which of the following can you do to reduce your fatigue when you drive?
a) Plan to take a break every few hours and get more exercise
b) Exercise right before you go to bed to wear yourself out
c) Wake up earlier
d) Drive through the night when there is less traffic congestion
These are created by taking out a critical part of a statement. Again, make sure the answers are similar so that it is more difficult to make a guess. You can make it a multiple choice format, or ask the learner to fill in the correct answer.
Example: Irritability and an increased risk of falls and accidents are considered _______________ effects of sleep debt.
When you are teaching about a process, asking learners to identify steps in the process or identifying a missing step works well. For this example, let's look at a question that relates to drug and alcohol testing because there are very defined processes in that topic.
Example: The three steps of the drug testing process include:
Give the learner a situation where they have to use the knowledge they have learned to make a decision. Below is an example relating to hazmat.
Example: You have a hazardous materials delivery through an unfamiliar part of the country. Who has the ultimate responsibility of making sure that you have the correct permits and are following any special routes?
b) Your carrier
c) Your dispatcher
These can be used when creating matching questions (match the definition to the symbol). They are great in multiple choice questions regarding identification.
Example: When you see this pictogram on a label, you know that it is:
a) An explosive material
b) A corrosive material
c) Gas under pressure?
How long should your test be?
The number of test questions on a test will vary based on the length of your content and the detail that your learners need to know. For a 30-minute session, I generally have about five or six learning objectives and about 10-15 questions.
Test length also depends on how much subject mastery you want people to have. For courses that focus on regulations, tests should be a little longer and more rigorous so that you can make sure that there is a good understanding.
I won't kid you, a lot of thought and time goes into creating meaningful learning objectives and test questions. And there will be some clunkers in the mix, so don't get despondent when people critique your questions. Like I said at the beginning, no one is rushing up to tell you what a fabulous test they just took!
Make sure that you listen to any feedback about test questions – when learners don't understand what a question means or think that it's vague or tricky, that's the time to look at the question again and double-check the objective you are trying to achieve.