Each month, the Learning Circuits Blog asks “The Big Question.” This month, the question is “What would you like to do better as a learning professional?”

My answer is, I would like to get better at responding to clients who set direction or request designs that are not in the best interest of their learners. Many clients think primarily in terms of content presentation and not in terms of generating meaningful learning activities. When clients request e-learning that is barely more than a narrated PowerPoint presentation, or have a vision for an instructor led course in which the instructor drones on for 7 or 8 hours while working her way through 500 PowerPoint slides, giving the client what he wants is not in the best interest of his learners. Yet, it can be very difficult to educate clients quickly enough to give them the necessary insight as to why what they’re asking for is suboptimal. When clients feel (rightly or wrongly) that they are “under orders” from higher management (usually not present at the meeting), the task can be very difficult. You have to somehow simultaneously educate the client and convince him that his boss will be ok with a more ambitious (and effective) design. Depending on the relationship between the client and his boss, and the culture of their organization, this may or may not be possible. When it’s not, it puts the instructional designer in a tough position. I would like to get better at avoiding that position.

In fact, I’d love to hear how you solve this problem. If you have an approach that’s working for you, please use the comment link, below, to share it.

Let’s start a conversation!

Advertisement

When I’m not teaching class, I work as a freelance instructional designer (ID). This year, I’ve been working most often as a contract ID for a San Francisco instructional design firm. Two weeks ago, I gave a presentation to the senior executives of this firm. I had just wrapped work on a classroom-based 1.5-day training course. The client was a large biotech company. Since the project successfully deployed a range of innovative activities, including an interesting serious games design, I was asked to present to the ID firm that managed the project.

I opened my talk with a few slides on Michael Allen’s criticisms of traditional “tell and test” learning. These criticisms are primarily that content-driven courses tend to be boring, and that the meaningful tasks aren’t usually covered until the course is almost over. Then I showed how Allen suggests fixing those problems by starting the course with a challenge that asks the learner to perform the final or near-final task that the course is teaching, while providing the “tell” portions of the course as supporting materials to help the learner master this challenge. By converting the “tell” portions of the course to subordinate material, Allen’s approach gives the learner a lot more motivation to process the course content than is typical in a more traditional content-centric course. I explained that it was with these ideas in mind that I approached the challenge of creating a highly activity-centric course for the biotech client.

At the end of my talk I listed a few resources my audience could check out if they wanted to learn more about this activity-centric approach. One of the links was to a then-upcoming webinar by Michael Allen. At that point, the Senior Director of Project Management turned to me and said, jokingly, “You should get a commission!”

I tell you this so you will know my bias—I’m a huge Michael Allen fan. The three books on e-learning that he’s written so far have been, by far, the most practical and interesting such books I have read. His latest, Designing Successful E-Learning, and his first, Michael Allen’s Guide to E-Learning, are particularly great. If, like me, you’ve spent the last several years designing corporate e-learning, these books will likely change your view about what good e-learning should strive to do and how it should go about accomplishing its goals. Allen’s principles, like “Put the learner at risk” or “Don’t start at the beginning” or “Test before telling” run counter to much ingrained practice in the industry. But radical though they are, the principles he lays out are, I think, exactly what is needed to shake e-learning out of its doledrums and take it to the next level where it can be of genuine value to the learner. Read these books and you’ll never want to design e-learning the “old” way ever again. And even though his focus is on e-learning, I find the core ideas in his books translate well to the design of highly interactive classroom courses too.

Last week, I attended Allen’s webinar. Now it’s online as an archive (see: Context, Challenge, Activity, Feedback). Check it out. It’s excellent, and the demo he shows is compelling. It doesn’t depend on fancy 3D animation or first-person shooter-style immersive graphics. It’s mainly constructed out of static photos and some clever recontextualization of simple multiple-choice questions. The power is in the instructional approach, not the delivery technology per se.

If you enjoy this webinar, you might like to view the archive of Allen’s earlier webinar on Stages of Change.

Ruth Clark and Richard Mayer, e-Learning and the Science of Instruction
Pfieffer, San Francisco, CA, 2003, 316 pp.

Many medical institutions and practitioners now practice a form of medical care called “evidence-based medicine.” In evidence-based medicine, the folk wisdom of doctors is not considered adequate. Instead, the question of which treatments are most effective for a given pathology must be answered by reference to the best available evidence, as published in reputable scientific journals.

Although they don’t use the term “evidence-based,” Clark and Mayer are nevertheless promoting a kind of “evidence-based instructional design.” As with evidence-based medicine, there are both pros and cons to this approach. On balance, though, e-Learning and the Science of Instruction presents with clarity a good sampling of research relevant to instructional designers. Some chapters also have useful things to say to other members of typical e-learning production teams, such as creative designers in charge of look and feel decisions.

Overall, the book is a wonderful summary of relevant research on learning, complete with recommendations for how to apply the research results to practical decisions that instructional designers must make, e.g., whether to use a voice-over narrator to read verbatim on-screen text or not; whether to describe pictures or animations with on-screen text alone, with audio narration alone, or with a combination of on-screen text and audio narration; whether to address the learner directly using a 2nd person narrative point of view, or to stick to a more formal 3rd person point of view, and so on.

The difficulty, not addressed in the book, is in deciding how generalizable the research results are. (This is also a problem in evidence-based medicine.) Using research results to defend designs that are insufficiently similar to those in the actual study runs the risk of turning the science of instruction into an authoritarian quagmire. Cornell University mathematician Steven Strogatz eloquently describes this problem in The End of Insight. Strogatz worries that, in the future, science may give us answers to certain problems without giving us an understanding of why those answers are true. For problems in human learning, because our understanding of the brain is so limited, we don’t have to wait for the future—we already have studies that tell us that certain things work better than others, without deep, insightful explanations as to why. As Strogatz writes, “When the End of Insight comes, the nature of explanation in science will change forever. We’ll be stuck in an age of authoritarianism, except it’ll no longer be coming from politics or religious dogma, but from science itself.”

To avoid this problem, we need to constantly re-evaluate what we are doing. We must use the evidence from learning research as a starting point, but not follow it slavishly if it seems to be leading us to design learning that isn’t maximally effective.

That said, I think e-Learning and the Science of Instruction is an extremely valuable resource, well worth the time of anyone tasked with designing online instruction.

Karpicke, J.D., and Roediger, H.L., III (2008). The Critical Importance of Retrieval for Learning. Science, 319, 966-968

This year I’ve been teaching a couple of Instructional Technologies classes at my local university, and one thing I’ve noticed is that beginning instructional designers tend to think that the content presentation portions of their designs are where all the learning takes place, while the tests are neutral events that merely assess what was learned.

In this fascinating report, published in the February 15, 2008 issue of the journal Science, Karpicke and Roediger show that testing is not a neutral event. In fact, for improving long-term retrieval, testing can be even more important than studying.

They divided learners into 4 groups. Each group was given a list of 40 Swahili vocabulary words, along with their English equivalents. Then, in a series of alternating sessions, learners in each group were asked to first study the vocabulary list, and then were tested on it.

The first group, designated ST, studied all 40 words during each study session and was tested on all 40 during each test session.

In the second group, designated SnT, learners dropped words from their subsequent study sessions after they were correctly recalled during a test session. However, they continued to be tested on all 40 words at every test session.

The third group, designated STn, studied all 40 words at every study session, but were tested only on those words not yet recalled in a previous test session.

In the final group, designated SnTn, words were dropped from both study sessions and subsequent test sessions as soon as they were recalled correctly in a test session.

After 4 {study session, test session} pairs, the ST group had experienced 160 study events and 160 test events. The SnT group experienced, on average, 76.8 study events and 160 test events. The STn group experienced 160 study events and, on average, 83 test events. And the SnTn group experienced, on average, 77.4 study events and 77.4 test events.

One week later, learners in each of the 4 groups were tested on the complete list of 40 words. Which group do you think performed best on this long-term recall test?

As the authors point out, one possibility is that, after a word has been correctly recalled, a learner’s study time might be better spent focusing on learning the remaining words in the list. Alternatively, perhaps the learner will forget the word if it’s dropped from the study session. If testing is a neutral event, then dropping words from test sessions should have no important impact.

What they found was that the two groups that were consistently tested on all 40 words showed a greater than 150% improvement in long-term recall, compared to the two groups who dropped terms from test sessions.

At the same time, there was practically no difference in long-term recall between the two high-scoring groups (ST and SnT), even though the ST group underwent about 80 more study events than the SnT group. So all that extra studying led to no real gain and was essentially wasted effort.

Similarly, there was practically no difference in long-term recall between the two low-scoring groups (STn and SnTn), even though the STn group underwent about 80 additional study events compared to the SnTn group. Again, all that extra studying led to no practical gain.

The authors conclude: “Repeated retrieval practice enhanced long-term retention, whereas repeated studying produced essentially no benefit.” [p. 967]

So testing is not a neutral event.

Welcome to Tales from the ID. My name is Ray Cole, and I am an instructional designer, working primarily for clients in the Computing, Manufacturing, and Biotech/Pharmaceutical industries, though I have done some work for companies in other industries on occasion.

I hope to use this blog to post items of interest to other instructional designers. Stay tuned—good stuff is coming.