Educational fallacies: post hoc, propter hoc

This weekend, I read a recent blog post at the Chronicle of Higher Education by Steven Conn on the "helicopter teacher." Conn's post is primarily a rant about what many might call "soft" teaching practices. It is filled with hyperbole and fallacies, and lacking both in nuance and in knowledge of the scholarly literature on teaching. I've decided not to post a direct response to such an argument. However, Conn's post, and many of the comments and discussions that have come in response to his post, expose a number of rampant fallacies in the way that we think about teaching, especially in higher education. So I've decided to write a series of blog posts on educational fallacies—things like "I covered it, I assessed it, therefore my students learned it," "a hard class is a rigorous class is a good class," or "what's good for the average is good for all." This is my first post in this series, addressing the fallacy:

"I did something; students learned something. Therefore, what I did caused the learning."

This fallacy, as I've heard it, usually comes from "traditional" faculty resistant to new methods, even when demonstrated in formal, peer-reviewed research. It goes something like "I've been teaching this way for x years, and my students turn out just fine." (Note the hint of survivor's bias.)

Now, informal experimentation happens all the time in teaching. It has to. And to some extent, academic freedom is about such curriculural experimentation. But that isn't what I'm talk about here.

In fact, I'm all for making changes semester-to-semester, year-to-year, and gauging the results. We can discover many wonderful new methods without an IRB-approved study by simply trying something new, and comparing the results to what we did the year before. More experimental rigor may be required before making large, generalizable claims. But most thoughtful teachers have improved their craft and their service to their students quite legitimately through such experimentation in their teaching.

However, the resistance to try something new simply because what we've always been doing produces good results is fallacious. Specifically, this is post hoc, propter hoc logic—the idea that when one thing is followed by another thing, that first occurrence caused the second. Simply because I teach in a certain way and my students demonstrate significant knowledge at the end of the course does not mean that my teaching led to their learning. We can all think of times as students when we learned the content of a course in a deep way in spite of the poor instruction we received. (We simply cared too much not to work hard enough to teach ourselves and/or learn from our peers.) We need the humility to recognize this possibility as teachers.

As in most examples of post hoc, propter hoc, there is more than one potential contributing factor to the result of student learning: the instructor's efforts, of course, but also the students' efforts, the textbook (and/or supporting media), the students' motivation, the level of interest and intelligence of fellow students, the location and time of the class, and—of course—the knowledge and abilities of students when they enter the class. Without accounting for these other factors, we cannot claim a causal relationship without major caveats and equivocations.

So when we find ourselves saying something like "I've been doing it this way for a long time, and it always gets good results," or "this is how my professor did it, and it worked great for me," let's check ourselves. Do we know the real cause of our students' success? Have we accounted for other factors?

Or are we arrogantly assuming that our students' learning is primarily due to our instructional prowess?