There are about four more weeks left in my stats class. I have to say that for all the hype, this class really wasn't that frightening. I was originally a bit worried about surviving the course in good stead, but all has gone wonderfully well. The main thing that has stood in my way was the one item I had anticipated: learning to use the statistical software.
The weekly assignments weren't particularly difficult---we were provided with enough "point and click" guidance to make things work. The only problem with that, of course, is that it doesn't foster a lot of learning. We have only one paper for this class and it is definitely a "rubber meets the road" sort of deal. We choose a particular research problem, create a survey, generate some fake data...and then analyze it. It's the analysis portion that has been eating my lunch the last few days. Because I created my own data set, I already knew how the analysis should skew, but I had to prove it. After several permutations of analysis (and a lot of swear words), I finally have the kind of output I need. I've managed to slay the dragon.
All of this has raised a more generic question for me about how people learn about the "reasonableness" of answers. When I taught chemistry, I was quite often amazed at some of the answers students calculated. How could they not have noticed that there was some sort of problem? Didn't it look off by a few orders of magnitude? With the statistical software I've been using, I have seen how easy it can be to just accept whatever output is spit out---it generates whatever you tell it to...so that must be correct, right? My guess is that most of my current classmates may be of that mindset---there is already talk of hiring "personal statisticians" (at $80/hr) to complete that part of their doctoral studies.
At first, my hunch was that people learn to evaluate the reasonableness of an answer through conceptual understanding...a deeper level of knowing the math. Now, I'm not so sure. I can't claim to have a very deep knowledge of the statistical concepts I'm wading through; but, something is still giving me a clue that all is or is not right with the world when I look at my data. Maybe learning to decide if you're at your final answer is just a reflective process---something that allows you to evaluate things in light of the original question and probe things that way. Has anyone seen any information on how we get students...how we learn...to look for and listen to that sort of cognitive dissonance? It would be great to find out how to foster and develop that.