Teacher Feedback VS Machine Feedback

Do students prefer to learn from a machine?

A bit of a professional disclaimer to this week’s geek study. I do not believe that technology is ever an effective substitute for good teaching (and this is a belief rooted in quite a few empirical studies as well). Technology is a tool. My interest is how tools are leveraged by teachers to help and increase learning by students. I found the results of this study to be surprising. I’d also hesitate to draw huge conclusions from a study at a large education college and apply those results to K through 12 education.

That said, this is interesting.

Why Students Prefer to Learn from a Machine

While this article’s headline is hyperbolic, it does share a fascinating study from The International Journal of English Studies. Education students were placed in two groups. In one group, students received feedback on writing assignments from live instructors. A comparison group received feedback from a software program called Criterion. The study’s intent: to identify differences in how students responded to different forms of feedback. In short, Would students receive feedback better from one or the other and how would they act on that feedback?

The results?

The computer program appeared to transform the students’ approach to the process of receiving and acting on feedback.Comments and criticism from a human instructor actually had a negative effect on students’ attitudes about revision and on their willingness to write, the researchers note. By contrast, interactions with the computer produced overwhelmingly positive feelings, as well as an actual change in behavior—from “virtually never” revising, to revising and resubmitting at a rate of 100 percent. As a result of engaging in this process, the students’ writing improved; they repeated words less often, used shorter, simpler sentences, and corrected their grammar and spelling. These changes weren’t simply mechanical. Follow-up interviews with the study’s participants suggested that the computer feedback actually stimulated reflectiveness in the students—which, notably, feedback from instructors had not done.

Why would this be? First, the feedback from a computer program like Criterion is immediate and highly individualized—something not usually possible in big classes like those at Alexandria University, the site of the study by El Ebyary and Windeatt. Second, the researchers observed that for many students in the study, the process of improving their writing appeared to take on a game like quality, boosting their motivation to get better. Third, and most interesting, the students’ reactions to feedback seemed to be influenced by the impersonal, automated nature of the software.”Annie Murphy Paul

There’s a lot to crunch through here. Part of me wonders if generational components affected the differences in responses? What social elements allow for students to respond better to a machine rather than an individual?

And does it really matter, as long as their writing improved?


Online Education and Seeing Her Flaws

When I first fell in love with online education my vision, which, I suppose, is normal at the start of any relationship, became exceptionally myopic. I’d develop some crazy fun and brilliantly educational lesson in Moodle, my students (most of them) would stay engaged, become rock stars, and ace every short cycle or state test placed in front of them. Never mind that when a sub brought them to the computer lab OR when I assigned online work for home the quality and quantity of work would generally be crap…or, more accurately, satisfy the “lowest common denominator” of a passing grade. We were in love. And when you’re first in love you ignore the blemishes and annoying habits.

Technoeducational utopianistas are still in love. These are people I enjoy reading and/or listening to like Clay Shirky, Salman Khan, and those founders of Udacity. And, while I sometimes find myself in their camp (it’s fun to envision educational revolution brought on by Moore’s Law), it doesn’t jive with what I’m seeing in the classroom or in the data. The more I read into their messages (and you can likely lump Arne Duncan, most left/right politicians, our president in this group) that technology will be the saving grace to poor performing students, the more I’m sensing a cop out to the real reasons why students aren’t performing well on those state tests: Poverty.

Freddie deBour has a series of posts on the “need for educational realism”. They’re great posts. Go read them (here and here for starters) for the full arguments and observations – but some gem quotes are as follows:

I’ve tried all number of ways to do that outside of class meetings– marking papers extensively, using Track Changes, real-time online collaboration– and it never, ever works. Most them don’t look, and most of them don’t care, unless there’s the basic human accountability of sitting down with them at a table and going through the changes together. That’s how I drag them to the skills they want.

I will have lost some of you with that verb. “Drag them! How presumptuous! That’s so insulting.” I assure you: no, it’s not. No, it’s not insulting to use the word “drag” to describe educating undergraduates. I promise you it’s not. Of course, there are in most classes one or two or three students who are both very bright and self-motivated. They’re wonderful to work with. But most students require a frankly endless amount of pushing, pulling, cajoling, motivating, and yes, dragging to competence. Some actively resist. I’m not complaining: this is what I love to do, and it’s why they pay me. I signed up to be a compositionist knowing that many, both within and outside of the university, see nothing to respect in the discipline. I did because I love teaching people to write and love researching ways to do it better. I’m just relaying reality, in context with an education media that simply doesn’t want to hear it: our college students are not an army of young autodidacts who are pursuing knowledge out of a love for learning. They just aren’t. They’re here, in very large measure, to collect a degree that they identify as being a largely or purely economic instrument. Who could blame them? That’s what their culture is telling them education is for: making money. So they proceed rationally from that premise.

Freddie deBour

The central, simple, most repetitive message of education (starting at much younger grade levels) is to make money is very true. I’m not going to make a value call on that message (perhaps a future post?). But that message has many implications on society and what society we become.

DeBoer continues:

So maybe you can see why I am so deeply frustrated with the Clay Shirky vision, which is really just the consensus view, and pretty much Obama’s major vision for the next era of the American economy. It’s a common saw: the next stage of American abundance requires all of our workers be educated, it’s too expensive to teach them in the conventional academic setting, and so we need to replace the physical university with online colleges, staffed by adjuncts teaching many sections of huge classes. And not only will we be erasing the very notion of individual instructor attention, we’ll be particularly targeting the most vulnerable, most difficult to educate students, the ones who now either never make it to college or drop out at huge rates. This is the perfect expression of an educational discourse that has no connection to the reality of what most schooling is like for most students.


Educating students takes a TON of work, especially when the students you teach are have no educational context from their families, are dealing with extreme poverty at home, and often time facing a bar that, realistically, they’re not all going to be able to jump. That’s not to say they won’t learn or can’t learn (I want to be exceptionally clear about that), but the vision being articulated by many education “leaders” is lacking, as deBour says, educational realism.

The people pushing this vision, tellingly, are almost exclusively people who have little to no connection to the day-to-day work of educating undergraduates in basic skills. Either the people arguing for this are journalists and pundits who have never educated, or they are deans and administrators who haven’t taught undergrads in 20 years, or they are celebrity intellectuals who barely teach and when they do, teach at elite institutions where only the most equipped to succeed are present. The greatest division in educational discourse today is not best understood as progressive vs. neoliberal or something similar. The greatest division, at all levels of education, is between those in the world of media and policy who assert that we have the ability to make miracles happen, and the educators who are actually out there, day-to-day, trying to get students to standards those students cannot meet. We can begin to let our policy discussions reflect on what’s actually happening in our actual schools, or we can continue to engage in pleasant fantasy.


Lots of ideas and concepts to mine here. More to come.


I Always Tried to Use Chicago Style

Technology creates anachronisms faster than we educators understand and realize.

Take the citation as an example. In high school I turned in papers MLA style (make sure to get the commas in the correct place). For graduate school it was ALA. For my undergraduate senior thesis I went with Chicago Style if only because it suited my desire to explore all kinds of thematic segues without distracting from my central thesis.

When I assigned history papers to my 8th graders, I always spent a good couple of lessons going over how to create a works cited page (MLA). This grew easier over time, as Microsoft and the web added programs that made the process stupid proof (it’s all algorithmic anyway). Getting my students to add quotation marks and citations in their actual texts was, in my opinion, more important than getting semicolons and underlines correct.

Why Do We Cite?

From a writer’s perspective, we cite to give credit where credit is due, to build arguments from other people’s logic, and to lend authority to the thesis we’re trying to prove.

From a teacher’s perspective, we insist our students cite for the above reasons and, honestly, it gives us one more area in a rubric we can associate with their grade (and who hasn’t docked a point or two for missing a comma…or had the conversation with a student on plagiarism).

Also, I think it may serve as an educational right of passage. Or maybe bragging rights among the nerds and academic go-getters. “My Bibliography was 30 pages long in Times New Roman Font Size 8″.

But Why Do We Cite this Way?

Or, more accurately, why do we cite this way still? The era where print copies of papers and essays is waning. Or, as Slate puts it in their excellent post “Stop Citing Your Papers. Start Hyperlinking“:

Conceived during an era when libraries were purely physical places, papers were composed by hand or typewriter, and professors returned graded assignments sprayed with red ink, current systems of citation in academic writing lack value in the digital age. While some will continue to hold out on digital documents, they are quickly becoming the minority as their peers take advantage of digital submission and grading as well as e-readers and computer programs that can easily adjust font, take notes, highlight text, and store (more legible) comments.

Digital documents can contain hyperlinks to the original source material. Which, from a reader’s (and teacher’s) perspective is much more valuable and useful. Think the essayist is making excellent points (so excellent that you start to have questions about their sources)? Follow the hyperlink trail! Think vast quantities are being copied and pasted? Look directly at the source (without having to Google the text)! Want to explore the bigger picture? Hyperlinking is much more effective and removing those barriers of exploration.

I’ve noticed that even when passages of text are not available for reading on the web, it’s becoming more common to simply link to the book title on Amazon (much to Amazon’s joy I’m sure).

MLA and ALA won’t disappear anytime soon. But they likely will as the digital world continues to make inroads on education.