Teacher Feedback VS Machine Feedback

Do students prefer to learn from a machine?

A bit of a professional disclaimer to this week’s geek study. I do not believe that technology is ever an effective substitute for good teaching (and this is a belief rooted in quite a few empirical studies as well). Technology is a tool. My interest is how tools are leveraged by teachers to help and increase learning by students. I found the results of this study to be surprising. I’d also hesitate to draw huge conclusions from a study at a large education college and apply those results to K through 12 education.

That said, this is interesting.

Why Students Prefer to Learn from a Machine

While this article’s headline is hyperbolic, it does share a fascinating study from The International Journal of English Studies. Education students were placed in two groups. In one group, students received feedback on writing assignments from live instructors. A comparison group received feedback from a software program called Criterion. The study’s intent: to identify differences in how students responded to different forms of feedback. In short, Would students receive feedback better from one or the other and how would they act on that feedback?

The results?

The computer program appeared to transform the students’ approach to the process of receiving and acting on feedback.Comments and criticism from a human instructor actually had a negative effect on students’ attitudes about revision and on their willingness to write, the researchers note. By contrast, interactions with the computer produced overwhelmingly positive feelings, as well as an actual change in behavior—from “virtually never” revising, to revising and resubmitting at a rate of 100 percent. As a result of engaging in this process, the students’ writing improved; they repeated words less often, used shorter, simpler sentences, and corrected their grammar and spelling. These changes weren’t simply mechanical. Follow-up interviews with the study’s participants suggested that the computer feedback actually stimulated reflectiveness in the students—which, notably, feedback from instructors had not done.

Why would this be? First, the feedback from a computer program like Criterion is immediate and highly individualized—something not usually possible in big classes like those at Alexandria University, the site of the study by El Ebyary and Windeatt. Second, the researchers observed that for many students in the study, the process of improving their writing appeared to take on a game like quality, boosting their motivation to get better. Third, and most interesting, the students’ reactions to feedback seemed to be influenced by the impersonal, automated nature of the software.”Annie Murphy Paul

There’s a lot to crunch through here. Part of me wonders if generational components affected the differences in responses? What social elements allow for students to respond better to a machine rather than an individual?

And does it really matter, as long as their writing improved?

First Thoughts on Google Classroom

Like receiving the golden ticket

Friday afternoon (timed for the most productive part of the day), Google turned on Classroom for me. My buddy and I got to exploring right away. For the most part, I’m very excited by what I see. I wanted to get a “at first glance” post up and running given the level of interest educators have towards classroom.

Up front nerdy confession: I love exploring Learning Management Systems. I fell in love with Moodle in the early aughts (and all its clicky / options glory). In addition to Moodle, I’ve taught with Blackboard, Schoology, and the WordPress flavored Learn Dash. Each have their strengths and weaknesses. Each their own cheerleaders.

Google Apps has always felt like an “almost there” LMS. You could cobble together a Google Site, play with some public folders, create some elaborate forms for hand-in processes. At first this was mildly annoying. Then it became extremely annoying as Google Apps pretty much took over all of my productivity and really started to take off in my school district.

With Classroom, Google takes the simplicity of their Apps and works them into an entry level / pseudo LMS. My guess is some folks might quibble over actually defining it as an LMS – but it’s evolving that way.

Opening "Stream" of the class (ie landing page)

Opening “Stream” of the class (ie landing page)

How does it work?

Basically, Google Classroom allows 2 functions:

  1. Discussions (via announcements or surrounding an assignment)
  2. Creating and grading of assignments

Given that Google Apps (Docs, Drive, YouTube) underpins those functions, you can actually get a lot mileage out discussions and assignments. As the teacher, you create the class (it comes with its own key code). A class can have multiple sections (think, 1st period). Students are added to the class either via your Google Contacts or by entering the key code.

Here’s the process flow for assignments:

  1. The teacher creates an assignment. The assignment can be spelled out in Google Classroom. You can also linked to uploaded files, Google Doc, URL, or YouTube Video.
  2. Students complete the assignment and turn them into the teacher. The assignment gets placed in a “classroom” folder in the teacher’s drive. The beauty of this is that all assignments are ordered and named clearly by Google. They can all be viewed in Google Drive.
  3. Teacher grades the assignment. Feedback can be provided in Google Classroom OR in the actual assignment if that assignment was created using Google Docs.
  4. Assignments are a “check in / check out” process. When a student turns in an assignment, they only have “view rights” to the assignment (they can no longer edit that assignment). Likewise, when a teacher passes the assignment back to the student, the teacher only has view rights. (This applies to Google Docs)

Classroom features a simple gradebook that can be downloaded as a CSV file.

Grade screen of an assignment

Grade screen of an assignment

It’s really about the flow.

Classroom really gets classroom work flow right. You finally can easily create a “paperless” classroom. Prior to classroom, organizing and tracking 150 history papers turned in by students was not a simple process to complete ONLY using Google Apps for Education (and we’d have to rely on traditional LMSs). Classroom makes it very simple to organize assignments without junking up your Google Drive or your Gmail.

Who is it for?

Honestly, Classroom is a great entry level LMS. It’s for teachers who are still dipping their toes in the tech, who are getting used to the collaborative nature of Google Apps, and who want something simple and easy to use. There are no bells and whistles. And I love that. You can connect the dots fast. Sometimes the best technology is the technology that takes a very little learning curve to climb.

And the downsides?

There are many, but at first glance two stood out. The first is that Google Classroom appears to very much run on a “real time” display (called the “Stream”). The landing page is like a Google Plus / Facebook page. This is great as many students will intuitively sense how classroom works. It’s a problem for structured / orderly teachers who want their class to have strong organization and instructor driven frameworks. In short, it’s harder to fashion a narrative.

As a quick example, there’s no area where you could place your syllabus where it wouldn’t get buried in the flow of announcements and assignments. This is an area that would be easy to fix and I wouldn’t be surprised if Google made a tweak to make this happen (fingers crossed).

Second, power LMS users will find Google Classroom lacking. There are many features I expect in an LMS (assessments – something Classroom does not directly have – immediately come to mind). The standard LMS tools of graded discussion boards, polls, content areas for creating lessons, etc. are not available (although you can argue that you can cobble together those functions using other Google Apps). Speaking personally, I’d hesitate using Classroom for a class taught purely online. But then, I like having strong control and options in developing my online content.

At the end of the day those limitations likely won’t matter to many teachers. Simple, functional, and reliable is key. If your district is a Google Apps for Education district and investing in Chromebooks, Classrooms will be an easy sell to your staff.







The Role of Chance in Outcomes

Are the successes in my life a result of ability (the quality I produce) or chance?

NPR kicked off a story about the Mona Lisa. It’s a popular painting – the most popular painting. Each year thousands swarm the Louvre in Paris to get a glimpse of her enigmatic smile. The Mona Lisa is famous.

But is she famous because she’s a spectacular work of art? Or is she famous because she’s the result of chance encounters (i.e. because she was acquired by the King of France, displayed at the Louvre, a product of a masterful inventor who was acknowledged as such by Italian authorities)?

This question – chance vs ability – has historically been untestable. We live in one reality and in this reality the Mona Lisa is famous.

Until Princeton professor Matthew Salganik decided to come up with a rather genious experiment. He would create multiple identical worlds online filled with the same works of art, get thousands of people to pick which work of art they liked best, and if the same works of art rose to the top in each world, then we could reasonably conclude it was the actual art work that determined its greatness.

The experiment

From NPR:

To test how much of success should be attributed to chance and how much to quality, Salganik created a website that randomly funneled the 30,000 teenagers he recruited online into nine identical worlds.

Each of these worlds exposed the teens to 48 songs from emerging artists — bands that hadn’t yet been signed so were totally unknown to the teens. The deal was that after listening to the songs, the teens could download the ones they liked best for free.

Now in one world — the control world — they couldn’t see which songs their peers were downloading so there was no social influence. But in the other eight, the teens could see which songs had been downloaded before, so they knew what other people thought was good.

“So we had the exact same 48 songs competing against each other, we had the exact same initial conditions, everything starts with zero downloads, and we have indistinguishable groups of participants, because they were randomly placed into the world,” Salganik says.

Good Art Is Popular Because It's Good. Right?

And what did he find?

Different songs become popular in different histories — and not in small ways, either.

….there are many different things that have an impact on success and failure — money, race and a laundry list of other things — and after this work, which one person in the field described as a seminal paper, Salganik went on to do similar studies with parallel worlds that suggest that quality does have at least a limited role. It is hard to make things of very poor quality succeed — though after you meet a basic standard of quality, what becomes a huge hit and what doesn’t is essentially a matter of chance.

Good Art Is Popular Because It's Good. Right?

What does this mean?

I think studies like this give us moments to pause, look at the world around us, and consider the implications and how these implications affect our approach to the people around us.

Take my life as example. In terms of the biggest element of chance – who my parents are – I scored the jackpot. Middle class, a reasonably healthy pool of genes, educated, and willing to throw considerable resources at the development of their children. I try not to confuse the fact that I was born on third base with those who had to run, on their own, from the start of a pitch.

Or consider my current position as Director of Technology. One reason I landed this wonderful gig was that members of Hamilton CSD viewed a presentation by me at an informational session. The curious thing about that day was that I wasn’t supposed to present. My boss at the time had pneumonia and I filled the spot. Pneumonia as a matter of a chance.

If you think that much of a persons success (or lack of success) is up to chance, perhaps it allows you to empathize with an individual better. Or as Salganik points out, you treat them better.