Showing posts with label evaluation. Show all posts
Showing posts with label evaluation. Show all posts

Thursday, March 19, 2015

Networked Learning as a pedagogy again

I can't quite escape from the ideas I was talking about in 2008. Media choice and the pedagogy of and around that choice impacts behaviour and somehow we're liable to miss that.
Scenario: Evaluation. You have a large class of students (say 200) and you want to get a good response-rate to the routine end-of-module evaluation. The organisation would quite like this process to cost nothing at all. So, from paper-based, or perhaps optical mark read forms, the organisation has moved to online evaluations. This has coincided with a dramatic drop in response-rates. The choice of media has distanced the student from anything but the slightest amount of pedagogic influence over them (a paper just published refers to this kind of thing as 'pedagogic distance' - see Westberry and Franken 2015 - I think it is a massive issue). Even in a large lecture setting, if a lecturer hands out paper forms and requests students to complete them, the students are likely to comply. This is likely to do with the social form of the lecture as an event within which students immediately cede control to the person managing the session. Once out of the lecture, the spell is broken and all kinds of things come between the student and the online form.
Given that it is not very manageable to troop them all off to an IT room, unless you have the facility and could tie the evaluation in with some other desirable learning opportunity.
You could try an audience response system but that would be expensive and quite clumsy to administer. Apart from anything else, an evaluation with more than a few summary questions is going to feel like an imposition on the students and would, in my view, be an abuse of the lecturer's pedagogic power. There have been many discussions about the issue of students using their mobiles in lectures.
I suggested projecting a very large QR Code that points to the online form. Google's URL shortener serves up a QR Code and still works even though it gets a bit blurry when enlarged. [Note of caution - check URL's unless you can absolutely trust the source - perhaps use something like https://scanurl.net - good to use a checker that looks harder than just what's at the other end of your click - thanks to Martha for suggesting this edit 😁)
Any student with a smartphone ought to be able to follow the link. Clearly that is fantasy. It would be a good idea to warn the students in advance or at least to ask them to ensure they have or know where to find a QR Code Scanner app.
Even if they don't complete the form there and then, at least their browser is at the right page and they may go on to complete it later.
So it is hoped that merely by using the lecture setting to display the QR Code and allocating a few minutes to complete the form will increase response rates. Whether this will work after the novelty factor wears off, and what proportion of students who are not capable of scanning the code will feel disenfranchised and possibly put in a worse evaluation than they might have otherwise...!!! As they say, that's an empirical question...
But has the lecturer's action of introducing the QR Code in this scenario promoted connections? Is that an example of networked learning pedagogy in action? The definition says nothing about having students having to be learning at a distance...
Then there's the whole thing about the way that my role here means that I get to sit in meetings and try and join the things that I hear with the 'stuff I know' to come up with useful suggestions. Its serendipity. If learning technologists want to influence learning and teaching practice, they may have to sit in and tune in to yet more meetings.

Westberry, Nicola, and Margaret Franken. “Pedagogical Distance: Explaining Misalignment in Student-Driven Online Learning Activities Using Activity Theory.” Teaching in Higher Education 20, no. 3 (April 3, 2015): 300–312. doi:10.1080/13562517.2014.1002393.

Tuesday, November 8, 2011

Evaluations and other articles in JIHE

Just reading a bit this morning in the Journal of the Internet and Higher Education for this month - quite a few papers in the alert caught my eye. The article on peer-assessment is similar to what we do in Year 2 with students although they undertake a group presentation. I note that they stop short, as we do, of getting students to mark each other's work, as a tutor would. This is an understandably conservative route, but I was a little disappointed... I also noted from Owsten et al's lecture capture paper that it seemed to be most used by lower performing students, but in a way that I would decry. These students are viewing the entire lecture, whereas the higher performers were locating the chunks they wanted to access again. The paper does not really help us to know which way to read this relationship and it continues to bemuse me. Would the low-achievers become high-acheiver if only they could take a better approach to their studies..?

In the conclusion to Dziuban and Moskal's paper I found this:
The class for many contemporary students is an increasingly complex network of interactions. Recently in discussing generational difference a student said to the authors, “You will learn by reading a book or the manual. We will learn through interaction with each other and the Internet.” (Hartman, Dziuban, & Brophy-Ellison, 2007). If this is true, then end-of-course evaluations offer very little opportunity for interaction. Perhaps one should consider new instructional evaluation protocols that are not summative and after the fact, but instead those that are more reflective and interactive. The opportunity cost involved would require a much greater investment of time and resources, but good news is that such an approach would increase the student voice in their learning and support a more collaborative educational environment.
I had not read this kind of philosophical rationale for moving to a more formative style of evaluation, which is greatly underused IMHO. Our evaluation forms have recently been under review in the light of the National Student Survey questionnaire. But it begs the question whether we can really account for the kinds of learning experience students are experiencing at an individual level when so much of it happens almost as a co-incidence of their ability to make connections (of all kinds). It must be our role as designers of education to do what we can to assure the potential for students to do that... embracing formative evaluation must be part of that...

Dziuban, C., & Moskal, P. (2011). A course is a course is a course: Factor invariance in student evaluation of online, blended and face-to-face learning environments. The Internet and Higher Education, 14(4), 236-241. doi:10.1016/j.iheduc.2011.05.003