Wednesday, 30 December 2015

eAssessment

Read the introduction up to page 14 the JISC publication ‘Effective Assessment in Digital Age’  and choose one case study that is most relevant to your educational area. Reflect on its implications for your area, by writing a reflective blog post and commenting on two others.
Working on this section of the course has highlighted something for me.

The complete focus in the unit is on HE and eLearning rather than any other - none of the examples were outside the HE space (allowing me I suppose to then contextualise it into my own). Up until now that hasn't been a particular problem for me (although it's irritating), but for a course on learning and digital technologies to spend so little time considering the school or early years sector is a lost opportunity. In this case the JISC report delivers a summary of several pieces of work on formative assessment for HE but neither mentions, nor references any of the research by William and Black, which is particularly relevant in the second case study I selected.

Case Study: Enhancing the experience of feedback University of Leicester (2010)


The case study concerns work in the distance learning course, MSc in Occupational Psychology. Audio recording was used to deliver feedback that was more personal and engaging for the student on their work, and a spin off benefit was the development of skills that would allow the faculty to make greater use of podcasting in the future.

Whether or not the case study illustrates a particular benefit of e-assessment is debatable.

The very fact of running the programme implies staff would have spent greater time giving feedback to students, as it was a project focus, and therefore the positive benefits may well have been just as great had the tutors left voicemail messages or simply written feedback. The fact that it is asynchronous and allows the students to access it when needed and return to it as needed is a positive.

If the work was simply replacing one feedback process (written feedback) with another (audio feedback) then it is I suppose a useful augmentation of the provision. Adding the audio in much smaller chunks within the document itself would seem to be an obvious and more useful approach as it then makes each piece of feedback rooted in the section of the work being discussed, but that is a very different assessment tool and would perhaps be better suited to earlier in the drafting and preparation cycle for the student.

In schools I have seen a similar, but more powerful technique used with the tool Movenote.

Movenote allows the user to set up a stack of pages (documents, slides, images) in advance, and then progress through them, adding audio commentary/ annotation or video. The end result is a URL that can be sent to the viewer. For e-assessement this goes beyond either the method used in the case study or the alternative I suggested.

  • Generic audio feedback for the document as a whole is not much more than a novel way of changing the emotional response and the workflow, not a new approach to feedback.
  • Smaller comments embedded in the document do not give the complete overview.

Within Movenote the tutor can deliver the audio feedback, but process through the document on screen, matching the delivery of the verbal feedback to the relevant section being discussed.

Movenote is very much the kind of useful tool that a learner can bring into their PLE and use within a cMOOC.

Case Study: Facilitating peer and self-assessment University of Hull and Loughborough University

URL http://www.webarchive.org.uk/wayback/archive/20140615043727/http://www.jisc.ac.uk/media/documents/programmes/elearning/digiassess_assessingselfpeers.pdf

A second case study that immediately caught my interest was this, as it seemed to offer two things I understand to be fundamental to good formative assessment practice and an area where e-assessment has massive potential to raise standards or achievement and improve learner experience.

Asking students to give their peers feedback both provides valuable responses and ideas to the student, and helps the giver of feedback to better understand and articulate the assessment criteria themselves - it follows that when you can better explain the criteria to gain a particular level, you are more likely to be able to achieve it.

The Universities used a system called WebPA and asked undergraduates to read and rate each others work numerically using a number of criteria.

This has the advantage of efficiency and providing students with the motive to sit down and read each other's work and respond by rating it against criteria. The case study doesn't go into how well the students understood the criteria, the level of motivation and time spent by students on the process or how reliably the peer assessment matched that from faculty staff.

I'm unsure why the complexity of putting a layer like WebPA between students and each other's work was required. Rather than have them assign numeric values would they not be better to have been asked to add a text comment against a simple framework for each? In that way not only would the student get useful feedback rather than a number (and to go back to the JISC research base, if they had referenced Black and William's work they would have found an immense amount of research to deter them from ever giving numeric values for feedback).

"Avoid grading.  Grades are consistently found to demotivate low attainers.  They also fail to challenge high attainers, often making them complacent.  So avoid giving a grade or mark except where absolutely necessary.  This is not easy to do on some courses.  However it is rarely necessary, and almost never desirable, to grade every piece of work."
Massively increasing the quantity of numeric grades and sub-grades awarded by using peer e-assessment will benefit the givers of those grades, but not the receivers. Far better to ask the students to spend more time on less work and give helpful feedback in the form of comments related to the assessment scheme.

Again, from within the school sector, an often-employed technique is to simply allocate students one or more peer reviewers for their work in progress or finished draft, and to ask them to share the work with those people with access rights set to 'comment only.'

Peer reviewers, having taken part in work to improve their ability to give useful feedback, and to make sure they understand enough about the assessment rubric to guide comments towards improvement that will lead to gains, leave comments on work that is designed to guide their partner to improve. In both the schools that I know use this approach in a routine way, it has been one of a number of strategies that has led to significant improvements in the quality of written work.

Reference:

Inside the Black Box, Black P and William D (1998)

Tuesday, 29 December 2015

xMOOCs and cMOOCs

"Briefly outline your views and reflections on the distinction between xMOOCs and cMOOCs. Write your reflections on a blogpost."


MOOCs are Massive

One thing that differentiates MOOCS from their predecessors are the scale they operate at. Whereas an institution based online course would be designed with a particular scale of enrolled students in mind, a MOOC is designed from the outset to scale to an undefined and potentially unlimited number of students.

An xMOOC is not dissimilar to the Virtual Learning Environment (VLE) as it represents the same model of operation - a single monolithic system - scaled to allow for use both in and beyond the institution. That doesn't mean the student cannot co-opt it to be part of their Personal Learning Environment (PLE)

A cMOOC is intended from the outset to form part of a PLE - it presents a way for a community of learners to connect and collaborate, but does so in a way that will scale massively.


MOOCS are Open

MOOCs differs from other online learning provision in the premise that they are open to external and internal students without limit.

For an xMOOC this implies that digital materials are made available for use by any potential student, but does not necessarily imply they are Open Educational Resources, or that any rights are given to reuse, modify or redistribute. So Open is in the sense of Access, not License.

For a cMOOC, by it's very nature materials have to be released to students to modify, extend, share and add to. So Open is very much in the sense of Access and License.


MOOCS are Online

Typically an xMOOC will be a single hosted platform, managed by the institution that offers the course.  Methods may be made available to integrate the xMOOC with other platforms or services used by the student (such as facilitating single sign on) but the priority it to ensure that every student sees and interacts with the material in the way the course designers intended.

A cMOOC is by definition a network of different platforms and services used and added to by the student. Students will see and interact with the course in the way they and their peers choose to.

MOOCS are Courses

An xMOOC is based on the premise that during the course the learner will acquire skills and knowledge through interaction with programme.

A cMOOC is intended to act as the network for a group or peers to co-construct their course and acquire knowledge and understanding through interaction.


References
Table showing the significant differences between xMOOCs and cMOOCs from Li, Olivier and Powell (2014) Beyond MOOCs: Sustainable Online Learning in Institutions. Centre for Educational Technology & Interoperability Standards (Cetis)
Yuan L, Powell S and Olivier B
Available at URL http://publications.cetis.ac.uk/2014/898 (accessed 29/12/15)
Characteristics of Massive Open Online Courses (MOOCs): A Research Review, 2009-2012 Kennedy J Journal of Interactive Online Learning Volume 13, Number 1, Spring 201 Accessed at URL: http://www.openeducationeuropa.eu/sites/default/files/asset/Characteristics-massive-online-courses-MOOCs.pdf ISSN: 1541-4914 (accessed 29/12/15).


Monday, 21 December 2015

MOOC is for?

MOOC? xMOOC? cMOOC? NOOC? GOOC?

Now we have COOC.

I can't argue with Peter Shake's sentiments in setting out a new name for something that sounds a lot like a constructivist MOOC (which from my limited knowledge was what MOOCS where supposed to be in the first place before people started calling big VLEs full of course content MOOCs). It does seem to me though that much as I thought it over complex, Conole's ideas about classifying these things might be more necessary that I first imagined.....

But I'd take issue with two things in his logic.

Firstly, the toolset he described as a COOC, is so radically different from the reality of the thing now commonly called a MOOC that it's a totally different animal. A COOC requires an entirely different set of tools and services to a massive set of linear online courses. So to describe it like "a MOOC but different'" is really misleading. It's a different model entirely.

Secondly, calling is anything [id]OOC will only make that confusion worse. Just makes you wish that the people who coined the term MOOC had the same level of (cough) vision (splutter) as Nottingham University in trademarking the term (and the opportunism to take some courses on their VLE and rebadge them as if they were new).

MOOCs - before talking about categorisation

Having happily submitted my first assignment I can start to move forward with the course again (I say happily both in the sense of being happy whilst submitting it, and as a result of so-doing!)

MOOCs and Open Educational Resources are two areas I'll be reading more about over the festive season, including different types of MOOC, and for entirely different reasons I'm mildly cynical about both.

I've never been hugely inspired by MOOCs as much of the promise and hyperbole is all too reminiscent of the hype and nonsense that surrounded local authority and regional (and national)  VLEs for schools. I worked hard in that area a few years ago and met many clever people who sincerely believed in the rhetoric, and others who were glad to sell them the necessary tools, and seeing the result was very depressing.

Channeling a huge government investment in online learning into something that will engage individual teachers and departments is not easy... or maybe it should have been if we'd started from a less procurement-led view and instead used a more organic and teacher and student-led model.

The experience of many of these, which became exercises in project management and justifying funding rather than pedagogy and school improvement left me with a deep distrust of grandiose visions based on "build it and they will come" (and that is perhaps why emotionally I'm much more relaxed and enthusiastic about Personal Learning Networks or rather a version of them that meets the needs of education for schools.

I hear good things about GLOW in Scotland (and less good ones) and similarly for Hwb (maybe more negative there) but frankly would not be enthusiastic to work with either as they currently stand.

So to properly view MOOCs and read about them with an open mind I need to leave significant amounts of  baggage at the door.

Thank heavens then for this deconstruction by Audrey Watters which pours far more healthy cynicism and awareness on the whole subject of the MOOC industry and visions for disrupting education by.... industrialising it. Having read it (and in particular enjoyed the comment about the History Channel which makes reading it worthwhile on its own) I feel ready to begin.
In this blog post at the NESTA website, Oiliver Quinlan introduces a paper called 'Digital learning technology: Converging promise and potential.' The premise of the paper - that there is a gap between the research into the impact of digital tools on progress, and the work of people seeking opportunities to use them.

Tuesday, 15 December 2015

Open Educational Resources

Read the Educause '7 things you should know about Open Educational Resources'. Summarise the key points in a blog post.

The Educause paper sets out succinctly an explanation of what OER is and some of it’s implications. It is very much focused on the HE sector, so in summarising I will attempt to add some reflections on implications in the school-sector.

OER are defined as resources for little or no cost for teaching, learning or research. I assume ‘low cost’ covers things like a provider charging a small fee for subscribers or members to maintain the library of content rather than to cover the cost of actually producing it. Typically OER materials are digital and the most common method of licensing them is under Creative Commons to support open or nearly open use (eg re-use, modification and how the original author should be credited). 

OER material can be produced at, or published by educational institutions, libraries, government, businesses or individuals. Indeed the key criteria is willingness to share although the motives for doing so will vary. Educause are talking at all times about open in the sense of open to the world, but in a school context, open within a network of schools is still a significant and important development.

People may of course publish material to promote a particular world-view or interpretation.

Materials are released under a license that sets out how it can be altered, attributed and distributed. They will typically be found in collections although that approach may change with time.

Course designers, teachers and of course end-users (learners) can access and use OER. Much OER is modular allowing it to be repurposed, combined applied in various ways. 

In some cases the original author may maintain the OER material, updating it as needed, and as technology opens up new ways of presenting it, or equally the community of users may well continue to adapt, improve and repurpose that original material.

MIT were one of the highest profile early adopters with the OpenCourseWare in 2002 (including roughly 2,000 MIT courses). Many colleges and universities (but a small proportion of the total) now put full sets of course material online. This is not the same as taking the course online – simply allowing people to use the content as they wish.

Educause say that OER material can be vetted and improved by a wider community but this is in part balanced by the greater freedom for individuals and groups to publish. Through recommendation and a distributed selection process better material will become more mainstream but to argue that overall quality will rise is in my opinion facile. OER is well matched to many approaches such as personal learning networks, MOOCs and other Web 2.0 technologies.

OER may well open up content creation to a wider audience by allowing many to create and share in smaller and more focused areas. However it may also depress the value given to content creation (as it is already out there and free) and reduce actual investment at institutional level.

OER can certainly lower the costs for students both by making good material available for free, and by reducing the notional ‘fair cost’ for content through competition.

Not all OER content will be good. Not all OER collections will present and make discoverable content in ways that are useful, but feedback loops will support and promote the good.

Similarly, OER material, like all other, will date. There is not the same impetus for an organisation to maintain freely published material – they can either allow their users to do the updating for them, or themselves use more recently published OER material from elsewhere. 

Selecting, repurposing and integrating together OER material requires high levels of knowledge and some skill. Assuming intellectual property and copyright concerns are addressed, the institution still needs to invest in a workforce able to create and use OER material.

Inevitably, organisations with a vested interest in educational content (universities and publishers) will seek to provide credibility to OER resources and collections, moderate them and add value to them. Whilst the development could be said to be disruptive, it isn’t a simple case of the users taking control of the means of publication and sweeping away the old system.

Educause describe a (radical) viewpoint in which OER allows learners the capability to construct their own course of study. I think it’s an interesting but perhaps flawed idea.

Another scenario is less disruptive where existing models continue enhanced by and integrated with OER. 

For schools there will a number of issues to consider in either using or sharing OER.  People like Josie Fraser (also a leader in the DigiLit project) have written at length supporting OER in the schools sector (see this guidance document for example) but it isn't something that makes the headlines - indeed the prevalent model is much more about closed educational resources - where a group of schools produce a strong set of materials based on what has been reviewed as best practice (usually post OfSTED inspection) as a lever to raise standards in the subject within the network as a whole. I think it would be wrong to criticise schools for following that route as the business of writing material and then putting it together as a complete course, is not the same as the one set out in the OER movement.
  • A school or group of schools will need to have considered rights and licensing, and the ownership of staff over materials they produce.
  • In many subjects the provenance and credibility of material needs to be carefully examined and a good QA process put in place to manage it
    • Where learners find their own material they need the digital literacy skills to spot material that is slanted or intended to push a particular view and take account of that
    • Departments need to be able to review material and make sure that what is being used represents the best possible selection
    • Institutions need to ensure they are compliant and staff need the digital literacy skills to understand the licenses for the material they are using

Sunday, 13 December 2015

You can ignore it but resistance is ultimately futile

I've nearly completed my first assignment for the course.  I've enjoyed doing it - an annotated bibliography related to the themes of the first unit. I chose to look in particular at research to inform eLearning strategy, and in particular how institutions should respond to the 'inevitable generational shift of the digital natives.'

The panic over how institutions and practitioners cannot possibly meet the needs of these transformed young people, immersed in digital technologies has been a source of frustration for me for some time partly because it is so patently not the reality in the classroom, and partly because it makes the job of arguing for change based on actually making better provision, improving learner experience, far harder because is winds people up with cod-science.

The process of reading research, looking at common threads and arguing against something has been an interesting and enjoyable one. However, inevitably arguing against things and taking a conservative position on them doesn't reflect the fact that actually I am really passionate about both the need to change education and to make use of new technologies to improve it (because we want to, because it will make things better, not because we are doomed by some made up generational shift).

So having got to the point that I'm trying to crunch the word-count to fit the assignment brief, which is perhaps not the most creative thing, it's a real pleasure to discover this video of a talk by Donald Clark at TEDx that reminds me of the positives of technology for learning and more importantly, the inevitability of change.


Donald promises to highlight "more pedagogical change in the last 10 years than the previous 1000" and perhaps doesn't deliver on that entirely, but he captures brilliantly the idiocy of repeating flawed models that were of a time where printing and broadcasting technology made sitting in a room talking to people, and then setting hard questions for them to resolve alone, the most viable solution. We live now in a world where even bad lectures can be improved simply by recording them and allowing the student to select sections, repeat them and review them, and skip the wasted bits.

His arguments, that change is inevitable, that not changing is ridiculous, does not contradict the assignment I'm trying to hone, but for a few moments I worried it might.

In the assignment I find myself arguing that a good strategy is based on establishing strong core provision and allowing people to then innovate around it, that change should be because we want to improve pedagogy and make the experience of being a student better, not the 'barbarians at the gates' narrative from Prensky and Tapscott.

In the first couple of weeks of the course we were asked to find and share two video clips that we felt really summed up the Learning Technology story so far. I wish I'd seen this clip earlier so I could have put forward this one (I do however disagree with Donald's comments about Brighton and Glasgow).

Two Sigma Improvement

I do enjoy a good provocative title.... "Education is failing technology" and this article by Mark E. Weston is both provocative and thought provoking.

I haven't posted anything on the blog for the last few weeks as I'm busy working on my first assignment but I thought this was worth sharing.

Weston sets out in his post a brief history of there work of Bloom and others in identifying ways for learning in a group setting to match the progress possible with one to one tuition. Many of the approaches referenced will be familiar to anyone who quotes Hattie from time to time.

Weston argues that the current set up in schools makes the burden of achieving that 2-sigma improvement (performing two standard deviations higher than their peers) too much about the actions of the individual teacher and not enough about the system as a whole.

People who listened to Professor David Hargreaves talk just about anytime in the last fifteen years would probably recognise the argument - as the students leave school full of energy and the staff stagger to the car park exhausted.

What I don't see in the article (and this is unfair as I don't think he set out to solve all the problems, just highlight them) are examples of the kind of technology education is supposedly failing, or even an indication of what that might be.