GoogleTranslate Service


What is Evaluation 2.0?

October 4th, 2011 by Graham Attwell

Graham Attwell interviews Jenny Hughes about Evaluation 2.0

Just what is Evaluation 2.0?

Evaluation 2.0 is a set of ideas about evaluation that Pontydysgu are developing. At its simplest, it’s about using social software at all stages of the evaluation process in order to make evaluation more open, more transparent and more accessible to a wider range of stakeholders. At a theoretical level, we are trying to push forward and build on Guba and Lincoln’s ideas around 4th generation evaluation which is a constructivist approach incorporating key ideas around negotiation, multiple realities and stakeholder engagement. But this is the first part of the journey – ultimately, I believe that e-technologies are going to revolutionise the way we think about and practice evaluation.

In what way do you think this is going to happen?

Firstly, the use of social media gives stakeholders a real voice – irrespective of where they are located.  Stakeholders can create and publish evaluation content. For example, in the past I might carry out some interviews as part of an evaluation. Sometimes I recorded it, sometimes I just made notes. Then I would try and interpret it and draw some conclusions about what it meant.  Now I set up a web page for each evaluation and I podcast the interviews using audio or video and put them on the site. (Obviously this has to be negotiated with the interviewee but so far, no one has raised any objections.) There is the usual comment box so any stakeholder with access to the site can respond to the interview, add their interpretations, agree or disagree with my conclusions and so on.

Secondly, I think it is challenging our perceptions of who are evaluators.  Everyone is now an evaluator. Think of the software that you use every day for on-line shopping from Amazon or Ebay or any big chain store. If I want to buy a particular product I check out what other people have said about it, how many of them said it and how many stars it has been given. These are called recommender systems and I think they will have a big impact in evaluation. We have moved from the paradigm of the ‘expert’ collecting and analyzing data into a world of crowd sourcing – harnessing the potential of mass data collection and interpretation.

Thirdly, the explosion of Web 2.0 applications has provided us with a whole new range of evaluation tools that open up new methodological possibilities or make the old ones more efficient.  For example, if I am at the stage of formulating and refining the evaluation questions – I put it out as a call on Twitter.  It’s amazing how restricting evaluation questions to 140 characters can sharpen them up!

I did an evaluation of a community capacity building project in an inner city area recently and spent quite a long time before I went to the first meeting walking around the streets, checking out the community facilities, the state of the housing, local amenities and so on, to get a ‘feel’ for the area – except I did it on Google Earth and with street-view on Google maps.  There are about 20 or so other applications I use a lot in evaluation but maybe they will have to wait for another edition!

Fourthly, I think the potential of Web 2.0 changes the way we can visualize and present data.  Why are we still writing long and indigestible text-based evaluation reports? Increasingly clients are preferring short, sharp evaluation ‘articles’ on maybe one outcome of an evaluation which they can find on a ‘newsy’ evaluation webpage – with hyperlinks to more detailed information or raw data or back up evidence if they want to check it out.  We can also create ‘chunks’ of evaluation reporting and repurpose them in different ways for different stakeholders or they can be localized for different cultures – for example, I have started doing executive summaries as downloadable podcasts.  I think evaluation 2.0 is about creating a much wider range of evaluation products.

Following on from that, I think Evaluation 2.0 breaks down the formative-summative divide and notions of ‘the mid-term report’ or ‘the ex-ante report’.  Evaluation 2.0 is continuous, it is dynamic and it is interactive.  For example, I use Googledocs with all my clients – I add them as readers and editors on all the folders that relate to their evaluations. At any time of the day or night they can see work in progress and add their comments. I keep their evaluation website up to date so they get evaluation information as soon as it is available.

So do you think all evaluators will have to move down this road or will there always be a place for evaluators using more established methods?

Personally, I think massive change is inevitable. Apart from anything else, our clients of the future will be the digital natives – they will expect it.

There will always be a role for the evaluator but that role will be transformed and the skills will be different. I think a key job for the specialist evaluator will be designing the algorithms that underpin the evaluation. The evaluator will also need to be the creative director – they will need skills in informatics, in visualizing and presenting information, the creative skills to write blogs and wikis. They will need networking skills to set up and facilitate online communities of practice around different stakeholder groups and the ability to repurpose evaluation objects.

The rules of engagement are also changing – in the past you engaged with a client, now you engage with a community.  We also have to think how stakeholder created content might change our ideas about copyright, confidentiality, ownership, authorship.

So do you think evaluators as we know them will become extinct!!

Well, as Mark Halper said

“Dinosaurs were highly successful and lasted a long time. They never went away. They became smaller, faster, and more agile, and now we call them birds.”

Tweetbacks/Trackbacks/Pingbacks

  1. […] Bridge to Learning – Educational Research So do you think evaluators as we know them will become extinct!! Well, as Mark Halper said The rules of engagement are also changing – in the past you engaged with a client, now you engage with a community. We also have to think how stakeholder created content might change our ideas about copyright, confidentiality, ownership, authorship. […]

  • Search Pontydysgu.org

    News Bites

    Zero Hours Contracts

    Figures from the UK Higher Education Statistics Agency show that in total almost 11,500 people – both academics and support staff – working in universities on a standard basis were on a zero-hours contract in 2017-18, out of a total staff head count of about 430,000, reports the Times Higher Education.  Zero-hours contract means the employer is not obliged to provide any minimum working hours

    Separate figures that only look at the number of people who are employed on “atypical” academic contracts (such as people working on projects) show that 23 per cent of them, or just over 16,000, had a zero-hours contract.


    Resistance decreases over time

    Interesting research on student centered learning and student buy in, as picked up by an article in Inside Higher Ed. A new study published in PLOS ONE, called “Knowing Is Half the Battle: Assessments of Both Student Perception and Performance Are Necessary to Successfully Evaluate Curricular Transformation finds that student resistance to curriculum innovation decreases over time as it becomes the institutional norm, and that students increasingly link active learning to their learning gains over time


    Postgrad pressure

    Research published this year by Vitae and the Institute for Employment Studies (IES) and reported by the Guardian highlights the pressure on post graduate students.

    “They might suffer anxiety about whether they deserve their place at university,” says Sally Wilson, who led IES’s contribution to the research. “Postgraduates can feel as though they are in a vacuum. They don’t know how to structure their time. Many felt they didn’t get support from their supervisor.”

    Taught students tend to fare better than researchers – they enjoy more structure and contact, says Sian Duffin, student support manager at Arden University. But she believes anxiety is on the rise. “The pressure to gain distinction grades is immense,” she says. “Fear of failure can lead to perfectionism, anxiety and depression.”


    Teenagers online in the USA

    According to Pew Internet 95% of teenagers in the USA now report they have a smartphone or access to one. These mobile connections are in turn fueling more-persistent online activities: 45% of teens now say they are online on a near-constant basis.

    Roughly half (51%) of 13 to 17 year olds say they use Facebook, notably lower than the shares who use YouTube, Instagram or Snapchat.

    The survey also finds there is no clear consensus among teens about the effect that social media has on the lives of young people today. Minorities of teens describe that effect as mostly positive (31%) or mostly negative (24%), but the largest share (45%) says that effect has been neither positive nor negative.


    Other Pontydysgu Spaces

    • Pontydysgu on the Web

      pbwiki
      Our Wikispace for teaching and learning
      Sounds of the Bazaar Radio LIVE
      Join our Sounds of the Bazaar Facebook goup. Just click on the logo above.

      We will be at Online Educa Berlin 2015. See the info above. The stream URL to play in your application is Stream URL or go to our new stream webpage here SoB Stream Page.

  • Twitter

  • RT @pete_wh For those interested in university architecture and open plan offices, my thesis Whitton, Peter David (2018)The new university: space, place and identity. Doctoral thesis (PhD), Manchester Metropolitan University. is available from MMu's e-space at ... e-space.mmu.ac.uk/620806/1/Ph…

    About 4 days ago from Cristina Costa's Twitter via Twitter for Android

  • Sounds of the Bazaar AudioBoo

  • Recent Posts

  • Archives

  • Meta

  • Upcoming Events

      There are no events.
  • Categories