GoogleTranslate Service


What is Evaluation 2.0?

October 4th, 2011 by Graham Attwell

Graham Attwell interviews Jenny Hughes about Evaluation 2.0

Just what is Evaluation 2.0?

Evaluation 2.0 is a set of ideas about evaluation that Pontydysgu are developing. At its simplest, it’s about using social software at all stages of the evaluation process in order to make evaluation more open, more transparent and more accessible to a wider range of stakeholders. At a theoretical level, we are trying to push forward and build on Guba and Lincoln’s ideas around 4th generation evaluation which is a constructivist approach incorporating key ideas around negotiation, multiple realities and stakeholder engagement. But this is the first part of the journey – ultimately, I believe that e-technologies are going to revolutionise the way we think about and practice evaluation.

In what way do you think this is going to happen?

Firstly, the use of social media gives stakeholders a real voice – irrespective of where they are located.  Stakeholders can create and publish evaluation content. For example, in the past I might carry out some interviews as part of an evaluation. Sometimes I recorded it, sometimes I just made notes. Then I would try and interpret it and draw some conclusions about what it meant.  Now I set up a web page for each evaluation and I podcast the interviews using audio or video and put them on the site. (Obviously this has to be negotiated with the interviewee but so far, no one has raised any objections.) There is the usual comment box so any stakeholder with access to the site can respond to the interview, add their interpretations, agree or disagree with my conclusions and so on.

Secondly, I think it is challenging our perceptions of who are evaluators.  Everyone is now an evaluator. Think of the software that you use every day for on-line shopping from Amazon or Ebay or any big chain store. If I want to buy a particular product I check out what other people have said about it, how many of them said it and how many stars it has been given. These are called recommender systems and I think they will have a big impact in evaluation. We have moved from the paradigm of the ‘expert’ collecting and analyzing data into a world of crowd sourcing – harnessing the potential of mass data collection and interpretation.

Thirdly, the explosion of Web 2.0 applications has provided us with a whole new range of evaluation tools that open up new methodological possibilities or make the old ones more efficient.  For example, if I am at the stage of formulating and refining the evaluation questions – I put it out as a call on Twitter.  It’s amazing how restricting evaluation questions to 140 characters can sharpen them up!

I did an evaluation of a community capacity building project in an inner city area recently and spent quite a long time before I went to the first meeting walking around the streets, checking out the community facilities, the state of the housing, local amenities and so on, to get a ‘feel’ for the area – except I did it on Google Earth and with street-view on Google maps.  There are about 20 or so other applications I use a lot in evaluation but maybe they will have to wait for another edition!

Fourthly, I think the potential of Web 2.0 changes the way we can visualize and present data.  Why are we still writing long and indigestible text-based evaluation reports? Increasingly clients are preferring short, sharp evaluation ‘articles’ on maybe one outcome of an evaluation which they can find on a ‘newsy’ evaluation webpage – with hyperlinks to more detailed information or raw data or back up evidence if they want to check it out.  We can also create ‘chunks’ of evaluation reporting and repurpose them in different ways for different stakeholders or they can be localized for different cultures – for example, I have started doing executive summaries as downloadable podcasts.  I think evaluation 2.0 is about creating a much wider range of evaluation products.

Following on from that, I think Evaluation 2.0 breaks down the formative-summative divide and notions of ‘the mid-term report’ or ‘the ex-ante report’.  Evaluation 2.0 is continuous, it is dynamic and it is interactive.  For example, I use Googledocs with all my clients – I add them as readers and editors on all the folders that relate to their evaluations. At any time of the day or night they can see work in progress and add their comments. I keep their evaluation website up to date so they get evaluation information as soon as it is available.

So do you think all evaluators will have to move down this road or will there always be a place for evaluators using more established methods?

Personally, I think massive change is inevitable. Apart from anything else, our clients of the future will be the digital natives – they will expect it.

There will always be a role for the evaluator but that role will be transformed and the skills will be different. I think a key job for the specialist evaluator will be designing the algorithms that underpin the evaluation. The evaluator will also need to be the creative director – they will need skills in informatics, in visualizing and presenting information, the creative skills to write blogs and wikis. They will need networking skills to set up and facilitate online communities of practice around different stakeholder groups and the ability to repurpose evaluation objects.

The rules of engagement are also changing – in the past you engaged with a client, now you engage with a community.  We also have to think how stakeholder created content might change our ideas about copyright, confidentiality, ownership, authorship.

So do you think evaluators as we know them will become extinct!!

Well, as Mark Halper said

“Dinosaurs were highly successful and lasted a long time. They never went away. They became smaller, faster, and more agile, and now we call them birds.”

Tweetbacks/Trackbacks/Pingbacks

  1. […] Bridge to Learning – Educational Research So do you think evaluators as we know them will become extinct!! Well, as Mark Halper said The rules of engagement are also changing – in the past you engaged with a client, now you engage with a community. We also have to think how stakeholder created content might change our ideas about copyright, confidentiality, ownership, authorship. […]

  • Search Pontydysgu.org

    News Bites

    Learning about technology

    According to the University Technical Colleges web site, new research released of 11 to 17-year-olds, commissioned by the Baker Dearing Educational Trust, the charity which promotes and supports University Technical Colleges (UTCs), reveals that over a third (36%) have no opportunity to learn about the latest technology in the classroom and over two thirds (67%) admit that they have not had the opportunity even to discuss a new tech or app idea with a teacher.

    When asked about the tech skills they would like to learn the top five were:

    Building apps (45%)
    Creating Games (43%)
    Virtual reality (38%)
    Coding computer languages (34%)
    Artificial intelligence (28%)


    MOOC providers in 2016

    According to Class Central a quarter of the new MOOC users  in 2016 came from regional MOOC providers such as  XuetangX (China) and Miríada X (Latin America).

    They list the top five MOOC providers by registered users:

    1. Coursera – 23 million
    2. edX – 10 million
    3. XuetangX – 6 million
    4. FutureLearn – 5.3 million
    5. Udacity – 4 million

    XuetangX burst onto this list making it the only non-English MOOC platform in top five.

    In 2016, 2,600+ new courses (vs. 1800 last year) were announced, taking the total number of courses to 6,850 from over 700 universities.


    Jobs in cyber security

    In a new fact sheet the Tech Partnership reveals that UK cyber workforce has grown by 160% in the five years to 2016. 58,000 people now work in cyber security, up from 22,000 in 2011, and they command an average salary of over £57,000 a year – 15% higher than tech specialists as a whole, and up 7% on last year. Just under half of the cyber workforce is employed in the digital industries, while banking accounts for one in five, and the public sector for 12%.


    Number students outside EU falls in UK

    Times Higher Education reports the number of first-year students from outside the European Union enrolling at UK universities fell by 1 per cent from 2014-15 to 2015-16, according to data released by the Higher Education Statistics Agency.

    Data from the past five years show which countries are sending fewer students to study in the UK.

    Despite a large increase in the number of students enrolling from China, a cohort that has grown by 12,500 since 2011-12, enrolments by students from India fell by 13,150 over the same period.

    Other notable changes include an increase in students from Hong Kong, Singapore and Malaysia and a fall in students from Saudi Arabia and Nigeria.


    Other Pontydysgu Spaces

    • Pontydysgu on the Web

      pbwiki
      Our Wikispace for teaching and learning
      Sounds of the Bazaar Radio LIVE
      Join our Sounds of the Bazaar Facebook goup. Just click on the logo above.

      We will be at Online Educa Berlin 2015. See the info above. The stream URL to play in your application is Stream URL or go to our new stream webpage here SoB Stream Page.

  • Twitter

  • Bundestagswahl 2017: Neues Deutschland - Kommentar - SPIEGEL ONLINE spiegel.de/politik/deutsc…

    About 36 minutes ago from Cristina Costa's Twitter via Twitter for iPad

  • Sounds of the Bazaar AudioBoo

  • Recent Posts

  • Archives

  • Meta

  • Upcoming Events

      There are no events.
  • Categories