GoogleTranslate Service


What is Evaluation 2.0?

October 4th, 2011 by Graham Attwell

Graham Attwell interviews Jenny Hughes about Evaluation 2.0

Just what is Evaluation 2.0?

Evaluation 2.0 is a set of ideas about evaluation that Pontydysgu are developing. At its simplest, it’s about using social software at all stages of the evaluation process in order to make evaluation more open, more transparent and more accessible to a wider range of stakeholders. At a theoretical level, we are trying to push forward and build on Guba and Lincoln’s ideas around 4th generation evaluation which is a constructivist approach incorporating key ideas around negotiation, multiple realities and stakeholder engagement. But this is the first part of the journey – ultimately, I believe that e-technologies are going to revolutionise the way we think about and practice evaluation.

In what way do you think this is going to happen?

Firstly, the use of social media gives stakeholders a real voice – irrespective of where they are located.  Stakeholders can create and publish evaluation content. For example, in the past I might carry out some interviews as part of an evaluation. Sometimes I recorded it, sometimes I just made notes. Then I would try and interpret it and draw some conclusions about what it meant.  Now I set up a web page for each evaluation and I podcast the interviews using audio or video and put them on the site. (Obviously this has to be negotiated with the interviewee but so far, no one has raised any objections.) There is the usual comment box so any stakeholder with access to the site can respond to the interview, add their interpretations, agree or disagree with my conclusions and so on.

Secondly, I think it is challenging our perceptions of who are evaluators.  Everyone is now an evaluator. Think of the software that you use every day for on-line shopping from Amazon or Ebay or any big chain store. If I want to buy a particular product I check out what other people have said about it, how many of them said it and how many stars it has been given. These are called recommender systems and I think they will have a big impact in evaluation. We have moved from the paradigm of the ‘expert’ collecting and analyzing data into a world of crowd sourcing – harnessing the potential of mass data collection and interpretation.

Thirdly, the explosion of Web 2.0 applications has provided us with a whole new range of evaluation tools that open up new methodological possibilities or make the old ones more efficient.  For example, if I am at the stage of formulating and refining the evaluation questions – I put it out as a call on Twitter.  It’s amazing how restricting evaluation questions to 140 characters can sharpen them up!

I did an evaluation of a community capacity building project in an inner city area recently and spent quite a long time before I went to the first meeting walking around the streets, checking out the community facilities, the state of the housing, local amenities and so on, to get a ‘feel’ for the area – except I did it on Google Earth and with street-view on Google maps.  There are about 20 or so other applications I use a lot in evaluation but maybe they will have to wait for another edition!

Fourthly, I think the potential of Web 2.0 changes the way we can visualize and present data.  Why are we still writing long and indigestible text-based evaluation reports? Increasingly clients are preferring short, sharp evaluation ‘articles’ on maybe one outcome of an evaluation which they can find on a ‘newsy’ evaluation webpage – with hyperlinks to more detailed information or raw data or back up evidence if they want to check it out.  We can also create ‘chunks’ of evaluation reporting and repurpose them in different ways for different stakeholders or they can be localized for different cultures – for example, I have started doing executive summaries as downloadable podcasts.  I think evaluation 2.0 is about creating a much wider range of evaluation products.

Following on from that, I think Evaluation 2.0 breaks down the formative-summative divide and notions of ‘the mid-term report’ or ‘the ex-ante report’.  Evaluation 2.0 is continuous, it is dynamic and it is interactive.  For example, I use Googledocs with all my clients – I add them as readers and editors on all the folders that relate to their evaluations. At any time of the day or night they can see work in progress and add their comments. I keep their evaluation website up to date so they get evaluation information as soon as it is available.

So do you think all evaluators will have to move down this road or will there always be a place for evaluators using more established methods?

Personally, I think massive change is inevitable. Apart from anything else, our clients of the future will be the digital natives – they will expect it.

There will always be a role for the evaluator but that role will be transformed and the skills will be different. I think a key job for the specialist evaluator will be designing the algorithms that underpin the evaluation. The evaluator will also need to be the creative director – they will need skills in informatics, in visualizing and presenting information, the creative skills to write blogs and wikis. They will need networking skills to set up and facilitate online communities of practice around different stakeholder groups and the ability to repurpose evaluation objects.

The rules of engagement are also changing – in the past you engaged with a client, now you engage with a community.  We also have to think how stakeholder created content might change our ideas about copyright, confidentiality, ownership, authorship.

So do you think evaluators as we know them will become extinct!!

Well, as Mark Halper said

“Dinosaurs were highly successful and lasted a long time. They never went away. They became smaller, faster, and more agile, and now we call them birds.”

Tweetbacks/Trackbacks/Pingbacks

  1. […] Bridge to Learning – Educational Research So do you think evaluators as we know them will become extinct!! Well, as Mark Halper said The rules of engagement are also changing – in the past you engaged with a client, now you engage with a community. We also have to think how stakeholder created content might change our ideas about copyright, confidentiality, ownership, authorship. […]

  • Search Pontydysgu.org

    News Bites

    Adult Education in Wales

    Learning and Work Institute is organising this year’s adult learning conference in partnership with the Adult Learning Partnership Wales. It will take place on Wednesday, 16 May 2018 at the Cardiff City Stadium.

    They say “Changing demographics and a changing economy requires us to re-think our approach to the delivery of learning and skills for adults. What works and what needs to change in terms of policy and practice?

    The conference will seek to debate how can we respond to need, grow participation, improve and measure outcomes for citizens, and revitalise community education.”


    Industry 4.0

    The UK Education Select Committee has launched an inquiry into the challenges posed and opportunities presented by the Fourth Industrial Revolution.The Committee is inviting written evidence on:

    • The interaction between the Government’s industrial, skills and digital strategies
    • The suitability of the current curriculum to prepare young people for the Fourth Industrial Revolution
    • The impact of the Fourth Industrial Revolution on the delivery of teaching and learning in schools and colleges
    • The role of lifelong learning in re-skilling the current workforce
    • Place-based strategies for education and skills provision; and
    • The challenges and opportunities of the Fourth Industrial Revolution for improving social justice and productivity

    The deadline for written submissions is Thursday 21 June 2018.


    Online Educa Berlin

    OEB Global (formerly Online Educa Berlin) has announced its Call for Proposals and the overall theme for 2018: Learning to Love Learning. The event will incorporate Learning Technologies Germany – a leading European exhibition on learning technologies in the workplace – for the first time this year. More details here.


    Barcelona to go Open Source

    The Spanish newspaper, El País, has reported that the City of Barcelona is in the process of migrating its computer system to Open Source technologies.

    According to the news report, the city plans to first replace all its user applications with alternative open source applications. This will go on until the only remaining proprietary software will be Windows where it will finally be replaced with a Linux distribution.

    To support the move, the city will employ 65 new developers to build software programs for their specific needs. they also plan the development of a digital market – an online platform – whereby small businesses will use to take part in public tenders.


    Other Pontydysgu Spaces

    • Pontydysgu on the Web

      pbwiki
      Our Wikispace for teaching and learning
      Sounds of the Bazaar Radio LIVE
      Join our Sounds of the Bazaar Facebook goup. Just click on the logo above.

      We will be at Online Educa Berlin 2015. See the info above. The stream URL to play in your application is Stream URL or go to our new stream webpage here SoB Stream Page.

  • Twitter

  • RT @NotRightRuth Zhang: Twitter is increasingly important for organising of LGBTQ clubs in Japanese universities. Majority of Twitter accounts explicitly invite interaction only from people who self-identify as LGBTQ, but others are open to allies etc. Some aren't explicit about focus. #isa18wcs

    About 6 hours ago from Cristina Costa's Twitter via TweetDeck

  • Sounds of the Bazaar AudioBoo

  • Recent Posts

  • Archives

  • Meta

  • Upcoming Events

      There are no events.
  • Categories