GoogleTranslate Service


#PLE2010 update – the outcomes of the review process

May 2nd, 2010 by Graham Attwell

A further update on planning and preparations for the PLE2010 conference. We received 81 proposals, far more than we had expected. And whilst very welcome, this has generated a lot fo work. Each proposal was assigned two reviewers from the conference Academic Committee. This has meant some members of the Committee being asked to review six papers which is quite an effort for which we are truly grateful

One of the main points made in feedback to us from the reviewers was that a 360 word abstract is too short to make a proper judgement. And indeed some submissions did not make full use of the 360 words. We produced criteria for the submissions which were used by some reviewers. Others disagreed with this approach. Stephen Downes, commenting on my last blog post about the conference, said:

  • the stated criteria, as listed in the post above, are actually longer than many of the abstract submissions. As such, the criteria were overkill for what was actually being evaluated.
  • the criteria do not reflect academic merit. They are more like a check-off list that a a non-skilled intake worker could complete. The purpose of having academics do the review is that the academics can evaluate the work on its own merit, not against a check-off list.
  • the criteria reflect a specific theoretical perspective on the subject matter which is at odds with the subject matter. They reflect an instructivist perspective, and a theory-based (universalists, abstractivist) perspective. Personal learning environments are exactly the opposite of that.
  • In other words, it is not appropriate to ask academic reviewers to bring their expertise the material, and to then neuter that expertise with overly perspective statement of criteria.

On the whole I think I agree with Stephen. But I am still concerned with how we reach some common understandings or standards for reviewing, especially in a multi-disciplinary and multi national context.

Following the completion of the reviews, the conference organising committee met (via Skype) to discuss the outcomes of the process. We did not have time to properly consider the results of all 166 reviews and in the end accepted to unconditionally accept any paper with an average score of two or more (reviewers were asked to score each submission on a scale ranging from plus to minus three). That accounted for twenty six of the proposals. Each of the remaining proposals was reconsidered by the seven members of the organising committee in the light of the feedback from the reviewers. In many of the  cases we agreed with their reviews, in some cases we did not. 30 of the proposals were accepted but we have asked the proposers to resubmit their abstract, feeling that improvements could be made in clarity and in explaining their ideas to potential participants at the conference.

We referred nine of the proposals, in the main case because whilst they seemed interesting proposals we did not feel they has sufficiently addressed the theme of the conference ie Personal Learning Environments. These we have asked to resubmit the abstract and we will review the proposals for a second time. In a small number of cases we have recommended a change of format, particularly for research which is still at a conceptual stage which we felt would be better presented as a short paper, rather than a full proceedings paper. And, following the reviews, we did not accept five of the proposals. Once more the main reason was their failing to9 address the themes of the conference.

I am sure we will have upset some people through this process. But the review process was if nothing else rigorous. the meeting to discuss the outcome lasted late into the evening the we were concerned wherever possible to be inclusive in our approach. We also decided not to use the automatic functionality of the EasyChair system for providing feedback on the proposals. the main reason for this was that we were very concerned that feedback should be helpful and constructive for all proposers. Whilst many of the reviews were very helpful in that respect, some were less so and thus we have edited those reviews.

Four quick thoughts on all this:

  • I am not sure that people spend enough time thinking about the calls for papers. What are the themes a conference is trying to address? How does my work contribute towards those themes.
  • I wonder if many academics struggle with writing abstracts. I was surprised how many did not use their full 360 words in their proposals. Abstracts are difficult to write (at least I find them hard) and perhaps our 360 word limit constrained many. However, it was surprising how many were not really clear in focus.
  • I am still concerned with how we can develop common understandings and standards between reviewers. Maybe we need some sort of discourse process between reviewers.
  • The task of providing clear feedback and judgement about proposals whilst still proving constructive and helpful feedback to proposers is not easy. Once more, this maybe something which needs to be addressed at a community level.

Tweetbacks/Trackbacks/Pingbacks

  1. […] PLE Konferenz in Barcelona – die Vorbereitungen Mai 2, 2010 Tags: BYOL, geotagging, google maps, MapaList, pechakucha, personal learning environment, PLE, PLE Konferenz, PLE2010, PLE_BCN, The PLE Conference by Ilona Buchem Die Vorbereitungen für die PLE Konferenz in Barcelona laufen auf vollen Touren. Die wichtigste Aufgabe der letzten Wochen war sicherlich der Review-Prozess. Mit den 81 Einreichungen aller Art, angefangen mit traditionellen Lang- und Kurzbeiträgen, über Demonstrationen, Poster, Workshops bis hin zu Round Tables, BYOL (Bring Your Own Laptop) und Learning Cafes,  waren sowohl die Mitglieder des Programmkomitees als auch das Orga-Team gut beschäftigt. Ich habe gerade mit MapAList eine Übersicht erstellt, woher überall aus der Welt die Beiträge angekommen sind. Es ist unglabublich, aber war, die erste PLE Konferenz hat in der kurzen Zeit eine globale Reichweite erreicht Mehr zum Review-Prozess direkt bei Graham Attwell. […]

  • Search Pontydysgu.org

    News Bites

    Erasmus+

    The European Commission has published an annual report of the Erasmus+ programme in 2018. During that time the programme funded more than 23,500 projects and supported the mobility of over 850,00 students, of which 28,247 were involved in UK higher education projects, though only one third of these were UK students studying abroad while the remainder were EU students studying in the UK. The UK also sent 3,439 HE staff to teach or train abroad and received 4,970 staff from elsewhere in the EU.


    Skills Gaps

    A new report by the Learning and Work Institute for the Local Government Association (LGA) finds that by 2030 there could be a deficit of 2.5 million highly-skilled workers. The report, Local Skills Deficits and Spare Capacity, models potential skills gaps in eight English localities, and forecasts an oversupply of low- and intermediate -skilled workers by 2030. The LGA is calling on the government to devolve the various national skills, retraining and employment schemes to local areas. (via WONKHE)


    Innovation is male dominated?

    Times Higher Education reports that in the UK only one in 10 university spin-out companies has a female founder, analysis suggests. And these companies are much less likely to attract investment too, raising concerns that innovation is becoming too male-dominated.


    Open Educational Resources

    BYU researcher John Hilton has published a new study on OER, student efficacy, and user perceptions – a synthesis of research published between 2015 and 2018. Looking at sixteen efficacy and twenty perception studies involving over 120,000 students or faculty, the study’s results suggest that students achieve the same or better learning outcomes when using OER while saving a significant amount of money, and that the majority of faculty and students who’ve used OER had a positive experience and would do so again.


    Other Pontydysgu Spaces

    • Pontydysgu on the Web

      pbwiki
      Our Wikispace for teaching and learning
      Sounds of the Bazaar Radio LIVE
      Join our Sounds of the Bazaar Facebook goup. Just click on the logo above.

      We will be at Online Educa Berlin 2015. See the info above. The stream URL to play in your application is Stream URL or go to our new stream webpage here SoB Stream Page.

  • Twitter

  • @HMRCcustomers just got an automated phone call *allegedly* from you saying my name is being used in some scam/fraud and that I should press 1 on my phone. I'm a sure I am not the only one reporting this, but just in case ...

    About 2 days ago from Cristina Costa's Twitter via TweetDeck

  • Sounds of the Bazaar AudioBoo

  • Recent Posts

  • Archives

  • Meta

  • Upcoming Events