Evaluation 2.0 in Regional Policy evaluation
I don’t have any answers to this – in some ways, that’s not for me to decide! I have mostly used Evaluation 2.0 stuff in the evaluation of education projects not regional policy. And my recent experience of the Cohesion Fund, ERDF, IPA or any of the structural funds is minimal. However, the ideas are generic and if people think that there are some they could work with, that’s fine!

That said, here are some suggestions for moving things forward – some of them are mine, most have been mooted by various people who have come to talk to me today (and bought me lots of coffee!)

Suggestions for taking it forward

• Set up a twitter hashtag #evaluation2.0. Well that’s easy but I don’t know how much traffic there would be as yet!
• Set up a webpage providing information and discussion around Evaluation 2.0. More difficult – who does that and who keeps it updated? Maybe, instead, it is worth feeding in to the Evalsed site that DG Regio maintain, which currently provides information and support for their evaluators. I gather it is under the process of review – a good opportunity to make it more interactive, to make more use of multimedia and with space for users to create content as well as DG Regio!
• Form a small working group or interest group – this could be formal or informal, stand alone or tied to their existing evaluation network. Either way, it needs to be open and accessible to people who are interested in developing new ideas and trying some stuff out rather than a representative ‘committee’.
• Alternatively, set up an expert group to move some ideas forward.
• Or how about a Diigo group?
• Undertake some small-scale trials with specific tools – to see whether the ideas do cross over from the areas I work in to Regional Policy.
• Run a couple of one-day training events on Evaluation 2.0 focusing on some real hands-on workshops for evaluators and evaluation unit staff rather than just on information giving.
• Check out with people responsible for evaluation in other DGs whether there is an opportunity for some joint development (a novel idea!) Unlike other ‘perspectives’ it is not tied to content or any particular theoretical approach.
• Think about developing some mobile phone apps for evaluators and stakeholders around content specific issues – I can easily think of 5 or 6 possibilities to support both counterfactual, quantitative approaches and theory-based qualitative approaches. Although the ideas are generic, customizing the content means evaluators would have something concrete to work with rather than just ideas.
• Produce an easy-to-use handbook on evaluation 2.0 for evaluators / evaluation units who want practical information on how to do it.
• Ring fence a small amount of funding to support one-off explorations into innovative practice and new ideas around evaluation.
• Encourage the evaluation unit to demonstrate leadership in new approaches – for example, try streaming a live internet radio programme around the theme of evaluation (cheap and easy!); set up a multi-user blog for people to post work in progress and interesting observations of ongoing projects using a range of media as well as text-based major reports; make some podcasts of interviews with key players in the evaluation of Regional Policy; set up a wiki around evaluation rather than having to drill down through the various Commission websites; try locating projects using GPS data so that we can all see where the action is taking place! Keep a twitter stream going around questions and issues – make use of crowd sourcing!
• Advertise the next European Evaluation Society biennial conference, in Helsinki, October 1st – 5th 2012 “Evaluation in the networked society: New concepts, New challenges, New solutions” (There you go Bob, I just did!)
• Broaden the idea of Evaluation 2.0 and maybe get rid of the catchphrase! We are already using the power of the semantic web in evaluation to mash open and linked data, for example. Should we be now be talking about Evaluation 3.0?? Or should we find another name – Technology Enhanced Evaluation? We could have TEE parties instead of conferences – Europe’s answer to the American far right ; )

P.S. Message to the large numbers of English delegates at the conference

When you left Heathrow yesterday to come to Brussels, I do hope you waved to the English Rugby team arriving home from the Rugby World Cup in New Zealand.

(Just as well this conference was not a week later or I’d have leave a similar message for the French delegates…..)

The Open Badges framework, say the project developers, is designed to allow any learner to collect badges from multiple sites, tied to a single identity, and then share them out across various sites — from their personal blog or web site to social networking profiles. The infrastructure needs to be open to allow anyone to issue badges, and for each learner to carry the badges with them across the web and other contexts.

Now some of the issues. I am still concerned of attempts to establish taxonomies, be it those of hierarchy in terms of award structures or those of different forms of ability / competence / skill (pick your own terminology). Such undertakings have bedeviled attempts to introduce new forms of recognition and I worry that those coming more from the educational technology world may not realise the pitfalls of taxonomies and levels.

Secondly is the issue of credibility. There is a two fold danger here. One is that the badges will only be adopted for achievements in areas / subjects / domains presently outside ‘official’ accreditation schemes and thus will be marginalised. There is also a danger that in the desire to gain recognition, badges will be effectively benchmarked against present accreditation programmes (e.g. university modules / degrees) and thus become subject to all the existing restrictions of such accreditation.

And thirdly, as the project roils towards a full release, there may be pressures for restricting badge issuers to existing accreditation bodies, and concentrating on the technological infrastructure, rather than rethinking practices in assessment.

Lets look at some of the characteristics of any assessment system:

This symposium will examine the paradoxes of giving and receiving online in education in a changing economic climate.  Each of the panelists will briefly address topic areas within the symposium theme, followed by an opportunity for present and at distance audiences to contribute, concluding with a 25 minute plenary discussion.

Symposium delegates will be provoked to reconsider the costs of participation online by paid and unpaid participants in ‘open’ discussion and sharing of resources.

Open Educational Resources exist within communities that create, use and sustain them (Downes 2007). When ‘communities’ in Higher Education break down due to redundancy and casualisation of labour what happens to OERs? Are they sustained? Can they reach out to other contexts?

All areas of education, including the school sector, currently face significant financial challenges and uncertainties. Institutions are increasingly reviewing the provision of devices and services, and looking at learner owned devices and commercially owned ‘free’ cloud-based services. What is the real price of an education system supported and transformed by embedded learning technologies?

Ownership in the age of openness calls for clarity about mutual expectations between learners, communities and ourselves. Ideas and content are shared easily through open platforms, and yet attributions can be masked in the flow of dissemination: does credit always go where it is due?

Openness in the production, sharing and reuse of education/resources is meaningless in the face of neoliberalism. Where coercive competition forms a treadmill for the production of value, openness/OERs are commodified. Control of the educational means of production determines power to frame how open are the relations for the production or consumption of educational goods or services, in order to realise value. The totality of this need, elicited by the state for capital, rather than the rights of feepayers, parents, communities or academics, shapes how human values like openness are revealed and enabled within HE.

Scarce research monies focus attention on impact factors, arguably stagnating practice. For publications, Open Access can increase wider societal impact but at the expense of career progression.We explore the tensions, paradoxes and professional costs on societal benefits, individual agency and academic progression.

Obviously this is a bit of a mash up proposal but it does raise a lot of questions. I think there is a tension between the idea of communities and institutions. Communities of practice, and for that matter the communities in which Open education Resources are being produced and shared, cross institutional boundaries. Furthermore the use of OERs may be within an institutional setting but may also be outside.

This again is reflected in recognition and reward structures. Whilst reward structures within institutions are based on either monetary compensation or in terms of progression, rewards within the community may reflect a wider understanding of recognition, especially respect or standing within that community. Does credit always go to who it should? Probably not, but this is taking a very individualistic view of research. Surely credit should be shared in the community rather than in the closed door offices of academic researchers.

Are OERs being commodified? Presumably the term “coercive competition” refers to the growing practice to require academics and researchers to publish their work as OERs. I don’t really understand what the authors mean in saying “The totality of this need, elicited by the state for capital, rather than the rights of feepayers, parents, communities or academics, shapes how human values like openness are revealed and enabled within HE.” Of course the idea of openness is being hijacked by institutions. But at the same time the movement towards openness is contradictory and I am not sure this is reflected in the abstract. Especially missing is a discussion of the nature of OERs in allowing reuse and modification and the impact this has on (commodified) relations of production and intellectual property. And at the same time, the spread of OERs is allowing new open forms of learning and knowledge production outside the confines of the institution. thus whilst the movement towards OERs may be becoming commodified the use of OERs is challenging traditional understandings of those very commodities.

I don’t think this reveals a paradox but a dialectical contradiction. Present schooling models of education are being found to be wanting. The discussion about open education is important in that it could provide an alternative to privatisation. The discussion over OERs forms an important part of this debate and to this extent the debate over this symposium is extremely interesting.

Last September, together with Jenny Hughes I undertook a literature review on new pedagogical approaches to the use of technologies for teaching and learning. You can access the full (86 pages) document below.

The research was commissioned by LLUK to feed into the review then being undertaken of teaching qualifications in the Lifelong Learning sector in the UK. The review was designed to ensure the qualifications are up to date and will support the development of the skills needed by the modern teacher, tutor or trainer.

However, we recognised that the gap in technology related skills required by teaching and learning professionals cannot be bridged by qualifications alone or by initial training and a programme of opportunities for continuing professional development (CPD) is also needed to enable people to remain up to date.

The literature review is intended to

• identify new and emerging pedagogies;
• determine what constitutes effective use of technology in teaching and learning
• look at new developments in teacher training qualifications to ensure that they are at the cutting edge of learning theory and classroom practice
• make suggestions as to how teachers can continually update their skills.

Pedagogical Appraches for Using Technology Literature Review January 11 FINAL 1

What has been learned in this process?

Firstly the model of courses which are free to participants but charge for institutional enrollment and for certification appears to be gaining traction. How far this can go depends I guess on the extent that participation (and recording of work) becomes recognised as achievement. It will also depend on how much value universities and other institutions think they can gain (or stand to lose) through such a model.

Secondly most of these programmes are using all manner of social software and Open Source applications. There seems to be a growing practice of hanging programmes together around open webinars, with students using their own blogs or other social software for their personal work. One of the less successful experiments seems to be attempts to integrate VLEs, especially Moodle, within MOOCs. Participants are being encouraged to develop their own Personal Learning Environments as part of the process.

Thirdly such initiatives place great emphasis on peer support for learning, with a greater or lesser extent of formal learning support and formalization of networks. One greatly encouraging development is the blurring of the boundary between teachers and learners. Another is the involvement of people form different organisations in leading, facilitating or stewarding such programmes. Most stewards or facilitators are not being paid, although I suspect at present this is being accepted by institutions as a legitimate part of their work as researchers. Whatever, this is resulting in a weakening of institutional boundaries and the emergence of stronger communities of practice.

There also seems to be considerable pedagogic innovation, with a willingness to explore new ways of learning. Especially encouraging is the use of multi media, which although promised in so many formal elearning programmes, has seldom really happened.

Now comes the big question. Can the experience gained from the MOOCs be extended to provide a transferable and scalable model for Open Education.

I’ve already talked about the issue of recognition which I see not so much as a question of assessment but of social recognition of achievement. But there are other open issues. How do we deal with language barriers? More critically, most participants in the early MOOCs seem to be professionals, teachers and researchers already engaged in online learning or multi media and / or students. In other words, people with a fair degree of competence in communicating through on-line media. The model is based on a large degree of self motivation and is reliant on learners being able to manage both their own learning and able to develop their own support networks. This is a pretty big limitation.

I see two ways to deal with this. One is to provide more formal and institutional support through participation in MOOCs becoming part of courses on which learners are already enrolled and their host institution providing support. This idea is already being suggested for the Change: Education, Learning and Technology MOOC. The second is through developing more fomalised individual and group mentoring and support systems. At the moment, we are tending to focus on presenters as the key people in facilitating the online programmes. But such a second layer of mentors could play the critical role, and providing such mentoring could be a key part of Continuing Professional Development for teachers and trainers. In other words, a win, win situation.

I fond it difficult to decide the standards or criteria I am reviewing against. How important is clarity of thinking, originality, creativity? How important is it that the author includes copious references to previous work? Are we looking for depth or breadth? How important is the standard of English, particularly for those writing in a second or third language?

In this world of social software the whole review process seems somewhat archaic. It relies very much on individuals, all working in isolation. People write an abstract according to a call for proposals (and I am well aware of how difficult it is to write such calls – unless of course it is one of these multi track conferences which just include everything!). The reviews are allocated to a series of individuals for blind review. They do their work in isolation and then according to often subjective criteria, the proposal is accepted or rejected.

OK, sometimes there is the opportunity to make a conditional acceptance based on changes to the proposal. and of course, you are encouraged to provide feedback to the author. But all too often feedback is limited and pressure of time prevents organisers allowing a  conditional acceptance.

How could social software help with this? As usual I think it is a socio technical solution we need to look for, rather than an adoption of technologies per se. Most conferences have adopted software to help with the conference organising and review procedures but as happens all to often that software has been developed to manage existing processes more efficiently with no thought into how we could transform practices.

One big issue is the anonymity of the review procedure. I can see many reasons to support this, but it is a big barrier to providing support in improving submissions. If we move to non blind reviewing, then we could develop systems to support a discourse between submitters and reviewers, where both become part of the knowledge creation process. and in added benefit of such a discourse could be to clarify and make transparent the criteria being used for reviews. reviewers would have more of a role as mentors rather than assessors or gatekeepers.

This would not really require sophisticated technological development. It would really just need a simple booking system to arrange for a review and feedback session, together with video, audio or text conferencing functionality. More importantly perhaps it might help us in rethinking the role of individual and collective work in the academic and scholarly forms of publishing and knowledge development. I suspect a considerable barrier is the idea of the ‘Doctor Father’ – that such a process would challenge the authority of professors and doctorate supervisors. My experience, based on talking to many PhD students, is that the supervisory role does not work particularly well. It was developed when the principle role of universities was research and was designed to induct students into a community of practice as a researcher. With the changing role of universities plus the fact that many students are no longer committed to a long term career in academia (even if they could get a job) such processes have become less than functional. Better I think to develop processes of support based on wider communities than the narrow confines of a single university department.

I have been working on data for the last couple of weeks. I am trying to build a TEBO – a Technology Enhanced Boundary Object (or objects) for explaining Labour Market data to Careers Advice, Information and Guidance (CAIG). Together with my colleagues from the Institute for Employment Research at Warwick University, I have been looking at TEBOs for some time.

Alan Brown explains the conceptual idea behind TEBOs:

The ideas of boundary crossing and tool mediation (Tuomi-Gröhn & Engeström, 2003; Kaptelinin & Miettinen 2005) and situated learning with a close alignment to the importance of a focus upon practice (Brown et al., 1989; Hall, 1996) informed considerations of the role of technologically-enhanced boundary objects in knowledge maturing processes in different contexts. One specific concern is to make visible the epistemological role of symbolic boundary objects in situations in which people from different communities use common artefacts in communication. A fruitful approach to choosing ways to develop particular boundary objects is to focus on what Onstenk (1997) defines as core problems: the problems and dilemmas that are central to the practice of an occupation that have significance both for individual and organisational performance — in this case the problems associated with providing advice relevant for career planning. One method this development project used was therefore to engage in a dialogue with guidance practitioners about common scenarios involving Labour Market Information (LMI) which could inform the development of prototype technologically-enhanced boundary objects (TEBOs). The development … was therefore informed by a consideration of the following issues:

• Importance of developing methods and strategies for co-design with users
• Need for conceptual tools to help people understand the models and ideas which are part of LMI
• Need for a more open pedagogy (than is typical of much existing technology-enhanced learning, and existing workplace training practice)
• A system in which boundary objects are configurable by end-users (practitioners) and by guidance trainers to be used in multiple ways
• Need to build an understanding of how TEBOs may be used in ways that are empowering for practitioners, and ultimately for clients too.

These concerns could be coupled with another set of issues concerning appropriate skill development:

• Need for time for people to interact, reflect, use concepts etc.
• Trying to reach a stage where practitioners have justifiable confidence in the claims they make and can exercise judgement about the value of information when faced with unfamiliar LMI
• Choosing between a range of possible use-contexts
• Decide how to employ support from communication and discussion tools
• Developing and transmitting Labour Market intelligence – importance of communicating to others
• Preconfigure certain ways of thinking through use of scenarios; discussions can point into and lead from scenarios.

In practice it is not so easy to develop such TEBOs. Identifying key problmes is probably the most useful approach. But then there is an issue in accessing different data to visualise as part of the process. A great deal of data is now publicly available. But I am no data specialist and have faced a steep learning curve in understanding and interpreting the data myself. then there is the issue of visualisation – I am mainly using Google Gadgets, although we are also working with Tableau (a powerful tool, but unfortunately only available for Windows) and IBM;s Many Eyes. All these tools are good, but are all extremely finicky about how the data is formatted. We are working with data in xls and Apple’s Numbers but I suspect longer term it would be better to use the Open Source R programming environment.

And the hardest task of all is the storyboarding. At the end of the day we are trying to tell stories with data: TEBOs are a storytelling and exploration approach to learning. So for each TEBO I intend to make a short video explaining the key concepts and showing the various visualizations. We will also provide access to the raw data and to static versions of the graphing, along with explanatory notes. And for each TEBO we will try to construct an interactive visualisation tool, allowing learners to play with the data and displays. I also want to try to build some sort of simulations using the Forio tool. No doubt there is better software (and if anyone has any ideas I would be very grateful). But I sort of feel that the more social software, open source or free tools we can use the better. We want to encourage people to do it for themselves. And they have no money to spend on fancy software tools.We cannot possibly provide access to visualisations of all the data available. But if we cane explain what is possible, hopefully interested CAIG professionals will start there own work. And then who knows – a Careers Guidance data store?

Instead I see two trends. With applications like Facebook, or whatever succeeds it, friends will return to being friends. People we know, people we want to socialise with, be it family and friends we see regularly face to face or friends in distributed networks.

The second will be the growth of social networks based on shared interests and shared practice. Of course this is nothing new. The early days of the web spawned many wonderful bulletin boards with graphics being based on the imaginative use of different text and fonts. Ning led to the explosion of community sites whilst it remained free. But now we are seeing the evolution of free and open source software providing powerful tools for supporting interest and practice based communities.

Cloudworks, developed by the UK Open University has now released an installable version of their platform. Buddypress seems to have developed a vibrant open source community of developers.And I am greatly impressed by QSDA, the Open Source Question and Answer System. Quora is all the hype now. But like so many of these systems, it will be overrun not so much by machine driven spam, but by the lack of a  shared community and purpose.

According to Ettiene Wenger, a community of practice defines itself along three dimensions:

• What it is about – its joint enterprise as understood and continually renegotiated by its members.
• How it functions – mutual engagement that bind members together into a social entity.
• What capability it has produced – the shared repertoire of communal resources (routines, sensibilities, artefacts, vocabulary, styles, etc.) that members have developed over time.

Open Source networking tools can allow us to support that shared repertoire of communal resources. I am working on the development of open and linked data for careers guidance and counselling. it is a fairly steep learning curve for me in terms of understanding data. And one of the bests sites I have found is Tony Hirst’s Get the Data site, only launched a week ago and based on the QSDA software, but already providing a wealth if freely contributed ideas and knowledge.

it is this sort of development that seems to me to be the future for social networking.

The occupation of the Conservative Party headquarters in London was an impressive piece of collective action so to all those involved in the organisation and to all those that turned up on the day, a message of support from Pontydysgu!

However, it did make me wonder how we ever used to do all this without mobile phones, computers or social networking media. Apart from using print media, I seem to remember a lot of organising time spent in public telephone boxes pressing button A and button B. In fact, one of my early ICT competences was learning how to tap the receiver rest up and down to mimic the operation of the dial in order to save the 4d (less than 2p) it cost.

Well I logged in or rather pressed a button saying something like login with Twitter and there I was. No filling in forms or making up passwords. And there straight away was a message for me:

Hi there! we have had a look at conferences your friends on twitter are going to, perhaps you might like to go too.

Yesterday, together with my colleague Jenny Hughes, I made a presentation to participants in the Critical Literacies course being run by Rita Kop and Stephen Downes as part of their ongoing research project on Personal Learning Environments.

The course blog says: “Technology has brought changes to the way people learn and some “critical literacies” are becoming increasingly important. This course is about these critical literacies. Critical, as the course is not just about finding out how to use the latest technologies for learning, but to look critically at the Web and its underlying structures. Literacies, as it is more about capabilities to be developed than about the acquisition of a set of skills. It is all about learning what is needed to develop confidence and competence, and to feel capable of negotiating an ever changing information and media landscape.”

Our presentation was on pragmatics. Pragmatics, we said is a sub field of linguistics which studies the ways in which context contributes to meaning.

Today we have made a short version of the presentation as a slidecast. In the presentation we explore different ideas about context in education. In the final part of the presentation we look at Personal Learning Environments and how they relate to issues of meaning and context.

The introductory and end music is from an album called Earth by zero-project. it can be downloaded from the excellent Jamendo web site.

Web 2.0 and Bricolage

Web 2.0 applications and social software mark a change in our use of computers from consumption to creation. A series of studies and reports have provided rich evidence of the ways young people are using technology and the internet for socialising, communicating and for learning. Young people are increasingly using technology for creating and sharing multi media objects and for social networking. A Pew Research study (Lenhart and Madden, 2005) found that 56 per cent of young people in America were using computers for ‘creative activities, writing and posting of the internet, mixing and constructing multimedia and developing their own content. Twelve to 17-year-olds look to web tools to share what they think and do online. One in five who use the net said they used other people’s images, audio or text to help make their own creations. According to Raine (BBC, 2005), “These teens were born into a digital world where they expect to be able to create, consume, remix, and share material with each other and lots of strangers.”

Such a process of creation, remixing and sharing is similar to Levi Struass’s idea of bricolage as a functioning of the logic of the concrete. In their book ‘Introducing Levi Strauus and Structural Anthropology’, Boris Wiseman and Judy Groves explains the work of the bricoleur:

“Unlike the engineer who creates specialised tools and materials for each new project that he embarks upon, the bricoleur work with materials that are always second hand.

In as much as he must make do with whatever is at hand, an element of chance always enters into the work of the bricoleur……

The bricoleur is in possession of a stock of objects (a “treasure”). These possess “meaning” in as much as they are bound together by a set of possible relationships, one of which is concretized by the bricoleur’s choice”.

Young people today are collecting their treasure to make their own meanings of objects they discover on the web. In contrast our education systems are based on specialised tools and materials.

Social networking

It is not only young people who are using social networks for communication, content sharing and learning. A further survey by Pew Internet (Lenhart, 2009) on adults use of social networking sites found:

• 79% of American adults used the internet in 2009, up from 67% in Feb. 2005
• 46% of online American adults 18 and older use a social networking site like MySpace, Facebook or LinkedIn, up from 8% in February 2005.
• 65% of teens 12-17 use online social networks as of Feb 2008, up from 58% in 2007 and 55% in 2006.
• As of August 2009, Facebook was the most popular online social network for American adults 18 and older.
• 10-12% are on “other” sites like Bebo, Last.FM, Digg, Blackplanet, Orkut, Hi5 and Match.com?

Lest this be thought to be a north American phenomena, Ewan McIntosh (2008) has provided a summary of a series of studies undertaken in the UK (Ofcom Social Networking Research, the Oxford Internet Institute’s Internet Surveys, Ofcom Media Literacy Audit).

The main use of the internet by young people, by far, is for learning: 57% use the net for homework, saying it provides more information than books. 15% use it for learning that is not ’school’. 40% use it to stay in touch with friends, 9% for entertainment such as YouTube.

Most users of the net are using it at home (94%), then at work (34%), another persons house (30%) or at school (16%). Only 12% use public libraries and 9% internet cafés. Most people’s first exposure to the web is at home.

A further survey into the use of technology for learning in Small and Medium Enterprises found few instances of the use of formal educational technologies (Attwell, 2007). But the study found the widespread everyday use of internet technologies for informal learning, utilizing a wide range of business and social software applications. This finding is confirmed by a recent study on the adoption of social networking in the workplace and Enterprise 2.0 (Oliver Young G, 2009). The study found almost two-thirds of those responding (65%) said that social networks had increased either their efficiency at work, or the efficiency of their colleagues. 63% of respondents who said that using them had enabled them to do something that they hadn’t been able to do before. The survey of based on 2500 interviews in five European countries found the following percentage of respondents reported adoption of social networks in the workplace:

• Germany – 72%
• Netherlands – 67%
• Belgium – 65%
• France – 62%
• UK – 59%

Of course such studies beg the question of the nature and purpose of the use of social software in the workplace. The findings of the ICT and SME project, which was based on 106 case studies in six European countries (Attwell, 2007) focused on the use of technologies for informal learning. The study suggested that although social software was used for information seeking and for social and communication purposes it was also being widely used for informal learning. In such a context:

• Learning takes place in response to problems or issues or is driven by the interests of the learner
• Learning is sequenced by the learner
• Learning is episodic
• Learning is controlled by the learner in terms of pace and time
• Learning is heavily contextual in terms of time, place and use
• Learning is cross disciplinary or cross subject
• Learning is interactive with practice
• Learning builds on often idiosyncratic and personal knowledge bases
• Learning takes place in communities of practice

It is also worth considering the growing use of mobile devices. A recent Pew Internet survey (Lenhart et al, 2010) found that of the 75% of teens who own cell phones in the USA, 87% use text messaging at least occasionally. Among those texters:

• Half of teens send 50 or more text messages a day, or 1,500 texts a month, and one in three send more than 100 texts a day, or more than 3,000 texts a month.
• 15% of teens who are texters send more than 200 texts a day, or more than 6,000 texts a month.
• Boys typically send and receive 30 texts a day; girls typically send and receive 80 messages per day.
• Teen texters ages 12-13 typically send and receive 20 texts a day.
• 14-17 year-old texters typically send and receive 60 text messages a day.
• Older girls who text are the most active, with 14-17 year-old girls typically sending 100 or more messages a day or more than 3,000 texts a month
• However, while many teens are avid texters, a substantial minority are not. One-fifth of teen texters (22%) send and receive just 1-10 texts a day or 30-300 a month.

Once more, of those who owned mobile phones:

• 83% use their phones to take pictures.
• 64% share pictures with others.
• 60% play music on their phones.
• 46% play games on their phones.
• 32% exchange videos on their phones.
• 31% exchange instant messages on their phones.
• 27% go online for general purposes on their phones.
• 23% access social network sites on their phones.
• 21% use email on their phones.
• 11% purchase things via their phones.

It is not just the material and functional character of the technologies which is important but the potential of the use of mobile devices to contribute to a new “participatory culture” (Jenkins at al). Jenkins at al define such a culture as one “with relatively low barriers to artistic expression and civic engagement, strong support for creating and sharing one’s creations, and some type of informal mentorship whereby what is known by the most experienced is passed along to novices… Participatory culture is emerging as the culture absorbs and responds to the explosion of new media technologies that make it possible for average consumers to archive, annotate, appropriate, and recirculate media content in powerful new ways.”

Thus we can see the ways in which technology and the internet is being used for constructing knowledge and meaning through bricolage and through developing and sharing content. This takes place through extended social networks which both serve for staying in touch with friends but also for seeking information and for learning in a participatory culture.

Having said that – in terms of format in seems to me that some of the submissions as full conference papers would have been better made under other formats. However, present university funding requirements demand full papers and inhibit applications for work in progress or developing ideas in more appropriate formats.

For the last two weeks I have been organising the review process. We promised that each submission would be blind reviewed by at least two reviewers. For this we are reliant on the freely given time and energy of our Academic Committee. And whilst reviewing can be a learning process in itself it is time consuming.

Submissions have been managed through th open source Easychair system, hosted by the University of Manchester. The system is powerful, but the interfaces are far from transparent and the help somewhat minimalist! I have struggled to get the settings in the system right and some functions seem buggy – for instance the function to show missing reviews seems not to be working.

Two lessons for the future seem immediately apparent. Firstly, we set the length of abstracts as a maximum of 350 words. Many of the reviewers have commented that this is too short to judge the quality of the submission.

Secondly is the fraught issue of criteria for the reviews. We produced detailed guidelines for submissions based on the Creative Commons licensed Alt-C guidelines.

The criteria were:

• Relevance to the themes of the conference although this does not exclude other high quality proposals.
• Contribution to scholarship and research into the use of PLEs for learning.
• Reference to the characteristics and needs of learners.
• Contribution to the development of learning technology policy or theory in education.
• Links that are made between theory, evidence and practice.
• Appropriate reflection and evaluation.
• Clarity and coherence.
• Usefulness to conference participants.

However, when I sent out the papers for review, whilst I provided a link to those guidelines, I failed to copy them into the text of the emails asking for reviews. In retrospect, I should have attempted to produce a review template in EasyChair incorporating the guidelines.

Even with such explicit guidelines, there is considerable room for different interpretation by reviewers. I am not sure that in our community we have a common understanding of what might be relevant to the themes of the conference or a contribution to scholarship and research into the use of PLEs for learning. I suspect this is the same for many conferences: however, the issue may be more problematic in an emergent area of education and technology practice.

We also set a scale for scoring proposals:

• 3 – strong accept
• 2 – accept
• 1- weak accept
• 0 – borderline
• -1 – week reject
• -2 – reject
• - 3 – reject

In addition we asked reviewers to state their degree of confidence in their review ranging from 4, expert, to 0, null.

In over half the cases where we have received two reviews, the variation between the reviewers is no more that 1. But there are also a number of reviews with significant variation. This suggest significant differences in understandings by reviewers of the criteria – or the meaning of the criteria. it could also just be that different reviewers have different standards.

In any case, we will organise a further review procedure for those submissions where there are significant differences. But I wonder if the scoring process is the best approach. To have no scoring seems to be a way fo avoiding the issue. I wonder if we should have scoring for each criteria, although this would make the review process even more complicated.

I would welcome any comments on this. Whilst too late for this conference, as a community we are reliant on peer review as a quality process and collective learning and reflection may be a way of improving our work.

Photo: James F. Clay

Here is the second in my series on informal learning. This is an extract from a a paper called ‘Rediscovering Apprenticeship?: A Historical Approach’ which I wrote in 1997. The paper, based on an interview with my father, provides a narrative account of an apprenticeship as a coach fitter in the Great Western Railway works in Swindon, England in the 1940s. The full paper can be downloaded at the bottom of this post.

“Apprentices were moved around between the gangs, usually spending about three months with a particular gang before the foreman would move them on. “Although it was not hard and fast we were moved in a fairly organised way. Apprentices always started on No 1 gang, which was a light job based in a siding outside the shop where we had to refurbish drop light windows. This meant removing the drop lights and the mouldings, which would be repaired inside, and then re-glazing the windows prior to fitting them back in the coach. After three months we were moved to a gang that undertook more complex work. The work got a little more complex with each move.

The chargeman for each gang was responsible for telling us what to do. Inside No. 7 shop we were mainly working on a bench. The men working on the next bench would show us how to do each job. When we were working outside the shop we were put with an individual tradesman who would teach us the job. Obviously some were better than others were”.

There was no written curriculum or even a list of skills or tasks that had to be learnt. “What we learnt depended totally on what a particular shop did. In fact our work was similar to a cabinetmaker. We were expected to achieve a tip-top finish. When we were working on the drop lights we would spend a whole day just sand papering – and then often the chargehand would make us do them all again. Time was not a problem – the question was quality”.

There were no written plans or procedures – learning was from practice. “Later I was sent ‘up the line’ to work outside the shop with the door gang. That was where I got my first interest in crossword puzzles. The tradesman was called Ted Quinn”. The door gang was responsible for hanging the interior doors in the carriages. “There is an art to hanging doors and making them slide – a knack to it. We had to screw a quarter inch rod with brackets above the door. The doors hang and rolled along the rod on wheels. If they were too high they would lift off the bottom guide rail, if they were too low they would stick or come off the rollers. Ted Quinn would get it right every time. He was fast enough that he would do a little work, then settle back to his crossword puzzles. If an apprentice mastered the skill he would be kept on the gang but some could never get the knack of it”.

New work, rather than repair and renovation, was highly regarded, mainly because it paid more. One of the best jobs was fitting the interior of compartments. This involved erecting the seats, interior panelling, mirrors, and putting up the net and blinds. Tradesmen would aim to complete one compartment each day. “Apprentices could be seen as a hindrance in this work – if you were not good you could slow the job down.” There were a number of specific skills to be mastered: “The best tradesman was a Hector Neaves. Everyone knew him as ‘one cut Neaves’. He would look at something and then cut a piece of wood which would fit first time nine times out of ten”.

There were no formal tests or assessment, neither was there any requirement to attend school. Those that did go to night-school could study for a National Diploma. This offered the opportunity on completion of apprenticeship to transfer to the Drawing Office, a position that was highly paid and the highest status. Many of the workers in the Drawing Office had passed their 11 plus and stayed on at school until the age of 16 prior to entering an apprenticeship. Few working class students went on to university. Other ‘grammar school boys’ joined the railway as clerks, a position which paid better and where they could “wear clean clothes”. Clerks also worked only 44 hours a week compared with 50 hours for tradesmen and labourers.”

Download the full paper here: apprenticeship_paper

Lets start with the old problems of Virtual Learning Environments – yes one problem is that they are not learning environments (in the sense of an active learning process taking place – but rather learning management systems. VLEs are great for enrolling and managing learners, tracking progress and completion and for providing access to learning materials. But the learning most often takes place outside the VLE with the VLE acting as a place to access activities to be undertaken and to report on the results. In terms of social learning, groups are usually organised around classes or assignments.

The idea of Personal Learning environments recognised three significant changes:

• The first was that of a Personal Learning Network which could be distributed and was not limited by institutional groups
• The second was the idea that learning could take place in multiple environments and that a PLE could reflect and build on all learning, regardless of whether it contributed to a course the user was enrolled on
• The third is that learners could use their own tools for learning and indeed those tools, be they online journals and repositries, networks or authoring tools, might also be distributed.

Then lest throw social software and Web 2.0 into the mix. This led to accordances for not just consuming learning through the internet, but for active construction and sharing.

This leads to a series of questions in developing both pedagogies and tools to support (social) learning (in no particular order):

• How to support students in selecting appropriate tools to support their learning?
• How to support students in finding resources and people to support their learning?
• How to support students in reporting or representing their learning?
• How to support students in identifying and exploring a body of knowledge?
• How to motivate and support students in progressing their learning?
• How can informal learning be facilitated and used within formal course outcomes?

How can we reconcile learning through communities of practice (and distributed personal learning networks) with the requirements of formal courses?

I am not convinced those of us who advocate the development of Personal Learning Environments have adequately answered those questions. It is easy to say we need changes in the education systems (and of course we do).

In one sense I think we have failed to recognise the critical role that teachers play in the learning process. Letsg o back to to Vykotsky. Vykotsky called those teachers – or peers – who supported learning in a Zone of Proximal Development as the More Knowledgeable Other. “The MKO is anyone who has a better understanding or a higher ability level than the leaner particularly in regards to a specific task, concept or process. Traditionally the MKO is thought of as a teacher, an older adult or a peer” (Dahms et al, 2007).

But the MKO can also be viewed as a learning object or social software which embodies and mediates learning at higher levels of knowledge about the topic being learned than the learner presently possesses.

Of course learners operate within constraints provided in part by the more capable participants (be it a teacher peer, or software), but an essential aspect of this process is that they must be able to use words and other artefacts in ways that extend beyond their current understanding of them, thereby coordinating with possible future forms of action.

Thus teachers or peers as well as technology play a role in mediating learning.

In terms of developing technology, we need to develop applications which facilitate that process of mediation. Some social software works well for this. If I get stuck on a problem I can skype a friend or shout out on Twitter, There is plenty of evidenced use of Facebook study groups. Yet I am not sure the pedagogic processes and the technology are sufficiently joined up. If I learn from a friend or peer, and use that learning in my practice, how does the process become transparent – both to myself and to others. How can I represent by changing knowledge base (through DIIGO bookmarks, through this blog?). And how can others understand the ideas I am working on and become involved in a social learning process.

I guess the answer lies in the further development of semantic applications which are able to make those links and make such processes transparent. But this requires far greater sophistication than we have yet achieved in developing and understanding Personal Learning Environments,

The summer school is being organised by the Stellar network and proposals were asked to explain how they contribute to the network’s three Grand Challenges:

• Connecting learners
• Orchestrating learning
• Contextualising learning environments

So here’s my proposal. I enjoyed writing it and if anyone else is interested in us running such a workshop juts get in touch.

Short description

The workshop will focus on the use of internet radio in education.

1) An exploration of the use of media (and particularly internet radio and television) for learning and shared knowledge developmentThis will include looking at issues such as:

a) The appropriation of media

b) The change from passive media to interactive Web 2.0 supported media and the changing distinctions between broadcaster/program planner and listener/consumer.

c) How media such as radio can support the development of online communities

d) The use of media to bridge contexts and provide spaces for exploration and shared meaning making.

2) A practical hands on session on how to plan develop and broadcast live internet media. This will include storyboarding, interviewing, finding Creative Commons licensed music, making jingles, mixing and post processing, directing and producing and using the technology for live broadcasts.

3) The third session is planned to take place in a lunchtime or evening session. This will be a live 45 minute to one hour broadcast “Sounds of the Bazaar – Live from Ohrid”. It is hoped to involve all summer school participants in the broadcast. The broadcast will be publicised in advance through iTunes, Facebook, Twitter and other social software platforms. It is also intended to use the boradcast to link to other researchers in TEL from around the world not able to be at the summer school. The programme will be recorded and made available through the Summer School web site, the Mature project web site, the Pontydysgu web site and through iTunes.

Contribution to the Grand Challenges agenda

The workshop is primarily designed to contribute to the Grand Challenge of Contextualising virtual learning environments and instrumentalising learning contexts.

Live internet radio provides both a shared context and space for learning, with universal reach outside of institutional or national boundaries, whilst at the same time allowing individual to collectively contribute to the development of shared artefacts, which in themselves can become part of the repertoire of a community of practice. Radio also offers a means of actively engaging learners in a community and through appropriation of what was a push (or broadcast) media, through merging with Web 2.0 tools and standards allows community participation and self expression.

James Turners says:

“According to Hoover, one way to ease the transition into real life development is to use an apprenticeship model. His book draws on his own experience moving from being a psychologist to a developer, and the lessons he’s learned running an apprenticeship program at a company called Obtiva. “We have an apprenticeship program that takes in fairly newcomers to software development, and we have a fairly loose, fairly unstructured program that gets them up to speed pretty quickly. And we try to find people that are high-potential, low credential people, that are passionate and excited about software development and that works out pretty well.”

Hoover bases his approach to apprenticeship on Vykotsky’s idea of a Significant Other Person who he describes as a mentor.

“For people that had had successful careers, they only point back to one or two people that mentored them for a certain amount of time, a significant amount of time, a month, two months, a year in their careers.”

He also points to the potential of a distributed community of practice for personal learning, including finding mentors outside a company the ‘apprentice’ is employed in.

For me personally, I wasn’t able to find a mentor at my company. I was in a company that didn’t really have that many people who were actually passionate about technology and that was hard for me. So what I did is I went to a user group, a local Agile user group or you could go to a Ruby user group or a .net user group, whatever it is and find people that are passionate about it and have been doing it for a long time. I’ve heard several instances of people seeking out to be mentored by the leader, for me that was the case. One of our perspective apprentices right now was mentored by the leader of a local Ruby user group. And that doesn’t necessarily mean you’re working for the person, but you’re seeking them out and maybe you’re just, “Hey, can you have lunch with me every week or breakfast with me every other week.” Even maybe just talking, maybe not even pairing. But just getting exposure to people that have been far on the path ahead of you, to just glean off their insights.

And he points out the value of being that Significant Other Person to those providing the mentoring.

At a certain point in your career, your priorities shift from learning being the most important thing, to delivering software is the most important thing, then mentoring becomes part of your responsibilities. It’s something you take on if you’re following the craftsmanship mentality of apprentice to journeyman to master. And transitioning from apprentice to journeyman, part of that is taking on more responsibility for projects and taking on more responsibility for mentoring.

Although there is no explicit reference to Vygotsky in James Taylor’s review of Hoover’s book, the Top five Tips for Apprentices correspond to Vygotsky’s model of learning through a Zone of Proximal Development.

1. Understanding where you’re at.
2. Find mentors who are ahead of you in the field
3. Find some peers to network with.
4. Perpetual learning.
5. Setting aside time to practice

I haven’t read the book but intend to. It is rare to find an such a model for learning in an advanced knowledge based industry like computing. And the drawing of parallels with the craft tradition of apprenticeship provides a potential rich idea for how learning can be organised in today’s society

I have never enjoyed reviewing papers. I worry that my own knowledge of the subject is often too little, still more that I only have an abstract idea of what comprises quality.

However, as a community building process, I find it more interesting. I haver been involved with VETNET for fourteen years. In the early days nearly everything used to be accepted. But as time went on a discussion emerged over improving the quality of VETNET and a formal review procedure was developed.

VETNET remains a somewhat traditional academic conference with paper and sumposium presentations. I suspect that the community’s desire for Vocational Education and Training to be taken seriously as a part of mainstream education research has tended to make us somewhat conservative in our approaches to formats and quality.

Over time as a community we have started defining quality indicators – even though they may be contested. We have had long debates about the relation between research focused on a particular system or country and its relation to wider European agendas. We have discussed how important the quality of language (English) is in assessing a contribution? Should leeway be given to emerging researchers to encourage them to contribute to the community? How important is a clear methodology when considering a submission?

This years debate has been over work in progress. It started innocuously enough with one reviewer emailing that he was concerned that many submissions referred to research which was not yet finished. Should we only consider completed research with clear results, he suggested? This provoked a flurry of replies with major differences between the reviewers. Some agreed with the original email; others (including myself) saw presentations based on work in progress as a potentially useful contribution to the community and a means of researchers testing their ideas in front of a wider international audience. In the normal way of things this debate will be reviewed at the VETNET board meeting at this years conference and revised guidelines agreed for next years conference.

In this way I think the review process does work well. It allows community rules and standards to emerge over time.

The other big change in the review system has been the use of an electronic reviewing system ‘conftool‘. The major benefit is to support the management fo the review process. VETNET receives some 120 proposals each year. The use of the system ensures every paper receives at least two reviews. More interestingly it makes transparent where there are disagreements between reviewers, providing a view showing the overall score for each proposal and the span between reviewer’s scoring. I was allocated nine proposals to review. Four of them have already been reviewed by a second reviewer. And somewhat to my surprise the span between my score and the other reviewers was small (the highest of the four was 1.7 (however the other reviewer has recommended rejection of this proposal and I have recommended acceptance!).

I welcome that when we have finished our reviews we are able to see other reviews of the same submission. This provides for me an opportunity for reflection and learning – and strengthens the potential of the academic review  becoming part of the process of community emergence.