Pontydysgu – Bridge to Learning

Have been in Brussels for the last two days – speaking at 9th European Week of Regions and Cities organized by DG Regio and also taking the opportunity to join other sessions. My topic was Evaluation 2.0. Very encouraged by the positive feedback I’ve been getting all day both face-to-face and through twitter. I thought people would be generally resistant to the idea as it was fairly hard-hitting (and in fairness, some were horrified!) but far more have been interested and very positive, including quite a lot of Commission staff. However, the question now being asked by a number of them of them is “How do we progress this?” – meaning, specifically, in the context of the evaluation of Regional Policy and DG Regio intervention.

Evaluation 2.0 in Regional Policy evaluation
I don’t have any answers to this – in some ways, that’s not for me to decide! I have mostly used Evaluation 2.0 stuff in the evaluation of education projects not regional policy. And my recent experience of the Cohesion Fund, ERDF, IPA or any of the structural funds is minimal. However, the ideas are generic and if people think that there are some they could work with, that’s fine!

That said, here are some suggestions for moving things forward – some of them are mine, most have been mooted by various people who have come to talk to me today (and bought me lots of coffee!)

Suggestions for taking it forward

• Set up a twitter hashtag #evaluation2.0. Well that’s easy but I don’t know how much traffic there would be as yet!
• Set up a webpage providing information and discussion around Evaluation 2.0. More difficult – who does that and who keeps it updated? Maybe, instead, it is worth feeding in to the Evalsed site that DG Regio maintain, which currently provides information and support for their evaluators. I gather it is under the process of review – a good opportunity to make it more interactive, to make more use of multimedia and with space for users to create content as well as DG Regio!
• Form a small working group or interest group – this could be formal or informal, stand alone or tied to their existing evaluation network. Either way, it needs to be open and accessible to people who are interested in developing new ideas and trying some stuff out rather than a representative ‘committee’.
• Alternatively, set up an expert group to move some ideas forward.
• Or how about a Diigo group?
• Undertake some small-scale trials with specific tools – to see whether the ideas do cross over from the areas I work in to Regional Policy.
• Run a couple of one-day training events on Evaluation 2.0 focusing on some real hands-on workshops for evaluators and evaluation unit staff rather than just on information giving.
• Check out with people responsible for evaluation in other DGs whether there is an opportunity for some joint development (a novel idea!) Unlike other ‘perspectives’ it is not tied to content or any particular theoretical approach.
• Think about developing some mobile phone apps for evaluators and stakeholders around content specific issues – I can easily think of 5 or 6 possibilities to support both counterfactual, quantitative approaches and theory-based qualitative approaches. Although the ideas are generic, customizing the content means evaluators would have something concrete to work with rather than just ideas.
• Produce an easy-to-use handbook on evaluation 2.0 for evaluators / evaluation units who want practical information on how to do it.
• Ring fence a small amount of funding to support one-off explorations into innovative practice and new ideas around evaluation.
• Encourage the evaluation unit to demonstrate leadership in new approaches – for example, try streaming a live internet radio programme around the theme of evaluation (cheap and easy!); set up a multi-user blog for people to post work in progress and interesting observations of ongoing projects using a range of media as well as text-based major reports; make some podcasts of interviews with key players in the evaluation of Regional Policy; set up a wiki around evaluation rather than having to drill down through the various Commission websites; try locating projects using GPS data so that we can all see where the action is taking place! Keep a twitter stream going around questions and issues – make use of crowd sourcing!
• Advertise the next European Evaluation Society biennial conference, in Helsinki, October 1st – 5th 2012 “Evaluation in the networked society: New concepts, New challenges, New solutions” (There you go Bob, I just did!)
• Broaden the idea of Evaluation 2.0 and maybe get rid of the catchphrase! We are already using the power of the semantic web in evaluation to mash open and linked data, for example. Should we be now be talking about Evaluation 3.0?? Or should we find another name – Technology Enhanced Evaluation? We could have TEE parties instead of conferences – Europe’s answer to the American far right ; )

P.S. Message to the large numbers of English delegates at the conference

When you left Heathrow yesterday to come to Brussels, I do hope you waved to the English Rugby team arriving home from the Rugby World Cup in New Zealand.

(Just as well this conference was not a week later or I’d have leave a similar message for the French delegates…..)

I have spent some time this morning thinking about the Mozilla Open Badges and assessment project, spurred on by the study group set up by Doug Belshaw to think about the potential of the scheme. And the more I think about it, the more I am convinced of its potential as perhaps one of the most significant developments in the move towards Open Education. First though a brief recap for those of you who have not already heard about the project.

The Open Badges framework, say the project developers, is designed to allow any learner to collect badges from multiple sites, tied to a single identity, and then share them out across various sites — from their personal blog or web site to social networking profiles. The infrastructure needs to be open to allow anyone to issue badges, and for each learner to carry the badges with them across the web and other contexts.

Now some of the issues. I am still concerned of attempts to establish taxonomies, be it those of hierarchy in terms of award structures or those of different forms of ability / competence / skill (pick your own terminology). Such undertakings have bedeviled attempts to introduce new forms of recognition and I worry that those coming more from the educational technology world may not realise the pitfalls of taxonomies and levels.

Secondly is the issue of credibility. There is a two fold danger here. One is that the badges will only be adopted for achievements in areas / subjects / domains presently outside ‘official’ accreditation schemes and thus will be marginalised. There is also a danger that in the desire to gain recognition, badges will be effectively benchmarked against present accreditation programmes (e.g. university modules / degrees) and thus become subject to all the existing restrictions of such accreditation.

And thirdly, as the project roils towards a full release, there may be pressures for restricting badge issuers to existing accreditation bodies, and concentrating on the technological infrastructure, rather than rethinking practices in assessment.

Lets look at some of the characteristics of any assessment system:

I am intrigued by this abstract is for a Symposium that will be presented at ALT-C 2011 by Frances Bell, Cristina da Costa, Josie Fraser, Richard Hall and Helen Keegan and is published on Frances Bell’s blog. I am reproducing it in full below.

This symposium will examine the paradoxes of giving and receiving online in education in a changing economic climate.  Each of the panelists will briefly address topic areas within the symposium theme, followed by an opportunity for present and at distance audiences to contribute, concluding with a 25 minute plenary discussion.

Symposium delegates will be provoked to reconsider the costs of participation online by paid and unpaid participants in ‘open’ discussion and sharing of resources.

Open Educational Resources exist within communities that create, use and sustain them (Downes 2007). When ‘communities’ in Higher Education break down due to redundancy and casualisation of labour what happens to OERs? Are they sustained? Can they reach out to other contexts?

All areas of education, including the school sector, currently face significant financial challenges and uncertainties. Institutions are increasingly reviewing the provision of devices and services, and looking at learner owned devices and commercially owned ‘free’ cloud-based services. What is the real price of an education system supported and transformed by embedded learning technologies?

Ownership in the age of openness calls for clarity about mutual expectations between learners, communities and ourselves. Ideas and content are shared easily through open platforms, and yet attributions can be masked in the flow of dissemination: does credit always go where it is due?

Openness in the production, sharing and reuse of education/resources is meaningless in the face of neoliberalism. Where coercive competition forms a treadmill for the production of value, openness/OERs are commodified. Control of the educational means of production determines power to frame how open are the relations for the production or consumption of educational goods or services, in order to realise value. The totality of this need, elicited by the state for capital, rather than the rights of feepayers, parents, communities or academics, shapes how human values like openness are revealed and enabled within HE.

Scarce research monies focus attention on impact factors, arguably stagnating practice. For publications, Open Access can increase wider societal impact but at the expense of career progression.We explore the tensions, paradoxes and professional costs on societal benefits, individual agency and academic progression.

Obviously this is a bit of a mash up proposal but it does raise a lot of questions. I think there is a tension between the idea of communities and institutions. Communities of practice, and for that matter the communities in which Open education Resources are being produced and shared, cross institutional boundaries. Furthermore the use of OERs may be within an institutional setting but may also be outside.

This again is reflected in recognition and reward structures. Whilst reward structures within institutions are based on either monetary compensation or in terms of progression, rewards within the community may reflect a wider understanding of recognition, especially respect or standing within that community. Does credit always go to who it should? Probably not, but this is taking a very individualistic view of research. Surely credit should be shared in the community rather than in the closed door offices of academic researchers.

Are OERs being commodified? Presumably the term “coercive competition” refers to the growing practice to require academics and researchers to publish their work as OERs. I don’t really understand what the authors mean in saying “The totality of this need, elicited by the state for capital, rather than the rights of feepayers, parents, communities or academics, shapes how human values like openness are revealed and enabled within HE.” Of course the idea of openness is being hijacked by institutions. But at the same time the movement towards openness is contradictory and I am not sure this is reflected in the abstract. Especially missing is a discussion of the nature of OERs in allowing reuse and modification and the impact this has on (commodified) relations of production and intellectual property. And at the same time, the spread of OERs is allowing new open forms of learning and knowledge production outside the confines of the institution. thus whilst the movement towards OERs may be becoming commodified the use of OERs is challenging traditional understandings of those very commodities.

I don’t think this reveals a paradox but a dialectical contradiction. Present schooling models of education are being found to be wanting. The discussion about open education is important in that it could provide an alternative to privatisation. The discussion over OERs forms an important part of this debate and to this extent the debate over this symposium is extremely interesting.

The proliferation of new technologies and internet tools is fundamentally changing the way we live and work. The lifelong learning sector is no exception with technology having a major impact on teaching and learning. This in turn is affecting the skills needs of the learning delivery workforce.

Last September, together with Jenny Hughes I undertook a literature review on new pedagogical approaches to the use of technologies for teaching and learning. You can access the full (86 pages) document below.

The research was commissioned by LLUK to feed into the review then being undertaken of teaching qualifications in the Lifelong Learning sector in the UK. The review was designed to ensure the qualifications are up to date and will support the development of the skills needed by the modern teacher, tutor or trainer.

However, we recognised that the gap in technology related skills required by teaching and learning professionals cannot be bridged by qualifications alone or by initial training and a programme of opportunities for continuing professional development (CPD) is also needed to enable people to remain up to date.

The literature review is intended to

• identify new and emerging pedagogies;
• determine what constitutes effective use of technology in teaching and learning
• look at new developments in teacher training qualifications to ensure that they are at the cutting edge of learning theory and classroom practice
• make suggestions as to how teachers can continually update their skills.

Pedagogical Appraches for Using Technology Literature Review January 11 FINAL 1

The idea of Open Education has come a long way in the last two years. Massive Online Open Courses are becoming more common (with the announcement of the “mother of all MOOCs” on Change: Education, Learning and Technology exciting great interest in the edu-blogosphere), conferences and seminars being streamed online and Open Educational Resources have entered the mainstream.

What has been learned in this process?

Firstly the model of courses which are free to participants but charge for institutional enrollment and for certification appears to be gaining traction. How far this can go depends I guess on the extent that participation (and recording of work) becomes recognised as achievement. It will also depend on how much value universities and other institutions think they can gain (or stand to lose) through such a model.

Secondly most of these programmes are using all manner of social software and Open Source applications. There seems to be a growing practice of hanging programmes together around open webinars, with students using their own blogs or other social software for their personal work. One of the less successful experiments seems to be attempts to integrate VLEs, especially Moodle, within MOOCs. Participants are being encouraged to develop their own Personal Learning Environments as part of the process.

Thirdly such initiatives place great emphasis on peer support for learning, with a greater or lesser extent of formal learning support and formalization of networks. One greatly encouraging development is the blurring of the boundary between teachers and learners. Another is the involvement of people form different organisations in leading, facilitating or stewarding such programmes. Most stewards or facilitators are not being paid, although I suspect at present this is being accepted by institutions as a legitimate part of their work as researchers. Whatever, this is resulting in a weakening of institutional boundaries and the emergence of stronger communities of practice.

There also seems to be considerable pedagogic innovation, with a willingness to explore new ways of learning. Especially encouraging is the use of multi media, which although promised in so many formal elearning programmes, has seldom really happened.

Now comes the big question. Can the experience gained from the MOOCs be extended to provide a transferable and scalable model for Open Education.

I’ve already talked about the issue of recognition which I see not so much as a question of assessment but of social recognition of achievement. But there are other open issues. How do we deal with language barriers? More critically, most participants in the early MOOCs seem to be professionals, teachers and researchers already engaged in online learning or multi media and / or students. In other words, people with a fair degree of competence in communicating through on-line media. The model is based on a large degree of self motivation and is reliant on learners being able to manage both their own learning and able to develop their own support networks. This is a pretty big limitation.

I see two ways to deal with this. One is to provide more formal and institutional support through participation in MOOCs becoming part of courses on which learners are already enrolled and their host institution providing support. This idea is already being suggested for the Change: Education, Learning and Technology MOOC. The second is through developing more fomalised individual and group mentoring and support systems. At the moment, we are tending to focus on presenters as the key people in facilitating the online programmes. But such a second layer of mentors could play the critical role, and providing such mentoring could be a key part of Continuing Professional Development for teachers and trainers. In other words, a win, win situation.

I don’t really know why, but I seem to be spending a lot of time at the moment reviewing proposals and contributions for conferences and publications. And whilst there is much to be learned from all the ideas being put forward it is time consuming and sometimes feels a very isolated and perhaps archaic process.

I fond it difficult to decide the standards or criteria I am reviewing against. How important is clarity of thinking, originality, creativity? How important is it that the author includes copious references to previous work? Are we looking for depth or breadth? How important is the standard of English, particularly for those writing in a second or third language?

In this world of social software the whole review process seems somewhat archaic. It relies very much on individuals, all working in isolation. People write an abstract according to a call for proposals (and I am well aware of how difficult it is to write such calls – unless of course it is one of these multi track conferences which just include everything!). The reviews are allocated to a series of individuals for blind review. They do their work in isolation and then according to often subjective criteria, the proposal is accepted or rejected.

OK, sometimes there is the opportunity to make a conditional acceptance based on changes to the proposal. and of course, you are encouraged to provide feedback to the author. But all too often feedback is limited and pressure of time prevents organisers allowing a  conditional acceptance.

How could social software help with this? As usual I think it is a socio technical solution we need to look for, rather than an adoption of technologies per se. Most conferences have adopted software to help with the conference organising and review procedures but as happens all to often that software has been developed to manage existing processes more efficiently with no thought into how we could transform practices.

One big issue is the anonymity of the review procedure. I can see many reasons to support this, but it is a big barrier to providing support in improving submissions. If we move to non blind reviewing, then we could develop systems to support a discourse between submitters and reviewers, where both become part of the knowledge creation process. and in added benefit of such a discourse could be to clarify and make transparent the criteria being used for reviews. reviewers would have more of a role as mentors rather than assessors or gatekeepers.

This would not really require sophisticated technological development. It would really just need a simple booking system to arrange for a review and feedback session, together with video, audio or text conferencing functionality. More importantly perhaps it might help us in rethinking the role of individual and collective work in the academic and scholarly forms of publishing and knowledge development. I suspect a considerable barrier is the idea of the ‘Doctor Father’ – that such a process would challenge the authority of professors and doctorate supervisors. My experience, based on talking to many PhD students, is that the supervisory role does not work particularly well. It was developed when the principle role of universities was research and was designed to induct students into a community of practice as a researcher. With the changing role of universities plus the fact that many students are no longer committed to a long term career in academia (even if they could get a job) such processes have become less than functional. Better I think to develop processes of support based on wider communities than the narrow confines of a single university department.

Today Google Labs released their new data visualisation store. Very impressive it is too, although it is not a straightforward task to register on the site, upload uses an XML format and you cannot download data. But the visualisation is pretty good and Google themselves have linked to a number of large Eurostat data sets.

I have been working on data for the last couple of weeks. I am trying to build a TEBO – a Technology Enhanced Boundary Object (or objects) for explaining Labour Market data to Careers Advice, Information and Guidance (CAIG). Together with my colleagues from the Institute for Employment Research at Warwick University, I have been looking at TEBOs for some time.

Alan Brown explains the conceptual idea behind TEBOs:

The ideas of boundary crossing and tool mediation (Tuomi-Gröhn & Engeström, 2003; Kaptelinin & Miettinen 2005) and situated learning with a close alignment to the importance of a focus upon practice (Brown et al., 1989; Hall, 1996) informed considerations of the role of technologically-enhanced boundary objects in knowledge maturing processes in different contexts. One specific concern is to make visible the epistemological role of symbolic boundary objects in situations in which people from different communities use common artefacts in communication. A fruitful approach to choosing ways to develop particular boundary objects is to focus on what Onstenk (1997) defines as core problems: the problems and dilemmas that are central to the practice of an occupation that have significance both for individual and organisational performance — in this case the problems associated with providing advice relevant for career planning. One method this development project used was therefore to engage in a dialogue with guidance practitioners about common scenarios involving Labour Market Information (LMI) which could inform the development of prototype technologically-enhanced boundary objects (TEBOs). The development … was therefore informed by a consideration of the following issues:

• Importance of developing methods and strategies for co-design with users
• Need for conceptual tools to help people understand the models and ideas which are part of LMI
• Need for a more open pedagogy (than is typical of much existing technology-enhanced learning, and existing workplace training practice)
• A system in which boundary objects are configurable by end-users (practitioners) and by guidance trainers to be used in multiple ways
• Need to build an understanding of how TEBOs may be used in ways that are empowering for practitioners, and ultimately for clients too.

These concerns could be coupled with another set of issues concerning appropriate skill development:

• Need for time for people to interact, reflect, use concepts etc.
• Trying to reach a stage where practitioners have justifiable confidence in the claims they make and can exercise judgement about the value of information when faced with unfamiliar LMI
• Choosing between a range of possible use-contexts
• Decide how to employ support from communication and discussion tools
• Developing and transmitting Labour Market intelligence – importance of communicating to others
• Preconfigure certain ways of thinking through use of scenarios; discussions can point into and lead from scenarios.

In practice it is not so easy to develop such TEBOs. Identifying key problmes is probably the most useful approach. But then there is an issue in accessing different data to visualise as part of the process. A great deal of data is now publicly available. But I am no data specialist and have faced a steep learning curve in understanding and interpreting the data myself. then there is the issue of visualisation – I am mainly using Google Gadgets, although we are also working with Tableau (a powerful tool, but unfortunately only available for Windows) and IBM;s Many Eyes. All these tools are good, but are all extremely finicky about how the data is formatted. We are working with data in xls and Apple’s Numbers but I suspect longer term it would be better to use the Open Source R programming environment.

And the hardest task of all is the storyboarding. At the end of the day we are trying to tell stories with data: TEBOs are a storytelling and exploration approach to learning. So for each TEBO I intend to make a short video explaining the key concepts and showing the various visualizations. We will also provide access to the raw data and to static versions of the graphing, along with explanatory notes. And for each TEBO we will try to construct an interactive visualisation tool, allowing learners to play with the data and displays. I also want to try to build some sort of simulations using the Forio tool. No doubt there is better software (and if anyone has any ideas I would be very grateful). But I sort of feel that the more social software, open source or free tools we can use the better. We want to encourage people to do it for themselves. And they have no money to spend on fancy software tools.We cannot possibly provide access to visualisations of all the data available. But if we cane explain what is possible, hopefully interested CAIG professionals will start there own work. And then who knows – a Careers Guidance data store?

The Guardian newspaper points to a so called ‘backlash’ against social networking, expressed in a number of recent academic studies and books. And to an extent, I agree. I suspect the novelty factor has worn off. That does not mean social networking is dead, far from it. But it does mean we are slowly evolving an ecosystem of social networking and I am not sure that the Facebook model, driven by the desire to monetarise a huge user base will survive in the long term.

Instead I see two trends. With applications like Facebook, or whatever succeeds it, friends will return to being friends. People we know, people we want to socialise with, be it family and friends we see regularly face to face or friends in distributed networks.

The second will be the growth of social networks based on shared interests and shared practice. Of course this is nothing new. The early days of the web spawned many wonderful bulletin boards with graphics being based on the imaginative use of different text and fonts. Ning led to the explosion of community sites whilst it remained free. But now we are seeing the evolution of free and open source software providing powerful tools for supporting interest and practice based communities.

Cloudworks, developed by the UK Open University has now released an installable version of their platform. Buddypress seems to have developed a vibrant open source community of developers.And I am greatly impressed by QSDA, the Open Source Question and Answer System. Quora is all the hype now. But like so many of these systems, it will be overrun not so much by machine driven spam, but by the lack of a  shared community and purpose.

According to Ettiene Wenger, a community of practice defines itself along three dimensions:

• What it is about – its joint enterprise as understood and continually renegotiated by its members.
• How it functions – mutual engagement that bind members together into a social entity.
• What capability it has produced – the shared repertoire of communal resources (routines, sensibilities, artefacts, vocabulary, styles, etc.) that members have developed over time.

Open Source networking tools can allow us to support that shared repertoire of communal resources. I am working on the development of open and linked data for careers guidance and counselling. it is a fairly steep learning curve for me in terms of understanding data. And one of the bests sites I have found is Tony Hirst’s Get the Data site, only launched a week ago and based on the QSDA software, but already providing a wealth if freely contributed ideas and knowledge.

it is this sort of development that seems to me to be the future for social networking.

Graham and I have just got back to Germany after a meeting of the Politics project team in Cardiff. We were following Wednesday’s demonstrations against the proposed hike in university fees live on TV at Cardiff airport – both of us getting very excited and cheering a lot.

The occupation of the Conservative Party headquarters in London was an impressive piece of collective action so to all those involved in the organisation and to all those that turned up on the day, a message of support from Pontydysgu!

However, it did make me wonder how we ever used to do all this without mobile phones, computers or social networking media. Apart from using print media, I seem to remember a lot of organising time spent in public telephone boxes pressing button A and button B. In fact, one of my early ICT competences was learning how to tap the receiver rest up and down to mimic the operation of the dial in order to save the 4d (less than 2p) it cost.

Last night I spent a hour or so playing with new social software startup, Lanyrd. And I love it. Why?

Well I logged in or rather pressed a button saying something like login with Twitter and there I was. No filling in forms or making up passwords. And there straight away was a message for me:

Hi there! we have had a look at conferences your friends on twitter are going to, perhaps you might like to go too.