Archive for the ‘Pedagogy’ Category

150 blogs on Learning Layers project – 200 altogether on Pontydysgu site

August 4th, 2016 by Pekka Kamarainen

I have come back from my summer break – but not back to work and normal business. During my holidays I had to run through a series of medical tests/investigations and now I am on sick leave for some time. I do not want to go into details – some investigations are yet to come – but I know enough that I have to take a break from my normal work. This gives me a reason to spell out some thoughts on my blogging on this site. It so happens that I have reached the milestone of 150 blogs on our ongoing EU-funded Learning Layers (LL) project and altogether the milestone of 200 blogs on Pontydysgu site.

In general, such numbers are not great achievements – veteran bloggers count their posts in thousands, not hundreds. And indeed, during my first years as a blogger I was not so successful in finding my approach and ways to work forward. With my first blog “I-Europe” I tried to stimulate a debate on European initiatives to promote vocational education and training (VET). Unfortunately, these entries were not so well grounded and attracted little attention. With my second attempt – with  my new blog “Working & Learning” – I tried get closer to the work of European projects and educational debates. Yet – for some time this remained at the level of irregular scraping. Some of the projects of that time were perhaps not that inspiring or they required blogging (or similar writings) on other platforms. Therefore, I had made some experiences but had not really found my own way of blogging.

This all changed with the start of the EU-funded Learning Layers (LL) project in 2012. The project has required us (ITB – research institute with focus on VET and learning in the context of work) to face new challenges. It has not been merely a matter of introducing new learning technologies and new learning concepts to the field (and study the impact). The project has been far more innovative in terms of exploring different options, involving users in co-design & co-development and in engaging us as VET researchers in different roles as co-developers, co-tutors and co-testers of new tools. From this perspective I have had the challenges and the opportunities to produce a more or less regular flow of blogs on new project activities, observations on parallel developments, links to inspiring research or to policies that have an impact on our work. And, moreover, the flow of blogs has not merely been recording of events, debates and happenings – they provide insights into our learning processes as research partners, developers and application partners. In particular they provide insights into our transformation from explorers to change agents and interpreters of the changes.

Having said all this I feel sad that I cannot continue with the intensive observation and documentation of field activities in the same way as I have done so far. From now on I have to take the role of listener and thinker. Perhaps that is also a positive turn in its way – after all, the rich project experience needs to be digested and interpreted in conceptual terms. And surely, our experiences as accompanying researchers differ from the traditional patterns of doing such research. But, as I said in the beginning, I have to take some time out of regular project work to get myself fit. Nevertheless, I will be around.

More blogs to come …

New book: Empowering Change in European Public Employment Services

July 18th, 2016 by Graham Attwell

employid bookReaders familiar with European Research projects will know how they work. The projects negotiate with the European Commission a DOW – Description of Work – which details the work to be undertaken in each year of the project. It is divided into discrete work packages. Every year the work package provides a (usually over lengthy) report on research and development undertaken which is then presented to a team of expert reviewers who can ‘pass’, recommend changes or ‘fail’ the report. Although obviously large scale multi national research projects need structures and plans. But all too often the work package structure separates research and development activities which should not be separated and the DOW become a restrictive ‘bible’, rather than a guide for action. And despite the large amount of work which goes into preparing the work package reports, they are seldom widely read (if indeed read at all), except by the reviewers.

In the EmployID project which is working with identity transformation in European Public Employment Services (PES), we are doing things differently. The work is structured though cross work package teams, who follow an adapted SCRUM structure. The teams are reviewed at face to face meetings and recomposed if necessary. And this year, instead of producing a series of Work package reports, the project partners have jointly contributed to a book – Empowering Change in Public Employment Services: The EmployID Approach which has just been published and can be downloaded for free.

The introduction to the 244 page PDF book explains the background to the work:

European Public Employment Services (PES) and their employees are facing fundamental challenges to the delivery of efficient and effective services and the need to change their strategies to combat high unemployment, demographic change in increasingly uncertain and dynamic labour markets. This does not only require developing new professional skills related to new tasks, but poses for more profound developmental challenges for staff members.

Three of these changes relate to understanding the changing world of work; a ‘turn’ towards coaching; and the increased importance of relations with employers. The staff need to learn new ways of working, with a major challenge being to enhance the power of collaborative (peer) learning in order to support staff in accomplishing their goals.

All these changes are linked to transforming professional identity, which requires learning on a deeper level that is often neglected by continuing professional development strategies. EmployID makes its contribution here; that PES practitioners’ learning related to professional identity transformation needs to be facilitated through social learning approaches and the nurturing of social learning networks, which include the following:

  • Reflection as a way of turning one’s own and others’ experiences into general insights on multiple levels, both from an individual and a collective perspective

  • Peer coaching as a way of helping learners in changing their behavior through a structured process

  • Social learning programmes as a way of engaging learners with new topics, other perspectives, and conversations around it.

Returning from Learning Layers Bristol meeting – Taking homework back to Bremen

June 23rd, 2016 by Pekka Kamarainen

In my previous blogs I reported on the preparations for the consortium meeting of  our EU-funded Learning Layers (LL) project that took place in Bristol during the last few days. Now I am on my way back and have some spare minutes to reflect on the baggage of homework that I am taking from the meeting back to office. In general we had a very productive meeting – so many ideas sparking up that it was good to have colleagues taking notes (on the spot and at the other end of online connection). Therefore I just make some short remarks, how our talks helped us to bring our work further: In particular I was happy to see that we are finding a way to present our results as a part of a common group picture – rather than as stand-alone results of different partners or work packages teams working on their own. Below some main points on this:

  1. Evaluation and documenting the impact: So far more attention has been given on the use of specific evaluation instruments (focus groups, complementary interviews, impact score cards, logdata on use of LL tools) and analysing data gathered with these instruments. Now we opened up this discussion to consider, how to use complementary evidence that is being gathered alongside the fieldwork in the sectoral pilots and in the co-design work. Here we worked with a set of transversal themes (such as digital transformation, adoption of innovation and changes in (informal) learning practices).  This has implications for the work of narrower ‘evaluation data’, complementary data and the impact scorecards.
  2. Presenting our R&D methodologies: We have already earlier agreed to report our results with a single deliverable – a website – and that one section should be dedicated to R&D methodologies. For this section some partners had prepared draft documents that shed light on different ‘local’, sectoral or technical aspects of our R&D work. In the light of these drafts we made clear progress in trying to open up certain contributions (such as co-design work) to be presented from the perspective of both pilot sectors – construction and healthcare. And we developed a better understanding how different activities carried out in the project can be presented as part of a coherent whole.
  3. Outlining ‘learning scenarios’: At different points of time our project had been working with different sets of ‘use cases’, ‘user stories’, ‘learning scenarios’ or ‘learning stories’. All these had been characterised by a preparatory and explorative phase of the project – presenting possibilities to work with the tools and learning arrangements that we were developing. Now it appeared that we are building learning scenarios that rely on ‘lead theories’ and on the way way have built upon them when developing tools and learning arrangements. Here we are drawing upon the transversal themes (mentioned in point 1. and on the more specific impact cards). This was reflected in a very specific set of ‘learning scenarios’ and tasks to draft them.
  4. Working further with the exploitation agendas: Here our colleagues Gilbert Peffer and Raymond Elferink presented a ‘generalised’ and at the same time well grounded model, how to adjust the prior partnership relations to new and renewed ones (with an exemplary start-up company for services in the centre). Alongside this example we also revisited the conclusions of the Aachen Integration Meeting on the co-management of the Open Source Software that has been developed in the context of the project. The most important point was that we found both models fully compatible with each other.

I guess this is enough for these spare minutes that I have had today. I am continuing my journey to Bremen (where I still have some meetings before I start my summer break).

More blogs to come …

Preparing for Learning Layers Bristol meeting – Part Two: Taking homework with me to Bristol

June 17th, 2016 by Pekka Kamarainen

In my previous blog I mentioned that  our EU-funded Learning Layers (LL) project will have its consortium meeting in Bristol next week. As preparation I have had a final run with reporting on activities that have taken place in the Construction pilot of the LL project – in particular with the deployment of the Learning Toolbox (LTB). In the previous post I tried to give a picture, what we are achieving altogether with our user engagement and tool deployment – enabling the users to become owners of innovation. Yet, at the same time I drew attention at the hurdles we still have to overcome to get the best out of the ongoing processes. Now, in this post I shift the emphasis to the agenda and to the topics the we are working with – to prepare the final deliverable of the project. However, I will not discuss in detail the plans we have – instead I try to put together my thoughts on what we (as the Construction pilot team) can contribute to some main points.

Below I try to outline my thoughts and a todo-list, how to proceed with them:

  1. Impact scorecards: We have earlier this year to use impact scorecards to present, what difference the introduction of LL tools has made in different pilot contexts. So far we in the construction pilot have not been rushing to draft them. However, due to our recent field activities we can give a far more differentiated picture with emphasis on different user groups of Learning Toolbox and on the role of complementary tools. However, we are looking forward to enrich the gallery later on with the results of our forthcoming workshops with construction companies (scheduled for September).
  2. Text for the section ‘research & development methodologies’: Here we need to give insights into the way in which the participative co-design activities and the contributions of accompanying research have nurtured each other. In particular we need to draw attention to some lead ideas and theoretical concepts that have characterised our work, such as  a) enhancing vocational learning as  action-oriented and self-organised learning, b) supporting the acquisition and exploitation of ‘work process knowledge’ and c) promoting co-design as social shaping of work, technology and work environment. Here, I have drafted the structure for our contribution and collected the key materials from our Theory Camp contributions, conference papers, LL website articles and contributions to Y2 and Y3 deliverables.
  3. Contributions to ‘Learning scenarios’: With the scenarios we want to highlight a) how our ‘lead theories’ have supported our development work and 2) how they help us to specify the potential and actual changes in the (informal) learning at working contexts. Here we are having a differentiated look at trainers in Bau-ABC (and their peer learning as change agents) and apprentices as users of new tools (and their insights into their role in vocational learning).
  4. Contributions to ‘Exploitation activities’: Here we can revisit the exploitation landscape (consisting of several spin-off or follow-up projects) that we presented last June in the Tallinn consortium meeting. As things stand now, most of these projects are going on and are looking for opportunities to introduce Learning Toolbox (and eventually other LL tools) in their contexts. This requires further talks on the partnership relations to be created with the tool developer teams and the new projects.

I guess this is enough for the moment. I have put down some of my thoughts and I need to work with them before the meeting and in the respective sessions. That is what will be there for – to achieve common results for the final phase of activities. I am looking forward to busy days in Bristol.

More blogs to come …

Zimmererblog is going on strong – Learning Layers’ multimedia training bears fruit

June 12th, 2016 by Pekka Kamarainen

In my latest blogs I have given progress reports on the construction pilot of our EU-funded Learning Layers (LL) project, mostly focusing on the integrative toolset Learning Toolbox (LTB). In addition I have provided a review on the progress we have achieved since the project consortium meeting in Tallinn one year ago. In this blog I will have a look at the main results of the earlier Multimedia Training Workshops (that the LL project organised for the full-time trainers (Lehrwerkmeister) of the construction sector training centre Bau-ABC in 2013-2014) – the trainers’ blogs. Here I will firstly focus on the most outstanding example, the Zimmererblog (Carpenters’ blog) of Bau-ABC trainer Markus Pape.

Zimmererblog – origins, development and impact

In the first Multimedia Training Workshops of the LL project in the years 2013-2014 the LL partners from Pontydysgu and ITB providedtraining for a group of voluntary Bau-ABC trainers. At that time we started by getting an overview of the general web tools and by making use of them. In this context the participating trainers created their own WordPress blogs. In the course of the training they developed their own pattern of working with blogs. Instead of keeping a diary or writing columns on different topics the trainers have transformed their blogs into their own ‘open educational resources’. In this process the trainer Markus Pape has been the pioneer and his Zimmererblog has become the most comprehensive one.

Looking at the structure – after the startpage – the main areas of the blog are the collections of project descriptions (worksheets) for each year of apprentice training. Then, the blog provides links to literature and other websites as well as an additional area for special techniques. Yet, the special trademark of this blog is that the pictures in the worksheets, in the special area and in the slideshow have been edited to make the site more attractive.

Looking at the impact, it is worthwhile to note that the Zimmererblog has from the very beginning on gained a wide popularity beyond the primary users – trainers and apprentices in Bau-ABC. The statistics reveal that it has been viewed from all over the world – although it is only available in German. Recently it has reached the milestone of 45.000 hits (the exact number being currently 45.103) and the interest is not dropping at all. In this respect the expression ‘open educational resource’ is justified.

Trainers’ blogs in the neighboring trades have also taken their place

Parallel to the Zimmererblog the trainers in some other trades (who had also attended the Multimedia Training Workshops) started to create similar blogs for their trades (or groups of trades). As a result there are three other blogs with similar structure in Bau-ABC:

  • The ‘Maurerblog’ (“Mauerwerksbau im Bau-ABC Rostrup. Backsteine und Mehr”) provides a similar set of learning resources for bricklayer apprentices and additional resources for skilled workers. Currently this site has reached 6.604 hits.
  • The ‘Tiefbaublog’ (“Tiefbau Bau-ABC Rostrup. Mach Dich schlau im Tiefbau”) provides a similar set of resources for three neighbouring trades – road-builders (Strassenbauer), pipeline-builders (Rohrleitungsbauer) and sewage-builders (Kanalbauer). In a similar way it provides additional info sheets and links to external resorces. Currently this site has reached 2.893 hits.
  • The ‘Brunnenbaublog’ (Brunnenbauer und Spezialtiefbauer) provides similar sets of resources for the neighbouring trades of well-builders (Brunnebauer) and for the tunnel-builders (Spezialtiefbauer). In addition, the blog provides further links to progression routes to higher education/qualifications (Duales Studium, Weiterbildung). In addition, the blog provides further sections for special themes, tables and instructions for health and safety. For this site we have not got the current statistics.

Here it is worthwhile to note that these blogs have been developed mainly for internal use in Bau-ABC. From that perspective they have been used rather well although their external impact has remained rather limited compared with the Zimmererblog and its impressive outreach.

– – –

Altogether the trainers’ blogs have already taken their place before the Learning Toolbox has been introduced. Now it is interesting to see, how these tools and instruments can best complement each other. Already in the ‘Theme Room” training workshops the trainers started developing thoughts in this respect. I am looking forward to the next steps.

More blogs to come …

 

Double Loop Learning and Learning Analytics

May 4th, 2016 by Graham Attwell

Another in this mini series on Learning Analytics. When looking at Work based learning, Double Loop Learning becomes particularly important. Double-loop learning is used when it is necessary to change the mental model on which a decision depends. Unlike single loops, this model includes a shift in understanding, from simple and static to broader and more dynamic, such as taking into account the changes in the surroundings and the need for expression changes in mental models.(Mildeova, S., Vojtko V. ,2003).

double loop learning

To remind readers again, in the EmployID European project we are aiming to support scalable and cost-effective facilitation of professional identity transformation in public employment services. And I would argue such identity transformation is based on refection on learning, on Double Loop Learning. Identity transformation necessarily involves the development of new metal models and new ways of looking at work based behaviours and practices.

So where does Learning Analytics fit into this? Learning analytics aims to understand and improve learning and the learning environment. This does not necessarily involve Double Loop Learning. For students feedback about their present performance may be enough. But if we aim for identity transformation and wish to improve the learning environment then we need a deeper interpretation of data. This has a number of implications in terms of designing Learning Analytics.

Firstly we have to have a very clear focus on what the purpose of the Learning Analytics is. Is it to find out more for example about informal learning in organisations or to inform L and D department staff about the Learning environment. Is it to help learners understand about their interactions with other staff or to examine their own dispositions for learning – and so on? Secondly – and crucially who is that data presented to users – be it learners or trainers. The existing parading for Learning Analytics presentations appears to be the dashboard. Yet in the LAk16 pre conference workshops there were a whole series of presentations where presenters invited participants to say what the graphics meant. And often we couldn’t. If LA professionals cannot interpret data visualisations then a leaner has little hope of making their own meanings. I am a little puzzled as to why dashboards have become the norm. And one of my major concern is that often it is difficult to understand the visualisation out of the context in which the learning exchange has happened. If Double Loop learning is to happen, then learners need to reflect in order to make meanings. And refection occurs best, I think, in the context in which it takes place.

technical-challenges-for-realizing-learning-analytics-11-638

Image: Ralph Klamma – http://www.slideshare.net/klamma/technical-challenges-for-realizing-learning-analytics

There are alternatives to the dashboard. For instance with EmployID we are developing real time discourse analysis and are also looking at providing dynamic prompts for reflection.>One final point. If we are aiming at using Learning Analytics for Double Loop Learning we need to find out what works and what does not. That means that any measure for Learning Analytics needs to be accompanied by well designed evaluation measures. All too often because LA collects data, it presumes to cover evaluation. Whilst both LA and evaluation may share data, they aim at different things.

Workplace Learning Analytics for Facilitation in European Public Employment Services

April 29th, 2016 by Graham Attwell

This week I have been at the pre-conference workshops for the Learning analytics conference in Edinburgh. This is my presentation at the workshop on Workplace Learning Analytics. And below is the abstract of my paper together with a link to download the full paper, if you should wish. In the next few days,  I will write up a reflection on the workshops, plus some new ideas that emerged from talking with participants.
Abstract

The paper is based on early research and practices in developing workplace Learning Analytics for the EU funded EmployID project, focused on identity transformation and continuing professional development in Public Employment Services (PES) in Europe. Workplace learning is mostly informal with little agreement of proxies for learning, driven by demands of work tasks or intrinsic interests of the learner, by self-directed exploration and social exchange that is tightly connected to processes and the places of work. Rather than focusing on formal learning, LA in PES needs to be based on individual and collective social practices and informal learning and facilitation processes rather than formal education. Furthermore, there are considerable concerns and restraints over the use of data in PES including data privacy and issues including power relations and hierarchies.

Following a consultation process about what innovations PES would like to pilot and what best meets their needs, PES defined priorities for competence advancement around the ‘resourceful learner’, self-reflection and self-efficacy as core competences for their professional identity transformation. The paper describes an approach based on Social Learning Analytics linked to the activities of the EmployID project in developing social learning including advanced coaching, reflection, networking and learning support services. SLA focuses on how learners build knowledge together in their cultural and social settings. In the context of online social learning, it takes into account both formal and informal educational environments, including networks and communities. The final section of the paper reports on work in progress to build a series of tools to embed SLA within communities and practices in PES organisations.

Download the paper (PDF)

Mobile Learning – the Dream goes on

February 29th, 2016 by Graham Attwell

“What killed the mobile learning dream?” asks John Traxler in an article for Jisc’s Digifest. John goes on to say:

Mobile learning was e-learning’s dream come true. It offered the potential for completely personalised learning to be truly any time, anywhere.

ltbInstead, we’ve ended up with mobile access to virtual learning environments that are being used as repositories. So, in practice, students reading their notes on the bus.

He’s right but I am not sure his reasons are sufficient. The main problem John sees is that when early projects were developed into mobile learning, they were based on supplying participants with digital devices. This was expensive and limited the scale and sustainability of such projects. Now new initiatives are emerging based on BYOD (bring your own Device). This is more sustainable but raises its own questions.

Bring your own device, enabling students to use their own equipment, introduces more questions: is there a specific range of technologies they can bring, what’s the nature of the support offered, and have we got a network infrastructure that won’t fall over when 20,000 students turn up with gadgets? What kind of staff development is needed to handle the fact that not only will the students turn up with many different devices but tomorrow they’ll have changed to even more different devices?

All this is true. And as we prepare to roll out the trial of our Learning Layers project funded Learning Toolbox (LTB) application we are only to aware that as well as looking at the technical and pedagogic application of Learning toolbox, we will have to evaluate the infrastructure support. The use of Learning toolbox has been predicated on BYOD and has been developed with Android, iOS and Microsoft versions. The training centre where the pilot will take place with some 70 apprentices, BauABC, covers a large site and is in a rural area. Telecoms network coverage is flaky, broadband not fast and the wireless network installed to support the pilots is a new venture. So many issues for us to look at there. However in terms of staff development I am more confident, with an ongoing programme for the trainers, but perhaps more importantly I think a more open attitude from construction industry trainers to the use of different technologies than say from university lecturers.

The bigger issue though for me is pedagogy. John hints at this when he talks about mobiles being used to access virtual learning environments that are being used as repositories. The real limitation here is not in the technology or infrastructure but a lack of vision of the potential of mobiles for learning in different contexts. Indeed I suspect that the primary school sector is more advanced in their thing here than the universities. Mobile devices have the potential to take learning into the world outside the classroom and to link practical with more theoretical learning. And rather than merely pushing learning (to be read on the bus although I have never quite understood why mobile learning vendors think everyone travels home by bus), the potential is to create a new ecosystem, whereby learners themselves can contribute to the learning of others, by direct interaction and by the sharing of learning and of objects. Dare I say it – Learning Toolbox is a mobile Personal Learning Environment (at least I hope so). We certainly are not looking to replace existing curricula, neither existing learning technologies. Rather we see Learning Toolbox as enhancing learning experiences and allowing users to reflect on learning in practice. In this respect we are aware of the limitations of a limited screen size and also of the lack of attraction of writing long scripts for many vocational learners. This can be an advantage. Mobile devices support all kinds of gesturing (think Tinder) and are naturally used for multimedia including video and photographs.

So what killed the mobile learning dream. Lack of understanding of its true potential, lack of vision and a concentration of funding and pilot activities with the wrong user groups. That is not to say that mobile learning cannot be used in higher education. But it needs a rethinking of curriculum and of the interface between curriculum, pedagogy and the uses of technology. So the dream is not dead. It just needs more working on!

If you would like to know more about Learning Toolbox or are interesting in demonstration or a pilot please contact me graham10 [at] mac [dot] com

The future of learning at work. How technology is influencing working and learning in healthcare: Preparing our students and ourselves for this future

February 16th, 2016 by Graham Attwell

Over the last few weeks I seem to have been bombarded with calls for submissions for conferences. Most I have ignored on the grounds that they are just too expensive. And if I can’t afford them, working as a relatively senior researcher with project funding, what hope do emerging researchers have of persuading their universities or companies to pay. But tto be honest I am bored with most of the conferences. Formal papers, formally presented with perhaps ten or twenty people in a session and very limited time for discussion. We know there are better ways of learning!

One conference I have submitted an abstract to is AMEE. – the International Association for Medical Education. Apart from short communications, research papers and PhD presentations AMEE invites posters, Pecha Kucha, workshops, points of view and organises a fringe to the conference. Sounds good to me and as you might guess I have submitted a point of view. Here goes (in 300 words precisely) ……

The future of work is increasingly uncertain and that goes just as much for healthcare as other occupations. An ageing population is resulting in increasing demand for healthcare workers and advances in technology and science are resulting in new healthcare applications. At the same time technology promises a revolution in self-diagnosis, whilst Artificial Intelligence and robots may render many traditional jobs obsolete.

So what can we say about healthcare skills for the future and what does it mean for healthcare education. Whilst machines may take over more unskilled work, there is likely to be increasing demand for high skilled specialist healthcare workers as well as those caring for the elderly. These staff need to be confident and competent in using existing technologies and adapting to technologies of the future.

They will need to be self-motivated lifelong learners, resilient and capable of coping with changing occupational identities. They will need to collaborate in multidisciplinary teams leading to a high premium on communication skills.

Present processes of education and training based predominantly on face-to-face courses cannot cope with the needs of lifelong learning. Learning needs to be embedded in everyday work processes. Technology is critical here; ubiquitous connectivity and mobile devices allow context-based learning. The same technologies can promote informal and social learning, learning from peers and sharing experience and knowledge in personal learning networks. Already there are many MOOCs dedicated to medical education. Healthcare professionals are using social media to build informal learning networks. But these are the exceptions not the norm. In the future machine learning algorithms can support individuals wishing to deepen their knowledge, VR to share experiences. Yet although there is a rich potential, medical educators have to steer the process. We need to know what works, what doesn’t, to evaluate, to share. That needs to start now!

Workplace Learning Analytics for Facilitation in European Public Employment Services

February 10th, 2016 by Graham Attwell

Along with colleagues from the EmployID project, I’ve submitted  a paper f to the workshop on Learning Analytics for Workplace and Professional Learning (LA for Work) at Learning Analytics and Knowledge Conference (LAK 2016) in April. Below is the text of teh paper (NB If you are interested, the orgnaisers are still accepting submissions for the workshop.

ABSTRACT

In this paper, we examine issues in introducing Learning Analytics (LA) in the workplace. We describe the aims of the European EmployID research project which aims to use

Image: Educause

technology to facilitate identity transformation and continuing professional development in European Public Employment Services. We describe the pedagogic approach adopted by the project based on social learning in practice, and relate this to the concept of Social Learning Analytics. We outline a series of research questions the project is seeking to explore and explain how these research questions are driving the development of tools for collecting social LA data. At the same time as providing research data, these tools have been developed to provide feedback to participants on their workplace learning.

1. LEARNING ANALYTICS AND WORK BASED LEARNING

Learning Analytics (LA) has been defined as “the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs.” [1]. It can assist in informing decisions in education education system, promote personalized learning and enable adaptive pedagogies and practices [2].

However, whilst there has been considerable research and development in LA in the formal school and higher education sectors, much less attention has been paid to the potential of LA for understanding and improving learning in the workplace. There are a number of possible reasons for this.

Universities and schools have tended to harvest existing data drawn from Virtual Learning Environments (VLEs) and to analyse that data to both predict individual performance and undertake interventions which can for instance reduce drop-out rates. The use of VLEs in the workplace is limited and “collecting traces that learners leave behind” [3] may fail to take cognizance of the multiple modes of formal and informal learning in the workplace and the importance of key indicators such as collaboration. Once more key areas such as collaboration tend to be omitted and in focusing on VLEs, a failure to include all the different modes of learning. Ferguson [4]) says that in LA implementation in formal education: “LA is aligned with clear aims and there are agreed proxies for learning.” Critically, much workplace learning is informal with little agreement of proxies for learning. While Learning Analytics in educational settings very often follow a particular pedagogical design, workplace learning is much more driven by demands of work tasks or intrinsic interests of the learner, by self-directed exploration and social exchange that is tightly connected to processes and the places of work [5].  Learning interactions at the workplace are to a large extent informal and practice based and not embedded into a specific and measurable pedagogical scenario.

Pardo and Siemens [6] point out that “LA is a moral practice and needs to focus on understanding instead of measuring.” In this understanding “learners are central agents and collaborators, learner identity and performance are dynamic variables, learning success and performance is complex and multidimensional, data collection and processing needs to be done with total transparency.” This poses particular issues within the workplace with complex social and work structures, hierarchies and power relations.

Despite these difficulties workplace learners can potentially benefit from being exposed to their own and other’s learning processes and outcomes as this potentially allows for better awareness and tracing of learning, sharing experiences, and scaling informal learning practices [5]. LA can, for instance, allow trainers and L & D professionals to assess the usefulness of learning materials, increase their understanding of the workplace learning environment in order to improve the learning environment and to intervene to advise and assist learners. Perhaps more importantly,  it can assist learners in monitoring and understanding their own activities and interactions and participation in individual and collaborative learning processes and help them in reflecting on their learning.

There have been a number of early attempts to utilise LA in the workplace. Maarten de Laat [7] has developed a system based on Social Network Analysis to show patterns of learning and the impact of informal learning in Communities of Practice for Continuing Professional Development for teachers.

There is a growing interest in the use of MOOCs for professional development and workplace learning. Most (if not all) of the major MOOC platforms have some form of Learning Analytics built in providing both feedback to MOOC designers and to learners about their progress. Given that MOOCs are relatively new and are still rapidly evolving, MOOC developers are keen to use LA as a means of improving MOOC programmes.  Research and development approaches into linking Learning Design with Learning Analytics for developing MOOCs undertaken by Conole [8] and Ferguson [9] amongst others have drawn attention to the importance of pedagogy for LA.

Similarly, there are a number of research and development projects around recommender systems and adaptive learning environments. LA is seen as having strong relations to recommender systems [10], adaptive learning environments and intelligent tutoring systems [11]), and the goals of these research areas. Apart from the idea of using LA for automated customisation and adaptation, feeding back LA results to learners and teachers to foster reflection on learning can support self-regulated learning [12]. In the workplace sphere LA could be used to support the reflective practice of both trainers and learners “taking into account aspects like sentiment, affect, or motivation in LA, for example by exploiting novel multimodal approaches may provide a deeper understanding of learning experiences and the possibility to provide educational interventions in emotionally supportive ways.” [13].

One potential barrier to the use of LA in the workplace is limited data. However, although obviously smaller data sets place limitations on statistical processes, MacNeill [14] stresses the importance of fast data, actionable data, relevant data and smart data, rather than big data. LA, she says, should start from research questions that arise from teaching practice, as opposed to the traditional approach of starting analytics based on already collected and available data. Gasevic, Dawson and Siemens [15]  also draw attention to the importance of information seeking being framed within “robust theoretical models of human behavior” [16]. In the context of workplace learning this implies a focus on individual and collective social practices and to informal learning and facilitation processes rather than formal education. The next section of this paper looks at social learning in Public Employment Services and how this can be linked to an approach to workplace LA.

2. EMPLOYID: ASSISTING IDENTITY TRANSFORMATION THROUGH SOCIAL LEARNING IN EUROPEAN EMPLOYMENT SERVICES

The European EmployID research project aims to support and facilitate the learning process of Public Employment Services (PES) practitioners in their professional identity transformation process. The aims of the project are born out of a recognition that to perform successfully in their job they need to acquire a set of new and transversal skills, develop additional competencies, as well as embed a professional culture of continuous improvement. However, it is unlikely that training programmes will be able to provide sufficient opportunities for all staff in public employment services, particularly in a period of rapid change in the nature and delivery of such services and in a period with intense pressure on public expenditures. Therefore, the EmployID project aims to promote, develop and support the efficient use of technologies to provide advanced coaching, reflection and networking services through social learning. The idea of social learning is that people learn through observing others behaviour, attitudes and outcomes of these behaviours, “Most human behaviour is learned observationally through modelling from observing others, one forms an idea of how new behaviours are performed, and on later occasions this coded information serves as a guide for action” [17]. Facilitation is seen as playing a key role in structuring learning and identity transformation activities and to support networking in personal networks, teams and organisational networks, as well as cross-organisational dialogue.

Social Learning initiatives developed jointly between the EmployID project and PES organisations include the use of MOOCs, access to Labour Market information, the development of a platform to support the emergence of communities of practice and tools to support reflection in practice.

Alongside such a pedagogic approach to social learning based on practice the project is developing a strategy and tools based on Social Learning Analytics. Ferguson and Buckingham Shun [18] say that Social Learning Analytics (SLA) can be usefully thought of as a subset of learning analytics approaches. SLA focuses on how learners build knowledge together in their cultural and social settings. In the context of online social learning, it takes into account both formal and informal educational environments, including networks and communities. “As groups engage in joint activities, their success is related to a combination of individual knowledge and skills, environment, use of tools, and ability to work together. Understanding learning in these settings requires us to pay attention to group processes of knowledge construction – how sets of people learn together using tools in different settings. The focus must be not only on learners, but also on their tools and contexts.”

Viewing learning analytics from a social perspective highlights types of analytic that can be employed to make sense of learner activity in a social setting. They go on to introduce five categories of analytic whose foci are driven by the implications of the changes in which we are using social technology for learning [18]. These include social network analysis focusing on interpersonal relations in social platforms, discourse analytics predicated on the use of language as a tool for knowledge negotiation and construction, content analytics particularly looking at user-generated content and disposition analytics saying intrinsic motivation to learn is a defining feature of online social media, and lies at the heart of engaged learning, and innovation.

The approach to Social Learning Analytics links to the core aims of the EmployID project to support and facilitate the learning process of PES practitioners in their professional identity development by the efficient use of technologies to provide social learning including advanced coaching, reflection, networking and learning support services. The project focuses on technological developments that make facilitation services for professional identity transformation cost-effective and sustainable by empowering individuals and organisations to engage in transformative practices, using a variety of learning and facilitation processes.

3. LEARNING ANALYTICS AND EMPLOYID – WHAT ARE WE TRYING TO FIND OUT?

Clearly there are close links between the development of Learning Analytics and our approach to evaluation within EmployID. In order to design evaluation activities the project has developed a number of overarching research questions around professional development and identity transformations with Public Employment Services. One of these research questions is focused on LA: Which forms of workplace learning analytics can we apply in PES and how do they impact the learner? How can learning analytics contribute to evaluate learning interventions? Other focus on the learning environment and the use of tools for reflection, coaching and creativity as well as the role of the wider environment in facilitating professional identity transformation. A third focus is how practitioners manage better their own learning and gain the necessary skills (e.g. self-directed learning skills, career adaptability skills, transversal skills etc.) to support identity transformation processes as well as facilitating the learning of others linking individual, community and organizational learning.

These research questions also provide a high level framework for the development of Learning Analytics, embedded within the project activities and tools. And indeed much of the data collected for evaluation purposes also can inform Learning Analytics and vice versa. However, whilst the main aim of the evaluation work is measure the impact of the EmployID project and for providing useful formative feedback for development of the project’s tools and overarching blended learning approach, the Learning Analytics focus is understanding and optimizing learning and the environments in which it occurs.

4. FROM A THEORETICAL APPROACH TO DEVELOPING TOOLS FOR LA IN PUBLIC EMPLOYMENT SERVICES

Whilst the more practical work is in an initial phase, linked to the roll out of tools and platforms to support learning, a number of tools are under development and will be tested in 2016. Since this work is placed in the particular working environment of public administration, the initial contextual exploration led to a series of design considerations for the suggested LA approaches presented below. The access to fast, actionable, relevant and smart data is most importantly regulated by strict data protection and privacy aspects, that are crucial and clearly play a critical role in any workplace LA. As mentioned above power relations and hierarchies come into play and the full transparency to be aspired with LA might either be hindered by existing structures or raise expectations that are not covered by existing organisations structures and process. If efficient learning at the workplace becomes transparent and visible through intelligent LA, what does this mean with regard to career development and promotion? Who has access to the data, how are they linked to existing appraisal systems or is it perceived as sufficient to use the analytics for individual reflection only? For the following LA tools a trade-off needs to be negotiated and their practicality in workplace setting can only be assessed when fully implemented. Clear rules about who has access to the insight gained from LA have to be defined. The current approach in EmployID is thus to focus on the individual learner as the main recipient of LA.   

4.1 Self-assessment questionnaire

The project has developed a self-assessment questionnaire as an instrument to collect data from EmployID interventions in different PES organisations to support reflection on personal development. It contains a core set of questions for cross-case evaluation and LA on a project level as well as intervention-specific questions that can be selected to fit the context. The self-assessment approach should provide evidence for the impact of EmployID interventions whilst addressing the EmployID research questions, e.g. the effectiveness of a learning environment in the specific workplace context. Questions are related to professional identity transformation, including individual, emotional, relational and practical development. For the individual learner the questionnaire aims to foster their self-reflection process. It supports them in observing their ‘distance travelled’ in key aspects of their professional identity development. Whilst using EmployID platforms and tools, participants will be invited to fill in the questionnaire upon registration and then at periodic intervals. Questions and ways of presenting the questionnaire questions are adapted to the respective tool or platform, such as social learning programmes, reflective community, or peer coaching.

The individual results and distance travelled over the different time points will be visualised and presented to individual participants in the form of development curves based on summary categories to stimulate self-reflection on learning. These development curves show the individual learners’ changes in their attitudes and behaviour related to learning and adaptation  in the job, the facilitation of colleagues and clients, as well as the personal development related to reflexivity, stress management and emotional awareness.

4.2 Learning Analytics and Reflective Communities

The EmployID project is launching a platform to support the development of a Reflective in the Slovenian PES in February, 2016. The platform is based on the WordPress Content Management System and the project has developed a number of plug ins to support social learning analytics and reflection analytics. The data from these plugins can serve as the basis for a dashboard for learners providing visualisations of different metrics

4.2.1 Network Maps

This plugin visualizes user interactions in social networks including individual contacts, activities, and topics. The data is visualised through a series of maps and is localised through different offices within the PES. The interface shows how interaction with other users has changed during the last 30 days. This way users can visually “see” how often they interact with others and possibly find other users with whom they wish to interact.

The view can be filtered by different job roles and is designed to help users find topics they may be interested in.

4.2.2 Karma Points

The Karma Points plugin allows users to give each other ‘Karma points’ and ‘reputation points’. It is based both on rankings of posts and of authors. Karma points are temporary and expire after a week but are also refreshed each week. This way users can only donate karma points to a few selected posts in each week. The user who receives a karma point gets the point added to her permanent reputation points.

4.2.3 Reflection Analytics

The Reflection Analytics plugin collects platform usage data and shows it in an actionable way to users. The purpose of this is to show people information in order to let them reflect about their behaviour in the platform and then possibly to give them enough information to show them how they could learn more effectively. The plugin will use a number of different charts, each wrapped in a widget in order to retain customizability.

One chart being considered would visualise the role of the user’s interaction in the current month in terms of how many posts she wrote, how many topics she commented on and how many topics she read compared to the average of the group.  This way, users can easily identify whether they are writing a similar number of topics as their colleagues. It shows change over time and provides suggestions for new activities. However, we also recognise that comparisons with group averages can be demotivating for some people.

4.3 Content Coding and Analysis

The analysis of comments and content shared within the EmployID tools can provide data addressing a number of the research questions outlined above.

A first trial of content coding used to analyse inputs into a pilot MOOC held in early 2015 using the FutureLearn platform resulted in rich insights about aspects of identity transformation and learning from and with others. The codes for this analysis were created inductively based on [19] and then analysed according to success factors for identity transformation. Given that identity transformation in PES organisations is a new field of research we expect new categories to evolve over time.

In addition to the inductive coding the EmployID project will apply deductive analysis to investigate the reflection in content of the Reflective Community Platform following a fixed coding scheme for reflection [20].

Similar to the coding approach applied for reflective actions we are currently working on a new coding scheme for learning facilitation in EmployID. Based on existing models of facilitation (e.g. [21]) and facilitation requirements identified within the PES organisations, a fixed scheme for coding will be developed and applied the first time for the analysis of content shared in the Reflective Community platform.

An important future aspect of content coding is going one step further and exploring innovative methodological approaches, trialing with a machine learning approach based on (semi-) automatic detection of reflection and facilitation in text. This semi-automatic content analysis is a prerequisite for reflecting analysis back to learners as part of learning analytics, as it allows the analysis of large amounts of shared content, in different languages and not only ex-post, but continually in real time.

4.4 Dynamic Social Network Analysis

Conceptual work being currently undertaken aims to bring together Social Network Analysis and Content Analysis in an evolving environment in order to analyze the changing nature and discontinuities in a knowledge development and usage over time. Such a perspective would not only enable a greater understanding of knowledge development and maturing within communities of practice and other collaborative learning teams, but would allow further development and improvements to the (online) working and learning environment.

The methodology is based on various Machine Learning approaches including content analysis, classification and clustering [22], and statistical modelling of graphs and networks with a main focus on sequential and temporal non-stationary environments [23].

To illustrate changes of nature and discontinuities at the level of social network connectivity and content of communications in a knowledge maturing process “based on the assumption that learning is an inherently social and collaborative activity in which individual learning processes are interdependent and dynamically interlinked with each other: the output of one learning process is input to the next. If we have a look at this phenomenon from a distance, we can observe a knowledge flow across different interlinked individual learning processes. Knowledge becomes less contextualized, more explicitly linked, easier to communicate, in short: it matures.” [24]

5. NEXT STEPS

In this paper we have examined current approaches to Learning Analytics and have considered some of the issues in developing approaches to LA for workplace learning, notably that learning interactions at the workplace are to a large extent informal and practice based and not embedded into a specific and measurable pedagogical scenario. Despite that, we foresee considerable benefits through developing Workplace Learning Analytics in terms of better awareness and tracing of learning, sharing experiences, and scaling informal learning practices.

We have outlined a pedagogic approach to learning in European Public Employment Services based on social learning and have outlined a parallel approach to LA based on Social Learning Analytics. We have described a number of different tools for workplace Learning Analytics aiming at providing data to assist answering a series of research questions developed through the EmployID project. At the same time as providing research data, these tools have been developed to provide feedback to participants on their workplace learning.

The tools are at various stages of development. In the next phase of development, during 2016, we will implement and evaluate the use of these tools, whilst continuing to develop our conceptual approach to Workplace Learning Analytics.

One essential part of this conceptual approach is that supporting learning of individuals with learning analytics is not just as designers of learning solutions how to present dashboards, visualizations and other forms of data representation. The biggest challenge of workplace learning analytics (but also learning analytics in general) is to support learners in making sense of the data analysis:

  1. What does an indicator or a visualization tell about how to improve learning?
  2. What are the limitations of such indicators?
  3. How can we move more towards evidence-based interventions

And this is not just a individual task; it requires collaborative reflection and learning processes. The knowledge of how to use learning analytics results for improving learning also needs to evolve through a knowledge maturing process. This corresponds to Argyris & Schön’s double loop learning [25]. Otherwise, if learning analytics is perceived as a top-down approach pushed towards the learner, it will suffer from the same problems as performance management. These pre-defined indicators (through their selection, computation, and visualization) implement a certain preconception which is not evaluated on a continuous basis by those involved in the process. Misinterpretations and a misled confidence in numbers can disempower learners and lead to an overall rejection of analytics-driven approaches.

ACKNOWLEDGEMENTS

EmployID – “Scalable & cost-effective facilitation of professional identity transformation in public employment services” – is a research project supported by the European Commission under the 7th Framework Program (project no. 619619).

REFERENCES

[1] SoLAR(2011).Open Learning Analytics: An Integrated & Modularized Platform. WhitePaper.Society for Learning Analytics Research. Retrieved from http://solaresearch.org/OpenLearningAnalytics.pdf

[2] Johnson, L. Adams Becker, S., Estrada, V., Freeman, A. (2014). NMC Horizon Report: 2014 Higher Education Edition. Austin, Texas: The New Media Consortium

[3] Duval E. (2012) Learning Analytics and Educational Data Mining, Erik Duval’s Weblog, 30 January 2012,  https://erikduval.wordpress.com/2012/01/30/learning-analytics-and-educational-data-mining/

[4] Ferguson, R. (2012) Learning analytics: drivers, developments and challenges. In: International Journal of Technology Enhanced Learning, 4(5/6), 2012, pp. 304-317.

[5] Ley T. Lindstaedt S., Klamma R. and Wild S. (2015) Learning Analytics for Workplace and Professional Learning, http://learning-layers.eu/laforwork/

[6] Pardo A. and Siemens G. (2014) Ethical and privacy principles for learning analytics in British Journal of Educational Technology Volume 45, Issue 3, pages 438–450, May 2014

[7] de Laat M. & Schreurs (2013) Visualizing Informal Professional Development Networks: Building a Case for Learning Analytics in the Workplace, In American Bahavioral Scientist http://abs.sagepub.com/content/early/2013/03/11/0002764213479364.abstract

[8] Conole G. (2014) The implciations of open practice, presentation, Slideshare, http://www.slideshare.net/GrainneConole/conole-hea-seminar

[9] Ferguson (2015) Learning Design and Learning Analytics, Presentation, Slideshare http://www.slideshare.net/R3beccaF/learning-design-and-learning-analytics-50180031

[10] Adomavicius, G. and Tuzhilin, A. (2005) Toward the Next Generation of Recommender Systems: A Survey of the State-of-the-Art and Possible Extensions. IEEE Transactions on Knowledge and Data Engineering, 17, 734-749. http://dx.doi.org/10.1109/TKDE.2005.99

[11] Brusilovsky, P. and Peylo, C. (2003) Adaptive and intelligent Web-based educational systems. In P. Brusilovsky and C. Peylo (eds.), International Journal of Artificial Intelligence in Education 13 (2-4), Special Issue on Adaptive and Intelligent Web-based Educational Systems, 159-172.

[12] Zimmerman B. J, (2002) Becoming a self-regulated learner: An overview, in Theory into Practice, Volume: 41 Issue: 2 Pages: 64-70

[13] Bahreini K, Nadolski & Westera W. (2014) Towards multimodal emotion recognition in e-learning environments, Interactive Learning environments, Routledge, http://www.tandfonline.com/doi/abs/10.1080/10494820.2014.908927

[14] MacNeill, S. (2015) The sound of learning analytics, presentation, Slideshare, http://www.slideshare.net/sheilamac/the-sound-of-learning-analytics

[15] Gašević, D., Dawson, S., Siemens, G. (2015) Let’s not forget: Learning Analytics are about learning. TechTrends

[16] Wilson, T. D. (1999). Models in information behaviour research. Journal of Documentation, 55 (3), pp 249-70

[17] Bandura, A. (1977). Social Learning Theory. Englewood Cliffs, NJ: Prentice Hall.

[18] Buckingham Shum, S., & Ferguson, R. (2012). Social Learning Analytics. Educational Technology & Society, 15 (3), 3–26

[19] Mayring, P. (2000). Qualitative Content Analysis. Forum Qualitative Sozialforschung / Forum: Qualitative Social Research, 1(2). Retrieved from http://nbn-resolving.de/urn:nbn:de:0114-fqs0002204

[20] Prilla M, Nolte A, Blunk O, et al (2015) Analyzing Collaborative Reflection Support: A Content Analysis Approach. In: Proceedings of the European Conference on Computer Supported Cooperative Work (ECSCW 2015).   

[21] Hyland, N., Grant, J. M., Craig, A. C., Hudon, M., & Nethery, C. (2012). Exploring Facilitation Stages and Facilitator Actions in an Online/Blended Community of Practice of Elementary Teachers: Reflections on Practice (ROP) Anne Rodrigue Elementary Teachers Federation of Ontario. Copyright© 2012 Shirley Van Nuland and Jim Greenlaw, 71.   

[22] Yeung, K. Y. and Ruzzo W.L. (2000). An empirical study on principal component analysis for clustering gene expression data. Technical report, Department of Computer Science and Engineering, University of Washington.http://bio.cs.washington.edu/supplements/kayee/pca.pdf

[23] Mc Culloh, I. and Carley, K. M. (2008). Social Network Change Detection. Institute for Software Research. School of Computer Science. Carnegie Mellon University. Pittsburgh, PA 15213. CMU-CS-08116.

http://www.casos.cs.cmu.edu/publications/papers/CMU-CS-08-116.pdf

[24] R. Maier, A. Schmidt. Characterizing Knowledge Maturing: A Conceptual Process Model for Integrating E-Learning and Knowledge Management In: Gronau, Norbert (eds.): 4th Conference Professional Knowledge Management – Experiences and Visions (WM ’07), Potsdam, GITO, 2007, pp. 325-334.

http://knowledge-maturing.com/concept/knowledge-maturing-phase-model

[25] Argyris, C./ Schön, D. (1978): Organizational Learning: A theory of action perspective. Reading.

  • Search Pontydysgu.org

    News Bites

    Peer Review

    According to the Guardian, research conducted with more than 6,300 authors of journal articles, peer reviewers and journal editors revealed that over two-thirds of researchers who have never peer reviewed a paper would like to. Of that group (drawn from the full range of subject areas) more than 60% said they would like the option to attend a workshop or formal training on peer reviewing. At the same time, over two-thirds of journal editors told the researchers that it is difficult to find reviewers


    Teachers and overtime

    According to the TES teachers in the UK “are more likely to work unpaid overtime than staff in any other industry, with some working almost 13 extra hours per week, according to research.

    A study of official figures from the Trades Union Congress (TUC) found that 61.4 per cent of primary school teachers worked unpaid overtime in 2014, equating to 12.9 additional hours a week.

    Among secondary teachers, 57.5 per cent worked unpaid overtime, with an average of 12.5 extra hours.

    Across all education staff, including teachers, teaching assistants, playground staff, cleaners and caretakers, 37.6 per cent worked unpaid overtime – a figure higher than that for any other sector.”


    The future of English Further Education

    The UK Parliament Public Accounts Committee has warned  the declining financial health of many FE colleges has “potentially serious consequences for learners and local economies”.

    It finds funding and oversight bodies have been slow to address emerging financial and educational risks, with current oversight arrangements leading to confusion over who should intervene and when.

    The Report says the Department for Business, Innovation & Skills and the Skills Funding Agency “are not doing enough to help colleges address risks at an early stage”.


    Skills in Europe

    Cedefop is launching a new SKILLS PANORAMA website, online on 1 December at 11.00 (CET).

    Skills Panorama, they say,  turns labour market data and information into useful, accurate and timely intelligence that helps policy-makers decide on skills and jobs in Europe.

    The new website will provide with a more comprehensive and user-friendly central access point for information and intelligence on skill needs in occupations and sectors across Europe. You can register for the launch at Register now at http://skillspanorama.cedefop.europa.eu/launch/.


    Other Pontydysgu Spaces

    • Pontydysgu on the Web

      pbwiki
      Our Wikispace for teaching and learning
      Sounds of the Bazaar Radio LIVE
      Join our Sounds of the Bazaar Facebook goup. Just click on the logo above.

      We will be at Online Educa Berlin 2015. See the info above. The stream URL to play in your application is Stream URL or go to our new stream webpage here SoB Stream Page.

  • Twitter

  • Sounds of the Bazaar AudioBoo

  • Recent Posts

  • Archives

  • Meta

  • Upcoming Events

      There are no events.
  • Categories