Published in Science!

Cover and pages from article in ScienceIn my mini-series of posts on papers there is a brand new one that I am rather proud of. The paper came out of conversations with Eileen Scanlon just after we heard that she was to become the new Regius Chair in Open Education. The focus was around the “known knowns” of open education, particularly as there has been a slightly strange period of reinvention of ideas for online and distance education around MOOCs. The idea of known knowns is actually quite useful (but the resonance with Donald Rumfeld’s statement is not) and so we have refined the approach leading to a short (2-page) article that is appearing in the Educational Focus section of Science Magazine. This reflects a very kind connection from Candace Thille (my co-Director on OLnet when she was with the Open Learning Initiative at Carnegie Mellon University) and the great support of Brad Wible the editor at Science.

Working with Eileen has been a very good experience and has taken us back to looking at the history and writings on what we know through 40 years of being The Open University, coping with the way things are changing but also the way things remain the same. The article is very short and had many references trimmed in the edit so I hope we can go on to produce a longer version giving more of the background. Brevity does mean focus, and I think that we have kept the four main lessons we wanted to communicate and also some of the lessons that matter more for science education (Eileen’s own focus area for many years).

The article also grew a stronger link to MOOCs. The first version tried hard not to mention them as there is actually much more going on in open education than just MOOCs. The title in the end though does capture some of the intent to encourage people to not reinvent, rather learn. The four lessons from the article can be summarised as:

  1. Build on distance learning pedagogy. Starting from classroom experience misses on what we know about engaging learners at a distance.
  2. Plan to help learners who need support. MOOCs most easily meets the needs of the experience but offers a path for those under served by other educational offerings. If we can push ourselves further to design for their needs then everyone will gain.
  3. Recognise the power of well-designed assessment. Self-directed learning needs integration and motivation to make sense for the learner. Bringing in assessment in intelligent ways can make all the difference to enable a learning experience to make sense for the learner and for society.
  4. Ensure quality by working together. Working in the open and online also means that we have different ways of working. Building a complete organisation, such as the OU, does not need to be the answer but there are three steps to quality that should be there: judge whether fit for broadcast, bring people together, and test the technology. In brief always ask at least one person whose opinion you respect “is this ok?”.

The article itself is now out in the 20 December edition of Science, though they have a relatively enlightened attitude to republishing so I think we will be able to provide open access to our article in time.

Old paper: Learning Design revisited

For this post I am going to revisit an old paper for reasons described in my previous post on old paper, new paper.

The paper is:

McAndrew, Patrick, Goodyear, Peter and Dalziel, James (2006) “Patterns, designs and activities: unifying descriptions of learning structures,” International Journal of Learning Technology, 2(2-3), pp. 216–242, [online] Available from:

Picture of the IJLT issue that the article appeared inIt appeared in a Special Issue (actually double issue) of IJLT which meant that there was some delay from writing to publication so the paper itself came out of some study leave time spent in Australia in 2004. I was very kindly hosted by Peter Goodyear at the CoCo research centre at The University of Sydney. The whole visit was wonderful in many ways. Part of the time was working with Peter and with James Dalziel from Macquarie University on the different perspectives we each brought to considering the ways that teachers design their materials. As the person with the study leave time I carried out the integrating task in the paper, but what was key was the brainstorming sessions in Peter’s office and a visit up to James’ base in Macquarie to see his work.

While our views overlapped, the paper presents three different perspectives which were each led by one of us: Peter on patterns, James on LAMs and myself on Learning Design. At the time IMS Learning Design was very visible and had expectations that it could provide an integrating framework from shared designs through to playable systems.

With the benefits of hindsight it might have been thought that we could say which perspective was the “winner”. But hey I am an academic so it is not going to be that simple. There has been some recent reflections on the over-enthusiasm for precise ways to describe learning designs, i.e. IMS Learning Design has not met all its hopes.  In the paper it says:

“… IMS Learning Design … may not provide significant support for exchange of understanding and reuse in a [way] that recognises adjustment to context and draws on the skills of both the original designer and those of the teacher involved in the reuse.”

Flow chart from Architecture to Pattern to Instance to Run

Figure 3 from the paper

I think that what has emerged is more a hybrid of the different approaches. Perhaps not as directly as suggested by the paper’s outlining of an alternative form of James Dalziel’s LAMs system, but with a broader aim to help share approaches and help people think of alternatives. It certainly should not be said that learning design has failed to have impact, especially here at the Open University.

After my work on the paper I have spent a period less directly involved in learning design research, focussing rather on open education, however things have certainly moved forward. The OULDI project which combined external support from Jisc and internal focus on updating module design inside the OU culminated in a set of tools and the running of the OLDS-MOOC (more on that in a paper and evaluation report). The publicly released tools demonstrate the hybrid approach: unlike IMS LD the output is not intended to be a full playable encoding of the “unit of learning”, it goes beyond patterns in using a range of tools to capture and represent the stages in a way that measures some of the impact, and while it mirrors LAMs in having a relatively easy to use toolset it does not integrate with a student facing component.

Flow from Design to Activity to Analytics to AnalysisThe OULDI work, not surprisingly, fits with the way we do things here. A team approach that needs clear steps (stage-gates) and multiple perspectives. Learning design is then communication but also an essential part of understanding expectations on learners which can lead into making effective use of learning analytics. In the 2013 Innovating Pedagogy report the point is made that there is a cycle from design to activity to analysis and back to design (the report was collaborative so not quite sure who to credit for this – maybe a mix of Doug Clow, Rebecca Ferguson and Mike Sharples) . Without the design part the chance to actually do something with what we measure is much reduced.

To bring things up to date learning design is a also a key element in the METIS project (see the recent METIS newsletter). There learning design is used to structure workshop designs and introduce the tools that help capture ideas. For the pilot workshop inside the OU the focus was on designing collaboration, not on learning design itself. Perhaps this demonstrates that the learning design approach has matured; no longer a novelty to be introduced (as it very much was in 2004) rather an assumed need to design learning in 2013.

This post started with a look back on my paper from nearly 10 years ago. It has ended with more connections to current work (of others more than myself) than I expected – and plenty of signs that Learning Design is of more importance now than then – but maybe no longer part of the hype cycle.

Old paper, New paper

Photo of the Spines of the Readings in books

Morgan-Kaufmann Readings in…

This is really the introduction to another couple of posts – one where I talk about a relatively old paper of mine (just about ready to go), and the other where I intend to discuss a fairly recent paper (when I write that one). So if you don’t want a post that describes why I posted other posts then don’t read this post read those posts :-).

As with many other people who work in UK academia a *lot* of time and effort has gone into the preparation of of information for the UK REF (Research Excellence Framework) which is just about finished with. The REF occurs every few years (last one was in 2007) and has many complications and aspects. One of these aspects is that it uses published pieces of work to allow the panels to assess the value of work. This means a list of upto 4 papers are submitted for each person and those papers must have appeared between January 2008 and December 2013 (though preparation time means that realistically the limit is earlier than that). Anyway that is already too much about the REF, though I suspect it will haunt us over the next year or so until the results flow back.

The consequence is that we tend to ignore papers we wrote before 2008, and in a year’s time we will be starting to pay less attention to the papers that made it into the 2013 submission as we start to collate evidence for the next time around. A year or two back we had a discussion in IET about this and a good idea emerged which was to gather together some of our best past papers and make a book of the collected works with a bit of narrative.  The idea was that we could and up with something similar to the Morgan-Kaufmann “Readings in …” series that I still have on my shelves from when I started my PhD – then it was a great convenience to get some of the best papers on a subject without standing for hours at the photocopier. (Martin Weller also had an interesting version where we could use his meta-journal approach to collect things – definitely a good way to go.) For whatever reason (probably that REF thing) we have not got around to actually building the collection but the idea has stuck with me.

So my next post is the “Old paper” one where I will revisit a paper that I am particularly proud of that was written in 2004 and appeared in 2006 … Learning Design revisited.

My “New paper” is a trickier choice, tempted by the very latest paper submitted and *about* to appear but not quite wanting to tempt fate on that. Or a couple of recent papers on the theme of openness, or when I finally got something published on Activity Theory, or getting a chance to work on the theme of accessibility, or … – so will see exactly what I do in the next couple of weeks.

Innovations, crowds and a bit of Direction

IET watermark consisting of letter ietI have just started a new role within the Institute of Educational Technology coming in as Acting Director as Josie Taylor steps down and retires. I have worked with Josie from my very first days here in IET 14 years ago. We constructed the funding bid to HEFCE for the Knowledge Network (or as we termed it then UNLOCK – University Networked Location of Community Knowledge) and set up the UserLab as a way to manage having several EU/internationally funded projects in the same area at the same time. Working closely with Josie as I have shadowed her over the last few weeks has been a great introduction and I am happy to say that I am able to take on IET in a good state.

Photo of Jennie Lee Building at The Open University in Milton KeynesIET has slimmed down to now have  about 90 staff in total across academics, researchers, academic related and support staff.  But IET’s research is healthy and the works it carries out inside the university seems more in demand than ever. All universities are having to cope with changes in how students view them, how they have come to study and the different options they now have. Perhaps the impact on study at the OU have been even bigger as it is a university that operates across all four nations as well as through Open Educational Resources. The OU needs to cope with various systems and in particular with the way part-time study has to adjust to the requirements for loans imposed on students from England. That is a major change that needs to be reflected on in terms of learning design, accessibility, data analytics and quality enhancement. All aspects where IET places a major role within the OU.

A big topic for this year is the coming together of learning opportunities around free resources. My research lies in this area, with the OER Research Hub, and the OU has brought together universities and other organisations from the UK and beyond in FutureLearn. This is an innovative way to lower the barrier to taking part in courses (in fact while typing this I though why not actually join the latest offering of a FutureLearn course from the OU on Ecosystems – the elapsed time from thought to registration was just under 2 minutes).

FutureLearn and its Massive Open Online courses (MOOCs) is only part of a broader approach to OER for the OU. There are also direct open courses, such as OLDSMOOC and H817Open (both operated out off IET) and continuing investment in OpenLearn, iTunesU and YouTube from the OU’s Open Media Unit. Not forgetting great BBC programmes.

Cover of Innovating Pedagogy reportThe work on the Innovating Pedagogy reports, which I was pleased to be able to be part oded helps us reflect on what this means for pedagogy. The 2013 report balances various options and considers 10 areas but one that is coming out as more prominent is how what we do with students is more and more overlapped with what is happening in the world. In the report this is seen as “Crowd-learning” and Mike Sharples in an article in the Times Higher speculates that perhaps 2013 is the year of the crowd. I had the chance to present on the Innovating Pedagogy report at the recent EADTU conference in Paris, already blogged by Leigh-Anne Perryman on the site. My own slides (below) covered the pedagogic lessons from the open universities and how the innovations we report are helping review these.

My MOOC learning so far: MITx, Google, Coursera and a little bit of edX

MITx 6.002x certificate

Last year I blogged about my early experience with the MITx course 6.002x Circuits and Electronics and always meant to follow it up with a further post to say how things ended up. The good news is that I received my certificate completing the course with (just) enough to gain an A grade and quite a satisfying feeling. As a leisure activity I enjoyed the challenge but rather worryingly found out that I responded more to the assessment driver than I expected – I think I might have been more pleased with myself if I had been cooler and more laid back about actually getting the (officially meaningless) certificate. One year later the content has faded away and I have not designed or analysed any electronic circuitry 🙂 but I am still very glad I took it.

Since then I have taken part as a learner in three further MOOCS.

  1. The Google Power Searching MOOC – this was interesting as it uses Google’s Course Builder. Much easier than the Maths of MITx 60002x and not that engaging. But it has meant I now use the advanced tools in Google a bit more than I did before so has made some impact on my work efficiency.
  2. The eDx course in programming in python CS50x. This is my “failure”. I made three attempts get started and got caught out by multiple logins so gave up at the first hurdle. Not completing puts me in with 90% of others (actually for this course via Katy Jordan’s useful summary of MOOC completion, in with 99.1% of the others).
  3. Coursera’s rerunning of Andrew Ng’s Machine Learning course. This is a classic alongside 6002x. Rather lighter though with a strong programming element. Very assessment driven. Two or more assignments every week which have to be in the same week the content and questions are available. Just finished and I made it to the end completing all assignments and programming. Overall slightly slicker than the earlier 6002x experience. But I will only watch videos on high speed and again ended up reading assignments and transcripts rather than watching videos. I *might* have an application for what I am learning in reanalyising some survey data but still not sure it quite fits with what I have learned.

Scatter diagram of completion of MOOCs v numbers enrolled

MOOCs continue to be interesting learning experiences. My use for slightly off the side professional development fits well with their strengths and seeing them as a leisure activity. The assessment driven group also shows up in the analysis reported in an interesting paper by Daphne Koller, Andrew Ng, Chuong Do, and Zhenghao Chen . Using techniques explained more fully in the Coursera course I have just take they demonstrate both a mathematical explanation of completion (i.e. to model those taking courses as two populations; those who complete, and those who don’t – the trick is to know which is which) and also cluster people into different groups based on factors such as how many video lectures have been viewed against the assignments completed..

Heat diagram of lectures attended v assessments completed

There are some signs that MOOCs may have peaked a bit. I think there is more to come and courses are evolving so that they will have a roll in learning in the future. What that future is I am less sure of, for me the link with formal is not clear – as a good way to carry out learning for interest they are fine – however that does not meet all of the hopes that we have for open learning.


An Agile Hypothesis Sprint: what do we know about learner performance?

The structure for the research for OER Reserach Hub is built around 11 hypotheses that we are testing through our work with collaborating organisations, fellowships and background studies. We are now at a stage where the collaborations are well underway and surveys are producing data and we want to bring together the views from the team around the hypotheses and reflecting on our research.

On an away day at the beginning of June we took some lessons from Agile Programming to become Agile Researchers and carried out an Hypothesis Sprint, involving Sprint Boards, T-shirt sized tasks, burndown velocity and mini-scrums!

Guided by Martin we picked one of our hypotheses and focused in on achievable tasks, reporting quick progress and then getting a reasonable result all in one morning. The method appears to work well and if it does give us more of what we need then expect we will expand…

View original post 188 more words

Starting out with MITx 6.002x

Typical signing up page

I have finally become a student again! What is more I am now enrolled with MIT(x) and it did not take a huge fee or any tough entry requirements. I am one of the 90,000 people who clicked the enroll button and signed in. For me there are several motivations:

  • I work on Open Educational Resources and feel I should experience what it is like to learn from them (I nearly did this with Sebastian Thrum’s AI course and regretted not going through when I spoke to an enthusiastic Seb Schmoller about his experience on the AI course).
  • I know some MIT folk and they come up with some interesting directions that it would be good to see in more detail. So this could feel like going in undercover and this way I needn’t feel bad about not doing the assignements but …
  • I nearly set out to do a course like this as an undergraduate. In the end I took Maths but always wondered if I should have taken electronic/physics. So I would like to do this for real.

So what has it been like? Well actually it has been quite hard! The level is fairly high and I have found in the first week that I have been integrating trig functions, differentiating using the product rule and writing rather a lot of symbols on pieces of paper! I have also found it quite rewarding – a bit like suduko on steroids. Puzzles that I really have to work at – and then carry on working on late at night. So far I have managed to come out the other side ok as luckily the “homeworks” all multiple attempts, I have a feeling that I have yet to get anything completely right on the first attempt.


The course itself has:

  • Videos to watch – these are sequenced and mixed with exercises. They are captioned but as Brandon Muramatsu points out in his post 6.002x Video: First Impressions they could have done this a bit better. Proper captioning would make quite a difference for me as I find I would rather read the words than listen and set things at 1.5x to help speed things up. (I think the team are listening though as it now remembers that I like video at 1.5x and I think at first I had to keep resetting that.
  • A huge textbook. This is a commercial book that you could buy but is available free. But in the free form it is quite annoying only page by page access and no downloading it all. I would really like this on my kindle and that is not possible, and I think actually the conditions would say that I cannot engineer it myself.
  • Discussions and a wiki with hints and feedbacks. The discussions are using a system that I think is called Askbot and so is based on Q&A rather than discussions as such. This might limit the chance for the community itself to form and is getting a bit unwieldy. There are also groups on other places such as facebook – but I have not explored those.
  • Exercises to work through. These typically ask questions and then you enter responses which are judged. Then you can look at the answers if you want or try again. The answers are often numeric.
  • Tutorials running through in videos worked examples. I have not watched many of these and it is quite hard to know what they will cover without watching them through.
  • Homework. These are similar to the exercises but you do not get the option to see the answer, These are fine – but they are quite challenging. Typically they extend well beyond the work covered in the videos and are difficult to match to the textbook as well.
  • Labs. These use a circuit simulator where you build up circuit diagrams and then calculate values or run a trace.

For me the homework has become the driver – quite revealing in itself though not a surprise to find assessment is taking over. The time I have spent was probably not far off the 10 hours in the first week but then a bit less as too much else has been going on in the second week.

I am enjoying doing the course though if the difficulty scales up I might fall by the wayside. I am not seeking to be critical in this post but I do agree with Brandon that a bit more effort to get the accessibility working better is needed. For the learning process I would normally expect stronger alignment between the materials and what is tested as well but actually having the homework as puzzles in themselves is quite interesting. I can then carry out a treasure hunt to see where the clues are hidden (in texts, videos, other exercises, discussions and out there in google search land).

I am not sure 6.002x should count as “Open Educational Resource” (OER), it is built on some materials already released on MIT’s OpenCourseWare site. But the addition of the textbook, the closed access to the course itself and that nothing in the design encourages transfer to another site make it more a free course than an open one.

At this point there is just time to sign up (first homework closes 18 March). So if you like mathematical puzzles, wonder if you should have done/should do electronics, and are interested in how open learning works this could be just the thing!