An Evaluation of the Course, Moderate and Facilitate Online (MFO)
Term 4, 2001

We got started with 22 eager participants - 18 from TAFE SA, 2 from TAFE in Victoria, and 1 from a local private provider.

This really was a fascinating journey for me. I have said to some of you in conversations that it was like letting others improvise on your own original composition. Those who write music will know that whenever others join in on an original composition it changes immediately, and the composer can be both fascinated and horrified at how the original piece is altered when others get their hands on it! Even though this course was an exercise in collaborative planning, the fact that I had thought about this course for many months I guess in a sense made it 'mine' to begin with. I was actually delighted by how the course evolved and changed, and I had to let go of any form of ownership as nearly every day in the first few weeks had me thinking "I hadn't thought of that."

For example, I had always thought that such a course would be pretty much run in the forums, and yes, that it would be principally text based - a style of learning that I am very happy with! After the first two weeks' activity (email games and group forum discussions) the first of the legitimate objections to the over reliance on text surfaced. I think we should be enormously grateful to Brian Gepp and others who prompted us to conduct discussions with more graphic input. The "Keeping the Words to a minimum": forum was a great success (more postings than any other forum) though I still think we still have a long way to go here. It was great to see all the pictures that people put up (my pix collection is much healthier now!) but I don't think we really came to grips with how (or is it possible?) to have a discussion on a topic that is done mainly with pictures. And I must say that I felt at a loss about how to make that happen. Maybe it was more the case that at this point in our collective professional lives we are now aware of this issue and the benefits will become apparent later in other forums?

I was somewhat aghast initially at the faith and trust you all put in me, the facilitator, in your comments in the course expectations forum. I honestly don't think I have done your expectations justice, but more about that later. What those comments certainly did do was make me promise to myself (and therefore to you the participants) that I won't let you down, and that this course would be a success.

THE EMAIL GAME

• very successful IMHO
  
• could have been conducted anonymously where all input goes directly to the facilitator who then posts only summaries to the group. This would have cut down on the amount of reading involved and people would not be influenced by what others were writing. I think I would do this next time though some people said they liked reading what others wrote as they were posted
  
• we could have used WebCT mail. The issue here (as it was throughout the course) is whether people like to have everything to do with the course sectioned off in the WebCT environment. I personally don't like using the WebCT mail tool very much, and my decision to use the Yahoogroups mail list was also to give people exposure to another means of communication which is widespread on the Net, and think about which option they preferred. This gave rise to an interesting discussion in the "Forums v Email" forum. One problem with the email.list that I did not anticipate is that many people did not do the course on their own pc, and had to fit it in wherever they were. This meant that they could not access their own email accounts and felt a bit excluded from much of the discussion. You can access mail and post to Yahoogroups from the web if you are registered with Yahoo, and next time I would provide explicit instructions on how to do this for the more mobile participants..
• some people commented that they would have preferred the final list to be put in headings for easier voting. My feeling on this (and many points during the course) was that I didn't want to pre-empt your choices but putting them into categories (I thought this night be me exercising too much control) but on balance it would seem that most people would have preferred some categorisation of the final list and I would probably do this next time.

THE GROUP FORUMS

• Here we hit the first hurdle. Now you were quite dependent on a smaller number of participants to get the most out of this activity. At this point it also became apparent that some people were not very au fait with WebCT and took little or no part. This left some groups with just 3 people in them.

• This was the first point in the course too where participants really were left to your own devices. Instructions were:

  "to get to know your group better….. I suggest that you begin my going back
  to the INTRODUCTIONS FORUM to see what people wrote and start from there.. It is your task to draw each other out, and find out what each of you know - have they had any experience with online facilitation/moderation before this course - as either learner or teacher? what have they experienced that works or doesn't work? can you see how it can be used in your teaching area or workgroup?"

• in all honesty I don't think this was done well by most groups. It may be that the task was too vague. It was certainly my intention to sit back this week and watch how people could engage in a forum discussion. See how well people were willing and able to probe, clarify, challenge and learn from each other. Quite a bit of swapping of info occurred but not much probing took place. Maybe there was simply not enough time to do this. For this to work everyone from your group had to get in to your forum early in the week and contribute, and this did not happen in many cases. I would congratulate group X though for making the best attempt at this activity, and they were one of only TWO groups who posted a summary.
• I would love to know what you think about this activity - why didn't it work for you? If the purpose is to get people really engaging in an online group discussion here, how would you do it? Should the questions have been more specific? (Again I didn't want to be too proscriptive.)
• my personal feeling that this kind of activity is precisely the kind that we need to get used to doing, and setting, for our students, if we are to break the pattern of dependence on the lecturer. I have a sneaking feeling (which may be incorrect) that one of the reasons this activity was not too successful was that it was more student centred than we are used to, and it was up to the participants to uncover a locus of interest and explore it.
• a definite plus from the groups though was that many used the group environment as a kind of home base for the rest of the course. Most of the groups remained active till about week 5 (when the online events took over)

RESOURCE COLLECTION (Week 3)

• task for the week was to locate two suitable resources on eModeration, annotate them, and submit them
• result: only 9 people submitted any resources, and while all of them are useful, some of them had little connection with eModeration per se
• I actually thought that this could easily be achieved in the allocated 3 hours/week and saw it as a week where people could easily achieve the task and have time spare to catch up with activities in weeks 1 and 2!
• so, what happened here?

ONLINE EVENTS (Weeks 4/5)

• these were slow to get started but once they did this is where I felt that the course was well and truly back on track.
  
• good range and variety
  
• lots of participation by those who were still on board

EVALUATION (Week 6 - still happening)

Other Issues

ASSESSMENT AND OUTCOMES

This was fascinating. I guess as it was a pilot, and also because I do have some sympathy with Thiagi's view that you run the course first then decide what the outcomes were, I had given little thought to this. For me the assessment was the taking part in a collaborative course building venture, and the outcome was the course we now have. I hadn't given any thought to learning outcomes for individuals. But as we progressed some raised the question of what constitutes 'successful completion'? And was RPL /RCC a possibility for some sections of the course? I still struggle to find the words to describe this but I'll try:

• this course was a group exercise. It was not meant to focus so much on individuals and how well those individuals performed tasks. (But perhaps it should?)
• every activity was designed to increase our collective knowledge in this area and build a course
  
• to contemplate RPL/RCC for skills gained in previous courses is not the point here. The focus was more how we function as a group in a collaborative environment where we are all dependent on the other for a successful outcome. For example, some may have been an excellent host of their online event - and many were - but did that same person bring those skills to bear in the collaborative forum discussion in week 2?
  
• in any event, IMHO we now have a great shell for a course so the final outcome is extremely positive!
• the task checklist served as a useful tool for everyone to monitor where they were up to, and allowed for some degree of self-assessment

DAILY DOSES

Let me say up front that I loved these, and the discussion they generated. I know some people found them an extra burden in their email inbox every day but I stress again they were optional. Participants were free to delete them and would not be disadvantaged in any way. And the fact that someone responded to almost every dose meant they were striking a chord for some. I'm sorry if they contributed to the overload factor for others but they were optional!

FACILITATION

Areas for improvement:

technical demands: I certainly underestimated the amount of time it would take simply building the site, arranging icons, uploading pages, etc etc. This was a necessary evil first time round but in subsequent runs of this course much of the resources would be there in the background and can just be released as appropriate. The time this all took is one of the excuses I am going to drag out when explaining why other things did not happen. For example:

forum summaries: several times throughout the course I asked people to present forum summaries. You may have noticed that the facilitator did NOT provide summaries of the excellent discussions in the forums (eg the Definitions forum). I intended to and should have, but I simply did not have the time in the end. Let it be known that I know it was an oversight and it won't happen in subsequent runs of this course! It really is a matter of practicing what you preach and in this respect I didn't.

lack of chats: I promised early in the course that I would schedule regular chats in the first 3 weeks of the course. I didn't. Again it was simply a matter of not having enough time. I had hoped in such chats to get feedback and ideas about the course, and to model some moderating techniques for chat that could have then been incorporated into the online events that people hosted later in the course.

reluctance to 'correct' peers: there were times when I was reluctant to be 'the teacher' and offer advice to people about things like forum summaries and running chat sessions. This was because I knew the people concerned, and because I had not modelled the sorts of strategies I was asking of others.

knowledge of participants: in general though, the fact that I knew most of the participants really worked in my favour most of the time. Because I knew people I did not have to be super sensitive in the language that I used. I could be more informal. If I had been facilitating a group of people I did not know, I would have thought much longer and harder about what language and tone I adopted. However,

offending people you know: I know that on a couple of occasions I did offend people on the course because I may have countered their arguments, or offered another perspective and perhaps used language that wasn't quite soft enough. I don't know. Maybe offending someone is inevitable in this situation - especially as it is online. As one of the daily doses said, "causing offence in this medium is inevitable."

Other

daily f2f contact with many participants: this was a joy and an absolute luxury. Most days as I moved around TAFE I would see people in the course and chat about what was happening and how it could be improved. This was a whole hidden dimension of building a course together that I had not anticipated and it was enormously valuable. So to all those who engaged in MFO chat around the corridors and offices over the last 6 weeks - thank you.

OVERALL

Someone in the their evaluation has already commented that the person who learnt most in this exercise was me, and they may be right. I did learn a lot - about moderating and facilitating online, people in groups, using WebCT, and plenty more. I just hope that participants learnt from it too. Term 4 was clearly not the best time to hold this course. The later the term got the harder it was for people to find the time to give the course justice. However I am sticking with the belief that it could have been done in 3 hours a week. Do you agree?

12 out of 22 finished (54%) - not a great outcome in terms of numbers, but as I said earlier, if you look at the course and structure we now have then this whole exercise has been enormously successful.

Watching the ebbs and flows of the course was interesting. It was never dormant, and what typically happened is that in any 2 or 3 day period a few people would be really active and would not be seen again for a week or so. But they would then be replaced by others who would become active for a few days. So it was always ticking over.

STATISTICS

Withdrawals (10 or 46%)

• 2 due to family reasons
  
• 5 due to workload pressures
  
• 1 due to change of job
  
• 2 unknown
• Completions (12 or 54%)

The MFO site:

• most number of entries to the course: Valda
  
• most postings read: Valda
  
• most postings written: Brian (the man who doesn't like text!)

Yahoogroups Mailing List

• 201 messages posted
  
• most posted (apart from facilitator): Carole
  
• messages posted by facilitator: 65

And how long did all this take me? At the time of writing the count is approx 45 hours. So if you embark on this kind of exercise allow at least 2 hours for every hour of course time for preparation and facilitation.