Thursday, May 21, 2009

Wk10_Evaluation Draft Plan - V2.0

Thank you everyone for your valuable feedback. I've read the feedback from Bronwyn and Simon, made some changes. My evaluation draft plan (2nd version) is published here

Detailed evaluation questions will be added into the plan later. Any feedback pls?


  1. Hi joy,

    How are you, hope howick class is not keeping you too busy.

    I just went through your plan, good choice - evaluating 2791, I think its true a lot of students are not able to get the bigger picture of what they are trying to learn. Your sub questions for the guidelines are appropriate to find the effectiveness of the course.

    Just ensure that students are aware that they are only evaluating unit 2791, rather than the whole cluster that involves 2791. :)

    Overall your plan is looking great

    Good luck with your evaluation


  2. thx Krishan. Good suggestion:-) I will focus only on Unit 2791. Might need ur further help while interviewing facilitators :-)


  3. Hi Joy
    This looks very good and well put together. I can understand the problem of completeing the activities yet not understanding the fundamentals. Learning objectives can also be difficult to understand at times so students ignor them. Good coice of guidlines and sub questions

    I am interested in the focus group, in how it works, how you select students? - please keep us posted.


  4. The following info is the feedback from Debra. I think it's very valuable and copied it here for 'permanently' keeping the info.



    Hi Joy

    I just tried to post comment on your Blog- but for some reason it "bounced me off"

    Hi Joy
    Enjoyed reading about your evaluation project. At UCOL we deliver the NCC programme in a 'blended' format, rather than all online- students work at a distance with the text book resources as well as face-to-face contact (in our community computing classrooms) if needed, plus 0800 phone support. And I agree, especially at Level 2-from what I hear this unit is certainly one that some students struggle with. Not enough opportunties to practice the intergration concept without fully appreciating the reasons for doing it. Guideline ST4 seems an excellent choice for exploring that issue.

    I have a question: By saying (in Background) that the students are 'missing the learning objectives- therefore they can't assess themselves'-do you mean they don't know why they would need to be intergrating data? They don't have any opportunity to acknowledge their prior knowledge in WORD, EXCEl and ACCESS (as already taught)?

    I also appreciate the difficulty you mention in sampling the students- you might not even get to see those students who have not had problems with the unit in the classrooms- as they are completing solely online, and aren't coming in for any extra assistance, I assume. I can see the importance of talking to other facilitators as 'experts' for part of your evaluation strategy- that information will be very useful, with the difficulties mentioned already in student sampling.

    Last comment- are you going to include some sort of general question on the level of computer literacy initially? This may have affected some students understanding on the unit standard how well did they did with each of the prior units for WORD /EXCEL / ACCESS may have had an effect?? (good for a comparison...)

    Will be keen to see how this develops as it is a good project Joy!


  5. Thx guys for ur feedback again. Appreciate ur support :-)

    Hi Pradeep, good question about the focus group in my evaluation plan:) I intend to select students who are currently doing the unit and those just finished it. I think it might be very interesting to get the feedback from them. But 'cause our students coming in their own timetable and at the moment not many of them are doing the module. So not sure whether I can gether them together. It's just an idea and I am interested to know what information will be collected through focus group. Hopefully I can organise one...

    Thank Debra. Our students are not given any learning objectives at the beginning/end of the course. Instead they are given step by step instructions. So they totally focus on completing the tasks following instructions without any independant thinking or exploring. They thought once they can do the tasks correctly and they meet the learning goals. That's why I said they cannot assess themselves whether they achieved the learning outcomes. Actually even the objectives are listed, like Pradeep mentioned above, students are likely ignoring them and diving into the instructions directly. So it's interesting to know whether the objectives really works, and how much they can help. Honestly most of the time I wont check the learning objectives:-(

    The online material is very easy to follow. We havent got any student has difficulties in finding and access the material. So I wont take account the computer literacy in this evaluation. But I do realise it may cause significant difference in other circumstances :-)

    It's a good idea to have a comparison of the effect of how well they done in Word, Excel and Access. In this case, due to the timeframe limitation, I try to keep the evaluation simple and small. If we have enough time and reasonable sample size, maybe we should consider the issue you raised ^_^

    Thx a lot, pp :-)


  6. Thanks a lot for sharing this amazing knowledge with us. This site is fantastic. I always find great knowledge from it.