Week 5 / 6_Evaluation Methods

Requirement for Week 5 & 6

Outline some information about the type of evaluation you believe will best suit your project with your reasoning.

Summarise the article(s) you have found (300 words) - full reference, evaluation design (paradigm, model, questions, methods), findings and how it aligns with the evaluation project you would like to conduct.
========================================

As mentioned in our reading material, to evaluate an e-Learning solution, the first important work to do is to create a practical evaluation plan. The key part of the planning is to decide using which methods of collecting data and evidence, and also to think about how to choose appropriate source of information, for example, collect the information from students, staff or documents.

As mentioned in my Week 4 post, I will evaluate how well students achieve their learning goals for a current online unit 2791-Intergrating data. The evaluation will be focused on effectiveness evaluation. Reeves and Hedberg ( 2003) states that the main focus of effectiveness evaluation is the difference between the KSAs (Knowledge, Skills and Attitudes) before and after the course.

The effectiveness evaluation in Six Facets of Instructional Product Evaluation listed several dimensions of effectiveness evaluation, which will help me in my plan greatly. I will stress on the following aspects of effectiveness evaluation: task-oriented, challenging, constructionist and formative in our learning environment.

Eclectic – Mixed Methods – Pragmatic Paradigm fits in my case. As stated in Project of Evaluation Toolkit :

It's best to use a number of data gathering techniques and/or sources of data to substantiate findings. This is known as a process of triangulation — the use of multiple investigative methods or information sources to home in on the question in focus.

Reeves and Hedberg (2003) emphasis that we should not rely on only one or two measurements while evaluate effectiveness of current learning system. We need to triangulate effectiveness by measuring it with as wide a variety of approaches as our resources allow. Therefore I will conduct a formative evaluation using multiple methods, e.g. questionnaire, interview, checklist, etc. And try to collect data from different groups such as students, facilitators and assessors.

Next challenge for me is implement the evaluation. Need to think about the advantages and disadvantages of different data collecting methods and carefully design the questions so appropriate data can be collected for further analysis. Acceptability, reliability and validity of measurement need to be considered thoroughly during the evaluation.

===========================================

Bibliography

Project of Evaluation Toolkit. (n.d.). Retrieved April 18, 2009, from University of Tasmania: http://www.utas.edu.au/pet/index.html

Reeves Thomas C., John G. Hedberg. (2003). Chapter 8 Effectiveness Evaluation. In J. G. Reeves Thomas C., Learning Systems Evaluation (p. 173). Englewood Cliffs: Educational Technology Publications.