In reflection... evaluation is a necessary process to establish or improve the effectiveness of a course.
As identified by Reeves and Hedberg (2006), preparing a plan helps you understand the size and scope of an evaluation project, and establishes interaction with stakeholders to help with identifying key questions and decisions.
The process has been informative and extremely challenging at times.
Working collaboratively has been interesting, sharing ideas and experiences however, being able to establish a regular 'meeting' time would have been more productive. Difficulties were experienced due to varying work demands. Problems with the LMS affected timelines and accessibility, and therefore continuity of the process.
I was surprised I hadn't received any comments on my blog until a colleague informed me that they would have to subscribe to blogger.com to be able to read my posts. I thought a link had been applied to Blackboard. Something to bear in mind if students were to set up an external blog.
In regard to our evaluation, the results identified the course had overall been effective.
One particular aspect the students identified as a concern was the slowness of the internet speed.
A positive result as to to our questions being answered as to whether we should continue with online delivery and adapt this approach into other subject areas.
Bates P., & Obexer R.(2005) .Evaluating student centred teaching and
learning strategies for Aviation students using a quality framework for online
learningenvironments. Retrieved from
Reeves, T. C., & Hedberg, J.G. (20080. Evaluating E-Learning : A user friendly guide. (In press)
Regarding Assessment 4, checking interaction on another blog/project - I have interacted, perhaps somewhat briefly on a couple of blogs, including projects but didn't make note of this.
Monday, December 19, 2011
Finalising the evaluation process...
As mentioned in previous blogs, the report by Bates and Obexer (2005), influenced to a certain extent the methodology used in our evaluation.
The report was based on a variety of theories combined (Gunawardena, 1996, Palloff & Pratt, 1999, Salmon, 2000, and Jonassen, 1998) which centred around the five principles of social presence, interaction, cognitive learning, and learner-centredness, in an online environment.
Bates & Obexer used an eclectic mixed-method appproach (Reeves, 1996) was used combining qualitative and quantitive methods, which was also our approach in order to triangulate our findings.
I thought it a useful approach to complete the two questionnaires, one earlier on in the course and then at the end, to evaluate students perceptions in view of teaching and learning strategies.
Our data collection also included a questionnaire, a student focus group to encourage further feedback and a peer review to gain a perception of the course, from a fellow tutor.
They also included an observation which was not possible with our evaluation as our evaluation was being conducted as a summative effectiveness evaluation, however, observation of student interaction with the course and materials were observed via our peer review.
The study involved a much greater student body, 49, as opposed to our 16 students. However, of the 49, only 12 students (24%) replied to the end of semester survey, whereas from our 16 students, 8 students (50%) replied.
The results of the analysis by Bates and Obexer are given in regard to the five principles of the framework:
Social presence
A strong social presence was provided, actively encouraging student participation. The students felt part of their learning community and enjoyed the online environment.
Interaction
A high level of interaction was indicated by the postings on the discussion board. They found found the discussion board effective as a learning tool.
Cognitive strategies
The students stated that the learning activities stimulated their learning, and that the content of the course authentic and well-structured.
Collaborative learning
Students didn't seem to enjoy this type of activity, although 50% did indicate the group assignment useful.
It was noted that the activity was very time consuming.
Learner-centredness
The purpose of putting the responsibility of learning into their own hands.
The students perceptions were mixed. The negotiation of marks identified 8.3% agreed it was useful for their learning, 16.7% not sure, and 58% disagreed or strongly disagreed with the statement.
It was suggested where some students are ready for a student-centred approach, others may need specific coaching.
The conclusion of the findings were not as clearly reported. One needed to read through the report to gain a clearer understanding. However, I found the report easy to read.
As mentioned in previous blogs, the report by Bates and Obexer (2005), influenced to a certain extent the methodology used in our evaluation.
The report was based on a variety of theories combined (Gunawardena, 1996, Palloff & Pratt, 1999, Salmon, 2000, and Jonassen, 1998) which centred around the five principles of social presence, interaction, cognitive learning, and learner-centredness, in an online environment.
Bates & Obexer used an eclectic mixed-method appproach (Reeves, 1996) was used combining qualitative and quantitive methods, which was also our approach in order to triangulate our findings.
I thought it a useful approach to complete the two questionnaires, one earlier on in the course and then at the end, to evaluate students perceptions in view of teaching and learning strategies.
Our data collection also included a questionnaire, a student focus group to encourage further feedback and a peer review to gain a perception of the course, from a fellow tutor.
They also included an observation which was not possible with our evaluation as our evaluation was being conducted as a summative effectiveness evaluation, however, observation of student interaction with the course and materials were observed via our peer review.
The study involved a much greater student body, 49, as opposed to our 16 students. However, of the 49, only 12 students (24%) replied to the end of semester survey, whereas from our 16 students, 8 students (50%) replied.
The results of the analysis by Bates and Obexer are given in regard to the five principles of the framework:
Social presence
A strong social presence was provided, actively encouraging student participation. The students felt part of their learning community and enjoyed the online environment.
Interaction
A high level of interaction was indicated by the postings on the discussion board. They found found the discussion board effective as a learning tool.
Cognitive strategies
The students stated that the learning activities stimulated their learning, and that the content of the course authentic and well-structured.
Collaborative learning
Students didn't seem to enjoy this type of activity, although 50% did indicate the group assignment useful.
It was noted that the activity was very time consuming.
Learner-centredness
The purpose of putting the responsibility of learning into their own hands.
The students perceptions were mixed. The negotiation of marks identified 8.3% agreed it was useful for their learning, 16.7% not sure, and 58% disagreed or strongly disagreed with the statement.
It was suggested where some students are ready for a student-centred approach, others may need specific coaching.
The conclusion of the findings were not as clearly reported. One needed to read through the report to gain a clearer understanding. However, I found the report easy to read.
Wednesday, December 14, 2011
We finally submitted the evaluation results for some feedback - and had a meeting discussing how the results could be collated for our evaluation report. This has probably been the most challenging course to stay focused on not only with the group work but also in regard to the time of year. We decided the report should be added to our pbworks site so that we could collaborate. This has definitely been an interesting exercise in education and online learning.
Monday, December 5, 2011
Identified in an earlier post literature such as Bates and Obexer influenced the methodology and ideas for our plan. The research was based on a framework outlining important components in an online environment (social presence, interaction, cognitive strategies, collaborative learning and learning centeredness). We felt this framework would be useful as a guide towards our own evaluation, although the aviation group was greater in number (49) to our 16.
Sunday, December 4, 2011
Looking back at my notes, I acknowledge looking forward to when we will be able conclude and submit our final assessment.
November 15 we were fixing up suggestions made to our draft plan......the problems experienced with our email service being down was frustrating (drawing our submission date out further!)
I was experiencing problems formatting but Sue managed to sort it out.
It was with great relief to hit the send button for submission :)
Work commitments have occupied much of my time of late..... the difficulties experienced maintaining sufficient contact to move ahead with our assessment has been a challenge to stay on track.
Perhaps I shouldn't need it at this level, but an email to remind me what week we were at would have been helpful - a gentle prompt to check the course blog.....
We had a very successful meeting with regard to re writing the questions - improving appropriateness/effectiveness, and to ensure fit to triangulate findings
Leanne was to organise students to complete survey and Gary to conduct interview.
I received an email from Sue today with the statistics of the online questionnaire completed, and suggested we have a meeting to organise how the results are going to be presented, and writing up our report.
November 15 we were fixing up suggestions made to our draft plan......the problems experienced with our email service being down was frustrating (drawing our submission date out further!)
I was experiencing problems formatting but Sue managed to sort it out.
It was with great relief to hit the send button for submission :)
Work commitments have occupied much of my time of late..... the difficulties experienced maintaining sufficient contact to move ahead with our assessment has been a challenge to stay on track.
Perhaps I shouldn't need it at this level, but an email to remind me what week we were at would have been helpful - a gentle prompt to check the course blog.....
We had a very successful meeting with regard to re writing the questions - improving appropriateness/effectiveness, and to ensure fit to triangulate findings
Leanne was to organise students to complete survey and Gary to conduct interview.
I received an email from Sue today with the statistics of the online questionnaire completed, and suggested we have a meeting to organise how the results are going to be presented, and writing up our report.
Subscribe to:
Posts (Atom)