Summary

As a response to improve the learning processes and opportunities for students, the School of Medicine recently piloted the CollabHero online platform to facilitate the provision of student peer feedback to other group members. Staff feedback when compared to prior methods was that it saved time, was easy to develop and set up tasks, simple to implement, and provided collation of data. It also had positive student responses at the user end with reports of easy to provide peer review feedback, clear instructions and further easy access the feedback that they received from their peers. As an online tool, CollabHero facilitates staff and student use of the system by not being bound by time and place. In addition, it provides other functions such as reporting on those students who may need a reminder to complete their tasks and allows for feedback of written work through online annotations.

The Story

Self and peer assessment has been considered as an innovative approach to assessment in that it activates student learning by making them assessors rather than the usual passive assessed. While there are various frameworks, self and peer assessment brings various developmental and learning opportunities for students (Adachi, Tai, & Dawson, 2018; Boud & Tyree, 1980). The benefits of self and peer assessment relates to what Boud and Soler (2016) described as sustainable assessment where assessment works as a cultivation of life-long learners. In self and peer assessment, students first need to critically engage with a set of standards or rubrics in evaluating their work and that of others. They then ought to carefully craft, receive and evaluate feedback to improve their future work. This process then allows them to develop other transferrable skills (i.e. graduate attributes) such as critical thinking, communication, teamwork and evaluative judgement skills (Tai, Canny, Haines, & Molloy, 2016). While there is a concern about the accuracy of student evaluation in disciplinary specific areas as they are considered to be novices in their fields, it is possible for students to assess each other and provide constructive feedback especially in the areas where teachers are less able to accurately assess – e.g. teamwork skills within project work. However, the design and implementation of effective self and peer assessment takes considerations for all factors to align – use of technology is certainly one such factor (Adachi, Tai, & Dawson, 2017).

Until recently those teachers wishing to employ self and peer feedback for students, have needed to rely on paper-based methods of data collection, followed by intensive data collation, interpretation and dissemination of outcomes. This was time consuming, required considerable effort and planning with chances of errors in transcribing the summative and formative feedback for dissemination of results to the students. CollabHero was developed out of the foreseen need to provide an online and easily managed system to facilitate peer feedback and evaluation. As a participant in Deakin’s Spark start-up incubator, CollabHero was successful in gaining funding to support development of the application in collaboration with various teams and expertise around Deakin (e.g. DLF, CRADLE). As an online system CollabHero provides a range of advantages and efficiencies in developing, administering and reporting on student peer feedback. As the system was new there was some training for staff involved and documentation and video guidance was provided for students so they had a visual guide on how to work through their feedback task.

The design of the system means there are only six steps to complete the setup of a feedback task in CollabHero. Firstly, the type of task is chosen where it is either an Individual or Group Task and the unit, title, description, and associated GLOs are provided. Next the type of feedback is chosen (intra or inter group) as well as the type of feedback (free annotations or guided prompts) and some administrative options are chosen. Next the guided prompts or questions are entered – these can be a text, numerical scale, or rubric response and you can enter as many as necessary. The next step is to add the students and groups who will be undertaking the task and this can be done manually or via spreadsheet upload. The penultima step require the entry of the schedule for when the task begins and ends and when feedback will be released to students. The final step provides a summary of confirmation screen so that all the details can be check before publishing the task. Being able to complete all this via an online form and being guided through the process with online prompts is an innovative and supportive way to ensure teachers can be confident that they have prepared their student feedback activity and task appropriately.

Three units in the School of Medicine participated in a pilot to use the CollabHero platform: in Trimester 3 2017, HMO201 Science of Vision 2, and in Trimester 1 2018, the units HMO101 – Principles of Optics, and HMM201 – Medical Nanotechnology. It was decided to keep the pilot limited to smaller numbers so as to mitigate risk, so only limited staff were invited to participate and approximately 250 students across the three units were involved. All units used CollabHero in slightly different ways for peer feedback.

HMO201 acknowledged that peer review is an appropriate way to provide the students feedback on their performance by their own team members with whom they have interacted during various team activities over the trimester. CollabHero is able to provide students with a platform for student-driven feedback as it is anonymous. In the Trimester 3 2017 pilot there were some initial stressors with students having to create accounts in the system before being able to access their tasks. Once in the system though, the students found the software easy to use and complete their review tasks. For the pilot in Trimester 1 2018, an integration with CloudDeakin had been developed and we were able to invite students to click a link from their Unit sites and go directly to their CollabHero tasks. This meant easier access to the feedback tasks and less help and support was required. The Unit Chair said: “CollabHero was easy to set up, easy to grade, and easy to distribute the formative comments.”

In HMM201 Medical Nanotechnology, there is a Group Project component which accounts for 30% of total marks of the Unit. Originally, the attendance and student peer assessment were all managed by paper, which was found to be very inefficient and error-prone.  CollabHero is an innovative software package that allows real-time tracking of attendance and time-sensitive processing of student peer-assessment.  More importantly, CollabHero allows the provision of a “big-picture” view of the attendance in each of the student working groups and real-time feedback of students within the group. The implementation of CollabHero in HMM201 has significantly improved students’ experience for this Unit and has saved a lot of time of manual processing of paper-form for the unit chair. The Unit Chair noted: “This is indeed a very user friendly and productivity enhancing programme which helps to deliver the course with significantly enhanced student experience.”

HMO101, 114 students were asked to grade their peers on a number of criteria on group involvement and contribution and also asked to write qualitative justification for their score. Being the first unit in optometry where the students do team work, HMO101 is a good opportunity to introduce the students to feedback and reflection on team work. Team work as well as feedback and self-reflection are key elements throughout the course, which is strongly based in team-learning and problem-based learning. The software made it easy for the students to give their feedback (see below) as well as for the unit chair to easily implement the software thanks to its link to CloudDeakin. The flexibility of the platform allowed the students to articulate their judgments on their group members in enough depth, while making sure that at least a minimum justification was given for each of the scores. The Unit chair could see the benefit in integrating this platform in CloudDeakin so that other units including team work and feedback can benefit from it, especially given the flexibility in the type of questions that this platform provides, which allows it to cater for different approaches to feedback.

The piloting of CollabHero in three units in the School of Medicine has demonstrated that the system has created an enhanced digital environment for student self and peer feedback. The system has provided a positive user experience for staff when setting up the tasks for students, and students have also found the process makes it easy to contribute feedback and also appreciated the way they could easily access the feedback that they received. As an online tool, CollabHero also facilitates staff and students’ use of the system by not being bound by time and place. Access to the tool can be anywhere and anytime and provides other functions such as reporting on those students who may need a reminder to complete their tasks. The impact on the time saved and ease of use and collation of data has positively benefited staff and student responses indicate that they too have experienced positive outcomes from the pilot. This all augers well for a more wide-spread adoption of the online environment for self and peer assessment, student learning and attainment of Deakin graduate learning outcomes such as communication, critical thinking, self-management and team work through further scaffolded feedback activities.

Supporting evidence for the impact and effectiveness of CollabHero for self and peer assessment. 

The table below from data collected on the student experience of using CollabHero in one unit (n=15). The results indicate that students found CollabHero easy to use and would be happy to use CollabHero for providing peer feedback again.

Below are statements from students about what they liked about their experience of using CollabHero and they confirm the data above.

  • We can give feedback on others work without them knowing who give the actual feedback for them. We can be honest about our opinions
  • It is simple due to its clearly written options.
  • The design is easy makes the information easy to understand.
  • Can be done alone which increases anonymity
  • It has data report that we can see in excel file
  • I can do it own time
  • Online aspect allows for easy access.
  • easily accessed and easy to use
  • being able to receive the feedback as well as give it.
  • The ability to give week to week reviews based on peer co-operation.
  • Giving feedback to group member
  • Just having to click on an option to answer questions
  • You can privately give feedback
  • Collabhero was convenient in that you were able to log in within the certain time period of the seminar to peer assess each week
  • it was simple and i liked how at the end you see what other people marked you


References

Adachi, C., Tai, J., & Dawson, P. (2018). A framework for designing, implementing, communicating and researching peer assessment. Higher Education Research & Development, 37(3), 453-467.

Adachi, C., Tai, J. H.-M., & Dawson, P. (2017). Academics’ perceptions of the benefits and challenges of self and peer assessment in higher education. Assessment & Evaluation in Higher Education, 1-13. doi:http://dx.doi.org/10.1080/02602938.2017.1339775

Boud, D., & Soler, R. (2016). Sustainable assessment revisited. Assessment & Evaluation in Higher Education, 41(3), 400-413.

Boud, D., & Tyree, A. (1980). Self and peer assessment in professional education: a preliminary study in law. J. Soc’t Pub. Tchrs. L. ns, 15, 65.

Tai, J. H.-M., Canny, B. J., Haines, T. P., & Molloy, E. K. (2016). The role of peer-assisted learning in building evaluative judgement: opportunities in clinical medical education. Advances in Health Sciences Education, 21(3), 659-676.

 

Bitnami