Keynote I - What Makes Good Feedback Good?

 Professor Margaret Price, Oxford Brookes University

The keynote speaker addressed student feedback in the context of a higher education environment, that is dynamic and subject to a large number of external and internal pressures. Within that context, there appears to be general consensus that feedback is a crucial component of learning and assessment, and that assessment itself is a key driver of learning. Professor Price pointed out that the discourse of assessment is relatively simple for such a complex subject and suggested a need for new and more descriptive terminology than exists at present. She also noted the added pressure of higher student expectations and a louder student voice from ‘customers’ paying high fees.

Oxford Brookes and Cardiff have collaborated on a new piece of research, examining what makes feedback good in the perceptions of the students and what domains influence these perceptions of ‘good’ and ‘bad’ in this context. The project used student researchers and a cross-discipline approach. Students taking part were asked to bring one piece of ‘good’ feedback and one ‘bad’. They were interviewed around these pieces and then the feedback was analysed. The domains used in the analysis covered quality, context, student development and expectations and were further divided into areas such as technical factors, particularity (i.e. personal/impersonal feedback), recognition of effort, assessment design (crucial), student resilience (can they accept criticism?), student desires (to learn or to achieve a high grade?) etc. The full report is available at http://www.brookes.ac.uk/aske/.

The research found that the domains overlapped and compensated for each other so that feedback that was poor in one domain might be good in another, and vice versa. Three important messages for those giving feedback that emerged were:

• Give it plenty of time
• Train, develop and support staff in giving feedback
• Limit anonymous feedback (personalised feedback scored highly)

Professor Price suggested that students need to develop their assessment literacy if they are to be able to gain the most from assessment and feedback. This can be done through developing their technical understanding of marking and grading, through self and peer assessment, and through an appreciation of what grading criteria actually mean. She pointed out that academics see hundreds of pieces of work and have a tacit understanding of what a 2:1 looks like, for example, but how are students to know this? Give them good examples was the answer and put them through stages of self-assessment, peer review, drafting, re-drafting and perhaps peer assisted learning where more experienced students support beginners and help them to develop their assessment literacy.

An overarching message resulting from the research was that ‘You don’t need to get it right all the time’: students are very forgiving of delayed feedback or low grades when they can see that there has been a real effort to engage with their individual piece of work. Examples of good practice are available at https://www.plymouth.ac.uk/whats-on/inclusive-assessment-in-practice-conference.

Finally – she concluded by saying that this should go beyond university and students should leave having developed these valuable self-evaluation skills to take with them into the future.

Report by Celia Cozens, e-Learning Content Manager, Centre for Academic Practice Enhancement (CAPE)

Track A2 - Can Assessment Literacy be Enhanced and does it Lead to Improved Student Performance? A Case Study of Year One Business and Management Students at Middlesex University Business School

Simon Roberts, Middlesex University, Business School
Ana Marinica, Centre for Academic Practice Enhancement (CAPE)
Karim Qizalbash, Learner Development Unit (LDU)

Simon Roberts started off the presentation by providing an overview of assessment literacy in the context of a project carried out in the Business School. The 12 week enhanced programme  and MBS0111 Preparing for Business have been evaluated through an assessment literacy lens and findings were discussed as part of the session. Students enrolled on the enhanced programme failed to meet their conditional offer with regard to required UCAS points and were offered a 12 week course that aimed at preparing them for University. The programme was designed in collaboration with the Learner Development Unit. Interestingly, it consisted of 12 hours of teaching in a week, 6 of which were delivered by the LDU in order to develop students’ academic writing skills.

Assessment literacy can be defined as students’ ability to translate and appreciate the relationship between students’ learning and assessment as well as assessment criteria, feedback practices and the level of assessment they are presented with. Ana also highlighted the importance of assessment in a student’s learning journey. Assessment has been looked at with regard to what students find to be important, how they spend their time and at the end how they see themselves as students and later on as graduates. The key issue identified in this project is the fact that students often don’t understand what a good piece of coursework is and what is expected from them, especially with regard to assessment criteria. 

Karim Qizalbash presented on how the team has employed a model by O’Donovan et al, (2008) - Approaches to Developing Student Understanding of Assessment Standards when evaluating the 12 week programme. The techniques Karim employed based on the feedback received from students at the beginning of the course were to make all the learning materials relevant to the assessment which included an essay and presentations. Formal 1-2-1 tutorials, general teacher lead instructions in seminars, individual/group tasks and peer review exercises were part of the 12 week programme.

The evaluation of the project consisted of both quantitative and qualitative data collection and two cohorts of students - enhanced and January start students were compared. The initial findings showed that the grades of the enhanced students were lower across all the assessments (with the exception of HRM1004), however the enhanced students had perceived higher clarity of the requirements of each assessment and were slightly more confident than their January start counterparts. 

Most of the discussions which took place during the workshop evolved around the implications and possible issues of managing the enhanced students and how the findings of this project can be disseminated and put into practice by the Programme Team. For further details, please have a look at the video and PowerPoint slides from this session.

Report by Natalia Czuba, Educational Technologist, Centre for Academic Practice Enhancement (CAPE)

Track B1 - Readiness for Direct Practice’: Using Video as a Tool to Assess Social Work Students

Helen Hingley-Jones, Middlesex University, School of Health and Education

Angus Macdonald, Centre for Academic Practice Enhancement (CAPE)

This session explored the ways in which video could be used as an assessment tool in evaluating the skills of social work students. We began by watching a video in which Helen Hingley-Jones described the context within which the project fits. Social work education has been under scrutiny and reform in recent years, leading to the development of the Professional Capabilities Framework.

This represents nine different domains of practice for social workers, at all stages of their career, from pre-training through to continuing professional development in advanced roles. Students undergoing training are required to reach a basic level of competence in these domains in a short, intensive period of time.

The need to evaluate students’ communication skills and demonstrate basic social work knowledge led to the development of a video interview process as part of the assessment of their readiness for direct practice. Students interact with actors based on a particular scenario and are assessed on different criteria using a rating scale. The criteria include building a rapport with the actor, evidencing good listening skills and acknowledging any potential risks raised during the interview.

Angus then talked to us about the use of video in e-learning, mentioning that Kay (2012) identifies the main areas in which it is often used, including lecture capture, supplementary materials, and worked examples - crucially, video for assessment was not identified. Angus argues that video can be used effectively for student presentations or recorded submissions, digital storytelling, and providing feedback. With regard to the video interview process, Cartney (2006) says that “Recording and playing back interviews with students has the potential to generate powerful learning experiences”, and suggests that video feedback is the most effective method for improving oral communication skills. The method was compared to the OSCEs (Objective Structured Clinical Examinations) which have been used since the 1970s and involve a similar kind of interview process; it is argued that the addition of recorded video makes this even more effective.

The interview day itself was described as very intense for both students and staff, and it was difficult to manage the sheer numbers of students. Emphasis was placed on the importance of supporting and communicating with students both inside and outside the room, and making sure that video equipment was setup correctly to ensure the process went as smoothly as possible, with as little distraction for the students as possible. That said, some students found being in front of a camera perturbing and it was suggested that students be given more opportunities to practice in this kind of environment before the event. Ultimately the session was a success and only four students failed on the day and three of these passed at re-sit.

In the tutors’ opinion the exercise was challenging but a realistic form of preparation for social work practice and there was evidence of students’ deep learning. They concluded that video is a meaningful and highly effective tool for teaching and assessment of practice skills.

Report by Paul Smith, Senior Educational Technologist, Centre for Academic Practice Enhancement (CAPE)

Track B3 - Using My Learning in the Assessment Cycle: How Hard Can it Be? Challenges and Benefits of Using VLEs for Assessment

Jas Ahmad, Middlesex University, Business School
Luiza Dantas, Centre for Academic Practice Enhancement (CAPE)

Jas and Luiza provided an energetic presentation on the benefits and versatility of integrating My Learning (Moodle) tools into the assessment cycle.

Luiza opened by describing the various stages of the assessment cycle; 1) specifying requirements, 2) setting assessment criteria and expectations, 3) supporting students through the assessment process, 4) submission of assignments, 5) marking and feedback, 6) returning marks and feedback, and 7) reflection. She identified that more focus and engagement is needed in the first 4 stages than is currently being given.

Jas then walked us through his approach to these 4 stages of the assessment cycle. Jas advocates the use of Breeze presentations.  Breeze is available freely online and allows the lecturer to upload Powerpoint slides coupled with short audio narration for each slide.  This allows Jas’ students to gain the basic information regarding assessment design at home, allowing for richer engagement and more informed questions during his face to face lectures.

Jas emphasised the need to ‘frontload’ your course with information and support.  Jas sets out his expectations early on, providing a recorded lectures, online announcements and engages with online discussions on how students should prepare for seminars, lectures and assignments.  In his experience, students routinely need the same guidance at the start of every term, so providing this information in a pre-recorded format saves valuable time and confusion in the long run.

Challenges to providing online guidance and support can include resistance from students who see more value in receiving the information in a face to face format. Gaining the students ‘buy in’ early on and continuously throughout the course is important for the success of online support.  It is also important to consider necessary adjustments for disabled students, if these students are not considered in the preparation stage of teaching then this can create a lot of back tracking and extra work later.

Jas empathised with colleagues who may be reluctant to try out technologies they are not familiar with due to fear of failure, and the potential for being caught on camera making a mistake. He recommends speaking to the camera as though you are speaking to a student in the room, if you trip up over a sentence or need to correct yourself it is not a big deal, and with time and practice confidence will grow.

Luiza closed the session by reminding the audience that the Centre for Academic Practice Enhancement is on hand to support any members of staff who would like to branch out and explore the potential of using My Learning in their assessment strategies.

Report by Jessica Isaacs, Online Learning Content Developer, Centre for Academic Practice Enhancement (CAPE)

Track C1 - Digital Stories - Everyone Has a Story to Tell… ‘Assessing Students Using Digital Stories and Bringing Out Their Creativity'

Alex Chapman and Luiza Dantas
Middlesex University, Centre for Academic Practice Enhancement (CAPE)

The theme of the session was digital storytelling and how this can be used as an assessment tool. Luiza Dantas introduced the presentation by explaining that stories are universal, used in all cultures and passed down from one generation to the next. We all know how to tell a story so how can we take this capability and use it for reflection and to present our work for assessment?

Using digital technology to add sound, colour and images to a story can bring it to life and make it engaging both for the storyteller and the intended audience. Luiza Dantas showed an example from YouTube of the story of Australian Aboriginal culture – images of the environment, the people and their art and work, backed with Aboriginal music, combined to give a rich and informative overview in a very short time. But though this was interesting and entertaining, we had to get on to the serious business of assessment – how could this be deep and rigorous enough for assessing our students? There were a few in the room who were definitely waiting to be convinced and some doubt was in the air.

The presenters put forward a very convincing argument for the benefits of using this tool. Luiza Dantas pointed out that – on the whole – students hate reflective writing. However, many of them are asked to produce a written reflection for assessment. Why not use a digital story, they asked. It is much more enjoyable for everyone and focuses the student fully on being concise, to the point, engaging, creative and reflective. What about presentations? How many students, particularly international ones, suffer the undue stress of standing up and talking, accompanied by ubiquitous PowerPoint slides? Why not get them to create a digital story instead? The same could be used to replace an Executive Summary; how much more creative and engaging that would be. Another idea was a digital CV. In this digital age, how impressive it would be to receive a digital story about your prospective employee – and you would know that they were equipped with the technological skills that are so important in 21st-century employment. The CV can be part of a person’s LinkedIn profile – the possibilities are endless … (you can tell I enjoyed this session).

Alex Chapman presented some excellent examples of students’ work, from the Business School and from a Mental Health programme. He pointed out (to the doubters in the room) that these assessments generally carry a small percentage of marks and are normally accompanied by a storyboard, and a written reflection as well as the story itself. He indicated the range of areas that could be assessed, including project planning, editing, research, critical analysis, originality, reflection, creativity and more.

Students were asked to feed back on the experience and while most started out as unsure or negative, afterwards they reported that it was exciting, enjoyable and skill enhancing, and they would like to see it used in more modules. A few questions arose, mostly to do with copyright issues and it was agreed that clear guidance should be given at the start of the process as this – like referencing – is all part of developing students’ academic integrity. Luiza Dantas concluded by pointing out that as well as developing all the core skills, digital stories also develop technical, visual and digital literacies – so important for employability. What’s not to like?

Assessing using digital stories from mdxaltc

Report by Celia Cozens, eLearning Content Manager, Centre for Academic Practice Enhancement (CAPE)

Track B4 - Drop-in Session: Hands-on Session Following on from track C1

Alex Chapman, Middlesex University, Centre for Academic Practice Enhancement (CAPE)

The drop-in session was a follow up from the morning session - Digital Stories – Everyone Has a Story to Tell by Luiza Dantas and Alex Chapman. This workshop focused on four stages of creating a digital story such as pre-production, production, presentation, as well as assessment and reflection. These stages were discussed in details and covered:
*gathering and researching multimedia in compliance with copyright laws
* writing a script
* planning the stories using digital storyboards tools (online and paper based)
* using technologies and various software that are free are accessible for students
* presentation and assessment of digital stories

Resources from the session: 
 
Guides and websites:

Kathy Schrock's Guide to digital stories
Digital Storytelling Tools for Educators by Silvia Rosenthal Tolisano
Digital storytelling site for students and educators

Software:
 Apple iMovie (Macintosh OS X)
 Audacity (Macintosh OS X and Windows)
 Microsoft Photo Story 3 (Windows)
 Windows Moviemaker 2.1 (Windows)
 Google story builder
 21 Free Digital Storytelling Tools For Teachers and Students

Free pictures:
Flickr: Advanced Search for Creative Commons only.
http://www.flickr.com 
Compfight: Search Creative Commons Flickr Images
http://compfight.com/
Pixabay
http://pixabay.com/
wikimedia commons
http://commons.wikimedia.org/wiki/Main_Page 

Free images
 http://www.freeimages.com/

Royalty-Free Music and Sounds:
http://digitalstorytelling.coe.uh.edu/page.cfm?id=23&cid=23&sublinkid=95
http://freesound.org
http://soundbible.com

The Centre for Academic Practice Enhancement (CAPE) provides staff support for use of Digital Stories - queries can be sent to elearning@mdx.ac.uk.

The Centre for Academic Practice Enhancement (CAPE) offers staff development across a range of areas relating to academic practice and learning technologies, please visit our pages in the 2015-2016 MDX Staff Development Brochure* the chapter called Developing Academic Practice which starts on page 19

(*This link directs you to a page on the Mdx Staff Intranet, you will need access to the University's internal server to view this).

Workshop 2- Appartunity or Appain?

Kate Brown, Sheila Cunningham, Jodie Ward, Kate Wilkinson and Jo Wilson, Middlesex University, School of Health and Education

Are apps in the classroom an opportunity or a pain? This was the question posed by this workshop with each group exploring answers for themselves. We were promptly organised into groups of 6-8 participants per table, all seated around board-style guided questions to allow us to explore the use of (mobile) applications at Middlesex. The workshop turned out to be as exciting as its title and setup hinted at, and was prepared with a good dose of creativity and carefully organised questions.

I’ll describe the experience at our table...

Participants were from different areas, with some qualifying themselves as ‘technology dinosaurs’, others quietly having used apps in the classroom for some time.  All of us seemed very keen to define what is meant by ‘apps’ (mobile apps only or others?), and the conversation quickly moved to practical considerations (‘but does this work on a Samsung tablet? ‘are all apps free?’ etc.) and then onto accessibility (‘would the library consider putting iPads on loan for students who do not have one?’). We also discussed the use of technology as a possible distraction in the classroom and how we can deal with this.

I particularly enjoyed how one academic talked about how he used 3D-visualisations of difficult concepts in his teaching as a way to break down a dry two-day course and as a way to make some of his presentations more vividly understandable (‘and your work is half done!’).  We eventually agreed that we needed to know where to ask for support and guidance (both CAPE Senior Academic Developers and School Librarians were referred to at that point) in order to be able to focus on pedagogical considerations and we talked about looking at apps in exactly the same way as you look at other new initiatives, with the same critical approach and part of an evaluative process.

We also touched on the potential of apps in an assessment context; we had more difficulty with this. We wondered whether they could help with developing certain aspects such as memorising skills. We also identified areas where we thought apps may currently be difficult to support assessment practice, such as in assessing creativity. One group wondered whether apps might be able to help you with minimising elements of subjectivity in the future and one group had come to the conclusion that we should develop our own apps, as a way to tailor them to our needs.

I was really impressed with the quality of the conversation in such as short space of time and with as few as 8 people from a range of areas. Definitely the highlight of the day for me.

Here are links to the articles and BBC item used during the session:

• Exploring Students' Mobile Learning Practices in Higher Education
• Mobile devices in medicine: a survey of how medical students, residents, and faculty use smartphones and other mobile devices to find information
• Developing confidence in the use of digital tools in teaching 
• The apps giving hospitals a cutting edge
• Study Skills Apps will be available from Middlesex University Library as of August 2015

Report by Pascale Colonna, Senior Academic Developer, Centre for Academic Practice Enhancement (CAPE)