Welcome to Middlesex ALTC 2015: Revisiting Assessment

The day was opened by our new Vice Chancellor, Professor Tim Blackman, who thanked all Middlesex staff for the warm welcome to his new role.

The Vice Chancellor shared his ambition for the future of the University and describes 'students first' as his mantra. He looks forward to seeing investment in 21st century teaching and dissemination of best practice, with ambition and innovation being at the heart of what we do.

Professor Blackman suggested that those working in higher education should move away from wishing for higher quality students and step towards a healthier appreciation for the diversity and creativity of the students we have, inspiring them to identify their existing assets and achieve their highest potential.

This year’s Annual Learning and Teaching Conference (ALTC) focused on assessment – proposing and sharing models of practice that demonstrate shared assessment literacies within academic communities of staff and students.

Our keynote speaker, Professor Margaret Price, National Teaching Fellow and Professor of Teaching and Learning at Oxford Brookes University, explored the importance of applying these shared literacies, along with other tactics, in formulating effective feedback.

The morning continued with colleagues from within and beyond the University demonstrating effective practice in applying assessment literacy, exploring creative assessment and considering the role of technologies in enabling alternative constructions of the assessment process.

Professor Paul Haywood, Deputy Dean of the School of Art and Design at Middlesex, opened the afternoon’s activities with reflection on the capacity of reconsidering assessment as the co-creation of content through collaborative processes – thus providing learners with alternative levers of control and influence.

The day closed with a drinks reception and the award of this year’s PGCHE prizes.

Track A1 - Learning About Assessment Literacy: A Case Study from the Assessment Literacy Project

 Nicky Spawls and Clare O'Donoghue, Middlesex University, School of Health and Education

How do you go about reviewing assessment literary in programmes that have diverse cohorts, widely different prior experiences of assessment and diverging expectations? This was the opening of the presentation by Nicky Spawls and Clare O’Donoghue from the School of Health and Education.

Nicky and Clare decided to review their programmes’ assessment practice and use this as an opportunity, taking a broader programme level approach of assessment. Nicky and Clare discussed how they questioned consistency of practice across modules, or they identified the need to clarify standards with students and went on to look at ways to explain the purpose of formative assessment to students.

Their review went back to basics: reminding ourselves of how daunting assessment can be; asking ourselves why an essay is indeed the best way to assess certain modules; or how to personalise assessment in large cohorts of 150-200 students. But also interesting questions addressed the need to discuss assessment amongst staff, recognising that assessment is not only about students.

Report by Pascale Colonna, Senior Academic Developer, Centre for Academic Practice Enhancement (CAPE)

Track A2 - Can Assessment Literacy be Enhanced and does it Lead to Improved Student Performance? A Case Study of Year One Business and Management Students at Middlesex University Business School

Simon Roberts, Middlesex University, Business School
Ana Marinica, Centre for Academic Practice Enhancement (CAPE)
Karim Qizalbash, Learner Development Unit (LDU)

Simon Roberts started off the presentation by providing an overview of assessment literacy in the context of a project carried out in the Business School. The 12 week enhanced programme  and MBS0111 Preparing for Business have been evaluated through an assessment literacy lens and findings were discussed as part of the session. Students enrolled on the enhanced programme failed to meet their conditional offer with regard to required UCAS points and were offered a 12 week course that aimed at preparing them for University. The programme was designed in collaboration with the Learner Development Unit. Interestingly, it consisted of 12 hours of teaching in a week, 6 of which were delivered by the LDU in order to develop students’ academic writing skills.

Assessment literacy can be defined as students’ ability to translate and appreciate the relationship between students’ learning and assessment as well as assessment criteria, feedback practices and the level of assessment they are presented with. Ana also highlighted the importance of assessment in a student’s learning journey. Assessment has been looked at with regard to what students find to be important, how they spend their time and at the end how they see themselves as students and later on as graduates. The key issue identified in this project is the fact that students often don’t understand what a good piece of coursework is and what is expected from them, especially with regard to assessment criteria. 

Karim Qizalbash presented on how the team has employed a model by O’Donovan et al, (2008) - Approaches to Developing Student Understanding of Assessment Standards when evaluating the 12 week programme. The techniques Karim employed based on the feedback received from students at the beginning of the course were to make all the learning materials relevant to the assessment which included an essay and presentations. Formal 1-2-1 tutorials, general teacher lead instructions in seminars, individual/group tasks and peer review exercises were part of the 12 week programme.

The evaluation of the project consisted of both quantitative and qualitative data collection and two cohorts of students - enhanced and January start students were compared. The initial findings showed that the grades of the enhanced students were lower across all the assessments (with the exception of HRM1004), however the enhanced students had perceived higher clarity of the requirements of each assessment and were slightly more confident than their January start counterparts. 

Most of the discussions which took place during the workshop evolved around the implications and possible issues of managing the enhanced students and how the findings of this project can be disseminated and put into practice by the Programme Team. For further details, please have a look at the video and PowerPoint slides from this session.

Report by Natalia Czuba, Educational Technologist, Centre for Academic Practice Enhancement (CAPE)

Track A3 - The Use of MCQs within Team Based Learning: Choosing the Right Approach to Foster Student Learning

Dr Venetia Brown, Dr Kevin Corbett, Icram Serroukh
Middlesex University, School of Health and Education

How do you sell the importance of research skills to nursing students? Nursing students know they are not going to be researchers. They can’t quite work out the relevance research has to nursing. So they hate it.

Venetia’s interest in team based learning started a couple of years ago after seeing Jenny Morris from Plymouth University present at a Nurse Education Today conference. Venetia was so impressed with the approach that she visited Plymouth and saw TBL in action. On returning to Middlesex she consulted the BSc Nursing programme teaching team.

Icram, a graduate teaching assistant has been working alongside Venetia, Kevin and the research methods teaching team to implement the TBL approach with the September 2015 nursing student cohort. Icram says TBL is a small group facilitation process that can be used with large cohorts (200 or more), where small permanent groups of 5 to 7 students work together. The permanency allows group cohesion to develop, ‘research has shown that after time, students become more concerned about the group’s performance rather than their own individual performance’. Although physical space is needed, the approach can be used in lecture theatres.

Is TBL a structured method of flipping the classroom? Does it portray the ideals behind ‘assessment for learning’? The first phase of TBL asks students to go through learning materials before the session, this phase is timetabled into the module and all learning materials are electronically made available through the module’s online space. Interestingly each resource has time recommendations and explicitly flagged up objectives so students understand what is required from them.

The second phase ‘readiness assurance’, consists of 5 sessions, the first session is an introductory session, used to familiarise students with the process of TBL. The remaining sessions takes up to 1 ½ hours. The first part of each session start off with an Individual Readiness Assurance Test (iRAT), a MCQ test, with 15 to 20 questions that goes towards 70% of the module’s summative assessment grade. The second part is the Team Readiness Assurance Test (tRAT), exactly the same iRAT test is taken by the group. Use of Immediate Feedback Assessment Technique (IF-AT) cards forefronts the team’s discussions of the iRAT test.

This approach generates student excitement around learning by introducing game-based or gambling-based approach to team learning. IF-AT cards are similar to lottery scratch cards, the teams must discuss and agree on a correct response before scratching. The correct response reveals a star and awards the maximum 4 points, however negative marking comes into play for incorrect responses. The discussion around the answer in addition to providing feedback for iRAT tests allows students to delve more deeply and critically into the question and share their individual understanding and knowledge with other team members. The tRAT team score contributes 30% of the module’s summative grade.

The final stage, the application of course concepts, can last between 2 or 4 hours. In this module this phase is formatively assessed and involves the group members collaboratively working on a case study. The same case study is used across all groups.

Watch the recording of this presentation to find out more …

Report by Asanka Dayananda, Senior Academic Developer, Centre for Academic Practice Enhancement (CAPE)

Track A4 - Assessment Literacy in Student Midwives

Jo Killingley, Middlesex University, School of Health and Education

As part of the revisiting assessment theme of the Learning and Teaching Conference 2015, Jo’s talk centred on providing prompt feedback to students immediately after one of the exams called the Objective Structured Clinical Examination, OSCE, which she carried out as a research project. Jo identified that:

• Feedback has to promote critical thinking and self-judgement
•   Feedback has to be behaviour focused
• Clarity and clear articulation – students want to be in control of their learning and be motivated through clear feedback. Feedback helps students take control of their learning.
•  Timeliness – it’s important that feedback provided is very close to the time of the event in       order for it to be effective 
• Coping with learning – feedback provokes high levels of anxiety 
• Feedback should be constructive. Destructive feedback can be more in the perception by the recipient than in the intended feedback content.

Jo described in detail how much dedication is required to support and mentor students through their learning progress on their Midwifery course and the value quality feedback played in this process.

Report by John Koushappas, Senior Educational Technologist, Centre for Academic Practice Enhancement (CAPE)

Track B1 - Readiness for Direct Practice’: Using Video as a Tool to Assess Social Work Students

Helen Hingley-Jones, Middlesex University, School of Health and Education

Angus Macdonald, Centre for Academic Practice Enhancement (CAPE)

This session explored the ways in which video could be used as an assessment tool in evaluating the skills of social work students. We began by watching a video in which Helen Hingley-Jones described the context within which the project fits. Social work education has been under scrutiny and reform in recent years, leading to the development of the Professional Capabilities Framework.

This represents nine different domains of practice for social workers, at all stages of their career, from pre-training through to continuing professional development in advanced roles. Students undergoing training are required to reach a basic level of competence in these domains in a short, intensive period of time.

The need to evaluate students’ communication skills and demonstrate basic social work knowledge led to the development of a video interview process as part of the assessment of their readiness for direct practice. Students interact with actors based on a particular scenario and are assessed on different criteria using a rating scale. The criteria include building a rapport with the actor, evidencing good listening skills and acknowledging any potential risks raised during the interview.

Angus then talked to us about the use of video in e-learning, mentioning that Kay (2012) identifies the main areas in which it is often used, including lecture capture, supplementary materials, and worked examples - crucially, video for assessment was not identified. Angus argues that video can be used effectively for student presentations or recorded submissions, digital storytelling, and providing feedback. With regard to the video interview process, Cartney (2006) says that “Recording and playing back interviews with students has the potential to generate powerful learning experiences”, and suggests that video feedback is the most effective method for improving oral communication skills. The method was compared to the OSCEs (Objective Structured Clinical Examinations) which have been used since the 1970s and involve a similar kind of interview process; it is argued that the addition of recorded video makes this even more effective.

The interview day itself was described as very intense for both students and staff, and it was difficult to manage the sheer numbers of students. Emphasis was placed on the importance of supporting and communicating with students both inside and outside the room, and making sure that video equipment was setup correctly to ensure the process went as smoothly as possible, with as little distraction for the students as possible. That said, some students found being in front of a camera perturbing and it was suggested that students be given more opportunities to practice in this kind of environment before the event. Ultimately the session was a success and only four students failed on the day and three of these passed at re-sit.

In the tutors’ opinion the exercise was challenging but a realistic form of preparation for social work practice and there was evidence of students’ deep learning. They concluded that video is a meaningful and highly effective tool for teaching and assessment of practice skills.

Report by Paul Smith, Senior Educational Technologist, Centre for Academic Practice Enhancement (CAPE)

Track B2 - Open Badges: Capturing and Rewarding Hidden Learning

Dr Simon Best, Middlesex University, Business School

John Parkinson, Centre for Academic Practice Enhancement (CAPE)

John Parkinson, Senior Academic Developer (CAPE) and Dr Simon Best, Senior Lecturer in the Business School, teamed up to tell us about digital badges; what they are, the results of a pilot implementation on a module (MGT3193 Business Start Up) and how they might be used going forward.

We were introduced to the idea that digital badges share many of the attributes of traditional badges earned in organisations like the Scouts and Guides, to identify and promote achievements that can be developed through some type of process. They can be seen as a type of “credentialing system” arising out of research into gamification of learning.

A main driver for the use of digital badges has been the Mozilla Foundation. After working with major organisations, such as NASA, they launched the OBI (Open Badge Infrastructure) in 2011, which provides a common system for the creation, issuance and verification if Digital Badges across a variety of platforms. Then in 2013 Mozilla created the Badge Backpack, an online space where badge recipients can store and present their badges to others.

The badge itself is a 250 x 250 pixel graphic, which can be encoded with metadata, such as the awarding institution and information about the skill achieved. The challenge is to create a meaningful visual within this size restriction. If schools want to consider using the Middlesex logo going forward, the institution would need to consider what is going out with that endorsement.  An awarded badge gets added to users’ individual online ‘back-packs’ and these can link to social media, such as Linked In.

“So what about the pedagogy?” poses John; after reviewing around 50 papers, he found little literature so far for their use in education but there is some work emerging around self-efficacy via motivation and assessment. Possibly badges can be considered as signposts for learning, sitting alongside the curriculum.

After a step by step demonstration of how easy it is to create and administer badges in My Learning, John handed over to Simon Best for a run through of how the pilot was applied to MGT3193. This module is taught using disruptive learning methodology, whereby students are not provided with texts or reading lists but spend the first few weeks learning what is expected of them, in order to work through four linked, progressive assessments; a reflective essay, a group progress presentation, a business plan and a final presentation that involves contributions from each member of the group.  The working groups are carefully constructed based on a combination of Belbin’s roles, what the students are majoring in and the spread of skills and knowledge. Their assessment is not based on right or wrong answers but whether they can acquire and apply tools and skills to support/back-up their arguments. Badges were awarded for; leadership, research and communication and if they acquired all three, they would automatically receive the forth (group leadership).

Initially Simon and John began the pilot thinking that badges would be useful for employability but at this stage of the project evidence is still being collated on the affordances of badges for professional credentialing. However, they do feel that there is something around cognitive ability and supporting assessment to be reviewed and explored in the future.

Simon reflected that “we don’t assess or reward an enormous amount of skills that the students pick up”, which prompted a question about using badges to capture the learning between modules, for example, being a student voice representative or giving a presentation.

To find out more about this innovative pilot, please watch the video of the session or review the slides via the links.

Report by Louise Merlin, eLearning Content Manager, Centre for Academic Practice Enhancement (CAPE)

Track B5 - Demo session: eMarking & eFeedback

Paul Smith and Kirsteen MacDonald

Middlesex University, Centre for Academic Practice Enhancement (CAPE)

This session included demos of a number of online tools to help with e-feedback and e-marking:

• How to use the Turnitin App for iPads for marking, and giving typed and audio feedback (which synchs to your Turnitin Assignments on My Learning)
• How to use the Grademark feature in Turnitin on My Learning
• How to annotate PDF's submitted to the Assignment tool to give feedback
• How to mark-up and add comments to Word documents submitted to the Assignment tool when giving feedback
• How to give video feedback through My Learning
• How to quickly and easily re-purpose and re-use quizzes for individuals and multiple groups

The Centre for Academic Practice Enhancement (CAPE) provides staff support for use of My Learning - queries can be sent to elearning@mdx.ac.uk

For help with all aspects of My Learning please visit My Learning Essentials - Staff

The Centre for Academic Practice Enhancement (CAPE) offers staff development across a range of areas relating to academic practice and learning technologies, please visit our pages in the 2015-2016 MDX Staff Development Brochure* the chapter called Developing Academic Practice which starts on page 19

(*This link directs you to a page on the Mdx Staff Intranet, you will need access to the University's internal server to view this).

Track C2: Developing quality feedback and assessment literacy in dance

 Julia K. Gleich
 Head of Choreography (London Studio Centre)

London Studio Centre’s shared their institution development of their Learning & Teaching Strategy 2013-18 focussing on assessment and feedback. The nature of practice at LSC makes use of intense continuous, day-to-day formative feedback by both tutors and peers, which ac
counts for unusual learning experience in HE. Addressing the high frequency of assessment and feedback is at centre of LSC’s culture of quality enhancement and review of assessment strategies that have led to significant successes in the National Student Survey.

The institution has reviewed strategies associated with developing effective feedback and assessment that include ways of creatively assessing creativity and identified points to consider in constructing effective feedback.

Identified Issues related to creative assessment:

·         How to “prepare for assessment” in a context of creativity

·         Prior learning

·         Tick-boxing / Goal-oriented-ness

·         Subjectivity

·         House styles

·         New ideas and risk-taking

                                                        

The staff at LSC also recognize that Assessment Literacy is a double-edged sword and if students are too focused on assessment, they will be unwilling to take risks and discover their own voices in a creative environment. In addition, Assessment tasks and criteria must be designed to reward even “inappropriate” choices as students re-invent themselves and develop new movement vocabularies, explore compositional strategies and form their artistic identities.

Constructing feedback

At LSC feedback is provided taking into consideration the points below:

·         Specificity

·         Constructive comments

·         Appropriate language

·         Transparency – see example below. If not clear from the starting point comments like “ student needs to be more analytical” – are not clear.

·         Enabling and encouraging/Positive but honest

·         Personal

The presentation also included seven principles of good feedback practice From Dr Nicol and Debra Macfarlane-Dick below:

“good feedback practice should facilitate the development of self-assessment (reflection) in learning, encourage dialogue, clarify what is good performance, deliver quality information about the students learning, provide information to teachers to shape learning and should be motivational for the students and create positive self-esteem.”

Creatively Assessing Creativity from mdxaltc

Report by Betty Sinyinza, Educational Tehcnologist, Centre for Academic Practice Enhancement (CAPE) 

Track C3 - Assessing Creativity

  Jon Mortimer, Middlesex University, School of Art and Design

Is transparency important for assessing creativity? Nationally, assessment and feedback is the lowest scoring category of the National Student Survey (NSS). For Jon’s programme their assessment and feedback satisfaction score has risen 25% over the last 3 years to 89%. Jon’s presentation describes how this was achieved.

‘How do you assess creativity?’ was the main problem that needed the teaching team’s attention. Assessment instils a pursuit of grades, a measure of how well you are doing compared to your peers and others who have come before you. How you as a teacher do you make this judgement when assessing creativity? And equally importantly how do you communicate this to students so they are aware of what is expected of them?

In Art and Design, students are commonly set design tasks to demonstrate their creativity. Jon says ‘in design it doesn’t matter how simple or detailed the question is there are thousands of possible answers…the job of the tutor is to help students think about the right questions to find the personal answers, that’s where the artist side of designing comes out’.

In the absence of clear assessment criteria for creativity, the tendency is for students to align their work to well received work of their predecessors and the creative inclinations of their assessors. This is clearly counterproductive and stifles creativity.

If you think about how students engage with assessment, students use their experience, design knowledge and creative talents. Creativity often involves asking increasingly complex questions, to produce a design, in most cases the final end product is arrived at after a process of changes in the designs over time. This design process needs to be acknowledged in assessment.

Jon says when assessing creativity there is a tension between connoisseurship and transparent assessment criteria. Jon describes the use of a marking rubric that includes the development process or the design journey in the assessment criteria. The assessment criteria has some degree of flexibility in that it is open to change through valid arguments raised by students at the start of the module.

Clearly informing students about the assessment criteria from the onset helps students approach assessment knowing what is expected of them. Students are asked to present their work, these are made available for both peer and staff feedback. This approach may also provide a way of reducing the subjectivity and bias in assessing creativity. Watch the presentation to see how the teaching team breaks up the assessment criteria to present it visually and use video feedback.

Jon mortimer - Assessing Creativity from mdxaltc

Report by Asanka Dayananda, Senior Academic Developer, Centre for Academic Practice Enhancement (CAPE)

Workshop 1- Changing Perceptions of Ourselves: How Graduate Academic Assistants are Altering the Learning Experience of Education Studies Students Through Assessment Literacy

Dr. Carole L Davis, Sandip K Gill and Kristína Repová
Middlesex University, School of Health and Education

Carole, along with Sandip and Kristina, facilitated a lively interactive session, with activities around the clarity of guidance and assessment criteria, the importance of the quality of feedback and peer and self-assessment. Using specific examples from the GAA’s sessions with Education Studies students,  to prompt thinking and discussion, the 50 or so participants were tasked with discussing and feeding back suggestions for how they might tackle the specific issues raised by students. These issues included self-doubt by a high achieving student and another students consistent inability to understand and respond to assessment tasks. The session also outlined some of the interesting approaches the department and GAA’s have developed that students were appreciative of, such as a book club for students to discuss readings together.

The video clip provides a snapshot of the liveliness of the session and wealth of discussion that arose.

Changing Perceptions of ourselves from mdxaltc