Kate Meier, Pam Parker and Della Freeth 34 J. of Practice Teaching & Learning 12(3), pp.xx-xx. DOI: 10.1921/7701240205. © w&b Mechanisms that support the assessment of interpersonal skills: A Realistic Evaluation of the interpersonal skills profile in pre-registration nursing students Kate Meier1 , Pam Parker2 and Della Freeth3 Abstract: In many health professions, experienced practitioners assess students’ or inexperienced practitioners’ clinical skills but do not formally or explicitly rate their interpersonal skills, even though it is often suggested that failing or struggling students have poor interpersonal skills. The Interpersonal Skills Profile (ISP) has been widely used in UK health care programmes. The tool allows assessors to select five statements from a list, which they feel reflect the student’s achievement. These are usually graded from fail to excellent. Using a Realistic Evaluation approach this study examined how the ISP was used to assess interpersonal skills in a university pre-registration nursing programme. The use of the ISP was investigated through interviews with clinical nursing mentors, practice education facilitators and education champions as well as a documentary analysis of student assessment booklets. The findings led to the development of three middle range theories which may be useful in other contexts. These focus on: 1) overt assessment of interpersonal skills, 2) providing support to mentors and 3) feedback and ‘feeding-forward’ to students. Keywords: practice assessment, nursing students, interpersonal skills, realistic evaluation, practice assessors, feedback 1. PhD Student, Learning Development Centre, City University 2. Assistant Director of Learning Development Centre, City University 3. Professor of Professional Education, Institute of Health Sciences Education, Barts and The London School of Medicine and Dentistry, Queen Mary University of London Address for correspondence: Dr. Pam Parker, Learning Development Centre City University, Whiskin Street, EC1V 0HB. p.m.parker@city.ac.uk Date of first (online) publication: Mechanisms that support the assessment of interpersonal skills: A Realistic Evaluation 35 J. of Practice Teaching & Learning 12(3), pp.xx-xx. DOI: 10.1921/7701240205. © w&b Introduction Healthcare practitioners require well-developed interpersonal skills, but these are difficult to identify and assess. One response has been the adoption of an Interpersonal Skills Profile (ISP) (Knight, 1998) to be completed by practice assessors. This paper draws from a doctoral study which used a Realistic Evaluation (RE) approach (Pawson & Tilley, 1997) to examine the use of an ISP to assess pre-registration (qualifying) nursing students at one higher education institute (HEI) in the United Kingdom (UK). RE asks ‘what works for whom and how’ through identifying contextmechanism-outcome (CMO) configurations which, as we shall discuss later, can support middle range theories (MRTs). This paper will summarise the difficulties of assessing interpersonal skills; describe the ISP and this study, which examined how the ISP was actually used by mentors in practice; then present three MRTs developed from the findings. While grounded in a particular case study, the MRTs are sufficiently abstracted to suggest what might work for other professions in practical settings with similar issues. Context Professional education requires some practical component in order to both teach practical skills and to socialise students into the profession (Smeby, 2007). Nursing is one of many professions in which a large proportion of the training is spent in practice. How this is enacted varies between professions and countries. Regardless of the form it takes, assessment of students in the work or practice setting is difficult to undertake and notoriously inconsistent (Allison & Turpin, 2004; Bogo et al., 2004). However, differences are not only between professions or HEI, even students who are on the same programme will have a wide range of experiences (Wilson & Scammell, 2010). Not only is it is difficult to standardise the student experience, but equally it is a challenge to prepare assessors to the same standards (Govaerts et al., 2011). However, even if all assessors could be adequately trained and prepared in assessment procedures, as Ginsburg and colleagues (2010) identify, subjectivity remains inevitable in the context of practice or the workplace. Therefore, they suggest, assessment should start with what the assessor actually observes, rather than trying to ‘standardise’ these perceptions away (p. 784). Within this Kate Meier, Pam Parker and Della Freeth 36 J. of Practice Teaching & Learning 12(3), pp.xx-xx. DOI: 10.1921/7701240205. © w&b already challenging assessment, the assessment of interpersonal skills and professional behaviours is even harder to capture (Hawkins et al., 2009). The mechanisms and MRTs identified in this study try to uncover what can support assessment of interpersonal skills, despite assessors’ variability. Though interpersonal skills are tenuous and difficult to describe, their importance is recognised by practitioners (Mann et al., 2005). Unlike exam or essay questions, interpersonal interactions are fleeting and unrecorded, we have only the recall of those involved to rely upon (Shapton, 2007). Personal attributes and communication skills clearly influence the assessment of students in the practical setting, mostly as part of the hidden curriculum (Hilton & Slotnick, 2005). However, explicit assessment of interpersonal skills is often missed, especially if assessment documents do not specifically include interpersonal skills, attitudes and behaviours (Regehr et al., 2007). Studies of medical residents suggest that identifying students with problems in interpersonal skills (that is poor initiative and impaired relationships with students, residents, faculty, and nurses) can be predictive for professional disciplinary action after qualification (Papadakis et al., 2005; Teherani et al., 2009). Problems with interpersonal skills are not uncommon; for example, in a survey of over 1,900 nursing mentors in the UK, 69% reported that ‘attitude (for example, work ethic, time keeping)’ was the worst area of student skill and aptitude (Gainsbury, 2010). Turning to the UK nurse education context for this study; from about six weeks into their programme, nursing students spend half their time in practice or work-based settings, usually spending from four to 12 weeks in each placement for a total of 2,300 hours over a three year programme (NMC, 2008a). During these periods in practice, students are expected to have the supervision of a mentor for 40% of the time. Mentors are clinical nurses who have received some preparation on education and assessment and work in the student’s practice setting (NMC & Mitchell, 2008), but who are given no extra time or financial compensation for mentoring students (Burke & Saldanha, 2005). Although the UK has no national licensing exam, each HEI has to ensure students meet the national requirements provided by the NMC (2008a). Mentors are responsible for assessing the practical component of the programme and HEIs provide practice assessment documents to support and regulate this process. HEIs also provide education programmes for mentors and training related to practice assessments. Mechanisms that support the assessment of interpersonal skills: A Realistic Evaluation 37 J. of Practice Teaching & Learning 12(3), pp.xx-xx. DOI: 10.1921/7701240205. © w&b The Interpersonal Skills Profile The Interpersonal Skills Profile Tool (ISP) was developed by an occupational therapy lecturer in the UK (Knight, 2003; 1998). The ISP attempts to explicitly assess students on interpersonal and professional aspects of their observed performance, using criterion referencing, rather than norm referencing based on the mentor’s personal experiences (van Mook et al., 2009). It comprises 40 statements about interpersonal skills, professionalism and engagement with the learning process, against which students’ performance in practice may be graded. These were based on comments drawn from assessors’ reports of students in occupational therapy. The ISP is currently in use in several UK HEI’s and several health care professions (for example, nursing, occupational therapy, paramedics), either in its entirety or in an adapted form. Therefore, evaluation of how it is actually used in practice is timely, to shed light on an important and challenging area of practice assessment. This study examines the use of the ISP to assess pre-registration nursing students at one large multi-site HEI in the UK, but the findings may also prove useful across other health and non-health related professions that have a practice component. The profile of statements selected by mentors enables students to be awarded a fail, pass, merit or distinction. There are three areas where some items are graded differently in different years of the programme to recognise developmental trajectories. For instance, item 8 ‘Needs to take responsibility appropriate for this level’ is graded as pass in the first year but invokes a fail grade in years two and three. Such items (8 through 13) are termed ‘borderline’ items. There are similar dual grading sections to allow for progression between pass and good (for example, item 25 ‘identifies own learning needs’), and good and excellent (for example, item 35 ‘shows a mature understanding’). The ISP in use during this study was very slightly modified from the original ISP (Knight, 1998). The HEI faculty removed one statement [makes effective use of opportunities and resources] and added space for the assessor to provide comments supporting the selection of each item, and for students to write a paragraph of self-assessment on the next page. The remaining 39 statements [see appendix] are graded fail, pass, good or excellent, and mentors are required to select five statements for each assessment to provide the student with feedback on interpersonal skills and aspects of professionalism. The ISP is one component in a more extensive practice assessment document. Kate Meier, Pam Parker and Della Freeth 38 J. of Practice Teaching & Learning 12(3), pp.xx-xx. DOI: 10.1921/7701240205. © w&b The study setting and data collection This study took place in an HEI that offers nursing programmes in all four fields of practice on the UK Nursing Register (Adult, Child, Learning Disability and Mental Health), with placement opportunities in each field offered in acute, long term and community settings with all degrees of acuity. The HEI had a complex practice document assessing clinical skills and proficiencies required by the NMC (2007). To develop an understanding of how the ISP was actually used in practice, three groups were approached for interviews, including practitioners from each field of practice. Practice education facilitators (PEFs)—whose role was to support all health care professionals involved with teaching and assessing students in the practical setting and who had wide experience of supporting mentors— were approached to gain a broad overview. Education champions (ECs)—who were lecturers responsible for supporting mentors and students in particular areas, like a hospital or area in the community— provided an HEI perspective. Mentors were invited to contribute interviews and students were invited to lend their practice assessment booklets so that copies of the ISP assessments could be taken. Seven PEFs, four ECs, 15 mentors and 20 students contributed to the study (see table). Most participants were from the Adult field of nursing. The practice assessment booklets contained examples of ISP use and comments by around 100 mentors. Participants Number of interviews/sets of documents PEFs Multidisciplinary and multi-field roles. 6 nurses and 1 allied health professional. 7 interviews (6 transcribed, one restricted to field-notes after recording failure) 3 follow-up interviews (PEFs 01-03) ECs Link to areas in all fields, more emphasis on Adult areas. Acute care (3), Community (1) 4 (all transcribed) Mentors Adult (10) & Child (4) all areas of acute trust including outpatients. Community Learning Disability (1) 15 (14 transcribed, one restricted to fieldnotes after recording failure) Documentary Analysis from two cohorts 09/07 and 03/08 Adult (14), Learning Disability (2) and Mental Health (4) 20 sets (16 complete for 8 modules—first two years of programme) Table 1. Research participants Mechanisms that support the assessment of interpersonal skills: A Realistic Evaluation 39 J. of Practice Teaching & Learning 12(3), pp.xx-xx. DOI: 10.1921/7701240205. © w&b Realistic Evaluation This study used a Realistic Evaluation (RE) approach (Pawson & Tilley, 1997) which links the evaluation to the context in which it was conducted and looks at what mechanism or combination of mechanisms may have contributed towards observed outcomes. The aim is to identify Context + Mechanism = Outcome (CMO) groupings from the data (see figure 1). Context refers not just the physical setting but also the culture and practices in which the research is conducted. The identified mechanisms— inferred from observations—are processes that may be triggered by an intervention (in this case use of an ISP) and, in interaction with features of the context, lead to certain outcomes {Wong et al., 2012, Medical Education, 46, 89-96}. Finally, outcomes refer to the observed patterns of events. The triggering of mechanisms can be complex and affected by many factors. The relationships between contexts, mechanisms and outcomes are also complex. The C+M=O formulation looks deceptively simple, but is a powerful heuristic device that prompts close examination of processes and possible causes for effects. ‘Outcome-patterns comprise the intended and unintended consequences of programmes, resulting from the activation of different mechanisms in different contexts.’ (Pawson & Tilley, 2004, p. 7). The purpose of data analysis in RE is to identify what makes up context, mechanism and outcome. Sometimes a context in one situation Figure 1 CMO configuration Adapted from Pawson & Tilley, 1997 p.58 Kate Meier, Pam Parker and Della Freeth 40 J. of Practice Teaching & Learning 12(3), pp.xx-xx. DOI: 10.1921/7701240205. © w&b can be a mechanism in another. For instance, in discussing fireworks, dry storage could be a context when looking at shipping but a mechanism for successful firing of the rocket. In RE, empirical data is used to refine (or reject) hypotheses that researchers bring to their studies (Pawson, 2003), building ever more robust CMO configurations which can be posited as MRTs (Byng et al., 2005; Pedersen & Rieper, 2008). In this study, a cyclical approach to analysis was adopted: each interview was transcribedand at least a preliminary analysis was undertaken before conducting the next interview. Data from interviews and documents were analysed retroductively (Kazi, 2003), searching for mechanisms that underpin how the ISP was used by mentors in practice and abstracting the mechanisms to identify what might work for other professions in similar practical settings. Four CMO configurations were identified with four main contexts, twelve mechanisms and two main outcomes. Through this back and forth process of inductive and deductive coding (Elo & Kyngas, 2008) and abstracting from the data, three MRTs of what works in an assessment tool to support assessment of interpersonal skills in a professional practice setting were identified. Middle range theories Middle range theories (MRTs) are common in nursing (Smith & Liehr, 2008) and are arguably the highest level of abstraction that can be developed from research into contextually bound and practice based questions. MRTs can organise a set of hypotheses and relate them to empirical findings in a way that others can find useful and may enable those findings to be used in other settings (Boudon, 1991). By grouping the mechanisms identified in the study into three ‘mechanism families’ related to an underpinning thread, three MRTs of how a tool such as the ISP can support the assessment of interpersonal skills can be proposed. First some specific contextual factors will be reviewed, followed by a brief discussion of outcomes. Quotations from mentors will be marked M; the abbreviations PEF and EC, which were introduced earlier, will be used for contributions from participants with these roles. Mechanisms that support the assessment of interpersonal skills: A Realistic Evaluation 41 J. of Practice Teaching & Learning 12(3), pp.xx-xx. DOI: 10.1921/7701240205. © w&b Enabling and disabling contexts identified in the study In a sense, each student is engaged in his or her own nursing programme, having worked with different clients, families, staff and in a variety of settings. Within those environments there are additional inconsistencies; being busy and under pressure are common complaints in many settings (Govaerts et al., 2011). PEF07 describes some of the many challenges that nursing mentors face in assessing pre-registration nursing students: P29: PEF07 (205:205) PEF07: I mean, sometimes because of the time frame, it is a forced relationship and the student is obviously there to learn […] they don’t know the student for very long they have to make a comment or judgement, […] it’s a huge responsibility then for the mentor because they can only go on the information that is provided to them or what they saw themselves I think people do feel uncomfortable […] This pressure can mean that mentors find it difficult to assess students after a short period of time. P7: M07 (75:75) M07: I think it is very difficult especially when they perhaps haven’t been here for a long period of time and you have got to assess that. Yes but at the same time I think it [the ISP] is a good thing to do definitely. I think it is very helpful because obviously just because you have…you can do certain tasks doesn’t mean you have the skills that you really need. Outside of the ISP itself, factors that enable the assessment of interpersonal skills are: the enthusiasm and engagement of the mentor; settings where mentors feel prepared and confident in their own judgement; and areas that welcome students. Other broader factors include the NMC’s Standards (2008b; 2006) and a push by both clinical areas and the HEI to better prepare mentors. In this study, interpersonal skills are seen as ephemeral, difficult to define and uncomfortable to assess. Furthermore assessors feel ill prepared to do so and are unsure that the assessment was even wanted by the HEI. Even confident, strong mentors feel challenged by the practice assessment document, and the pressures of work and time. Thus, disabling factors for assessment of interpersonal skills are the practice setting, an assessors’ own training, experience and level of confidence, and the nature of what Kate Meier, Pam Parker and Della Freeth 42 J. of Practice Teaching & Learning 12(3), pp.xx-xx. DOI: 10.1921/7701240205. © w&b is being assessed. Situational factors are also important. In placements of a week or two in duration, the nuanced grades of interpersonal skills are challenging to assess. Awarding good and excellent may also be a challenge as some of the higher items (for instance, confidence in decision making and being innovative) may be difficult to achieve in specialist areas. Outcomes Prior to the introduction of the ISP in the placement documentation, interpersonal skills may have been addressed; however, participants suggest it was not clearly done, there was nowhere to document it and students may have been unaware of the assessment. The main outcome of the ISP, identified in the study, is that interpersonal skills are overtly assessed. Everyone involved in assessment after the introduction of the ISP is aware that interpersonal skills are actually being assessed and can identify what is expected. Furthermore, the assessment enables students to change (improve their skills) and/or challenge their assessments. Middle range theory 1: making it overt What the ISP seems to be doing is making the assessment of interpersonal skills overt and thus usable. This transparency can create a potential for learning dialogue between assessor and student (Gillespie, 2005). Though some items are ambiguous (for example, what exactly constitutes a ‘pleasant and approachable manner’, item 23?), the presence of descriptors in the document opens up an awareness of an expectation and a possibility for the mentor and student to discuss these expectations. In this study, overt-ness is achieved through three mechanisms (explicit, clarity, levelling) discussed below. Mechanisms are denoted by italics and will be expanded upon in turn. By being explicit, the ISP brings the assessment of interpersonal skills to the surface of the practice learning experience. Assessors can see that they are expected to comment on aspects such as maturity (items 13, 24 and 35), teamwork (items 22, 28 and 36) or responsibility for learning (items 6 and 27). PEF07: I think it [the ISP] probably highlights that it is just as important for mentors Mechanisms that support the assessment of interpersonal skills: A Realistic Evaluation 43 J. of Practice Teaching & Learning 12(3), pp.xx-xx. DOI: 10.1921/7701240205. © w&b and students to be aware that students that can equally fail on clinical skills as well as interpersonal skills whereas before I think 95% mentors really focused on clinical skills and I think they forgot about the interpersonal […] so I suppose it just makes people aware, and raises the profile, and I think that it has gotten them to question their own opinions and interpersonal skills as well, of their own personal judgement really. Though some mentors report interpreting the borderline items differently and documentary analysis reveals that evidence for choices was heterogeneous, the mechanism of clarity is important. The ISP is in the assessment document, for formative and summative use in every part of the programme, making it apparent to all that interpersonal skills are an important part of the assessment. Not only is the need to assess interpersonal skills seen but mentors can clarify students’ progress both to help them develop and for summative feedback. MA07: …[it is] helpful to the students really because it is pointing out things they need to improve on so that they are more self-aware. I mean I have had to speak to both of them [current students] about certain issues that weren’t necessarily a problem but things that they needed to improve on really…it [the ISP] is motivational I suppose. Finally there is a levelling mechanism that does not overcome but can minimise the great variability between practical experiences. All placements in the pre-registration nursing programme are required to assess using the ISP, regardless of field of practice, acuity of the setting, and year of the programme. This means that interpersonal skills are not linked to a particular aspect or type of nursing but reinforces that interpersonal skills permeate all of nurses’ (or any) professional practice. While assessment remains a local practice (Knight & Yorke, 2008), the ISP can provide some consistency to assess–at least the lower scoring items (as identified in enabling and disabling contexts above)—in a variety of settings. Abstracting from the particular context of pre-registration student nurses in one HEI, MRT 1 is proposed as: Making requirements about assessing interpersonal skills clear to students and assessors, can enable overt assessment and learning. Kate Meier, Pam Parker and Della Freeth 44 J. of Practice Teaching & Learning 12(3), pp.xx-xx. DOI: 10.1921/7701240205. © w&b Middle range theory 2: providing support This group of six mechanisms seems to revolve around supporting the mentor, both practically and emotionally, to complete the assessment. A place to document is a pragmatic mechanism that literally allows assessors to capture the transitory and intangible aspects of practical education. Circling an item on the ISP and writing a comment or providing evidence may assist in assessing interpersonal skills and in documenting conversations. Previously, mentors reported that they could not fail students if the problem was not in the documentation. EC4: I think before the documentation we had was that if you’re not assessing something in a way it’s hard to address it […] it was sort of a generic progression rather than perhaps identifying any particular areas, so there were students who would turn up late, those students who would be off sick a lot, there would be students who were quite negative, but there was no way sort of purposively for mentors to write it and so in a way not having to raise it they didn’t […] although those things area quite important to anybody who’s a trained nurse there was nowhere of putting it in our previous documentation where now that we’ve got it, it makes it a very real issue now, that actually as a mentor it is about professionalism as well as delivering clinical care. The prompt mechanism lies between practical and emotional support. By existing in every practice document twice, the ISP serves as a visual reminder that the assessment must take place. PEF08: I think it gives them support but it also gives them a prompt, whereas they might have just kind of let things slip before, they are not able to as such here, they’ve got to do this assessment… Prompting is important in that assessors may not consciously look at attitudes and behaviours if they do not feel they form part of the assessment. Enabling functions on two levels. Firstly, in this study, mentors found it easier to select items from a list than to have to generate their own responses. PEF02: I think probably it’s like a, it’s easy isn’t it, writing a learning contract you’ve got to have the time. The ward staff are phenomenally busy, when you’ve got a failing student and it’s phenomenally hard, especially some of the ones we’ve had recently so to be able to write a number and just write a brief thing kind of gives a Mechanisms that support the assessment of interpersonal skills: A Realistic Evaluation 45 J. of Practice Teaching & Learning 12(3), pp.xx-xx. DOI: 10.1921/7701240205. © w&b nice segment (sic) section to be able to do it, so yes they say they like it they find it easy, they find it useful. The second practical aspect of enabling is simplicity. Research has demonstrated that assessors frequently do not use lengthy or complicated assessment documentation (Calman et al., 2002; Hunt et al., 2011; Huybrecht et al., 2011), a finding supported in this study by one mentor who suggests that detailed assessments do not get done as well or as often. M05: […] I think it is going to be harder and harder to make the time and the documentation tends to get more detailed or precise, it is not going to be done either one, as thoroughly or two, as often […]. Simple tools support the assessment process and allow mentors to decide what is important to note about the student’s behaviour. Three further mechanisms in this grouping provide emotional support. The ISP gives mentors a distance from which to assess interpersonal skills. The one-to-one relationship between mentors and students and the faceto-face verbal assessment is emotionally difficult. M15: […], obviously it’s my opinion but it’s not my exact comment [the ISP], whereas if I was to write a comment saying, ‘difficulty with this that and the other’ it would feel that I was personally attacking them, where that way it’s still personal but it takes a little bit of the personal edge away. Mentors work closely with students and need to continue to do so after the assessment period. The ISP provides a way for them to initiate difficult conversations, without it seeming like they are personally attacking the student. Permission and legitimises are two closely aligned mechanisms that function by reinforcing to the mentor that they are allowed to assess such subjective aspects as interpersonal skills and that they are not alone in doing so. Permission relates to all mentors. PEF02: […] because I think as I said at the very beginning when you see [the ISP] in black and white, it gives people permission to identify what the issue is. Whereas legitimise supports mentors lacking confidence. PEF08: They [mentors] love it, for the same reason I do, it’s that you’ve always had Kate Meier, Pam Parker and Della Freeth 46 J. of Practice Teaching & Learning 12(3), pp.xx-xx. DOI: 10.1921/7701240205. © w&b these students who are borderline and you know, yes, if you are a very strong person you can do something about it but I think on the whole people aren’t experienced enough, a very small percentage of mentors would actually be experienced enough and mentally strong enough to deal with it’. By including items related to motivation, confidence and response to criticism in the tool with evident consequences the academic institution is sending a clear signal to practice assessors that failure, or reward, is an assessment option. As discussed above, an outcome of these mechanisms is that they increase mentor confidence in assessing interpersonal skills. However, increased mentor confidence is itself also a mechanism that supports mentors to overtly assess interpersonal skills. The second proposed MRT is: A tool to assess interpersonal skills that is provided in practice assessment documentation can support assessors both practically/physically and emotionally. Middle range theory 3: Feedback-feed forward The ISP is used both for formative and summative assessment. Some theories of assessment suggest that different tools should be used as midpoint and final assessments have different purposes (Galbraith et al., 2011). However, in practice, where the assessors have little access to training, it is simpler to use the same tool for both assessments (Dalton et al., 2009). Ideally, if a midpoint assessment is completed, the student can use the formative written and verbal feedback, to improve their skills (Fotheringham, 2011), and as a benchmark for what they might achieve in the summative assessment. Furthermore, the student can use the feedback from one placement to prepare for the next. M01: But this, I do feel that this document does help, but it also helps to motivate as well. It is nice to get feedback about how you are doing. KM: So you think the students who are at the better end feel motivated. M01: Definitely, even those who aren’t the fact that I have seen an improvement in their performance because of this...(taps the ISP). For the mentor and the student, the consequences of assessment can clearly be seen; some items will result in a fail while, others indicate superior performance. In this study mentors reported enjoying being able to reward Mechanisms that support the assessment of interpersonal skills: A Realistic Evaluation 47 J. of Practice Teaching & Learning 12(3), pp.xx-xx. DOI: 10.1921/7701240205. © w&b good practice and thought it could stimulate students’ motivation. M05: Actually as a progressive thing as well, if someone sees that they are improving and going up the scale that is more motivational. So it can be more motivational as well. Students often find it difficult to hear and respond constructively to feedback (Eva et al., 2011) and the intention of the mentor, to ‘feed-forward’ to the summative or next assessment can be lost (Knight, 2006). This difficulty in assimilating critical feedback means that the mechanism of evidence is also important. Because there is a place to document the reasons behind the choice of items selected, there is a record for students to return to and potentially learn from or to challenge if it is perceived of as unfair. From the documentary analysis it could be seen that evidence was not always well executed and assessments could be seen to be done late in the placement or both the formative and summative on the same day. However, the requirement to provide evidence for the selection of items supports a level of transparency and clarity [see MRT 1] that allows the student to either accept the comments or to challenGE The third proposed MRT is: Overt feedback and written evidence that has clear consequences can feed-forward and be motivating to students. It can allow students to assimilate feedback and develop their practice; or constructively challenge potentially biased feedback. Limitations of the study There were limitations of this doctoral research in which the first author was not only conducting a study but learning how to do so. The most relevant here is that students were not interviewed. Though their practice documents were obtained, the outcome that students could change or challenge came from the words of their assessors (mentors), supporters (ECs) and observers (PEFs). Further study of the use of the tool involving observations of midpoint and summative assessments with follow up interviews could provide a richer picture of the experience of being assessed and a way to interrogate the MRTs and mechanisms developed. Kate Meier, Pam Parker and Della Freeth 48 J. of Practice Teaching & Learning 12(3), pp.xx-xx. DOI: 10.1921/7701240205. © w&b Recommendations for practice The ISP is a practical tool, and is already in use in several institutions and professional programmes. The recommendation of this study is that some sort of overt assessment of interpersonal skills should exist in the practice documentation alongside practical skills and knowledge assessments [MRTs 1 and 2]. Furthermore the feedback from this assessment should be both verbal and written so that students can both digest it and have evidence to challenge any potentially unfair assessments [MRT 3]. Conclusion Interpersonal skills form an important part of professional education and are notoriously difficult to assess. Through being clear, explicit and providing emotional and practical support through distance and a place to document, an assessor can initiate difficult conversations that might serve either to help the student improve, or if necessary support the difficult process of failing the assessed practice. A vital part of assessment is written evidence so that students can understand and respond to feedback by changing their practice or challenging a potentially biased or unfair assessment. Traditional summative assessments often measure something other than what evaluators are actually looking for (Broadfoot, 2001) and not everything worth learning is measurable; the purpose of assessment should be to support learning (Neighbour, 2003). Instead of trying to eliminate differences and variability between practice settings and individual mentors, the ISP can capture their assessments with the safety net that biased assessments can be picked up and challenged. Reflecting on the different agendas of practical and academic learning, Millar (1985) said: ‘nurses rely on implicit knowledge, nurses see that nursing is what nurses do, while educationalists see it as what nurses ought to do’ (cited in Andrews & Jones, 1996). In its simplicity and overt place in the documentation, the ISP allows nurses to decide what nursing is and make that knowledge explicit, in effect handing interpersonal skills assessment from the HEI over to the practitioner, while providing safeguards for all stakeholders. Mechanisms that support the assessment of interpersonal skills: A Realistic Evaluation 49 J. of Practice Teaching & Learning 12(3), pp.xx-xx. DOI: 10.1921/7701240205. © w&b References Allison, H. and Turpin, M.J. (2004) Development of the student placement evaluation form: A tool for assessing student fieldwork performance. Australian Occupational Therapy Journal, 51, 125-132 Andrews, M. and Jones, P.R. (1996) Problem-based learning in an undergraduate nursing programme: A case study. Journal of Advanced Nursing, 23, 357-365 Bogo, M., Hughes, J., Regehr, C. et al. (2004) Toward new approaches for evaluating student field performance: Tapping the implicit criteria used by experienced field instructors. Journal of Social Work Education, 40, 417-426 Boudon, R. (1991) What middle-range theories are. Contemporary Sociology, 20, 519-522 Broadfoot, P. (2001) Liberating the learner through assessment. In J. Collins and D. Cook Learning: Influences and outcomes, London : Paul Chapman Publishing/ Open University Press Burke, A. and Saldanha, M. (2005) NMCconsultationonastandardtosupportlearning and assessment in practice. London : NMC Byng, R., Norman, I., and Redfern, S. (2005) Using realistic evaluation to evaluate a practice-level intervention to improve primary healthcare for patients with long-term mental illness. Evaluation, 11, 69-93 Calman, L., Watson, R., Norman, I., Redfern, S. and Murrells, T. (2002) Assessing practice of student nurses: Methods, preparation of assessors and student views. Journal of Advanced Nursing, 38, 516-523 Dalton, M., Keating, J. and Davidson, M. (2009) Development of the Assessment of Physiotherapy Practice (APP): A standardised and valid approach to assessment of clinical competence in physiotherapy. Brisbane : Griffith University Elo, S. and Kyngas, H. (2008) The qualitative content analysis process. Journal of Advanced Nursing, 62, 107-115 Eva, K.W., Armson, H., Holmboe, E.etal. (2011) Factors influencing responsiveness to feedback: On the interplay between fear, confidence and reasoning processes. Advances in Health Science Education, 17, 15-26 Fotheringham, D. (2011) The role of expert judgement and feedback in sustainable assessment: A discussion paper. Nurse Education Today, 31, e47-e50 Gainsbury, S. (2010) Mentors passing students despite doubts over ability. Nursing Times, 106, 1-3 Galbraith, R., Clyman, S.G. and Melnick, D.E. (2011) Conceptual perspectives: Emerging changes in the assessment paradigm. in J.P. Hafler Extraordinary Learning in the Workplace. Dordrecht : Springer (pp. 87-100) Gillespie, M. (2005) Student-teacher connection: A place of possibility. Journal of Kate Meier, Pam Parker and Della Freeth 50 J. of Practice Teaching & Learning 12(3), pp.xx-xx. DOI: 10.1921/7701240205. © w&b Advanced Nursing, 52, 211-219 Ginsburg, S., McIlroy, J.H., Oulanova, O., Eva, K. and Regehr, G. (2010) Towards authentic pursuit of clinical evaluation: Pitfalls in the pursuit of competency. Academic Medicine, 85, 780-786 Govaerts, M.J.B., Schuwirth, L.W.T., van der Vleuten, C., and Muijtjens, A.M.M. (2011) Workplace-based assessment: Effects of rater expertise. Advances in Health Science Education, 16, 151-165 Hawkins, R.E., Katsufrakis, P.J., Holtman, M. and Clauser, B.E. (2009) Assessment of medical professionalism: Who, what, when, where, how and why? Medical Teacher, 31, 348-361 Hilton,S.R.andSlotnick,H.B.(2005)Proto-professionalism:Howprofessionalisation occurs across the continuum of medical education. Medical Education, 39, 58-65 Hunt, L.A., McGee, P., Gutteridge, R. and Hughes, M. (2011) Assessment of student nurses in practice: A comparison of theoretical and practical assessment results in England. Nurse Education Today, in press Huybrecht, S., Loeckx, W., Quaeyhaegens, Y., De Tobel, D. and Mistiaen, W. (2011) Mentoring in nursing education: Perceived characteristics of mentors and the consequences of mentorship. Nurse Education Today, 31, 274-278 Kazi, M. (2003) Realist evaluation for practice. British Journal of Social Work, 33, 803-818 Knight, J. (1998) A fieldwork award: Rewarding excellence in practice. British Journal of Occupational Therapy, 61, 316-320 Knight, J. (2003) Assessment of fieldwork practice: The student experience. Journal of practice teaching in health and social work, 5, 39-60 Knight, P. (2006) The local practices of assessment. Assessment and Evaluation in Higher Education, 31, 435-452 Knight, P. and Yorke, M. (2008) Assessment close up. The limits of exquisite descriptions of achievement. International Journal of Educational Research, 47, 175-183 Mann, K.V., Ruedy, J., Millar, N. and Andreou, P. (2005) Achievement of noncognitive goals of undergraduate medical education: Perceptions of medical students, residents, faculty and other health professionals. Medical Education, 39, 40-48 Neighbour, R. (2003) Reflections on the ethics of assessment. Education for Primary Care, 14, 406-410 Nursing and Midwifery Council. (2006) Standards to Support Learning and Assessment in Practice: NMC standards for mentors, practice teachers and teachers. London : NMC Nursing and Midwifery Council (2007) Mapping of Essential Skills Clusters to the Mechanisms that support the assessment of interpersonal skills: A Realistic Evaluation 51 J. of Practice Teaching & Learning 12(3), pp.xx-xx. DOI: 10.1921/7701240205. © w&b Standards of proficiency for pre-registration nursing education (NMC 2004). Annexe 3 to NMC Circular 07/2007, 9 Nursing and Midwifery Council. (2008a) StandardsofProficiencyforPre-Registration Nursing Education. London : NMC Nursing and Midwifery Council. (2008b) Standards to Support Learning and Assessment in Practice. (2nd ed.) London : NMC NursingandMidwiferyCouncil,andMitchell,D.(2008) AReviewofPre-Registration Nursing Education: Report of consultation findings. 1-137 Papadakis, M.A., Teherani, A., Banach, M. et al. (2005) Disciplinary action by medical boards and prior behavior in medical schools. The New England Journal of Medicine, 353, 2673-2682 Pawson, R. (2003) Nothing as practical as good theory. Evaluation, 9, 471-490 Pawson, R. and Tilley, N. (1997) Realistic Evaluation. London: Sage Pawson, R. and Tilley, N. (2004) Realist Evaluation. www.communitymatters. com.au/RE_chapter.pdf Pedersen, L.H. and Rieper, O. (2008) Is Realistic Evaluation a realistic approach for complex reforms? Evaluation, 14, 271-293 Regehr, G., Regehr, C., Bogo, M., and Power, R. (2007) Can we build a better mousetrap? Improving the measures of practice performance in the field practicum. Journal of Social Work Education, 43, 327-343 Shapton, M. (2007) Failing to fail students in the caring professions: Is the assessment process failing the professions? Journal of Practice Teaching and Learning, 7, 39-54 Smeby, J.-C. (2007) Connecting to professional knowledge. Studies in Higher Education, 32, 207-224 Smith, M.J. and Liehr, P.R. (2008) Understanding middle range theory by moving up and down the ladder of abstraction. In M.J. Smith and P.R. Liehr Middle range theory for nursing. (2nd ed.) New York : Springer (pp. 13-32) Teherani, A., O’Sullivan, P., Lovett, M., and Hauer, K.E. (2009) Categorization of unprofessional behaviours identified during administration of and remediation after a comprehensive clinical performance examination using a validated professionalism framework. Medical Teacher, 31, 1007-1012 van Mook, W.N.K.A., van Luijk, S.J., O’Sullivan, H. et al. (2009) General considerations regarding assessment of professional behaviour. European Journal of Internal Medicine, 20, e90-e95 Wilson, K. and Scammell, J. (2010) Opportunities and challenges of grading performance in practice. In M. Hammick and C. Reid Contemporary Issues in Assessment in Health Sciences and Practice Education. London: Higher Education Academy (pp. 94-111) Kate Meier, Pam Parker and Della Freeth 52 J. of Practice Teaching & Learning 12(3), pp.xx-xx. DOI: 10.1921/7701240205. © w&b