STEVEN KEEN, KEITH BROWN, JANE HOLROYD, AND EMILY ROSENORN-LANNG 44 Social Work & Social Sciences Review 16(1) pp.xx-xx. DOI: 10.1921/1203160302 Evaluating the impact of the IPOP (Improving Personal and Organisational Performance) programme: An introductory leadership and management development module for social work managers Steven Keen1 , Keith Brown1 , Jane Holroyd3 , and Emily Rosenorn-Lanng4 Abstract: A host of recent UK social work publications have highlighted the critical importance of leadership and management development for social work managers. The lack of specialist leadership and management development is evidenced by the General Social Care Council’s figures on the uptake of post-qualifying leadership and management programmes. There is a limited amount of published research on the impact of this type of professional learning as most studies focus on programme delivery rather than on their impact on practice. Hence, this paper will report on an evaluation of the practicerelated impact of an introductory leadership and management programme, using data from pre- and post-programme questionnaires (n=75), follow up telephone interviews at 3-months (n=24) and an initial analysis of submitted assignments (n=32). Those data are drawn from consenting social work managers across 5 widespread local authorities in England. The programme, Improving Personal and Organisational Performance, is taught over 4-days in two, 2-day blocks and focuses on three areas of development: self-leadership, communication and resilience as a leader. A third-party testimony process,alongsideassignmentsubmission,ensurestheinvolvementof,andvalidationfrom,theemployer. The findings of the evaluation indicate positive, statistically significant evidence of initial programme impact on managers and longitudinal examples of actual programme impact on managers, their teams, organisations and services. The implications of these findings will be considered in relation to the new Professional Capabilities Framework. Keywords: continuing professional development; post-qualifying social work; leadership and management; evaluation; impact; self-leadership. 1. Senior Lecturer in Research, National Centre for Post-Qualifying Social Work, Bournemouth University 2.ProfessorofPost-QualifyingSocialWork,Director,NationalCentreforPost-QualifyingSocialWork 3. IPOP Unit Leader, National Centre for Post-Qualifying Social Work 4. Research Assistant, National Centre for Post-Qualifying Social Work Address for correspondence: Dr Steven Keen BA (Hons), MA, Ph.D, Senior Lecturer in Research, National Centre for Post-Qualifying Social Work, Bournemouth University, Christchurch Road, Bournemouth BH1 3LT. skeen@bournemouth.ac.uk THE IPOP (IMPROVING PERSONAL AND ORGANISATIONAL PERFORMANCE) PROGRAMME 45 Introduction Since 2009, UK social work publications have highlighted the critical importance of future leadership development for social work managers. Laming (2009) pointed out how serious social work case reviews cite the significance of effective leadership and management in the same year as the Social Work Task Force recommended the creation of dedicated programmes of training and support for frontline social work managers (DCSF, 2009). Professional leadership and the taking of responsibility for the professional learning and development of others via management, supervision, mentoring, and the like, then became one of nine capabilities within the new ProfessionalCapabilitiesFrameworkforsocialworkers(TCSW,2012).EileenMunro’s (2011) review of child protection recommended the appointment of a principal, local authority, child and family social worker that is both a senior manager and yet remains involved in frontline practice. Under the new Ofsted (2012) framework for the inspection of local authority children’s services, the quality of leadership and management is one of three key areas that inspectors will judge. That leadership and management development is crucial for the future of social work is not in doubt – the exact nature and potential impact of this development is (Watson, 2008; Leinster, 2010; Hafford-Letchfield & Bourn, 2011). In response to the above political and professional agenda, Learn to Care, the professional association for workforce development managers in social care commissioned Holroyd and Brown (2011) to develop a leadership pathway for social work managers. The IPOP (Improving Personal and Organisational Performance) (Holroyd, 2012) programme is the starting point of this pathway and its evaluation is the subject of this paper. IPOP has been designed as an introductory unit for social work managers with little or no experience of formal leadership and management development and leads to the award of a Graduate Certificate in Leadership and Management within the National Centre for Post-Qualifying Social Work at Bournemouth University. Post-qualifying leadership and management development In the early 1990s, the regulatory body for social work at the time introduced a framework for post-qualifying education and training providing two levels of professional award – a post-qualifying award (for most social workers) and an advanced award for specialist studies (see Taylor et al. 2010). This framework was in part based on the American idea of competence-based education (Yelloly, 1995) and has been superseded twice since then; first, by a revised post-qualifying framework in 2007 (GSCC, 2004), and second, by the recent Professional Capabilities Framework for social workers highlighting, as the title suggests, a move away from competenceSTEVEN KEEN, KEITH BROWN, JANE HOLROYD, AND EMILY ROSENORN-LANNG 46 based education (TCSW, 2012). The revised post-qualifying framework for social workers provides the immediate background to this study as it includes a range of specialist, higher specialist and advanced programmes of study for child care, adult or mental health social workers and/or those working as practice educators or managers (GSCC, 2004). Empirical evidence about the effectiveness of this type of leadership and management development programme, ‘its prescribed curriculum and outcomes, is not strong…there is insufficient knowledge in the field about social work managers’ pivotal but challenging role to improving relationships within and between the organisations working together to provide more seamless, responsive and integrated services’ (Hafford-Letchfield & Bourn, 2011, p.499). Therefore, ‘research and debate in the area of social work management and leadership is still in its infancy’ (HaffordLetchfield & Lawler, 2010, p.5). McAllan and MacRae’s (2010) evaluation of a leadership development programme for actual and prospective social work managers within a single Scottish council is a rare empirical example. What evidence there is often refers to localised evaluations of (non-leadership and management) postqualifying social work programmes (for example, Bailey, 2002; Bailey et al., 2003; Brown & Keen, 2004; Brown et al., 2008; Doel et al., 2008; Webber et al., 2010; Chapman & Morris ,2011; Kelly & Jackson, 2011; Platt, 2011; Webber & Salter, 2011). Indeed, as Carpenter (2011, p.137) points out ‘there is, as yet, little evidence of changes in behaviour as a result of social work education’ in general. Perhaps this lack of evaluative evidence should not surprise us, as enrolments in the revised post-qualifying social work leadership and management specialism only reached 133 candidates (5% of total post-qualifying candidates) in 2009-10 (GSCC, 2011). In total, about 360 candidates have enrolled on the leadership and management specialism since 2007 (GSCC ,2011) with 97 (27%) of these candidates studying with the National Centre for Post-Qualifying Social Work at Bournemouth University. Given the estimate that 14% of social workers are likely to hold managerial positions (TIC 2012) – equivalent to about 12,000 social work managers – this level of investment in leadership and management development and its evaluation thereof is paltry. More recently, however, the number of candidates enrolling on leadership and management programmes at Bournemouth University has increased markedly since 2010 (n=314). The evaluation of leadership and management development should be an integral part of its improvement. To show evidence of in-depth impact beyond learners’ reactions, it is essential for evaluations to move towards assessing changes in behaviour and organisational practice, and benefits to users and carers (McAllan & MacRae, 2010; Carpenter, 2011; Holroyd & Brown, 2011). Within this context, therefore, the overarching aim of this study has been to evaluate the impact of IPOP on the practice of social work managers, and more specifically to seek out evidence of changes in managerial behaviour and organisational practice across a number of widespread organisations. THE IPOP (IMPROVING PERSONAL AND ORGANISATIONAL PERFORMANCE) PROGRAMME 47 IPOP Originally designed for managers with little or no experience of leadership and management development (as opposed to leadership and management experience in general), IPOP is an introductory 40 level H or 6 credit unit (equivalent to the academic standard of an undergraduate student’s final year). Candidates can continue with four, further 20 credit units to exit with either a Graduate Diploma in Leadership and Management or top-up their previous qualifications to receive a BA (Hons) in Leadership and Management. Therefore, IPOP is the starting point of a pathway of leadership and management development (Holroyd & Brown, 2011). The IPOP programme is based on relevant frameworks and recommendations (for example: DCSF, 2009; GSCC, 2007; Leadership Academy, 2011; TCSW, 2012) and discussions with local employers, and focuses on three key areas of development, each in part influenced by the theories and practice of Neuro-linguistic Programming (NLP): • Self-leadership • Written, verbal and non-verbal communication • Resilience NLP has three components: neuro – how we use our neurology to think and feel; linguistic – how we use language to influence others and ourselves; and programming – how we act to achieve the goals we set (Walker 2004). Chosen in this study for its emphasis on ‘change’, NLP has gained significant recent attention for example as part of the positively evaluated Fast Track leadership programme for teachers (Jones & Attfield, 2007). IPOP managers are encouraged via a focus on NLP techniques, to explore their leadership and management capability, starting with themselves, in part by developing an action plan focusing on two (personal and organisational) objectives and critically reflecting upon their impact and experience via a two-part assignment: • Part A – (2,000 to 3,000 words) Action plan with critical analysis of chosen outcomes • Part B – (3,000 to 4,000 words) Reflection and critical analysis of the implementation of the above aligned action plan A ‘third party’ testimony, usually written by a manager’s line manager, provides evidence and confirmation of the practical application and impact of the objectives within the action plan. The third party testimony handbook, a general programme handbook and a reader and guidance on additional learning needs and information skills are sent prior to the start of the programme. The concept of self-leadership is a new approach defined as ‘the process of influencing oneself’ to create the direction and motivation needed to perform (Manz & Neck, 2009, p.4). It builds on previous concepts of self-awareness (see Neill, 2009), self-management (Politis, 2005) and distributed leadership (see Gray et al., STEVEN KEEN, KEITH BROWN, JANE HOLROYD, AND EMILY ROSENORN-LANNG 48 2012) and centres on changing the behaviour of managers as a result of changing they way they think and communicate (Manz & Neck, 2009; Holroyd & Brown, 2011). Therefore, self-leadership is not just about self, but encompasses how a manager works more effectively within teams and organisations and so recognises the often artificial distinctions found in the literature between ‘leadership’ and ‘management’ – both are required in social work (Jones, 2010; Holroyd & Brown, 2011). Method Junior and senior managers of adult and children’s services (n=75, out of a total of 98) from seven IPOP cohorts spread across five widespread local authorities (and three other independent, voluntary and private sector organisations) in England consented to take part in a prospective, three-stage evaluation – after reading an information sheet containing details about the purpose and process of the evaluation (including statements about anonymity, confidentiality and complaints), and signing a consent form: Stage 1: pre-post questionnaires Stage 2: follow up telephone interviews at 3-months Stage 3: an analysis of submitted assignments Similar to McAllan and MacRae’s (2010) methodology of questionnaires and follow-up interviews, IPOP questionnaires, administered before the start of the programme and at the completion of teaching on day 4, used a mix of 4, 5 and 10 point rating scales with open questions to allow for programme evaluation over a six week period. Questions and statements mirror course content, for instance, around resilience, communication, and building and managing relationships, and also cover the organisation and delivery of the programme. Pre-post data from 75 managers was summarised by cohort and as a group of cohorts, using simple frequencies and descriptive statistics (that is, mean, median and mode), content analysis and other statistical measures where appropriate (that is, Wilcoxon signed rank test, KruskalWallis test, and regression analyses). This data acted as a building block for Stage 2 of the evaluation as the telephone interviews provided an opportunity to follow up areas of statistically significant change to gain concrete examples of programme impact. Audio-recordedtelephoneinterviewswithasub-sampleof24consentingmanagers took place about three months after IPOP teaching and lasted between 7 and 25 minutes. Each recording has been studied at least once in order to produce detailed notes on the programme in general and examples of IPOP-related impact over time – and the framework of this content analysed using a series of charts (Ritchie & Lewis, 2003). THE IPOP (IMPROVING PERSONAL AND ORGANISATIONAL PERFORMANCE) PROGRAMME 49 Stage 3 involved an initial review of managers’ submitted assignments (n=32) from Agency 1 and Agency 3. The emphasis of the assignment is on translating IPOPrelated learning to the work environment with a focus on ‘change’ – encouraging managers to specify objectives or targets they are currently working on or which may have already been identified as part of a performance development appraisal. A more detailed analysis of a larger number of submitted assignments is the subject of another manuscript, however, this initial review includes the grouping of key objectives and written feedback from third party testimonies and IPOP managers. The three stages of this evaluation took place between February 2011 and June 2012. Results As the focus of this study is to evaluate IPOP and its impact on practice, this section is divided into two main parts after details about the profile of managers – course evaluation and further course impact. Italicised text refers to verbatim, anonymised quotations, some of which have been tidied to improve readability without losing meaning. Profile of managers The sample of 75 managers from the seven IPOP cohorts consenting to take part in the evaluation contained just over three-quarters of the total attendees (see Table 1). Although originating from five widespread local authorities (and three other agencies) in England, over half of this sample works for a large council in the South of England. The vast majority of completed questionnaires and interviews derive from managers working in agencies 1-3 and approximately one-quarter of the total attendees consented to the telephone interview. Table 1 Number of Managers by Agency Agency IPOP Attendees Questionnaire Sample Interview Sample Agency 1 46 41 10 Agency 2 29 16 7 Agency 3 14 12 5 Agency 4 5 5 1 Agency 5 1 0 0 Other 3 1 1 Totals 98 75 24 STEVEN KEEN, KEITH BROWN, JANE HOLROYD, AND EMILY ROSENORN-LANNG 50 The characteristics of this sample are broadly similar to all candidates completing other leadership and management programmes at BU in that the majority are female (69%), aged over 41 (60%), have at least a degree level qualification (60%), have no additional learning needs (88%) and class their ethnicity as ‘white’ (97%). Course evaluation Managers evaluated IPOP positively – 94% rated it ‘excellent’ or ‘good’ (see Table 2). No manager rated IPOP as poor. Managers were also asked to agree or disagree with a series of more detailed assertions around programme delivery. Nearly all managers (range 96%–100%) agreed or strongly agreed with the nine statements listed in Table 2. Virtually 60% (p<0.01) of the variance in overall course evaluation ratings may be predicted by managers’ expectations of ‘the subject content and range of topics’. There is no significant difference in the average ratings by cohort (Kruskal-Wallis test) – put another way, these positive findings are consistent across each cohort. Even when the course had not met prior expectations in regards to content the most sceptical of managers had still found IPOP of benefit: I have to say that I felt quite negative about this being an appropriate course for me as it did not cover what I thought it would i.e. role expectations of me and what was being put on me in my new role. Now I can see the benefit of looking at me first in order to look beyond. From the open responses to the post-course questionnaire, managers appeared to appreciate three core aspects of IPOP: • The communicative ability and motivational style of the course tutors (n=21) the tutor – very engaging, knowledgeable and helpful … the comfortableness of the course i.e. the easy and calm manner in which the course materials are delivered. • Course content (n=15) As expected, given much of the variance in overall course evaluation ratings is predicted by managers’ expectations of ‘the subject content and range of topics’ • Group participation (n=6) However, managers across all cohorts (n=34) believed they required more ‘time’ to reflect on, discuss and apply the learning, and practise the taught techniques. For example, managers would have appreciated more opportunities for: longer discussion on specific topics i.e. some topics, advanced questioning skills I feel could have been discussed further. reflection on the learning – if a study day took place between the blocks of teaching, that would have helped. THE IPOP (IMPROVING PERSONAL AND ORGANISATIONAL PERFORMANCE) PROGRAMME 51 Table2:OverallCourseEvaluation Excellent(1)Good(2)Fair(3)Poor(4)Mean Overall,IwouldevaluatetheIntroductionto LeadershipandManagementcourseas….. CountValidN%CountValidN%CountValidN%CountValidN% 3251%2743%46%00%1.6 StronglyAgree(1)Agree(2)Disagree(3)StronglyDisagree(4)Mean CountValidN%CountValidN%CountValidN%CountValidN% Thesessionswerewellstructuredand coherent4063%2438%00%00%1.38 Theteachingwasclearandinformative4367%2133%00%00%1.33 Thesubjectcontentandrangeoftopics metmyexpectations2742%3148%69%00%1.67 Thetopicsweredealtwithatthe appropriatelevelandinsufficientdepth2844%3352%35%00%1.61 Thetraineractivelyinvolvedmeinthe learningprocess5281%1219%00%00%1.19 Iwassatisfiedwiththepaceoflearning3859%2438%23%00%1.44 Newand/orcomplexideaswere explainedclearly4164%2336%00%00%1.36 Ifeelabletoexpressmyprofessional reasoning/judgmentintheassignment2553%2145%12%00%1.49 Thecoursehasmademefeelmore confidentinthesubject3047%3250%23%00%1.56 STEVEN KEEN, KEITH BROWN, JANE HOLROYD, AND EMILY ROSENORN-LANNG 52 linking the learning more specifically to our roles. Indeed, managers (n=28) viewed workload pressures and their lack of ‘time’ at work as the foremost constraint to the application of IPOP related learning: working in a chaotic environment means that you sometimes cannot stop and reflect on inappropriate actions. it’s not having the time at work to focus on changes and how to improve the team and organisation’s performance goals. A smaller number of managers would have liked more opportunities to explore the expectations of the assignment (n=6) and ‘management and leadership strategies’ outside of NLP (n=5). Not surprisingly, a similar number of managers believed there should have been less emphasis on theories (n=7) and more of a balance between teaching content, tasks, the assignment and re-capping (n=6). For example: On this course there should have been less… theory and more opportunity to consolidate learning intensity – I feel it should have been delivered over 6, not 4 days’ The IPOP team have already responded to many of these comments in the form of updates to the handbook, teaching content and employer briefing documents about the expectations and time commitment required. Nevertheless, when invited in the post-course questionnaire to respond to an open question about what they had gained most from IPOP, responses could be grouped into two main themes – increases in self-awareness and confidence (n=48) and ‘how to use’ certain techniques taught on the programme in practice (n=21). When asked to provide one example of how their practice might alter as a result of IPOP, again managers across all cohorts highlighted two key expected changes: • The use of techniques, taught on the programme, in everyday practice (n=45) e.g. ‘I will think more carefully about how I communicate with different staff and organisations to ensure I get what is needed’. • Increases in self-awareness and confidence (n=32) e.g. ‘I will have more confidence in my own practice; being able to challenge people’s statements’. These key themes of increased self-awareness, confidence and communicative ability are confirmed by an ‘all cohort’ analysis of responses to 19 ‘pre and post’ statements and questions (see Table 3). At the end of the four days of IPOP teaching, THE IPOP (IMPROVING PERSONAL AND ORGANISATIONAL PERFORMANCE) PROGRAMME 53 managers indicated highly statistically significant (Wilcoxon signed ranks test; p<0.01) increases: • in levels of general self-awareness and confidence in their work role • in their perceived ability to communicate non-verbally, lead change through people and create a strong learning climate • in feeling less overwhelmed in dealing with conflict at work. When compared with the beginning of the IPOP programme, these changes represent relative impact shifts of between 7-12% over a ~6 week period. Other statistically significant (p<0.05) changes include increases in the perceived ability of managers to communicate orally and in improving future performance to make a difference to individual service users. At this juncture, it is important to highlight that these ‘pre-post’ course questionnaire findings are based on managers’ perceived abilityandintentions.Thefollowing‘Furthercourseimpact’sectioncontainsextended evidencegleanedfromananalysisofsubmittedassignmentsandtelephoneinterviews completed approximately three months after IPOP teaching, of how the practice of managers’ had actually changed. Further course impact The telephone interviews commenced with an open question to IPOP managers about the programme in general, before focusing on more detailed descriptions and explanations of IPOP related personal and organisational impact. From these responses a number of key themes emerged: • IPOP has been delivered by ‘enthusiastic’, ‘knowledgeable’ and ‘inspiring’ tutors – positively described as ‘interesting’ and ‘enjoyable’ • Itispositivelycontrastedwithotherpublicsectormanagementcourses,supported by a wide consensus of IPOP’s ‘engaging’, ‘practical’, ‘fresh’, and ‘innovative’ nature • There is general recognition that the learning from the course is embedded in the assignment – even if managers are anxious, shocked or surprised at, or would have liked more detailed information on, having to complete it • IPOP had enabled sustained improvements in managers’ self-awareness, confidence and communicative ability. Managers talked openly about concrete examples of IPOP-related impact. These examples are numerous and varied and typically contain evidence related to the key themes from the questionnaire findings, that is the impact of IPOP on managers’ selfawareness,self-confidenceandcommunicativeability.These‘personal’improvements STEVEN KEEN, KEITH BROWN, JANE HOLROYD, AND EMILY ROSENORN-LANNG 54 Table 3 Pre and Post Questionnaire Analysis (All Cohorts) Scale 1 = Lowest , 10 = Highest Asymp Sig. Mean Relative N Z (2-tailed) Difference. Change In general, how self-aware do you feel? 74 -4.002 .00** 7% 9% Please rate your current level of job satisfaction 75 -.629 .529 -1% -2% How would you rate your current level of stress at work? 75 -.271 .787 0% -1% Please rate your level of confidence in your current work role 74 -3.352 .01** 5% 7% Oral Communication 75 -1.987 .047* 4% 5% Written Communication 75 -1.818 .069 3% 4% Non-verbal communication 75 -2.825 .00** 5% 7% Leading change through people 75 -4.212 .00** 7% 11% Creating a strong learning climate 75 -4.056 .00** 8% 12% Scale 1=Strongly Agree, 4=Strongly Disagree Asymp Sig. Mean N Z (2-tailed) Difference. I feel I am able to protect my well-being at work 64 -.153 .88 0% I sometimes feel overwhelmed in dealing with conflict at work 64 -2.560 .01** 3% I feel I have the confidence to succeed in any problems at work64 -.688 .49 0% I feel there is strong professional leadership within my team 64 -.182 .86 0% I feel able to influence the professional leadership agenda 64 -.105 .92 0% I feel I am able to hold people to account 64 -1.512 .13 -1% I feel I work hard with a common sense of purpose 64 -1.213 .23 -1% I feel I contribute well to creating trust between staff 64 -1.091 .28 -1% I feel I work well in partnership with other organisations 64 -.894 .37 -1% I feel able to improve future performance to make a difference to individual service users 64 -1.964 .05* -1% N = numbers of managers. Z = Wilcoxon signed ranks test statistic. Asymp. Sig (2-tailed) = statistical significance (2-tailed). * = probability of less than 0.05 or p<0.05. ** = probability of less than 0.01 or p<0.01 Mean difference = the difference between the pre- and post- scores aggregated over all cohorts Relative change = the difference between pre- and post- course ratings relative to the starting point THE IPOP (IMPROVING PERSONAL AND ORGANISATIONAL PERFORMANCE) PROGRAMME 55 often acted as catalysts toward organisational impact, including impact on users of services. The most effective way of displaying this type of example of impact is in the form of vignettes, using the actual words of the managers themselves. Three vignettes have been chosen from the telephone interviews to illustrate the actual and concrete nature of these changes and to highlight the important association between increases in self-awareness, communicative ability and self-confidence – and the impact of these changes on the managers’ practice and their organisation. Agency 1. Vignette Hannah (pseudonym) believed that her previous (pre-course) relationship with her manager had been having a negative impact: it left me feeling, and a number of my colleagues, that she [the manager] didn’t appreciate the impact she was having on us with regard to disempowering us, when we were able to make our own decision. I regarded myself as very able and very self-aware. Yet, I felt I was at a point where I’d really been disempowered and couldn’t make the smallest decisions, and so I wanted to deal with that relationship and also the relationship with the group of staff that I was managing. Hannah expressed a desire to become more assertive in the relationship with her manager. The course had played a key part in realising this objective. it was the behaviour that was really important to me, that learned behaviour – how to use the language – what you pick up on – how NLP works…that was the important part for me…it enabled me to enter into dialogue with my line manager…she had, for example, wanted me to be more assertive…rather than saying to her ‘is this alright?’…I then made a concerted effort to do things differently…it has shifted… Raising this personal issue with her line manager led to Hannah being able to work in a more appropriate way with her colleagues. Agency 2. Vignette Ella (pseudonym) believed that the course had given her increased confidence to deal with the difficult aspects of her role. The case study below illustrates how she has been able to harness this increased confidence to deliver significant organisational impact: At a meeting I had with other service managers in my organisation ... one of my aims was to eliminate the use of agency staff within the organisation, and to do that I needed to liaise with other service managers and talk about sharing staff etc...[as using agency staff is] financially not really viable long term and also for continuity of care... [it’s] more beneficial for the people we support if they have a regular staff team in place....the teams are very small so we’re quite STEVEN KEEN, KEITH BROWN, JANE HOLROYD, AND EMILY ROSENORN-LANNG 56 vulnerable to needing extra help...so we’re trying to work with other managers to utilise other staff as well as building the teams to cover each other...I did plan a little in advance of a meeting with one particular service manager...I noticed she was mentioning about the fact that staff had not previously enjoyed working across different departments because of the different job roles, and I noticed how she kept on touching the back of her neck, and obviously I picked up it was something she wasn’t comfortable with talking about...I definitely wouldn’t have even noticed that before the course...I was able to use the advanced questioning model to get the rest of the information. It had been a couple of members of staff that had found it difficult, for x number of reasons, and we were able to work around that, so rather than it being a ‘no go’, I was able to go a bit deeper...we were able to come up with a plan that avoided any issues that staff had found difficult before... We were able to work around it and say if you can be flexible for us, we can be flexible for you...there has been a really good result...in all but one of our services we’ve not had agency for several number of weeks...Neuro-linguistic Programming has been useful in that...it has helped to put things in place and action them. Agency 3. Vignette Rosie (pseudonym) reported that when she moved into the management role she did not receive any formal training and had relied on her own life experience to help her to manage conflict and to supervise staff. She has used specific techniques learnt on the programme e.g. matching words and discovered that these techniques had given her ‘almost instantaneous results’. Rosie recalled recent events where she had used these techniques to deal with conflict within the team: There were some very conflict filled situations, which I would have floundered in a year or six months ago, but using these techniques actually surprised me about how useful they were, and they allowed me to understand people better and just how we all think about mapping and where we are. Rosie’s team is currently undergoing their third restructure in the last three years – this had led to major changes in management structures throughout the department. As a result there are some individuals who are unhappy about the changes, and having joined a new team in the current climate, this has proved challenging (initially). Without IPOP Rosie does not feel she would have been able to have resolved the situation quite so quickly or effectively. I don’t think I could have managed it [the restructure] as well without this course behind me, as I have been able to use all of the techniques or most of them to stay measured, to feel that I could cope with the situation, I used all of the things I have been taught. I have been trying to implement them. It’s not always easy when some flash point happens that you weren’t expecting, but I’ve managed to pull it together and bring the teams around really quite nicely now. Without the training I am not sure that I would have been able to resolve the situation so quickly or effectively. THE IPOP (IMPROVING PERSONAL AND ORGANISATIONAL PERFORMANCE) PROGRAMME 57 Initial review of submitted assignments The final aspect of the three-stage evaluation was an analysis of 32 submitted assignments. Managers developed an action plan as part of the assignment focusing on two objectives (personal and organisational). The two most popular objectives centre on communication: • Improving supervision within the organisation (n=16 objectives) • Improving personal communication (n=15 objectives) Other objectives focused on improving managers’ confidence as leaders (n=6 objectives); improving self-awareness (n=4 objectives); more effective line management of others (n=2 objectives); and improving the effectiveness of meetings (n=2 objectives). Single-themed objectives ranged from ‘improving work-life balance’ to ‘working better with partner organisations’ and ‘service redesign’ work. Examples of comments from IPOP managers during their assignments included the following statements: Team members have seen a change in my behaviour; they state I am more responsive, focused and approachable. An increased awareness of myself, my impact on others and how I can positively influence them have all come about through my learning with this work. CompletingtheassignmenthasdemonstratedthatIhavedevelopedandimprovedmypersonal and professional organisational performance by developing different strategies to improve my communication techniques and managing relationships more successfully with colleagues. These are complimented by further comments from the third party testimony in which line managers confirm the practical application of the objectives within the action plan, again confirming the development of self-awareness, confidence and communicative ability: in my opinion (the individual’s) competence has been extended and improved through the process of this assignment and their abilities have shone through. there is significant post-qualifying development. The student has confronted issues they have always found difficult. I feel that (the individual’s) new learned approach to communication with their staff will continue to reap benefits for the department and herself. I would like to extend this learning to other managers. STEVEN KEEN, KEITH BROWN, JANE HOLROYD, AND EMILY ROSENORN-LANNG 58 Discussion and implications The majority of studies evaluating social work education have not attempted to measurechangesinbehaviour(Carpenter,2011).Thisisthefirstknownstudyaiming to evaluate the impact of an introductory post-qualifying social work management development programme by seeking out evidence of self-reported changes in managerial behaviour and organisational practice. IPOP evaluated positively; highly significant increases in perceived levels of general self-awareness (see McAllan & MacRae,2010),confidenceatworkandinmanagers’perceivedabilitytocommunicate non-verbally, lead change through people and create strong learning climates are apparent at the end of the taught component. These changes represent an impressive relative impact shift of between 7-12% over a short ~6 week period and after just 4 days of teaching. These perceived changes have been confirmed by actual examples of IPOP related impact from both telephone interviews and assignments at three months; that is, that IPOP had enabled self-reported improvements in managers’ selfawareness, confidence and communicative ability – and that these changes had led to sustained examples of organisational impact (also see McAllan & MacRae, 2010). To increase our confidence that these differences are attributable to IPOP we could have designed this pre-test, post-test evaluation with a no-intervention or otherintervention control group and random allocation – both impractical options, given that employers were paying for their managers to complete IPOP. This is probably why there has been ‘no examples of randomised controlled trials of educational interventions in social work’ (Carpenter, 2011, p.133). The mixed method approach of using follow up telephone interviews and an analysis of assignments to gain actual examples of personal and organisational impact is a far more appropriate technique for aetiological purposes (for example, see McAllan & MacRae, 2010). However, the main limitations of this evaluation relates to the sample and the self-report nature of the data. While IPOP managers (Table 1) come from both children’s and adult services in five widespread local authorities and three independent, voluntary and private sector organisations, about half the managers and four (out of seven) IPOP cohorts hail from onelocalauthority.Sincethisevaluation,IPOPhascontinuedtobeaccessedbyfurther local authorities (n=5), independent, voluntary and private sector organisations (n=6) and managers (n=69) with similar results. In other words, over time and as the response rate to the evaluation moves above 77%, it should be possible to demonstrate increased confidence in the wider impact of IPOP. It has also not been possible, because of the sensitive nature of the questionnaires, to link programme outcomes with the experience or position of IPOP managers. Although the data in this study is self-reported, line managers do provide evidence and confirmation of the practical application and impact of the objectives within the action plan in their ‘third party’ testimony. Whilst the characteristics of this sample are broadly similar to all candidates THE IPOP (IMPROVING PERSONAL AND ORGANISATIONAL PERFORMANCE) PROGRAMME 59 completing other leadership and management programmes at BU when comparisons are made with national data (GSCC, 2011), it would appear that male managers (31% vs 20%), white managers (97% vs 81%), those managers with at least a degree level qualification (60% vs 40%) and managers with additional learning needs (12% vs 7%) are over-represented in the IPOP sample. The implications for the PCF and future management development of social work managers in their 40s and 50s who for the most part are likely to have initially qualified at diploma, not degree, level and have additional, most likely to be dyslexia related, learning needs, are considerable. In part, this is why nearly 60% of the variance in overall course evaluation ratings may be predicted by managers’ expectations of the subject content and range of topics. Put another way, programme content needs to be relevant to the work of social work managers. The concept of self-leadership is, therefore, highly likely to be critical to the future development of individuals with management responsibility in social work (Skills for Care, 2008; Hafford-Letchfield & Bourn, 2011; Holroyd & Brown, 2011; Leadership Academy, 2011). The skill and experience of any tutors, and the assessment process, should also not be underestimated. In this case, the development of an action plan focused on the critical reflection and analysis of two (personal and organisational) objectives. The fact that the employer confirms this evidence in their ‘third party’ testimony gives credence to the objectives themselves but also verifies to the employer the usefulness of the programme (McAllan & MacRae, 2010). Again, without this perceived relevance or usefulness, the purchase of future programmes is likely to reduce. Perhaps this is why the investment in, and take-up of, post-qualifying social work management courses has been so low (see GSCC, 2011) given the rhetoric (for example, Ofsted, 2012); that is, they are just not relevant to the practice of social work managers? Leinster’s study of 56 social work managers in Ireland found that none had received social work management development specific to their roles. Therefore, much social work leadership and management development appears to be of an inhouse nature, that is, with no assessment (e.g. McAllan & MacRae, 2010) or generic, for example an MBA (Master’s Degree in Business Administration). Purchasers are none the wiser as to the impact of these programmes on changes in behaviour and organisational practice or indeed whether there is any benefit to the well-being and quality of life of the people they serve. Though in the leadership and management field, this is not unusual (see Armstrong & Fukami, 2009; Ashworth et al., 2010). General evidence about the impact and effectiveness of leadership and management programmes and their content is weak (Hafford-Letchfield & Bourn, 2011; HaffordLetchfield & Lawler, 2010). Wealsobelievesocialworkmanagersarecryingoutforleadershipandmanagement development(McAllan&MacRae,2010)thatisrelevanttothemselves,theiremployer and the user (Leinster, 2010) – development that is assessed in a meaningful way and evaluated – development that is appropriate and practical for those with additional learning needs – development that changes their practice and, therefore, STEVEN KEEN, KEITH BROWN, JANE HOLROYD, AND EMILY ROSENORN-LANNG 60 the practice of others for the common good. Investment in this type of leadership and management development for the remaining 11,000 plus social work managers to fulfil the intentions of the PCF, in particular the ‘professional leadership’ and ‘context and organisations’ domains (TCSW, 2012) is crucial for the development of the profession and the people they serve: These are ways of thinking and this is language I wouldn’t have used…before I started the course. Considering I’ve been in the work for 25 years plus and I’m in my 50s and we’re only three months in, well I think that’s a considerable impact on the way I’m thinking and the way I’m talking. Acknowledgements Thank you to the 75 social work managers for taking part in this evaluation. KB and JH conceived the study. KB, JH and SK designed the study. SK drafted the paper; KB, JH and ER-L have subsequently revised it for intellectual content. JH and Vicky Ross delivered the IPOP programme. ER-L designed the tables and co-ordinated the majority of the data collection and statistical data analysis, supported by SK, Lucy Morrison, Angela Warren and Andrew Harding. SK and Andrew Harding analysed the interview data and JH completed the assignment analyses. SK, KB, ER-L and JH have interpreted the data. References Armstrong, S. and Fukami, C. (2009) Past, present and future perspectives of management learning, education and development. in S. Armstrong and C. Fukami (Eds.) The Sage Handbook of Management Learning, Education and Development. London: Sage, (pp.1-22) Ashworth, R., Boyne, G. and Entwistle, T. (2010) Reflections on theories of public service improvement. in R. Ashworth, G. Boyne, and T. Entwistle Public Service Improvement. Theories and evidence. Oxford: OUP (pp.205-217) Bailey, D. (2002) ‘Training together – part two: An exploration of the evaluation of a shared learning programme on dual diagnosis for specialist dug workers and Approved Social Workers. Social Work Education, 21, 6, 685-699 Bailey, D., Carpenter, J., Dickinson, C., and Rogers, H. (2003) Post Qualifying Mental Health Training. London: National Institute of Mental Health Brown, K. and Keen, S. (2004) Post-qualifying awards in social work (Part 1): necessary evil or panacea. Social Work Education 23, 1, 77-92 Brown, K., McCloskey, C., Galpin, D., Keen, S., and Immins, T. (2008) Evaluating the impact THE IPOP (IMPROVING PERSONAL AND ORGANISATIONAL PERFORMANCE) PROGRAMME 61 of post-qualifying social work education. Social Work Education, 27, 8, 853-867 Chapman, J. and Morris, M. (2011) Post-qualifying students’ experiences of implementing skills acquired from a ‘hearing voices’ module into routine clinical practice. Journal of Mental Health Training, Education and Practice, 6, 2, 85-92 Carpenter, J. (2011) Evaluating social work education: A review of outcomes, measures, research designs and practicalities. Social Work Education, 30, 2, 122-140 DCSF (Department for Children, Schools and Families) (2009) Building a Safe, Confident Future: The final report of the Social Work Task Force. London: DCSF Doel, M., Nelson, P., and Flynn, E. (2008) Experiences of post-qualifying study in social work. Social Work Education, 27, 5, 549-571 GSCC (General Social Care Council) (2004) Revised Post-Qualifying Framework for Social Work Education and Training. London: GSCC GSCC (2007) Leadership and Management Specialist Standards. London: GSCC GSCC (2011) A Report on Social Work Education in England: Data pack. Rugby: GSCC Hafford-Letchfield, T. and Lawler, J. (2010) Guest editorial: Reshaping leadership and management: The emperor’s new clothes? Social Work and Social Sciences Review, 14, 1, 5-8 Hafford-Letchfield, T. and Bourn, D. (2011) ‘How am I doing?’: Advancing management skills through the use of a multi-source feedback tool to enhance work-based learning on a post-qualifying post-graduate leadership and management programme. Social Work Education, 30, 5, 497-511 Holroyd, J. (2012) Improving Personal and Organisational Performance in Social Work. London: Sage Holroyd, J. and Brown, K. (2011) Leadership and Management Development for Social Work and Social Care: Creating leadership pathways of progression. Birmingham: Learn to Care Jones, R. (2010) Managing at a distance in social work and social care. Social Work and Social Sciences Review, 14, 1, 59-75 Jones, J., and Attfield, R. (2007) Flying High: Some leadership lessons from the Fast Track Teaching Programme. Reading: CfBT Education Trust Kelly L. and Jackson, S. (2011) Fit for purpose? Post-qualifying social work education in child protection in Scotland. Social Work Education, 30, 5, 480-496 Laming, H. (2009) The Protection of Children in England: Action plan. London: TSO Leadership Academy (2011) Leadership Framework. London: Department of Health Leinster, J. (2010) Social work management in Ireland: time for education and training. Social Work and Social Sciences Review, 14,2, 73-94 Manz, C. and Neck, C. (2009). Mastering Self-Leadership: Empowering yourself for personal excellence. Englewood Cliffs, NJ: Prentice Hall McAllan, W. and MacRae, R (2010) Learning to lead: evaluation of a leadership development programme in a local authority social work service. Social Work and Social Sciences Review, 14, 2, 55-72 Munro E (2011) The Munro Review of Child Protection. Final report. A child centred system. London: TSO Neill, M. (2009). Supercoach – 10 secrets to transform anyone’s life. London: Hay House STEVEN KEEN, KEITH BROWN, JANE HOLROYD, AND EMILY ROSENORN-LANNG 62 Ofsted (The Office for Standards in Education, Children’s Services and Skills) (2012) Framework for the inspection of local authority arrangements for the protection of children. Available from www.ofsted.gov.uk/resources/framework-for-inspection-of-localauthority-arrangements-for-protection-of-children [Accessed 11 June 2012] Platt, D. (2011) Assessments of children and families: Learning and teaching the skills of analysis. Social Work, 30, 2, 157-169 Politis, J. (2005). Dispersed leadership predictor of the work environment for creativity and productivity. European Journal of Innovation Management, 8, 2, 182-204 Ritchie, J. and Lewis J (2003) Qualitative Research Practice. London: Sage Skills for Care (2008) Mapping of Leadership and Management Standards for Health and Social Care: Leadership and management product 3. (2nd ed.) Leeds: Skills for Care Taylor, B.J., Mullineux, J.C., and Fleming, G. (2010) Partnership, service needs and assessing competenceinpost-qualifyingeducationandtraining.SocialWorkEducation,29,5,475-489 TCSW (The College of Social Work) (2012) Professional Capabilities Framework for Social Workers. Available from www.collegeofsocialwork.org/pcf.aspx [Accessed 6 June 2012] TIC(TheInformationCentre)(2012)PersonalSocialServices:StaffofSocialServicesDepartments, England, as at September 2011. London: The Health and Social Care Information Centre Walker, L. (2004) Changing with NLP: A casebook of neuro-linguistic programming in medical practice. Oxford: Radcliffe Medical Press Watson, J. (2008) ‘The times they are a changing’: Post-qualifying training needs of social work managers. Social Work Education, 27, 3, 318-333 Webber, M., Currin, L., Groves, N., Hay, D., and Fernando, N. (2010) Social workers can e-learn: evaluation of a pilot post-qualifying e-learning course in research methods and critical appraisal skills for social workers. Social Work Education, 29, 1, 48-66 Webber, M. and Salter, L.C. (2011) Gearing practitioners up for research: evaluation of a pilot online research training course for social workers. Research, Policy and Planning, 28, 3, 185-197 Yelloly, M. (1995) Professional competence and higher education. in M. Yelloly and M. Henkel (Eds.) Learning and Teaching in Social Work. Towards Reflective Practice. London: Jessica Kingsley.