A WHOLE FAMILY APPROACH TO DESIGNING AN EVALUATION FRAMEWORK 63 Social Work & Social Sciences Review 17(2) pp.63-92. DOI: 10.1921/990317030205 How far does a whole family approach make a difference: Designing an evaluation framework to enable partners to assess and measure progress? Nigel Malin1 , Jane Tunmore2 , and Angela Wilcock3 Abstract: This article reports findings from a study to support the inclusion of a Whole Family Approach(WFA)withinpolicyandprovisionforchildrenandfamiliesconductedinonelargenorthern local authority in England. In recent years mainly from research and good practice elsewhere WFA had been seen to offer opportunities to focus on shared needs, developing strengths and assessing risk factors. The principal aim was to design an evaluation framework to enable partners to assess and measure progress in the delivery of a WF strategy. Methods included individual interviews with professionals and managers (N=22) on knowledge and experience of WFA, for example their understandingofmulti-agencywork,alongwiththeirevidenceofadoptingaWFAapproach.Findings demonstrate the process of how an evaluation framework was constructed based on adapting preexisting outcome-focused ‘models’ used to evaluate functions of partnership-working along with indigenous sources. The first type of ‘model’ entailed a number of dimensions including Vision and Strategy, Partnership Dynamics, Impact and Performance Measurement. The second had two key features: it drew upon the idea of realist evaluation, a paradigm used by practitioner researchers where the focus is upon identifying mechanisms that explain how an action affects outcomes in particularcontexts;andthe‘model’hadbeenappliedextensivelytoananalysisoffamilyintervention projects. What has emerged is an evaluation framework characterised by a number of key ‘signifiers’ each of which is populated by a series of questions. The framework embraces introducing changes to the culture of planning and delivering services, placing building family strengths at centre-stage. Keywords: outcome evaluation; whole family; family intervention; multi-disciplinary working; service delivery 1. Professor of Social Policy 2. Principal Lecturer in Social Work 3. Research Associate Address for correspondence: Faculty of Education and Society, Chester Rd Campus, University of Sunderland, Sunderland SR1 3PZ. nigel.malin@sunderland.ac.uk Date of first (online) publication: Acknowledgements: We would like to thank the Institute of Local Governance (ILG), based at the University of Durham, for funding the project. NIGEL MALIN, JANE TUNMORE, AND ANGELA WILCOCK 64 Introduction This study based within a single northern local authority had the aim of producing an Evaluation Framework for a Whole Family Strategy, and included (i) carrying out astock-takeofcurrentworkingpractices,inparticularpartnershiparrangementsand relationships;and(ii)accessingandadvisingongoodpracticefromcomparativework and relevant research conducted elsewhere. Why create an evaluation framework? This had two expressed goals, firstly to test the validity of the terminology and operational definition; secondly to provide a means, for managers and professionals, to assess the effects and effectiveness of using a WFA defined as an innovation, an intervention, policy, practice or service. Of major concern to all stakeholders was a need to establish a method for the identification and measurement of both impact and outcomes of different interventions leading to a positive organisational culture. The Think Family Literature Review of Whole Family Approaches (WFA) published in 2008 (Cabinet Office: Social Exclusion Task Force) and Think Family Pathfinders (Kendall et al, 2010) showed positive changes in families at risk who received a whole family package of support, also in how professionals delivered support resulting in reduced levels of need. Resources which strengthen families within communities can be explored using different theoretical frameworks, including social capital (Putnam, 2000; Sennett,2003) and family resilience (Kalil,2003; Mackay, 2003; Quinton, 2004). This Think Family literature has illustrated a momentum towards WFA within policy and provision in relation to a number of service user groups; and concluded that such interventions were still in their infancy and required further evaluation focused on professional and agency competency. Using a WFA has been seen to offer opportunities to focus on shared needs, develop strengths and address risk factors that could not be dealt with through a focus on family members as individuals. A key theme in applying this model left to the discretion of each Local Authority appears to be the adaptation of existing systems such as Family Intervention Projects (FIPs) and the Common Assessment Framework (CAF). The 2010 Spending Review set out more details of the Coalition’s plans to establish local Community Budgets to fund family interventions for families with multiple problems (HM Treasury, 2010). This approach was characterised by new forms of financing multi-agency interventions. Families with multiple problems are a diverse group of families defined, by the Coalition, as families suffering from five or more of the following problems: no adult in paid work; family living in poor quality or over-crowdedhousing;noadultwithanyqualifications;motherhavingmentalhealth problems; one adult or child with a limiting illness; disability or infirmity; relative household poverty; family not able to afford a number of food and clothing items; or at least one child in the family involved in the youth justice system or having a clinical behavioural problem. In September 2011, the DfE estimated that there were 117,000 such families in England. These families, it was argued, are often involved with a number of adult and children’s services. A WHOLE FAMILY APPROACH TO DESIGNING AN EVALUATION FRAMEWORK 65 The previous Labour Government developed family intervention projects and services for these families - multi-agency responses coordinated by specialist lead professionals who worked intensively and assertively with families and addressed multiple needs and problems in the family. Intensive family intervention projects (FIPs) became a flagship mechanism within New Labour’s anti-social behaviour and social policy programmes and the number of these projects increased substantively. A national network of fifty-three FIPs in England during 2006-7 was established as part of the Respect Action Plan (Respect Task Force,2010); and the Youth Task Force Action Plan established fifty-two Challenge and Support Projects and twenty Intensive Intervention Projects (DCSF,2008). There is now a substantial evaluative literature on intensive familyintervention projects (Dillane et al, 2001; Jones et al, 2006; Nixon et al, 2006,2008; White et al, 2008; Pawson et al, 2009; Dixon et al, 2010; Renshaw and Wellings, 2010; Flint et al, 2011). These evaluations have included quantitative and qualitative methodologies and economic cost-benefit analysis. Thisresearchevidencesuggeststhattheseprojectsachievesignificantandpositive outcomes and deliver considerable public cost savings. However, both intensive family intervention projects and evaluations of them have been subjected to critique (Garrett,2007; Gregg, 2010) on the basis that the research techniques deployed and the presentation of results has been flawed or limited and have exaggerated the positive and progressive outcomes associated with the projects. Despite considerable controversyovertheputativeeffectivenessofsomeoftheseprojects,theUKCoalition Government has continued to support investments in multi-agencyintensivefamily services (Home Office 2010; Ministry of Justice, 2011; Home Office, 2012). The Coalition set up Community Budget pilots in 16 areas covering 28 Local Authorities in England as a new approach to financing these type of services. Community Budgets provide Local Authorities with considerable autonomy to decide on the model and type of approach to take, and to fund different types of help and support for families rolling out this approach nationwide from 2013/14. In taking forward proposals for the ‘social investment’ market, the Coalition has looked into ways in which local businesses can contribute to Community Budgets and help fund projects for ‘troubled families’ via Social Impact Bonds, whereby businesses would secure an additional return on their ‘investment’ when projects meet their targets and objectives. A more integrated and focused approach to supporting families with multiple and complex needs allegedly has been a key Coaltion priority, leading in 2012 to the creation of a Troubled Families Unit at the Department for Communities and Local Government (DCLG). The first stage of the Programme (2012-014) has been the delivery element of this work aimed nationally to ‘turn around’ (i.e. this means getting adults into work, children into school and reducing crime and anti-social behaviour) the 120,000 ‘troubled families’ identified by the Unit. The intention has been to reduce costs and improve outcomes through effective targeted intervention, supported by a results-based funding scheme. ‘Troubled families’ were defined by the DCLG as households who: have members who are involved in crime and NIGEL MALIN, JANE TUNMORE, AND ANGELA WILCOCK 66 antisocial behaviour, have children who are not in school, have an adult who is on out-of-work benefits; and cause high costs to the public purse. In the Local Authority which provides the focus of this study there was an additional agreement to consider families who are dealing with domestic violence, substance misuse (both adults and young people) and child protection issues. The Troubled Families Programme runs on a payment-by-results basis, with £4,000 available from Government for each family that is ‘turned around’ (i.e. they achieve the stated outcomes defined by the DCLG). In the said Local Authority the requirement was to achieve successful outcomes with at least 805 families - an indicative number of troubled families in the area; and the DCLG expected each participant Local Authority to identify and work with at least one third of eligible families in the first year (2012/13), 50% in year 2 and the final 15% in year 3. What is a whole family approach (WFA)?: A stock-take of current working practices A snapshot of what is in place based on oral evidence: The aim of the WFA is to improve outcomes for vulnerable children, young people and adults through better coordination of the support that they receive from existing services. The WFA, at its most general level, means that when practitioners are working with families, staff whose primary focus is the adult(s) will give due regard to the needs of the child(ren), and any impact on them from adult behaviours or difficulties, and the practitioner whose primary focus is the child(ren) will consider the needs of any vulnerable adult in the family..... .This approach should be applied across universal, targeted and specialist services for adults and children ... (and) build on preventative work through the Common Assessment Framework (CAF), and Team around the Family (TAF). The involvement of children’s centres, locality-based working and other preventative services, is essential to the success of adopting this approach. (Draft City Council Children’s Services Guidance, March 2013) One of the aims of this data-gathering was to provide an evidence-based account of current practices including both management, professional and other staff perspectives and ideas to form a baseline for developing the WFA strategy and evaluation framework. This was achieved using both focus groups and individual interviews with a variety of staff across the Local Authority. Undertaken towards the end of 2012 this focused on gathering their knowledge and experience of WFA, along with some of their aspirations for the kind of changes which need to be implemented around the culture of working practices. A WHOLE FAMILY APPROACH TO DESIGNING AN EVALUATION FRAMEWORK 67 Main questions posed to staff • Are you able to identify specific examples of where WFA is working effectively at present ? • What is your role in WFA? • How far is multi-agency work embedded in services, and how has a WFA benefited from its predecessor Think Family(TF)? • Will the approach link adult and children’s services which largely work separately? • What would you regard as useful evidence to demonstrate that services have adopted successfully a WFA? • What are the key factors characterising working practices? • In implementing WFA which particular aspects need to be most protected given the current need for making efficiency savings? • What do you think are the main barriers to introducing /sustaining a WF Strategy? • What may be an expected outcome of using a WFA? Summarising the responses Most responses contained an expressed attitude towards making the case for a more generic type of community support worker using ‘assertive outreach’ skills and with proven ability to engage effectively with situations of increasing complexity and challenge, for example those with multi-agency input. There’s a need to retain /enhance specialist skills not see WF as a move towards generalist practitioner. The focus should be on a broader model of strengthening families which is very inclusive, not just ‘high end’ but ‘much wider e.g. Sure Start. Families aren’t traditional anymore, so we need to consider what we mean by family, and consider all individuals e.g. grandparents. We have not addressed challenges for young adults whilst they were children and now they are the next set of parents. The focus should be on how to engage families. There’s a need for a community worker with an ear to the ground to link with more agencies e.g. Gentoo, Salvation Army; as we’re trying to understand family circumstances and I don’t think we’re successful in that. Person x has a presenting problem, so we now must ask ‘what is the impact on the wider family?’. We cannot anymore just respond to individual issues without looking at what is going on more generally. NIGEL MALIN, JANE TUNMORE, AND ANGELA WILCOCK 68 What about our current resources e.g. Common Assessment Framework (CAF)? There is the CAF but people don’t choose to complete or complete it inadequately; then think that if they complete you can hand over and park the problem, not take responsibility. Or if you complete it you get stuck with the whole thing, or at least if you offer to take the lead role. Staff are reluctant hence if WF then time needs to be spent on how to remove barriers. The CAF is much criticised; there are fewer requests for statementing. Those who undertake CAF need to think WF, not just children, now it’s about the family team. Based on the CAF we’dmeasureprogressevery4weekse.g.safeguarding(towardsoutcomes)butit’stheprocesses driving progress nothing more. CAF has produced good, new partnerships, real buy-in to the overall assessment process, e.g. good training by Gentoo. It is about the fact that there is a process to follow that you all adhere to and you all own that process - you might not have a good working relationship with those people, and think well that is children’s services and it’s a pain I don’t properly understand etc, if you feel you don’t add value to that then you are not going to adopt it’. For example: ‘Woman arrested for shoplifting/routinely tested for drugs. She had 2 children with a disability – it was all becoming too much for her. It was the Turning Point worker who shared the information with us, so we were able to provide some support around the WF - it was an adult issue but it was the medical condition of the children that was part of the problem. Gentoo deal with people when there is a household issue, e.g. when family has risk of eviction, also household adaptations. What has been learned from other pilot initiatives? The lead person working on a child/adult poverty project brought agencies together through the school but only 10 or so 20 families. All the money spent on poverty strategies, all the talk-shops etc., and still reluctance to give families cash. If we spend 2k on x families we can make a saving of y, return on investment equals a strong outcomes model suggesting financial benefits of early intervention. WFA should be an extension of the work of Children’s Centres. We need to move further into a position of integrated working, breaking down all those barriers in terms of structures, and say ‘what is best for the family?’ in this situation. Need in future less complex and specialist services with focus on skills and knowledge within the family - resilience, resources, social capital. The criteria we choose to use for assessment will shape eligibility. Is there anyone else in the family who can offer support? Would this have an impact on a parent with a mental health issue? How are other family members coping? All the agencies take a problem-solving multi-agency approach, however they do not discuss domesticviolenceas(ExtraordinaryLocalMulti-AgencyPracticeTeam)theprocessdealsonly withhigh-riskvictims,andaconcernisneglectofmedium/standardriskvictims’.Forexample: A WHOLE FAMILY APPROACH TO DESIGNING AN EVALUATION FRAMEWORK 69 ‘We might have Mr Smith and he /his children are suffering Anti-Social Behaviour. So we think we need to get into more detail around that. An email would be sent to all partners- ‘what do you know about this person?’ They do a risk assessment to check that person’s vulnerability so I think that would involve children’s services/adult services and different people coming round the table. The man/father might be suffering a nervous breakdown, and say it’s down to anti-social behaviour if kids are kicking a ball against his house; but actually there may be an issue here around child care, or mental health, so it is getting the right people round the table. So that in the past it might have been the neighbourhood relations team. There’saperceptiblelackofcoordinationarounddecision-makinggroupse.g.MARAC(MultiAgency Risk Assessment Conference) which represents a set of processes to go through to supportvictim/childrenandwhatagenciescandoaroundmanagingtheperpetrator,anholistic approach needed. We need to have an inroad on GPs, so adult who goes with a substance abuse issue and GP neglects his/her children; and refers to another service but that’s not the answer. A new professional identity? Appropriate skills? It is a mindset rather than a process thing, it’s a cultural change where we need a family assessment, presenting the child but then think WF. The purpose is to give families confidence, information to resolve their own problems, the aim is to produce families that are resilient, with skills and knowledge, able to do things for themselves. Changes frequent in the way services are run, some professions are stronger and retain their own line-management e.g. educational psychology., and now we have locality teams and they are managed operationally and it is to try and break down these barriers; which is ‘well we work in this way’ . Almost impossible to break down barriers through act of will when professions only regard their professional body as the fount of wisdom and will not defer to will of majority, leadership issues left unresolved. Need to maintain their original level of expertise but now ‘skilled up’ to look at the broader issues that families might have. So now we have a family team for 0-19 years, which includes youth workers. Getting someone in to understand what the issues are, this is central, also successfully recruiting and mentoring volunteers to work with families with complex needs. 3rd sector partners are the best providers; they are engaged and commissioned to deliver a service, like ‘Turning Point’. When working with a family it is about establishing what is a good outcome for them. The more successful people - in engaging families - tend to be the lower level family befrienders rather than the professional. They have an opportunity to link with a family where there is reluctance to engage with e.g. a professional. So a lower level befriender can be the link for professionals to work behind the scenes, now we are looking to develop community befrienders not just in our own family team so for local residents you know people feel they are not from any ‘organisation. NIGEL MALIN, JANE TUNMORE, AND ANGELA WILCOCK 70 How do we achieve effective information-sharing: A professional dilemma? People say nothing rather than anything because of fear of litigation, big issue is regarding safeguarding and the legal/ethical side of the process. Do people trust one another in order to work together, how is information stored, who has access to what? (We) can’t achieve effective multi-disciplinary working because of poor information-sharing. Big problem is the PCT; also no single system in social care, the pupil-based system is only accessible to some people. Yes, you may be able to access information via someone but it’s easier if it was readily available. Data-sharing protocols represent a huge issue, also sharing information across separate local authorities in the same region A barrier is the power of a profession, e.g. medical profession, also social work who hold onto certain types of information and do not see that others, such as untrained or differently trained staff, have a right to view this information. A practical dilemma is do I offer advice or do I send the person somewhere else? This is the difficulty, it’s about meeting the needs of people whom you might not necessarily have seen as your client group, whether you decide to gather information about this family and what are the boundaries to your responsibilities? For instance we are starting to deal with worklessness by encouraging families to put their child into childcare, so they can start to look at jobs, training etc...We are working with Job-Centre Plus, the lone parent adviser manager. The Health/Social Care legislation calls for huge re-structuring and demands more joint working which has to be brought together with the WF agenda. A continuum of integrated intervention to improve outcomes: Who takes overall responsibility? Early intervention is everybody’s business as is viewing the bigger picture. From early intervention we look at it from the earliest point start in a child’s life, because that is where you make the biggest impact. Parenting is a real, real issue, so often you need to protect parenting as a resource. Staff have a lot of passion to help the young people whom they work with, and naturally wish to look at their wider circumstances. I’ve tried to discourage people from passing on a case to someone else but to take ownership, not just say ‘I only deal with children ... Someone who said ‘well I have been to see this young person and when I have looked their attendance was appalling in secondary school and they have this and that problem; and really people should have been doing something with them before now’. So I said to her ‘Did you carry out an assessment to try and put something in place?’ She just looked at me. So it is about when I find out something where reaction should not be ‘it’s not my responsibility’ A WHOLE FAMILY APPROACH TO DESIGNING AN EVALUATION FRAMEWORK 71 but I’m going to take ownership for that person. It is taking ownership and that is another barrier which needs to be addressed. As in nursing it’s untrained staff who often do not show any ‘professional responsibility’, because that is how they see themselves and it is the line-manager who should think on their behalf and advise them accordingly. Don’t prioritise presenting problems but consider the impact on the wider family. There is an urgent need to formulate outcomes .... diverse skills are needed on the part of project workers to provide ‘assertive’ approaches commensurate with improved outcomes for vulnerable children, young people and adults. Good practice from comparative work and relevant research conducted elsewhere Model A The following framework draws upon two key sources: one based on a set of ‘underlying values and principles integrated into practice and promoting a reflective approach’ (Kahan et al, 2008); the other ‘based on action planning constructed through action research’ (Atkinson, 2005, 2006). Both have been applied empirically to adopting a partnership/multi-agency working approach, and the second of these frameworks includesastructureusingDimensions,Sub-DimensionsandAssessmentCriteria(seeFigure 1). Using this framework analysis, (see for example Atkinson, (2005) a description of each dimension is provided both to define it and to give a sense of its scope. Each dimensionoftheframeworkisfurtherbrokendownintosub-dimensionsasalogicalway of analysing the dimension and is reflective of key components of how the partnership - or in this case - the strategy operates. For each sub-dimension a range of critical and focused questions and/or evidence has been identified which would form the basis for conducting the evaluation. Sullivan and Skelcher (2002:185-207) provide a useful overview of approaches to the evaluation of partnership strategies over the last two decades; and identify a number of approaches to evaluating public policy collaboration which include: • value for money evaluation which emphasises questions of economy, efficiency and, to a lesser extent, effectiveness; • outcome-focused evaluation which gives greater emphasis to the assessment of the outcomes of collaborative activity and offers greater flexibility in how these outcomes are achieved; • process-outcome evaluation which examines the process of implementing an NIGEL MALIN, JANE TUNMORE, AND ANGELA WILCOCK 72 initiative in order to understand whether and how the objectives of the initiative were met and in elaborating the circumstances in which particular interventions take effect; • stakeholder or 'interactive' evaluation which requires the consideration of a range of stakeholders’ views, since different stakeholders will have differential access and influence over the evaluation process; and evaluation of • collaborative mechanisms which focuses on the assessment of the means of collaboration i.e. the partnership itself. Figure 1 High-level schema of the evaluation framework (Atkinson et al, 2006) Thecomplexityofmulti-agencyworkingrequirestheutilisationandintegrationof Dimension Descriptionofthe“dimension of the evaluation framework Dimensions are key areas of thepartnershipthathavebeen identified as having sufficient weight to warrant evaluation Sub-dimensions Identification and description of the “sub- dimensions” of the evaluation framework. Each dimension is broken down into sub-dimensions as a logical way of analysing the dimension and reflection of key componentsof how the partnership operates. Assessment of sub-dimensions Ident i f ic at ion of key evaluation questions and/ or evidence required for each sub-dimension of the evaluation framework elementstakenfromtheaboveresultinginaframeworkcontainingsevendimensions: • Dimension 1 – Impact • Dimension 2 – Vision and Leadership • Dimension 3 - Partnership Dynamics • Dimension 4 – Strategy and Performance Measurement • Dimension 5 – Influencing • Dimension 6 – Participation; and • Dimension 7 – Cost Effectiveness A high-level overview of the framework is shown in Figure 2 below: A WHOLE FAMILY APPROACH TO DESIGNING AN EVALUATION FRAMEWORK 73 This section describes each of the dimensions of the evaluation framework in detail. Dimension 1- Impact This dimension of the framework is designed to determine the extent to which the partnership has added value and achieved a greater impact than would have been achieved without its existence necessitating the establishment of causal links betweentheinterventionsmadethroughimplementingthestrategyandtheperceived resulting outcomes. It is assessed in terms of six sub-dimensions: quality; innovation; integrated service delivery; changes to existing services; resources; and efficiency, and translates into a series of evidence-based questions, including: • To what extent has the strategy brought about an improvement in the quality of services which would not otherwise have been achieved? • Has the strategy been innovative in the development of new services or approaches which would not otherwise have been introduced? • From the perspective of service users, has multi-agency working resulted in improved and integrated service delivery on the ground? Dimension 2: Vision and leadership This dimension of the framework is designed principally to determine the extent to which the strategy has been able to develop a shared and cohesive vision as an D 2 VISION & LEADERSHIP D3 PARTNERSHIP DYNAMICS D4 STRATEGY AND PERFORMANCE MEASUREMENT D1 IMPACT D7 COST EFFECTIVENESS D6 PARTICIPATION D5 INFLUENCING NIGEL MALIN, JANE TUNMORE, AND ANGELA WILCOCK 74 outcome of effective leadership. It is assessed in terms of three key sub-dimensions: future orientation; making it happen; and creating opportunities to lead; and translates into a series of evidence-based questions, including: • Isthestrategyfuture-orientatedwithkeyindividualswhocanexerciseleadership and create a vision through personal skills - rather than position or power – to catalyse, champion and nurture collaboration between individuals and organisations, and secure the necessary resources? • Are there key individuals in place to make it happen, possessing the skills to establish, facilitate and co-ordinate collaboration? • Have leadership opportunities been created at all levels to empower and facilitate different individuals from a range of organisations to take up leadership positions? Dimension 3 – Partnership dynamics This dimension of the framework is designed to determine the extent to which the strategy has developed appropriate structures, processes, resources and a culture inductive to collaboration. It is assessed in terms of six sub-dimensions: structure and processes; trust; commitment to an ethos of collaborative working; communication; learning; and capability; and translates into a series of evidence-based questions, including: • Does the strategy have in place appropriate organisational structures and processes to deliver partnership activities? • Has trust been built amongst individuals, organisations and stakeholders to facilitate collaboration? • Is there a commitment to an ethos of collaborative working evidenced by shared values and common goals, the decentralisation of decision-making and the development of new roles and relationships? • Are the purpose, achievements and needs of the strategy being effectively communicated and promoted internally and to key external target audiences / stakeholders? Dimension 4 – Strategy and performance measurement This dimension of the evaluation framework is designed to determine the extent to which processes for strategic and performance measurement have been embedded within the strategy and the degree to which they are effective. It is assessed in terms of five sub-dimensions: developing a strategic vision; setting objectives and performance targets; formulating a plan to achieve those objectives and performance targets; implementing and executing this plan; and evaluating performance and reformulating the strategic plan and/or its implementation; and translates into a series of evidence-based questions, including: A WHOLE FAMILY APPROACH TO DESIGNING AN EVALUATION FRAMEWORK 75 • Have multi-agency partnerships developed a strategic vision based on identified need including a clearly charted path as to how this will be achieved? • Has the strategic vision been translated into challenging and specific strategic objectives and performance targets- i.e. results and outcomes? • Are processes in place to ensure that the strategy is flexible and adaptable? Dimension 5 – Influencing This dimension of the evaluation framework is designed to determine the extent to which the creation of the strategy has enhance the joint understanding of the political, organisational and funding context in which the strategy operates and how effectively it influences at different levels to bring about change. It can be assessed in terms of three sub-dimensions: influencing government departments and funders; influencing partner organisations; and influencing other relevant partnerships and initiatives; and translates into a series of evidence-based questions, including: • Isthereevidenceofthestrategybeingabletoinfluencegovernmentdepartments/ funders in terms of the way they work; policy and strategy development; and funding and resource deployment? • Is there evidence of the strategy being able to influence partner agencies in the mainstreaming of service provision, for example for children and families in their overall planning and service delivery processes and in the deployment of resources? Dimension 6 – Participation This dimension of the evaluation framework is designed to determine the extent to which the strategy actively promotes the involvement of family members and/or their representative bodies and communities as stakeholders in collaborative action. It is assessed in terms of six sub-dimensions: membership; community development; consultation with users; communication; generating evidence and knowledge; and reduction in social exclusion; and translates into a series of evidence-based questions, including: • To what extent are family members and/or their representative bodies and communities involved in development and implementation processes, via membership of groups etc? • What is the extent of consultation and user involvement in decision-making about strategic plans, services and policies? • How does the strategy tap into community and user involvement to generate evidence and knowledge to for example gauge the experiences of those using services? NIGEL MALIN, JANE TUNMORE, AND ANGELA WILCOCK 76 Dimension 7 - Cost effectiveness This dimension of the evaluation framework is designed to identify the extra benefits that accrue from the strategy implementation. It will therefore be necessary to quantify both direct and opportunity costs incurred through, for example partnership working and what is saved to the public purse by providing support and services to individual family members. The purpose of the costing exercise is to demonstrate that the strategy is proactive in monitoring the costs of working in partnership, to enable the strategy to weigh the costs identified against achievements and thus be able to substantiate how it is generating value for money to a range of funders and stakeholders. Model B The above framework combines an evidence-based, action planning approach setting out underlying values and principles to achieve relevant integrated processes and outputs e.g. performance measurement; vision, leadership/governance and participation. The following ‘model’ framework advances on the first by using a similar action research or co-production approach to focus strategically on project outcomes. For example it builds upon the first in terms of the processes deployed by gathering evidence to understand the impact of a specific intervention. This evaluation framework ‘model’ is divided into three main constituents each of which may influence the impact of an organisation’s strategy in adopting a Whole Family (WF) approach: 1. Contexts The circumstances that form the setting for a project intervention are conceptualised as operating in a form of sealed vacuum, in which social influences are expunged or ‘controlled’ as variables (see Blamey and Mackenzie, 2007; Flint 2011, 2012; Batty, 2014). The individuals subject to family interventions using a WF approach are regarded therefore as autonomous individuals, rather than as being formed in and through relationships (Morris and Featherstone, 2010). Interventions in families as a result of partnership-working are an attempt to exert control and recognition of the importance of sociability and the situated nature of action, context and setting (Aldridge et al, 2009); such that socially constructed space appears as a neglected dimension in understanding family projects (see Murray and Barnes, 2010). Outreach models of family intervention projects may be, to some extent, plugged into these social worlds through the primary mechanism of domestic visits to family homes despite the terminology of ‘intensive intervention’, workers will spend a maximum of a few hours a week with family members (Batty and Flint, 2012). Previous evaluations of intensive family intervention projects have identified the A WHOLE FAMILY APPROACH TO DESIGNING AN EVALUATION FRAMEWORK 77 complexity, multiplicity and fluidity of the vulnerabilities and issues experienced by the families and the causal factors underpinning problematic conduct. These factors combine with the relationships influencing individuals to form the individual-level characteristics and factors which have been identified as the most significant in determiningtheoutcomesofatherapeuticintervention(RenshawandWellings,2010). Examples of such contextual factors are a lack of resources and skills particularly in relation to parentingstrategies (Respect Task Force, 2006; HM Government , 2010); but family members may not have basic literacy and numeracy skills required to navigate some key interactions. The families may be in substantial financial debt and poverty and may not have access to required transport resources (Flint, Batty, Parr, Casey and Nixon, 2011; Batty and Flint, 2012). Additionally the families may also lack abasicknowledgeofavailableservices, or of how to access and interact with service providers,ornothavetheconfidencetobeassertivewiththeseprovidersifnecessary. A focus on a parental skills deficit is appealing to policy narratives, in part because it tends to individualise problems, locate the responsibility within individuals and offers a visible path to resolution, through retraining or the inculcation of such skills (see for example the Coalition’s Troubled Families programme). Establishing a WF approach will be challenged by the limited capacity of projects to respond to primary causal factors of vulnerability or problematic behaviour, including the health of families, mental or physical illness, disability, substance addiction and misuse, risky sexual activity, poor diets and lack of exercise. 2. Mechanisms How does the evaluation framework capture the different types of support provided to families by project workers? Intensive family intervention projects are premised on the relationship and dynamics between a family and a key worker or workers. The importance of relationships and empathy for productive engagement for productive engagement is well-established (Forrester et al, 2008; Malin and Morrow, 2009; Hughes, 2010; Gerdes, 2011). Previous evaluations have similarly identified the centrality of project workers to the outcomes of family intervention projects and that the relationship between the project workers and families has been a primary factor in the achievement of positive change (Nixon et al, 2006; White et al, 2008; Pawson et al, 2009; DfE,2011; Malin and Morrow, 2012). Although these evaluations have identified and discussed the key attributes and practices of project workers, Figure 3 below attempts to provide a more systematic and sequential classification of their key roles. NIGEL MALIN, JANE TUNMORE, AND ANGELA WILCOCK 78 Figure3:Keyattributesandpracticesofprojectworkers(BattyandFlint,2012) A WHOLE FAMILY APPROACH TO DESIGNING AN EVALUATION FRAMEWORK 79 Stage 1 The process of engagement lies at the heart of the WF approach i.e. securing family member involvement; and is a strong determinant of any of the outcomes set (Batty,2014). There needs to be agreed a single point of contact to identify families. Once a referral of a family to a project has been accepted, the first role of the project worker is to attempt to secure the engagement of the family with the support being offered. The challenges that project workers may face in securing rapport, trust and participation in project interventions is paramount. Levels of engagement may vary considerablyamongfamilymembers,andthatfacilitatingengagementisafluctuating process present throughout the period of the intervention. One challenge is when to close a case as changes in family circumstances occur; also many families want engagement as the process of building relationships requires careful consideration. Stage 2 Selection and support of the project key worker. Choosing a key worker will require a family’s consent. Although family views should be taken into account, the family should not have a veto, or an ultimate right to ‘de-select’ /exclude a key-worker outright. Key-workers need to receive appropriate training and to possess the right skills-beingclearaboutyourprofessionalroleandconfidentaboutyourcontribution withinthatrole;seektounderstandthetasksofotherprofessionalsandorganisations and respecting their roles; using social work skills appropriately such as empathy, clarifying and challenging; seek to develop trust through timely, open and honest communication, fulfilling promises and acting with integrity. Key workers require a degree of autonomy, need to demonstrate resourcefulness and be given a small case-load due to the size and complexity of families. Stage 3 The role of assessment is significant in the conceptualisation of family problems and consequent support needs and the extent to which family lives are opened up for examination and intervention. It is necessary not to rely on a single mode of assessment and not to be intrusive with assessments at the beginning of the process, ie families don’t like forms/assessments and ‘referral language’. The overall process demands a simple system that gathers family strengths; and which needs to be visual to demonstrate change; also a formal recognition that assessment information is ‘live’ and is continuously updated. Assessment may also be defined by reviews of family progress or evaluation of their conduct, and is therefore a continual mechanism. Project workers will utilise an initial period following referral to clarify the issues facing a family and link these to the development of a support plan. The intensity of the relationship that is often established between workers and families expands the conventionalscopeofassessment,suchthatworkersobserve,throughdomesticvisits, the dynamics within the private space of households (see Flint, 2012; Batty, 2014) , and undertake a form of excavation in which issues from parents’ own childhoods NIGEL MALIN, JANE TUNMORE, AND ANGELA WILCOCK 80 become evident. This suggests considerable power for project workers to perceive family needs beyond the relative superficiality of official assessment mechanisms. However, such a process which may last for the duration of an intervention may result in a shift of emphasis in interventions and an increase in the support being requested from other agencies. This may not always be welcomed, particularly as the disparity with initial referral rationales and assessed needs increases. Stage 4 The process of multi-professional decision-making is where different professions have distinct roles, statutory or otherwise, in relation to the decision and where there may be a statutory or policy requirement for partnership in specific areas of practice (Taylor, 2010; Quinney and Hafford-Letchfield,2012). A key practice skill in inter-professional working is to respect the various roles and not to demean any, to challenge opinions on occasion as well as seeking compromises. A family profile needs to be established, containing information about who is currently working with the family, including agreement on a method of gathering evidence to consider the effects of different interventions in the life of the family, along with a formal centralised data-base with degree of public access; ultimately to create a process where any collection of information becomes focused, including highlighting key information which gets shared. Stage 5 The development of a family support /life plan and the use of a contract with family members emphasises within policy narratives the contractual basis of the relationship between project workers and the subject of interventions; and its efficacy reflects the quasi-legal and economic rationality premises that underpin governmental understandings, for example through the advocating of formal sanctions (see Cabinet Office, 2011; DfE, 2011; DCLG 2012). From existing literature it would appear that the function and provision of support e.g. emotional, practical, financial are the most contentious; and this typology of direct support, referral to other services and advocacy usually needs to be progressed simultaneously, and comprises a diverse range of potential actions requiring discrete forms ofskills and knowledge. The plan needs to be a two-way agreement, based on a pledge of consent. All agencies should work towards one plan with milestones to measure outcomes (e.g. focus on what the family sees as its main objectives). There should be a menu of options so that the family understands what is available, including opportunities for innovation within the current range of provision. There needs to be clarity around what is said to the family about the possible consequences. Recognition that some families will struggle to achieve/fall - issue of expectations.Individualsneedtobeawareofthesanctionsimposedbyagenciesoutside of the pledge/plan. The flip-side is towards rewarding families – most families want to change;andtheremaybeanimperativetoconsiderhowmoneycouldbemadeavailable from the programme to encourage families to work towards rewards. A WHOLE FAMILY APPROACH TO DESIGNING AN EVALUATION FRAMEWORK 81 Stage 6 The final role of project workers is exit planning. This is essential given the emphasis on the sustainability of positive outcomes, but is often the least developed element of projects (Nixon et al, 2006; Pawson et al, 2009; Batty, 2014). Exit planning is also strongly influenced by the status and power of projects within the network of agency relationships, given that it is usually a process of transferring responsibility and resource expenditure from the projects to other organisations. 3. Outcomes A number of factors have been identified as being linked to positive outcomes including independence, the combination of support and enforcement, flexibility, multi-agency working, a whole family and holistic approach and the quality and commitment of project workers (Barnes et al, 2009; Duffy, 2010; Wright et al, 2010; Batty, 2014). Previous critiques of intensive family intervention projects have all made claims for their outcomes (or lack of outcomes) and interpreted the data and evidence from a spectrum of perspectives. These range from claims that projects ‘work, change and make communities safer’ (Brown, 2009) to arguments that projects comprehensively fail to address anti-social behaviour and mental health issues (Gregg, 2010). There have been four main sources of data produced: • economic cost-benefit analysis of inputs, outputs and the imputed savings from outcomes. These are based conventionally on decision-modelling techniques, allowing a modelling of different pathways through the intervention and an estimating of thecostsandoutcomesofdifferentsocialcareinterventions.Theywillinvolvethe use of cost-benefit, cost-effectiveness or cost-minimisation analysis, depending on the nature of available data (see Knapp et al 2011, Squires and Tappenden 2011). • quantitative analysis of measurable ‘hard’ outcomes, such as reductions in antisocial behaviour or improved school attendance • quantitative analysis of ‘softer’ subjective elements, for example proportions of families feeling their situation or health had improved • qualitative data drawing upon the perspectives of families, project workers and agency officers to identify a diverse set of outcomes. NIGEL MALIN, JANE TUNMORE, AND ANGELA WILCOCK 82 Figure 4: A typology of intensive family intervention project outcomes (Flint et al, 2011) Achieving change ‘Hard’ outcomes Improved education (attendance and attainment) Entry to training or employment Reduction or cessation of risky behaviours (drugs, alcohol, sexual, peer groups) Reduction or cessation of anti-social or criminal behaviour Prevention of entry to criminal justice system Prevention of eviction or children being taken into care Transformative ‘Soft’ outcomes Improved self-confidence and self esteem Improved mental and physical health Improved interfamily relationships and dynamics Improved social and personal skills Raised aspirations Stabilising Improving stability: Maintaining domestic environment Maintaining family relationships and dynamics Maintaining family relationships with agencies and services Managing relationships with peer groups and neighbours Ensuring attendance at school and keeping appointments Ensuring attendance at support service sessions Limiting of drug and alcohol misuse Limiting of risky sexual behaviour The above typology of outcomes demonstrates, for example, how a WF project’s influence on families and efficacy as a mechanism of policy-making may be understood and assessed. These are limited to positive outcomes. There may of course be no change in a family as a result of an intervention and interventions could exacerbate problems for families, although there is no research evidence of this (see Garrett, 2007; Gregg, 2010, for a commentary of possible negative impacts). Outcomes may differ also between family members, including between children and adults. A WHOLE FAMILY APPROACH TO DESIGNING AN EVALUATION FRAMEWORK 83 The broad categories of outcomes identified in Figure 4 are as follows: • Stabilising outcomes relate to stabilising a family or improving stability within the family unit. This sequentially follows stages of crisis management, creating a context and a space (emotional, social and physical) which enables families to begin to address underlying causal factors, to access or engage more beneficially with a range of services, to undertake the learning and practice of new strategies and to focus beyond immediate daily management of severe challenges or problems. • Transformativeoutcomesrelatetoachievingchangedeterminablefromanoriginal ‘baseline’ position at the point of referral and that will significantly reconfigure the position and circumstances of families or individual family members. Assessments of outcomes need to capture ‘journeys of progress’ or ‘distance travelled’ and reductions in risk factors and stabilising impacts in addition to more measurable quantitative outcomes ( Batty, 2014). Finally, this evaluation ‘model’ attempts to reveal the challenges, complexities and diverse skills required by project workers, which are often simplified (and undervalued) in limited notions of ‘challenging’ families and ‘assertive’ approaches (Batty and Flint, 2012). The Troubled Families Programme (DCLG,2012) takes a holistic family approach, bringing together local services and appointing a key worker. However its focus on payment by results, reinforcing the notion of hard outcomes, overlooks the importance of the learning and soft outcomes that provide the building blocks for the future (Batty, 2014). In particular, project workers often facilitate a comprehensive, nuanced and accurate assessment of families’ actual complex problems, needs and dynamics that is not generated through any other technique of intervention. However, these workers often have an identity limited to a transitional, initial stage of the process, reflected in their relatively low status and power within local regimes of governance; especially but not exclusively where projectsaredeliveredby3rd sectororganisations,asthepresentCoalitiongovernment has been actively promoting (Ministry of Justice, 2010). NIGEL MALIN, JANE TUNMORE, AND ANGELA WILCOCK 84 Creating an evaluation framework Apart from providing a management tool, the following framework presents an opportunity for greater ‘democratic’ engagement towards findingsolutionsthatbuild on family strengths. A desired cultural change is about giving a ‘voice’ to individuals within families as each person has a learned capacity to make changes in their lives. (The full framework is shown in Appendix 1.) The five dimensions are as follows: Converting policy to practice This dimension deals with the extent to which there is consensus on the nature of the problem and appropriate responses, and the extent to which the policy issue is novel. This acknowledges the level of bureaucracy, professionalism and importance placed on evidence reviews by policy-makers in power. Vision, leadership and cultural change The good manager must give a shareable, linguistic formulation to already shared feelings, arising out of shared circumstances, done by use of metaphors rather than by reference to any already existing theories (Shotter, 1993). A leader at work is one who gives others a different sense of the meaning of that which they do by recreating it in a different form, a different ‘face’. A property of sense-making is that human situations are progressively clarified (Weick,1995). Strategy, performance measurement and partnership dynamics This dimension lays emphasis on mutual benefit , trust and reciprocity as highly relevant , thereby potentially acting wherein the whole becomes greater than the sum of the parts . In this case where there is a commitment to an ethos of collaboration, each partner in sharing their ideas, knowledge and resources stands to gain from the additional ideas, knowledge and resources that other members of the partnership bring to it by offering partners opportunity to influence each other to behave differently, allowing partners to achieve their goals more economically and effectively (Gambetta, 1988). Participation, engagement and multi-disciplinary focus This dimension recognises the extent of democratic openness, norms of consultation and ‘special interests’ of the range of actors involved, including the relative interest of service users, private shareholders, the public sector, unions, professional associations, civil society and other donors in the policy process. Impact, influencing and outcomes This dimension covers different types of outcome - crisis management, stabilising or transformative. It is essential that the full range of potential outcomes is recognised, e.geffectsofspecificinterventions,effectsofenforcementactionlinkedtoreductionof A WHOLE FAMILY APPROACH TO DESIGNING AN EVALUATION FRAMEWORK 85 criminal /anti-social behaviour; and econometric assessments limited to measurable quantitative indicators e.g. improved education, entry to training or employment. Dissonance theory focuses on post-decisional efforts to revise the meaning of decisionsthathavenegativeconsequences.Peoplestartwithanoutcomeinhand,and then render that outcome sensible by constructing a plausible story that produces it. Concluding comment The above evaluation framework primarily consists of questions which managers and professionals need to reflect on regularly if they are to deliver a successful Whole Family (WF) Strategy. The inclusion of specific items came as a result of indigenous sources such as considering staff views, experiences, and perspectives; and followed a pre-existing typology used to measure concepts of partnership-working and family intervention, projects, both of which were conducted as a summative and formative evaluation (process and outcome). Moving towards a WF approach might be viewed as an extension of a form of ‘family casework’, reminiscent of the recommendations of the Seebohm Report (1968) which argued for a ‘single door on which to knock’. Social work for example has always been regarded as predominantly the expression of values or the observance of principle, e.g confidentiality, the principles of ‘acceptance’ and self-determination. The move to a multi-purpose Social Services Department (SSD) at that time gave real flexibility in moving resources from one use to another; it began the notion of Personal Social Services (PSS) as an instrument in the pursuit of equality and invigorated the social work profession in search of a clear philosophy. Similarly WFA is a response to a number of factors – inadequate range and level of service, poor coordination, problems of access for families, and insufficient adaptability. Whereas organisational change seemed to be the preferred solution in the early 1970s, a change in the ‘cultural system’ (Boyle et al, 2010 ) setting legitimate goals, and the technology that determines the means available for reaching them would appear more instinctively relevant today. Possibly the main features of an effective WFA are (1) participation, engagement and multi-disciplinary focus and (2) impact and outcomes, as these two dimensions may contribute more to a changing culture (attitudes, behaviour). For (1) partnerships need to be translated into specific objectives with shared understanding of why some existing partnerships have broken down. It is not enough to try and build trust among individuals and organisations through imposing or simply talking about shared values, common goals, and offering some decentralisation of decisionmaking, closer working relationships are required grounded on sharing information, joint-working on tasks, also mentoring and reflection (2) The phrase ‘a continuum of integrated intervention’ was used by one interviewee referring to the linking, coordinating of various parts of the process; emphasising family strengths but NIGEL MALIN, JANE TUNMORE, AND ANGELA WILCOCK 86 bringing into equal participation in membership. In her research Batty (2014) concludes that continuous learning during engagement with Intensive Intervention Projects can lead to soft outcomes which enable future positive change in the lives of individuals. Questions 32, 33 above consider what might be achieved in the absence of WFA or the opportunity cost, that is the loss of other possible benefits when a WFA is chosen, for example what results from highlighting dependencies rather than strengths. Describing a WFA in the form of a ‘continuum’ recognises individualised family interventions where adjacent elements are not perceptibly different from each other but where extremes are quite distinct. The common practice is for WFA to be used primarily as a management tool, yet for ‘hands-on professionals’ it may be for gathering evidence e.g keeping a diary, to support their role in decision-making and innovation. References Aldridge, J, Shute, J, Ralphs, R and Medina, M (2009) Blame the parents? Challenges for parent-focused programmes for families of gang-involved young people. Children & Society, 25, 5, 371-381 Atkinson, M (2005) The development of an evaluation framework for partnership working, Journal of Business Research Methodology, 3, I, 1-10 Atkinson, M (2006) The Development of an Evaluation Framework for Partnership Working. British Journal of Management, 7, 1, 75-86 Batty, E and Flint, J (2012) Conceptualising the contexts, mechanisms and outcomes of intensive family intervention projects. Social Policy and Society, 11:3, 345-358 Batty, E (2014) Learning and soft outcomes: Evidence from intensive intervention projects. Social Policy and Society, 13, 3, 357-369 Blamey, A and MacKenzie, M (2007) Theories of change and realistic evaluation: Peas in a pod or apples and oranges? Evaluation, 13, 4, 439-55 Boyle, D, Coote, A, Sherwood, C, and Slay, J (2010)RightHere,RightNow:TakingCo-production into the Mainstream. Discussion Paper: NESTA www.nesta.org.uk Brown, G (2009) Speech to Labour Party Conference , Brighton, 29th September 2009. The Guardian. [http;//www.guardian.co.uk/politics/2009/september/29] Churchill, H (2012) Family support and the Coalition: Retrenchment, refocusing and restructuring, Social Policy Review 24 , 35-54 in Kilkey, M, Ramia, G and Farnsworth, K (Eds). Bristol: Policy Press Department for Children, Schools and Families (2008) Youth Task Force Action Plan: Give respect, get respect, youth matters. London: DCSF Department for Children, Schools and Families (2010) Disadvantaged Communities to Get More Support. London: DCSF Department for Communities and Local Government (2012) Working with Troubled Families: A WHOLE FAMILY APPROACH TO DESIGNING AN EVALUATION FRAMEWORK 87 A guide to the evidence and good practice. London: DCSF Dillane, J, Hill, M, Bannister, J and Scott, S (2001) Evaluation of the Dundee Families Project. Edinburgh; Scottish Executive, Dundee City Council and NCH Action for Children Dixon, J, Schneider, V, Lloyd, C , Reeves, A, White, C, Tomaszewski, W, Green, R and Ireland, E (2010) Monitoring and Evaluation of Family Interventions. London: Department for Education Flint, J (2011) The role of sanctions in intensive support and rehabilitation: Rhetoric, rationalities and realities. British Journal of Criminal Justice, 9, 1/2, 55-67 Flint, J, Batty, E, Parr, S, Nixon, J and Sanderson, D (2011) Evaluation of Intensive Intervention Projects. London: Department for Education Flint, J, Batty, E, Parr, S, Casey, R and Nixon, J (2011) Evaluation of Rochdale Family Project. Sheffield: CRESR [www.shu.ac.uk/research/cresr/rochdale-families.html] Forrester, D, Kershaw, S, Moss, H and Hughes, L (2008) Communication skills in child protection: How do social workers talk to parents? Child and Family Social Work, 13, 41-51 Gambetta, D (1988) Trust: Making and breaking cooperative relations. Oxford: Blackwell Garrett, P.M (2007) ‘Sinbin Solutions’: The ‘pioneer’ projects for ‘problem families’ and the forgetfulness of social policy research. Critical Social Policy, 27, 2, 203-30 Gerdes, K (2011) Importance of Empathy for Social Work Practice: integrating new science, Social Work, 56, 2, 141-8 Gregg, D (2010) Family Intervention Projects: A classic case of policy-based evidence. London: Centre for Crime and Justice Studies HM Treasury (2011) Budget 2011. (HC 836) London: HM Treasury Home Office (2010) Drug Strategy 2010 Reducing Demand, Restricting Supply, Building Recovery: Supporting people to live a drug free life. London: Home Office [https://www.gov.uk/ government/publications/drug-strategy-2010--2] Home Office (2011) More Effective Responses to Anti-Social Behaviour. London: Home Office Jones, A, Please, N, Quilgars, D and Sanderson, D (2006) Addressing Anti-Social Behaviour: An independent evaluation of Shelter Inclusion Project. London: Shelter Kahan, B (2008) Excerpts from Review of Evaluation Frameworks. Reina, SK: Kael Consulting/ Saskatchewan Ministry of Education [http://driverspack.org/download/review-ofevaluation-frameworks-idm-best-practi/] Kalil, A (2003) FamilyResilienceandGoodChildOutcomes:Areviewoftheliterature. Wellington. New Zealand: Centre for Social Research and Evaluation Kendall, S, Rodger, J and Palmer, H (2010) Redesigning Provision for Families with Multiple Problems: An assessment of the early impact of different local approaches. London: Department for Education Knapp, M, Bauer, A, Perkins, M, and Snell, T (2011) Building Community Capacity: Making an economic case. London: Think Family Local Act Personal Partnership MacKay, R (2003) Family resilience and good child outcomes: An overview of the research literature. Social Policy Journal of New Zealand, 20, 98-118 Malin, N and Morrow, G (2009) Evaluating the role of the Sure Start Plus Adviser in providing integrated support for pregnant teenagers and young parents. Health and Social Care in NIGEL MALIN, JANE TUNMORE, AND ANGELA WILCOCK 88 the Community, 17, 5, 495-503 Malin, N and Morrow, G (2012) Evaluating Sure Start: Inter-professionalism and parental involvement in local programmes, (revised ed.) London: Whiting and Birch Ministry of Justice (2010) Breaking the Cycle: Effective punishment, rehabilitation and sentencing of offenders. London: Ministry of Justice Morris, K and Featherstone, B (2010) Investing in children, regulating parents, thinking family: a decade of tensions and contradictions. Social Policy and Society, 9, 4, 557-66 Murray, L and Barnes, M (2010) Have families been rethought? ethic of care, family and ‘whole family’ approaches. Social Policy and Society, 9, 4, 533-44 Nixon, J, Parr, S, Hunter, C, Myers, S , Sanderson, D and Whittle, S (2006) Anti-Social Behaviour Intensive Family Support Projects: An evaluation of six pioneering projects. London: Dept for Communities and Local Government Nixon, J, Parr, S, Hunter, C, Myers, S, Sanderson, D, and Whittle, S (2008) The Longer Term Outcomes for Families who had worked with Intensive Family Support Projects, London: Dept for Communities and Local Government Pawson, H, Davidson, E, Sosenko, F, Flint, J, Nixon, J, and Sanderson, D (2009) Evaluation of Intensive Family Support Projects in Scotland. Edinburgh: Scottish Government Putnam, R (2000) Bowling Alone. New York: Simon & Schuster Quinton, D (2004) Supporting Parents: Messages from research. London: Jessica Kingsley. Renshaw, J and Wellings, S (2010) Catch22 Intensive Intervention Projects: Interim Report, Catch 22. Report of the Committee on Local Authority and Allied Personal Social Services (Seebohm Committee) (1968) Cmnd 3703. London: HMSO Respect Task Force (2010) Respect Action Plan. London: Respect Task Force Sennett, R (2003) Respect:Theformationofcharacterinaworldofinequality. London: Allen Lane Squires, H and Tappenden, P (2011) Mathematical Modelling and its Application to Social Care. SSCR Methods Review 7. London: NIHR School for Social Care Research Sullivan, H and Skelcher, C (2002) Working Across Boundaries: Collaboration in public services. Basingstoke: Palgrave Macmillan Taylor, B (2010) Professional Decision-Making in Social Work Practice. www. learningmatters. co.uk Weick, K (1995) Sensemaking in Organizations. London; Sage White, C, Warrener, M, Reeves, A, and La Valle, I (2008) Family Intervention Projects: An evaluation of their design, set-up and early outcomes. London: DCSF/Communities and Local Government A WHOLE FAMILY APPROACH TO DESIGNING AN EVALUATION FRAMEWORK 89 Appendix 1 AN EVALUATION FRAMEWORK TO ENABLE PARTNERS TO ASSESS AND MEASURE PROGRESS IN THE DELIVERY OF A WHOLE FAMILY (WF) STRATEGY A. Converting Policy to Practice. Does the WF Strategy draw upon evidence-based solutions which: 1. Focus on the root causes of disadvantage/family breakdown discourse - worklessness, educational failure, parenting and early child development deficiencies and health inequalities? 2. Promote a more socially inclusive society - social and cultural cohesiveness, marginalised social groups; and preserve continuity of race, culture, language and education for children separated from their parents in the care system? 3. Develop the strengths and resources of each family to address risk factors, create services that are personalised and tailored to an individual or family; and to minimise the focus on family members as individuals? 4. Help to create permanence and continuity for children; embrace solutions to end child poverty by supporting workforce reform and integrated working; and refocus resources to investments in education particularly for young children, to help disadvantaged families improve their prospects? B. Vision, Leadership and Cultural Change: 5. Is the WF strategy future-oriented with key individuals who can exercise leadership and create a vision, through personal skills rather than position or power, to nurture collaboration between individuals and organisations and secure necessary resources? 6. Are there key individuals in place possessing the skills to establish, facilitate and to coordinate the WF approach? 7. Have leadership opportunities been created to empower and to facilitate individuals from a range of different organisations to take up leadership positions? NIGEL MALIN, JANE TUNMORE, AND ANGELA WILCOCK 90 To what extent does a WF approach : 8. Embody sufficient opportunities for mentoring and supervision, staff appraisal and CPD, particularly for any volunteers involved? 9. Tackle the challenges of building up to a positive culture – developing a ‘mindset’ to unlock the right processes that can bring about cultural change; and which can result in a common culture shared by all in the service of ‘putting the family first’ ? 10. Develop a set of fundamental standards, easily understood and accepted by families, the public and those working in the service? 11. Provide a professionally-endorsed and evidence-based means of compliance with these fundamental standards which can be understood and adopted by the staff who have to provide the service? 12. Takestepstomoveawayfromabusiness-type,target-led,managementstyled culture to one that focuses on the needs of families and those who seek to support them? C. Strategy, Performance Measurement and Partnership Dynamics: 13. Have WF agency partnerships developed a strategic vision based on identified need including a clearly charted path as to how this will be achieved? 14. Have these partnerships been translated into specific strategic objectives and performance targets ie results and outcomes (see section E below)? 15. Are processes in place to ensure that the strategy is flexible and adaptable? 16. Does the WF approach address the strengths and weaknesses of existing systems of partnership e.g. CAF, FIP? 17. Hastrustbeenbuiltamongstindividuals,organisationsandstakeholders to facilitate collaboration? 18. Is there a commitment to an ethos of WF collaborative working evidenced by shared values and common goals, e.g. decentralisation of decision-making, development of new roles and relationships? 19. Does the WF approach enable compilation of a Family Profile (or equivalent) highlighting demographic data, any issues within the family, and which organisations are in contact with the family and the nature of their involvement? A WHOLE FAMILY APPROACH TO DESIGNING AN EVALUATION FRAMEWORK 91 D. Participation, Engagement and Multi-Disciplinary Focus: 20. To what extent are family members and/or their representative bodies and communities involved in development and implementation processes, via membership of groups etc? 21. What is the extent of consultation and user involvement in decisionmaking about strategic plans, services and policies? 22. How does the strategy tap into community and user involvement to generate evidence and knowledge to, for example, gauge the experiences of those using services? To what extent does a WF approach : 23. Ensure that staff are skilled up to look at broader issues, ie develop specialist expertise over/above original learned skills? 24. Enable a process of engagement with the family ie securing family member involvement; and where agency workers are able to secure rapport, trust and participation in any agency intervention? 25. Involve a family’s consent in choosing a key worker, e.g. although family views should be taken into account, the family should not have a veto, or an ultimate right to ‘de-select’ /exclude a key worker outright ? 26. Allow for assessment/review on a continuing basis - it is necessary not to rely on a single mode of assessment and not to be intrusive with assessments at the beginning of the process i.e. families don’t like forms/assessments and ‘referral language’? 27. Develop multi-agency, multi-disciplinary involvement with a focus on decision-making; and to seek to equalise the contribution, including power and status, of different professions, groups and agency representatives – where ‘different professions have distinct roles, statutory or otherwise, in relation to the decision and where there may be a statutory or policy requirement for partnership in specific areas of practice’ (Taylor, 2010: 48)?28. 28. Strengthen coordination of services and support to individual families, including showing respect for the various roles of different professionals/agencies, not to demean any, to challenge opinions on occasions as well as seeking comprises? 29. Promote effective information-sharing e.g. data-sharing protocols, and to reduce the impact of any processual or legal barriers? 30. Resultinaninterventionor‘package’ofsupportwhichisflexible,targeted, coordinated, containing milestones; and which entails direct support - emotional, practical, financial; referral to other services and advocacy? NIGEL MALIN, JANE TUNMORE, AND ANGELA WILCOCK 92 E. Impact, Influencing and Outcomes: 31. Does the WF approach work towards creating a ‘continuum of integrated intervention’? 32. To what extent does the WF strategy bring about an improvement in the quality of service/support to families that would not otherwise be achieved? 33. Is the WF strategy being innovative in the development of new services or approaches which would not otherwise be introduced? 34. From the perpective of individual families, does the WF approach result in improved/integrated service delivery on the ground? 35. Is there evidence of the WF strategy being able to influence local government departments/funders in terms of the way they work eg policy/strategy development, funding/resource deployment? 36. Is there evidence of the WF strategy being able to influence partner agencies in the mainstreaming of service provision e.g. children’s centres, early years speech/language provision , health visiting? 37. What types of outcomes for individual families result from adopting a WF approach – eg crisis management, stabilising, or transformative? • Crisis management : relationship breakdown, offending incidents, conflict with neighbours or peers, increased use of drugs/alcohol, ill-health -mental and physical, emotional breakdown or fragility, pregnancy or risky sexual behaviour, escalating child protection or domestic violence risks or incidents • Stabilising : maintaining domestic environment, maintaining family relationships and dynamics, maintaining relationships with peer groups and neighbours, ensuring attendance at school and keeping appointments, limiting of drug and alcohol abuse, limiting of risky sexual behaviour • Transformative: ‘hard’ outcomes- improved education (attendance and attainment), entry to training or employment, reduction or cessation of risky behaviours, prevention of entry to the criminal justice system, prevention of eviction or children being taken into care; ‘soft’ outcomes -improved self-confidence and self-esteem, improved mental and physical health, improved inter-family relationships and dynamics, improved social and personal skills, raised aspirations