Evaluation of Family Preservation and Reunification Programs: Final Report: Volume 1: Study Overview

1. Study Overview

[Main Page of Report | Contents of Report]

Contents

Endnotes

1.1 Background

In 1980, the Adoption Assistance and Child Welfare Act of 1980 (P.L. 96-272) required states to make "reasonable efforts" to prevent children from entering foster care and to reunify children who were placed out of the home with their families. A major focus of policy and planning in state child welfare systems was the development of family preservation programs. The emphasis on family preservation culminated in 1993 in the Family Preservation and Family Support provision of the Omnibus Budget Reconciliation Act (OBRA) (Title IV, subpart 2 of the Social Security Act), which encouraged states to institute or further develop family preservation and family support.

As part of the legislation, the Department of Health and Human Services (DHHS) was authorized to set aside funds to evaluate state family preservation and family support programs. In support of this, DHHS funded three separate studies in September 1994:

· Family Preservation and Family Support Services Implementation Study. This study was awarded to James Bell Associates and is a process analysis of the implementation of the legislation, focusing on the types of programs developed and the barriers encountered. The interim report, "Family Preservation and Family Support (FP/FS) Services Implementation Study," was released March 1999. Special topic reports were completed in 2001 and a final report on implementation should be complete in December 2003.

· National Evaluation of Family Support Programs. This study was awarded to Abt Associates, Inc. and is an outcome evaluation of family support programs. Volume A, a meta-analysis evaluation of family support, and Volume B, a research studies final report, were both completed in April 2001.

· The Evaluation of Family Preservation and Reunification Services. This study was awarded to Westat, Chapin Hall Center for Children, and James Bell Associates, and is the subject of this report. It is an outcome evaluation of family preservation and reunification programs.

The three projects are designed to be complementary. Although each focuses on a different aspect of the 1993 legislation, taken together they represent a comprehensive examination of the programs authorized.

More recently, the enactment of the Adoption and Safe Families Act of 1997 (P.L. 105-89) changed and clarified a number of policies established in the 1980 Act with a renewed emphasis on safety, permanency, and adoption. This legislation placed Federal family preservation initiatives under the rubric of "Promoting Safe and Stable Families" and extended funding for FY 2001. The law made safety of children the paramount concern in service delivery. The law increased the need to understand how family preservation services strengthen families and prevent foster care placement and subsequent abuse and neglect allegations.

Public Law 107-133, the "Promoting Safe and Stable Families Amendments of 2001" was signed into law in January 2002. This legislation reauthorized family preservation services through 2006. Additionally, the legislation authorized the Court Improvement Program, and offered states flexibility in defining family preservation services to allow states to support infant safe haven programs and strengthen parental relationships and promote healthy marriages.

Concurrent with the development of legislation have been program initiatives in family preservation at the state and local levels. Since the 1970s, a number of programs have been developed to provide services to children and families who are experiencing serious problems that may eventually lead to the placement of children in foster care or otherwise result in the dissolution of the family unit. Although these programs share a common philosophy of family- centered services, they differ in their treatment theory, level of intensity of services, and length of service provision. Three models emerged (Nelson et al., 1990):

1. Crisis intervention model. This model, based on crisis theory and intervention, stresses the situation of everyday people confronted with unstable and unsecure circumstances from precipitating events, and the belief that symptoms can be worked through in a brief amount of time (Barth, 1990). Crisis theory also holds that those experiencing a crisis - that is, families about to have a child placed in foster care - will be more amenable to receiving services and learning new behaviors (Nelson et al., 1990, citing Kinney et al., 1988). Homebuilders, a foster care placement prevention program developed in 1974 in Tacoma, Washington, is the prototype program for the crisis intervention model. The program calls for short-term, time-limited services provided to the entire family in the home. Services are provided to families with children who are at risk of an imminent placement into foster care. Social learning theory also plays a part in defining the Homebuilders program, providing the theoretical base for interventions employed (Nelson et al., 1990). Social learning theory stresses that behavior, beliefs, and expectations influence each other in a reciprocal manner, and rejects the belief that changes in thinking and feeling must precede changes in behavior (Barth, 1990). Concrete and supportive services are an important element of the Homebuilders program. Key program characteristics include: contact with the family within 24 hours of the crisis, caseload sizes of one or two families per worker, service duration of four to six weeks, provision of both concrete services and counseling, and up to 20 hours of service per family per week (Nelson et al., 1990).

2. Home-based model. This model focuses on the behavior of the family overall, how members interact with one another, and attempts to change the way in which the family functions as a whole and within the community. Programs using the home-based model stress longer-term interventions based on family systems theory. The FAMILIES program, which began in Iowa in 1974, is the original program using the home-based model. Under the original program in Iowa, teams of workers carry a caseload of 10 to 12 families whom they see in the families' homes for an average of four and one-half months. Both concrete and therapeutic services are provided (Nelson et al., 1990).

3. Family treatment model. This model focuses less on the provision of concrete and supportive services and more on family therapy (Nelson et al., 1990, citing Tavantzis et al., 1986). Services are provided in an office as well as in the home and are less intensive than those using the crisis intervention model. The Intensive Family Services (IFS) Program, which began in Oregon in 1980, is based on the family treatment model. The IFS program also uses family systems theory, which views individual behavioral problems as a reflection of other family problems. Therefore, treatment focuses on the family as a whole. Workers carry a caseload of approximately 11 families. Services are provided for 90 days with weekly followup services provided for three to five and one-half months (Nelson et al., 1990).

Over the years, various states have adopted these family preservation models, sometimes with variations. The growth in family preservation can be partly attributed to early evaluations that were "unequivocally positive and reported high placement prevention successes" (Bath, Howard, and Haapala, 1993). Primarily, these studies only measure family outcomes such as placement prevention for families who receive the treatment. No comparison was made to families who did not receive the services. It was assumed that nearly all children would be taken into foster care placement. However, it cannot be assumed that a high proportion of children receiving family preservation services were at imminent risk without observing the experiences of a comparison group that did not receive the intervention. More recent studies using experimental designs have shown that most of the cases referred were not at imminent risk of placement, as many children in the control groups did not become part of the foster care population.

Although many nonexperimental studies have suggested that high percentages of families remain intact after intensive family preservation services, the results of randomized experiments are mixed. Seven of eleven studies reviewed in A Synthesis of Research on Family Preservation and Family Reunification (Littell and Schuerman, 1995) found that the programs did not produce significant overall reductions in placement. In less than half of the control or comparison cases, placements did not occur within a short period of time after group assignment, which suggests that these programs were generally not delivered to families with children at risk of placement. When the risk of placement among family preservation clients is low, it is unlikely that a program will demonstrate significant reductions in placement.

Despite these findings, placement prevention remains a primary goal of family preservation programs. A review of family preservation programs was conducted in 1995 as part of the Evaluation of Family Preservation and Reunification Services. Information from that study was updated in 1997. As part of the update, 32 family preservation state coordinators were asked if placement prevention was the primary purpose of their program. The majority (78 percent) indicated that it was still the primary purpose, with the remaining coordinators identifying child safety (18 percent) and family functioning (4 percent) as the primary purpose. These goals broaden when county public agency and family preservation administrators were asked about the objectives of local family preservation progress. From the 32 states, 58 county public agency administrators and family preservation program administrators were asked to describe their family preservation objectives. Of the 58 administrators contacted, most offered multiple service objectives. The most frequently reported objective was placement prevention, followed by strengthening families and child and family safety. The purpose of the Evaluation of Family Preservation and Reunification Services is to test whether these service delivery objectives are attained.

[Go To Contents]

1.2 Study Objectives

The Evaluation of Family Preservation and Reunification Services is intended to estimate the impact of family preservation and reunification services. The design of the evaluation was guided by the following objectives:

The evaluation was conducted through randomized experiments in four family preservation sites: Kentucky, New Jersey, Tennessee, and Pennsylvania and the evaluation of an earlier implemented reunification program in New York City. The classic experimental design of this study is the best way to determine causal connections between interventions and outcomes. The control group received the "regular services" of the child welfare system; it was not a no-treatment control group. We studied the effects of the experimental services relative to ordinary services, i.e., services that would have been provided in the absence of family preservation services.

1.2.1 Site Selection and Recruitment

Site selection was based on a number of criteria, including selecting programs which were based on well-articulated theories, in place long enough to operate in the way expected by program managers, consistently implemented, and with sufficient numbers of families to provide adequate sample sizes. It was also important that programs have a primary focus on a population of children involved in abuse and neglect reports and that key policymakers, managers, and line staff were willing to allow evaluation. Initially, it was proposed that of the six sites to be evaluated, at least two would be placement prevention programs, two broader family preservation programs, and two reunification programs.

Emphasis was placed on selecting well-defined programs and those with characteristics useful for the development of knowledge (e.g., serving clientele with substance abuse problems). It was decided to evaluate three programs that use relatively "pure" versions of the Homebuilders model of service. These include Memphis, Tennessee; Louisville and Lexington, (8) Kentucky; and seven counties in New Jersey. The fourth family preservation site, Philadelphia, has a program in which the goal of family preservation services is defined more broadly than prevention placement, compares family preservation services to less intensive in-home services, and has an explicit focus on substance abuse.

Our program review established that there were few reunification programs, and those that existed served small numbers of clients. Most reunification programs were part of family preservation programs and served families after discharge from foster care. We decided to examine the HomeRebuilders reunification program in New York City, by conducting the data collection for the experiment started by the New York State Department of Social Services. We were not able to identify a suitable site for a second experimental evaluation of reunification.

1.2.2 Sample Size

Each site was evaluated separately. We initially set a goal of 500 cases in each site, about 250 in each group. To detect a difference of 15 percentage points between the experimental and controls groups in such characteristics as placement rates with a probability of 0.8 (directional hypothesis, centered on 50%) we would require a total of about 275 cases in both groups. We set our goal higher in order to be able to do some subgroup analyses with adequate power. Initially we hoped to enroll 500 families in each site over a one-year period. However, the sample accumulation in sites in this report, Kentucky, New Jersey, Tennessee, and Philadelphia was slower than expected. A 349-case sample size was achieved in Kentucky after enrolling families for two years. In New Jersey, 442 net cases were enrolled over an 18-month period and in Tennessee, 147 net cases were enrolled over a 21-month period. In Philadelphia, we obtained a sample of 353 cases over a 26-month period.

[Go To Contents]

1.3 Data Elements and Measures

Outcome measures relate to the goals of the programs and require multiple measures, including placement, subsequent maltreatment, family problems, and child and family functioning. Outcome measures are the heart of the experiment, but other types of measures were also needed in order to carry out the study and to more fully understand the observed overall impact in specific sites. Other measures include mediating and conditioning variables. Mediating variables reflect intervening factors that may be the underlying mechanism for achieving change in the more general outcomes, including parents' coping skills, the family's social isolation or embeddedness, and the general quality of interactions in the home environment. There is not always a clear dividing line between mediating and outcome measures. Moreover, an outcome in one realm may be a mediator in another. For instance, adequacy of the parent's attention to a child's health may be considered an outcome as itself, but it is also a key mediating variable in relation to other outcomes.

Measures that may "condition" the effects of the treatment, such as demographic and household composition variables, were examined for their potential influence. For example, family preservation services may emerge as more effective for families with certain characteristics (e.g., single parent families or families with younger children). We also used check measures to ensure that the treatment that was intended actually occurred and to determine whether control group families received services that are supposed to be reserved for members of the experimental group. Finally, the study used service variables to identify at the program level those variables necessary for understanding the results at the family level.

1.4 Data Sources

To obtain these measures, we used multiple data sources, including administrative data, interviews with investigating workers, caseworkers and caretakers, and qualitative data collection on program operation and context.

For family preservation/placement prevention sites, the study used a longitudinal design in which caretakers were interviewed at three points in time: when they entered the study, at the end of services, and at one year after entry to the study. Caseworkers were interviewed at two points in time, when the family entered the study and at the end of services. Investigating workers completed a self-administered form as quickly after assignment as possible. They were asked to provide a description of the allegation and the investigation findings. Caseworkers were asked to provide information on the actual services provided during in-person contacts with the family during treatment for the experimental cases and during a comparable time period for the control cases. Administrative data on placement and subsequent maltreatment were collected for 18 months after enrollment on each case.

An interim evaluation report was released in October 2000. The interim report presented description, service, and outcome analyses for the Homebuilders study sites. This report expands on the interim report by including description, service, and outcome analyses of the non-Homebuilders site. Additionally, analyses on sample attrition, social support, investigating worker questionnaires, staff questionnaires, and secondary analyses are included in this report. (9)

To preserve the distinct nature between the Homebuilders programs (Kentucky, New Jersey, and Tennessee) and non-Homebuilders programs (Philadelphia), the description and analysis are presented separately. This report consists of three volumes. The Executive Summary and Study Overviews are provided in both Volumes One and Two. In addition, each volume provides the following:

Volume One - Study implementation, descriptions of each study site, and a description of the families for the Homebuilders sites.

Volume Two - Services for the Homebuilders sites, outcome analysis for the Homebuilders sites, description and analysis on the Philadelphia family preservation, attrition analysis for the study; social support; investigating worker questionnaire analysis; staff questionnaire analysis; and study conclusions.

Volume Three - Appendices A through K, which include study protocols, forms, secondary analysis and questionnaires.

[Go To Contents]

References

Barth, Richard P. (1990) "Theories Guiding Home-Based Intensive Family Preservation Services," in J.K. Whittaker, J. Kinney, E.M. Tracey, and C. Booth, eds., Reaching High-Risk Families: Intensive Family Preservation in Human Services. New York: Aldine de Gruyter, 1990.

James Bell Associates. (1997). "Interim Report: The Family Preservation and Family Support Services (FP/FS) Implementation Study Volume I," submitted to DHHS, ACF, contract no. 105-94-8103.

Nelson, K. E.; Landsman, M. J.; and Deutelbaum, W. (Jan.-Feb. 1990) "Three Models of Family-Centered Placement Prevention Services." Child Welfare vol. 69, 3-21.

Nelson K. (1994). Family-based Services for Families and Children at Risk of Out-of-Home Placement. Child Welfare Research Review I, 83-108.

Westat, Chapin Hall Center for Children, and James Bell Associates. (1998). "A Review of Family Preservation and Reunification Programs" submitted to DHHS, ASPE, contract no. HHS-100-94-0020.

Westat, Chapin Hall Center for Children, and James Bell Associates. (1998) "Evaluation of the New York City HomeRebuilders Demonstration," submitted to DHHS, ASPE, contract No HHS-100-94-0020.

[Go To Contents]

Endnotes

8. Lexington, Kentucky, remained in the study only a short time. Further details on Lexington are presented in Chapter 3, Kentucky Overview.

9. As to be expected with any program, some of the families assigned to family preservation programs did not receive the services or received a minimal dosage of the services. In addition, a small number of the families in the control group were actually provided family preservation services. To address these issues, analyses were conducted in which these cases were dropped (secondary analysis).


Where to?

Top of Page | Contents

Main Page of Report | Contents of Report

Home Pages:
Human Services Policy
Assistant Secretary for Planning and Evaluation (ASPE)
U.S. Department of Health and Human Services (HHS)