Section i behavioral assessment
1 like650 views
This document provides a compendium of assessment tools for evaluating programs aimed at preventing youth violence. It begins with an introduction that describes the importance of conducting outcome evaluations to identify effective violence prevention strategies. The introduction outlines the key components of comprehensive evaluations, including formative, process, and outcome evaluations. It then provides 10 steps for conducting outcome evaluations, such as clearly defining the problem and intended outcomes, selecting valid and reliable measures, and disseminating findings. The bulk of the document consists of four sections that describe over 100 assessment tools for measuring various constructs related to youth violence, such as attitudes, behaviors, cognitive skills, and environmental factors. Each tool is summarized with details about its scales, target population, reliability, and references.
Section i behavioral assessment
- 1. Measuring Violence-Related Attitudes, Beliefs, and Behaviors Among Youths: A Compendium of Assessment Tools U.S. Department of Health & Human Services CENTERS FOR DISEASE CONTROL AND PREVENTION
- 2. Measuring Violence-Related Attitudes, Beliefs, and Behaviors Among Youths: A Compendium of Assessment Tools Compiled and Edited by Linda L. Dahlberg, PhD Susan B. Toal, MPH Christopher B. Behrens, MD Division of Violence Prevention National Center for Injury Prevention and Control Centers for Disease Control and Prevention Atlanta, Georgia 1998
- 3. Acknowledgments In 1992 and 1993, the Centers for Disease Control and Prevention funded 15 evaluation projects whose primary goal was to identify interventions that change violence-related attitudes, beliefs, and behaviors among children and youths. The following investigators and program staff from these projects have made numerous contributions to the field of violence prevention and have been instrumental to the development of this document. We thank them for making this document possible and for helping to move the field of violence prevention forward. J. Lawrence Aber Henry (Hank) Atha Kris Bosworth Edward DeVos Dennis D. Embry Leonard Eron Dorothy Espelage Albert D. Farrell Daniel J. Flannery Robert L. Flewelling Vangie A. Foshee Roy M. Gabriel Nancy G. Guerra Marshall Haskins Tony Hopson Cynthia Hudley L. Rowell Huesmann Kenneth W. Jackson Russell H. Jackson Steven H. Kelder Molly Laird Gerry Landsberg Linda Lausell-Bryant Fletcher Linder Christopher Maxwell Aleta L. Meyer Helen Nadel Pamela Orpinas Mallie J. Paschall Christopher L. Ringwalt Tom Roderick Faith Samples Mark Spellmann David A. Stone Patrick Tolan Rick VanAcker Alexander T. Vazsonyi William H. Wiist ii
- 4. Contents Acknowledgments . . . . . . . . . . . . . . . . . . . .ii How To Use This Compendium . . . . . . . . . . . .1 How This Compendium Is Organized Choosing the Right Instrument Introduction . . . . . . . . . . . . . . . . . . . . . . . . .5 Why Outcome Evaluations Are So Important Components of Comprehensive Evaluations Ten Steps for Conducting Outcome Evaluations Future Considerations Section I Attitude and Belief Assessments . . . . . . . . .13 Section II Psychosocial and Cognitive Assessments . . .47 Section III Behavior Assessments . . . . . . . . . . . . . . . .141 Section IV Environmental Assessments . . . . . . . . . . . .243 iii
- 5. This compendium provides researchers and pre- vention specialists with a set of tools to evaluate programs to prevent youth violence. This com- pendium is only a first step, however. New measurement tools must be developed, existing tools must be improved, and all such measures must be made available to those of you working in the field of youth violence prevention so that the vast array of prevention programs now being used can be criti- cally reviewed and evaluated. If you are new to the field of youth violence prevention and unfamiliar with available measures, you may find this com- pendium to be particularly useful. If you are an experienced researcher, this compendium may serve as a resource to identify additional measures to assess the factors associated with violence-related behavior, injuries, and deaths among youths. Although this compendium contains more than 100 measures, it is not an exhaustive listing of available measures. Some of the more widely used measures to assess aggression in children, for exam- ple, are copyrighted and could not be included here. Other measures being used in the field, but not known to the authors, are also not included. You will find that many of the measures included in this compendium focus on individuals’violence-related attitudes, beliefs, and behaviors. These measures will be particularly useful if you are evaluating a school-based program or a community-based pro- gram designed to reduce violence among youths. Few scales and assessments address family, eco- nomic, or other community factors. It is our goal to include such measures in future editions of this compendium. Most of the measures in this compendium are intended for use with youths between the ages of 11 and 20 years, to assess factors such as attitudes toward violence, aggressive behavior, conflict resolution strategies, self-esteem, self-efficacy, and exposure to violence. The compendium also con- tains a number of scales and assessments developed for use with children between the ages of 5 and 10 years, to measure factors such as aggressive fan- tasies, beliefs supportive of aggression, attributional biases, prosocial behavior, and aggressive behavior. When parent and teacher versions of assessments are available, they are included as well. How This Compendium Is Organized The Introduction, beginning on page 5, provides information about why outcome evaluations are so important and includes some guidance on how to conduct such evaluations. Following the Introduction, you will find four sections, each focus- ing on a different category of assessments. Each section contains the following components: • Description of Measures. This table summa- rizes key information about all of the assessments included in the section. Each assessment is given an alphanumeric identifier (e.g., A1, A2, A3) that is used repeatedly throughout the section, to guide you through the array of assessments provided. The table identifies the constructs being measured (appearing in alphabetical order down the left- hand column), provides details about the characteristics of the scale or assessment, identifies target groups that the assessment has been tested with, provides reliability and validity information where known, and identi- fies the persons responsible for developing the scale or assessment. When reviewing the Target Group information, keep in mind that we have included only those target groups we know and that the reliability information 1 How To Use This Compendium
- 6. pertains specifically to these groups and may not apply to other groups. When reviewing the Reliability/Validity information, you will notice that several measures are highly reli- able (e.g., internal consistency > .80) whereas others are minimally reliable (e.g., internal consistency < .60). We included measures with minimal reliability because the reliability information is based, in some cases, on only one target group from one study; these mea- sures may be more appropriate for a different target group. We also included measures with limited reliability with the hope that researchers will try to improve and refine them. Evidence of validity is available for only a few of the measures included in this compendium. • Scales and Assessments. The items that make up each assessment are provided, along with response categories and some guidance to assist you with scoring and analysis. In the few instances where scales have been adapted, the most recent (modi- fied) version is presented. We also have provided information on how to obtain per- mission to use copyrighted materials. In most cases, we have presented individual scales rather than the complete instruments because instruments generally are composed of several scales. This approach increases the likelihood that the scales’ test properties will be altered. Nonetheless, we did this because the field is very new and thus has produced few standardized instruments with established population norms for a range of target audiences. • References. This list includes citations for published and unpublished materials pertaining to original developments as well as any recent adaptations, modifications, or validations. In the few instances where scales have been adapted, references for the most recent (modi- fied) version are provided. To obtain information about the original versions, please contact the developers and refer to any relevant references cited. Choosing the Right Instrument Developing instruments that are highly reliable, valid, and free of any bias is not always possible. Carefully choose among the measures included in this document. The criteria on the facing page may assist you in making this selection. As with any research effort, consider conducting a pilot test to minimize problems and to refine the instrument. 2
- 7. 3 Criterion Rating Inter-item correlation Alpha-coefficient Test-Retest Reliability Convergent Validity Discriminant Validity Exemplary average of .30 or better .80 or better Scores correlate more than .50 across a period of at least 1 year. Highly significant correlations with more than two related measures. Significantly different from four or more unrelated measures. Extensive average of .20 to .29 .70 to .79 Scores correlate more than .40 across a period of 3-12 months. Significant correlations with more than two related measures. Significantly different from two or three unrelated measures. Moderate average of .10 to .19 .60 to .69 Scores correlate more than .30 across a period of 1-3 months. Significant correlations with two related measures. Significantly different from one unrelated measure. Minimal average below .10 < .60 Scores correlate more than .20 across less than a 1 month period. Significant correlations with one related measure. Different from one correlated measure. General Rating Criteria for Evaluating Scales Source: Robinson JP, Shaver PR, Wrightsman LS. Measures of personality and social psychological attitudes. San Diego, CA:Academic Press, Inc., 1991.
- 8. This compendium of assessment tools is a publication of the National Center for Injury Prevention and Control of the Centers for Disease Control and Prevention. Centers for Disease Control and Prevention Jeffrey P. Koplan, MD, MPH, Director National Center for Injury Prevention and Control Mark L. Rosenberg, MD, MPP, Director Division of Violence Prevention W. Rodney Hammond, PhD, Director Editing services were provided by the Office of Communication Resources, National Center for Injury Prevention and Control. Editing: Valerie R. Johnson Production services were provided by Parallax Digital. Graphic Design and Layout: Jeff Justice Caroline Kish Cover Design: Jeff Justice Cover Photography: AKid’s World, James Carroll—Artville, LLC, 1997 Suggested Citation: Dahlberg LL, Toal SB, Behrens CB. Measuring Violence-Related Attitudes, Beliefs, and Behaviors Among Youths: A Compendium of Assessment Tools. Atlanta, GA: Centers for Disease Control and Prevention, National Center for Injury Prevention and Control, 1998.
- 9. 5 Youth violence is a serious public health prob- lem in America. Despite a recent decline in homicide rates across the United States,1 homicide continues to claim the lives of many young people. The human and economic toll of violence on young people, their families, and society is high. Homicide is the second leading cause of death for persons 15- 24 years of age and has been the leading cause of death for African-Americans in this age group for over a decade.2 The economic cost to society associ- ated with violence-related illness, disability, and premature death is estimated to be in the billions of dollars each year.3 Researchers and prevention specialists are under intense pressure to identify the factors that place young people at risk for violence, to find out which interventions are working, and to design more effec- tive prevention programs. Across the country, primary prevention efforts involving families, schools, neighborhoods, and communities appear to be essential to stemming the tide of violence, but we must have solid evidence of their effectiveness. To find out what works, we need reliable and valid measures to assess change in violence-related atti- tudes, beliefs, behaviors, and community factors. Monitoring and documenting proven strategies will go a long way toward reducing youth violence and creating peaceful, healthier communities. Why Outcome Evaluations Are So Important Despite the proliferation of programs to prevent youth violence, we have yet to determine the most effective strategies for reducing aggression and vio- lent behavior. We know that promising programs exist, but evaluations to confirm positive effects are lacking.4-6 In their desire to be responsive to con- stituents’concerns about violence, schools and communities often are so involved with prevention activities that they rarely make outcome evaluations a priority. Such evaluations, however, are necessary if we want to know what works in preventing aggression and violence. In the area of youth vio- lence, it is not enough to simply examine how a program is being implemented or delivered, or to provide testimonials about the success of an inter- vention or program. Programs must be able to show measurable change in behavioral patterns or change in some of the attitudinal or psychosocial factors associated with aggression and violence. To demon- strate these changes or to show that a program made a difference, researchers and prevention specialists must conduct an outcome evaluation. Components of Comprehensive Evaluations Evaluation is a dynamic process. It is useful for developing, modifying, and redesigning programs; monitoring the delivery of program components to participants; and assessing program outcomes. Each of these activities represents a type of evaluation. Together, these activities compose the key compo- nents of a comprehensive evaluation. • Formative Evaluation activities are those undertaken during the design and pretesting of programs.7 Such activities are useful if you want to develop a program or pilot test all or part of an intervention program prior to imple- menting it routinely. You can also use formative evaluation to structure or tailor an intervention to a particular target group or use it to help you anticipate possible problems and identify ways to overcome them. • Process Evaluation activities are those under- taken to monitor program implementation and coverage.7 Such activities are useful if you Introduction
- 10. 6 want to assess whether the program is being delivered in a manner consistent with program objectives; for determining dose or the extent to which your target population participates in the program; and for determining whether the delivery of the program has been uniform or variable across participants. Process or moni- toring data can provide you with important information for improving programs and are also critical for later program diffusion and replication. • Outcome Evaluation activities are those undertaken to assess the impact of a program or intervention on participants.7 Such activities are useful if you want to determine if the pro- gram achieved its objectives or intended effects—in other words, if the program worked. Outcome evaluations can also help you decide whether a program should be con- tinued, implemented on a wider scale, or replicated in other sites. Ten Steps for Conducting Outcome Evaluations Outcome evaluations are not simple to conduct and require a considerable amount of resources and expertise. If you are interested in conducting an out- come evaluation, you will need to incorporate both formative and process evaluation activities and take the following steps: • Clearly define the problem being addressed by your program. • Specify the outcomes your program is designed to achieve. • Specify the research questions you want the evaluation to answer. • Select an appropriate evaluation design and carefully consider sample selection, size, and equivalency between groups. • Select reliable and valid measures to assess changes in program outcomes. • Address issues related to human subjects, such as informed consent and confidentiality. • Collect relevant process, outcome, and record data. • Analyze and interpret the data. • Disseminate your findings, using an effective format and reaching the right audience. • Anticipate and prepare for obstacles. Define the problem. What problem is your pro- gram trying to address? Who is the target population? What are the key risk factors to be addressed? Youth violence is a complex problem with many causes. Begin by focusing on a specific target group and defining the key risk factors your program is expected to address within this group. Draw evidence from the research literature showing the potential benefit of addressing the identified risk factors. Given the complexity of the problem of youth violence, no program by itself can reasonably be expected to change the larger problem. Specify the outcomes. What outcome is your program trying to achieve? For example, are you try- ing to reduce aggression, improve parenting skills, or increase awareness of violence in the community? Determine which outcomes are desired and ensure that the desired outcomes match your program objec- tives.Aprogram designed to improve conflict resolution skills among youths is not likely to lead to an increased awareness of violence in the commu- nity. Likewise, a program designed to improve parenting skills probably will not change the interac- tions of peer groups from negative to prosocial. When specifying outcomes, make sure you indicate both the nature and the level of desired change. Is your program expected to increase awareness or skills? Do you expect your program to decrease neg- ative behaviors and increase prosocial behaviors? What level of change can you reasonably expect to achieve? If possible, use evidence from the literature for similar programs and target groups to help you determine reasonable expectations of change.
- 11. 7 Specify the questions to be answered. Research questions are useful for guiding the evaluation. When conducting an outcome evaluation of a youth violence prevention program, you may want to determine the answers to three questions: Has the program reduced aggressive or violent behavior among participants? Has the program reduced some of the intermediate outcomes or mediating factors associated with violence? Has the program been equally effective for all participants or has it worked better for some participants than for others? If multi- ple components of a program are being evaluated, then you also may want to ask: Have all components of the program been equally effective in achieving desired outcomes or has one component been more effective than another? Select an appropriate evaluation design. Choose an evaluation design that addresses your evaluation questions. Your choice in design will determine the inferences you can make about your program’s effects on participants and the effective- ness of the evaluation’s various components. Evaluation designs range from simple one-group pretest/posttest comparisons to nonequivalent con- trol/comparison group designs to complex multifactorial designs. Learn about the various designs used in evaluation research and know their strengths and weaknesses. Special consideration should be given to sample selection, size, and equivalency between groups as part of your evaluation plan. Outcome evaluations are, by definition, comparative. Determining the impact of a program requires comparing persons who have participated in a program with equivalent persons who have experienced no program or an alternative program.7 The manner in which partici- pants are selected is important for the interpretation and generalizability of the results. Sample size is important for detecting group differences. When estimating the sample size, ensure the sample is large enough to be able to detect group differences and anticipate a certain level of attrition, which will vary depending on the length of the program and the evaluation. Before the program is implemented, make sure that the treatment and control/comparison groups are similar in terms of demographic charac- teristics and outcome measures of interest. Establishing equivalency at baseline is important because it helps you to attribute change directly resulting from the program rather than change resulting from an extraneous factor. Choose reliable and valid measures to assess program outcomes. Selecting appropriate measure- ment instruments—ones that you know how to administer and that will produce findings that you will be able to analyze and interpret—is an impor- tant step in any research effort. When selecting measures and developing instruments, consider the developmental and cultural appropriateness of the measure as well as the reading level, native lan- guage, and attention span of respondents. Make sure that the response burden is not too great, because you want respondents to be able to com- plete the assessment with ease. Questions or items that are difficult to comprehend or offensive to par- ticipants will lead to guessing or nonresponses. Subjects with a short attention span or an inability to concentrate will have difficulty completing a lengthy questionnaire. Also consider the reliability and validity of the instrument. Reliable measures are those that have stability and consistency. The higher the correlation coefficient (i.e., closeness to 1.00), the better the reliability. A measure that is highly reliable may not be valid. An instrument is considered valid if it mea- sures what it is intended to measure. Evidence of validity, according to most measurement specialists, is the most important consideration in judging the adequacy of measurement instruments.
- 12. 8 Address issues related to human subjects. Before data collection begins, take steps to ensure that participants understand the nature of their involvement in the project and any potential risks associated with participation. Obtaining informed consent is necessary to protect participants and researchers. Obtaining permission from participants eliminates the possibility that individuals will unknowingly serve as subjects in an evaluation. You may choose to use active informed consent, in which case you would obtain a written statement from each participant indicating their willingness to participate in the project. In some cases, you may decide to use passive informed consent, in which case you would ask individuals to return permission forms only if they are not willing to participate in the project. Become familiar with the advantages and disadvan- tages of both approaches. Once you have secured informed consent, you also must take steps to ensure participants’anonymity and confidentiality during data collection, management, and analysis. Collect relevant data. Various types of data can be collected to assess your program’s effects. The out- come battery may be used to assess attitudinal, psychosocial, or behavioral changes associated with participation in an intervention or program. Administering an outcome battery alone, however, will not allow you to make conclusions about the effectiveness of your program.You also must collect process data (i.e., information about the materials and activities of the intervention or program). For example, if a curriculum is being implemented, you may want to track the number of sessions offered to participants and the number of sessions attended by participants, as well as monitor the extent to which program objec- tives were covered and the manner in which information was delivered. Process data allow you to determine how well a particular intervention is being implemented as well as interpret outcome findings. Interventions that are poorly delivered or implemented are not likely to have an effect on participants. In addition to collecting data from participants, you may want to obtain data from parents, teachers, other program officials, or records. Multiple sources of data are useful for determining your program’s effects and strengthening assertions that the program worked. The use of multiple sources of data, how- ever, also presents a challenge if conflicting information is obtained. Data from records (i.e., hos- pital, school, or police reports), for example, are usually collected for purposes other than the evalua- tion. Thus, they are subject to variable record-keeping procedures that, in turn, may pro- duce inconsistencies in the data. Take advantage of multiple data sources, but keep in mind that these sources have limitations. Analyze and interpret the data. You can use both descriptive and inferential statistical techniques to analyze evaluation data. Use descriptive analyses to tabulate, average, or summarize results. Such analyses would be useful, for example, if you want to indicate the percentage of students in the treat- ment and comparison groups who engaged in physical fighting in the previous 30 days or the per- centage of students who reported carrying a weapon for self-defense. You also could use descriptive analyses to compute gain scores or change scores in knowledge or attitudes by subtracting the score on the pretest from the score on the posttest. You could extend the descriptive analyses to examine the rela- tionship between variables by utilizing cross-tabulations or correlations. For example, you might want to determine what percentage of students with beliefs supportive of violence also report engaging in physical fights. Inferential analyses are more difficult to conduct than descriptive analyses, but they yield more infor- mation about program effects. For example, you could use an inferential analysis to show whether differences in outcomes between treatment and com- parison groups are statistically significant or whether
- 13. 9 the differences are likely due to chance. Knowing the change scores of the treatment or comparison groups is not as useful as knowing if the change scores are statistically different. With inferential sta- tistical techniques, evaluators can also take into account (i.e., statistically control for or hold con- stant) background characteristics or other factors (e.g., attrition, program dose, pretest score) between the treatment and comparison groups when assess- ing changes in behavior or other program outcomes. Regardless of the statistical technique you use, always keep in mind that statistical significance does not always equate with practical meaningful signifi- cance. Use caution and common sense when interpreting results. Many statistical techniques used by researchers to assess program effects (e.g., analysis of variance or covariance, structural equation, or hierarchical linear modeling) require a considerable amount of knowledge in statistics and measurement. You should have a good understanding of statistics and choose techniques that are appropriate for the evalu- ation design, research questions, and available data sources. Disseminate your findings. This is one of the most important steps in the evaluation process. You must always keep program officials abreast of the evaluation findings, because such information is vitally important for improving intervention pro- grams or services. Also communicate your findings to research and prevention specialists working in the field. Keep in mind that the traditional avenues for disseminating information, such as journal articles, are known and accessible to researchers but not always to prevention specialists working in commu- nity-based organizations or schools. When preparing reports, be sure to present the results in a manner that is understandable to the tar- get audience. School, community, and policy officials are not likely to understand complex statisti- cal presentations. Reports should be brief and written with clarity and objectivity. They should summarize the program, evaluation methods, key findings, limi- tations, conclusions, and recommendations. Anticipate obstacles. Evaluation studies rarely proceed as planned. Be prepared to encounter a number of obstacles—some related to resources and project staffing and others related to the field investi- gation itself (e.g., tension between scientific and programmatic interests, enrollment of control groups, subject mobility, analytic complexities, and unforeseeable and disruptive external events).8 Multiple collaborating organizations with competing interests may result in struggles over resources, goals, and strategies that are likely to complicate evaluation efforts. Tension also may exist between scientists, who must rigorously document interven- tion activities, and program staff, who must be flexible in providing services or implementing inter- vention activities. During the planning phases of the evaluation, scientific and program staffers must have clear communication and consensus about the evalu- ation goals and objectives, and throughout the evaluation, they must have mechanisms to maintain this open communication. Future Considerations The field of violence prevention needs reliable, valid measurement tools in the quest to determine the effectiveness of interventions. In past years, researchers in violence prevention have looked to the literature for established measures and have modified them accordingly to assess violence- related attitudes and behaviors. These adaptations have sometimes yielded satisfactory results, but in other cases, the measures have not yet proven to be very reliable. Researchers have also tried to develop new measures to gauge skill and behavior changes resulting from violence prevention interventions. For example, a number of researchers have developed
- 14. 10 measures to assess conflict resolution skills. Many of these measures also require further refinement and validation. We also need to develop measures that go beyond assessing individuals’attitudes, beliefs, and behaviors. Research findings indicate that a number of complex factors—related not only to individuals but to the broader social environment as well— increase the probability of problem behavior during adolescence and young adulthood. Peers, families, schools, and neighborhoods are all involved in shap- ing attitudes and behaviors related to youth violence. Measures to assess how these factors are related to violence need to be developed, refined, and made available to researchers and prevention specialists. To ensure that the instruments we use are cultur- ally appropriate, we must involve a wide range of target groups. Violence cuts across all racial and eth- nic groups and is especially prevalent among African-American and Hispanic youths. Some of the more standardized instruments that have been adapted for use in violence prevention efforts, how- ever, were not developed specifically for use with minority populations. Thus, the items contained in some of the more standardized instruments may not be culturally or linguistically appropriate for minor- ity populations. One final problem we must address is the lack of time-framed measures that can be used for evalu- ation research. To assess the effectiveness of an intervention, we must be able to assess how a partic- ular construct (e.g., attitudes toward violence or aggressive behavior) changes from one point in time to another point in time following an intervention. Instruments that instruct respondents to indicate “usual behavior,” or to “describe or characterize the behavior of a child or teenager” are not likely to pre- cisely measure behavior change. Instruments that instruct respondents to consider behavior “now or in the last six months” are also not precise enough to measure behavior change. The field of violence prevention is still very new, and we must make progress on several fronts. New tools must be developed and existing tools need to be improved. More importantly, researchers and specialists dedicated to the prevention of youth violence must have access to the many measurement tools that have been developed. We hope that increased use of and experience with these measures will help to validate them and will expand our knowledge about effective strategies to prevent youth violence.
- 15. 11 References 1. Centers for Disease Control and Prevention. National summary of injury mortality data, 1985- 1995. Atlanta, GA: National Center for Injury Prevention and Control, 1997. 2. Anderson RN, Kochanek KD, Murphy SL. Report of final mortality statistics, 1995. Monthly Vital Statistics Report 1997; 45,11(2 Suppl). 3. Miller TR, Cohen MA, Rossman SB. Victim costs of violent crime and resulting injuries. Health Affairs, 1993;12(4):186-197. 4. Altman E. Violence prevention curricula: sum- mary of evaluations. Springfield, IL: Illinois Council for the Prevention of Violence, 1994. 5. Cohen S, Wilson-Brewer R. Violence prevention for young adolescents: the state of the art of pro- gram evaluation. Washington, DC: The Carnegie Council on Adolescent Development, 1991. 6. Tolan PH, Guerra NG. What works in reducing adolescent violence: an empirical review of the field. Boulder, CO: The Center for the Study and Prevention of Violence, Institute for Behavioral Sciences, University of Colorado, 1994:1-94. 7. Rossi PH, Freeman HE. Evaluation: a systematic approach. 5th Edition. Newbury Park, CA: Sage Publications, 1993. 8. Powell KE, Hawkins DF. Youth violence preven- tion: descriptions and baseline data from 13 evaluation projects. American Journal of Preventive Medicine 1996;12(5 Suppl).