SOP - Evaluation




Purpose for SOP Evaluation

Successful training, coaching and use of Safety Organized Practice require the establishment of an overarching research and evaluation framework. The existing SOP research and evaluation strategies are aimed at assisting counties to create the systemic infrastructure necessary for realizing and sustaining Safety Organized Practice across all levels of the workforce. The information collected is analyzed and reported to ensure consistent practice and program fidelity.

Outcomes to Examine Related to the Training
and Implementation of Safety Organized Practice
Improved completion and accuracy of SDM ToolsIdentify SDM completion measures SafeMeasures®
Measure Accuracy/Consistency Case Review Tool
Utilization of SOP tools and skill enhancementMeasures of SOP tools utilization Worker Training Survey, Six month follow up survey, Case Review Tool
Practice ProfilesPractice Profiles
Worker Satisfaction Survey of CoachingWorker Satisfaction
Improved identification of danger & safetyWorker Training Survey (skill and knowledge questions)Worker Training Survey
Case Review ToolCase Review Tool
Practice ProfilesPractice Profiles
Improved parent/family engagementSurvey questions on engagement, involvement in case planning and understanding of CWS involvementFamily Survey
Practice ProfilesPractice Profiles
Case Review ToolCase Review Tool
Increased client advocacyPractice ProfilesPractice Profiles
Family SurveyFamily Survey
Reduced disproportionalityIn care and entry rates by race/ethnicitySafeMeasures
Fewer children in out of home care & long term foster careNumber in out of home care & percentage in permanent placement (excluding guardian)SafeMeasures
Reduced recurrence of maltreatmentCFSR measureSafeMeasures
Increased timeliness of reunifications Percentage of exits that are due to reunificationsSafeMeasures
Reunification within 12 months (CFSR measure, entry cohort)
Reduced re-entry into careCFSR measureSafeMeasures

[back to top]


Tools to Evaluate Safety Organized Practice

Practice Profiles

Click here to view/save the Practice Profile (PP)
Click here to view/save the PP FAQs

UC Davis continues to develop practical approaches to assessing whether staff members are implementing key elements of SOP or some form of safety-oriented practice. Presently we are working on finalizing the development of SOP fidelity of tools for Safety Mapping, Incorporating the Child’s Prospective, Harm and Danger Statements, Safety Planning, Safety Networks and Safety Goals. Following the development of the practice profiles of SOP tools, we will begin defining the critical components of SOP practices. These critical components of SOP consist of the broader themes that make up the core philosophy of SOP. Just as the SOP practice profiles break down developing skill to use a specific tool or method at the micro level, the critical components of SOP capture the various avenues in which a practitioner can enhance their ability in a particular area. The critical components are:

Advocacy/Communication: Such as recognizing individual or group needs, providing intervention on behalf of a client or client group, communicating with decision makers and initiating actions to secure or enhance a needed service, resource, or entitlement; in part accomplished by creating collaborative Safety Goals.

Cultural Humility: Interacting with families holding assumptions, respecting and learning from the unique characteristics and strengths of the family while acknowledging and honoring the diversity within and across cultures, and applying these skills to the partnership with the family and the options made available to them.

Engagement/Involvement: Effectively joining and partnering with the family to establish common goals concerning child safety, well-being, and permanency; could be accomplished by implementing Safety Mapping, Three Houses.

Critical Thinking (Assessment): Gathering information about reported concerns and family needs, evaluating the relevance of that information as well as identifying family strengths and community resources that may be applied to address those concerns and needs; could be accomplished by using Structured Decision Making and in formulating Harm and Danger statements, Three Houses, Safety Mapping.

Evaluation: Monitoring outcomes of service plans and system programs to determine if the desired goals are being achieved and if not, to use this information to reconsider goals and strategies developed in the planning phase or services and resources identified in the implementation stage.

Fidelity measurement tool(s), research brief once the tools are piloted.

Data collection design and measures:

  • Practice profiles for tools: completing the last Ppractice Pprofile Safety Goals
  • Once the final Safety Goal profile is complete we will begin piloting the tool with pPractice lLeaders and sSupervisors to start. The purpose this initial pilot is to obtain feedback on the definitions, scaling systems, and the overall ease of use.

Case File Review

This tool includes key elements of Safety Organized Practice in three areas: Engagement, Critical Thinking and Increased Safety. To complete the case review tool, both computerized case records (e.g. in automated child welfare information systems such as CWS/CMS) and paper files are examined. Before completing the case review tool, the reviewer talks with the caseworker about where information about Safety Organized Practice (SOP) may be found. Case information may be in more than one paper file depending on the case and the county. Different workers also may have different ways of documenting child, family, and safety network input in case notes. Scales are provided for each SOP element to define a progression of skill from a basic or “emergent” level through an advanced or “distinguished” level mirroring the framework used in California’s Practice Profiles. Scale points are defined generally as follows:

  • Emergent practice - There is evidence in the case files that the worker has begun to use the practice or tool; “it’s there.”
  • Accomplished practice - There is evidence that the worker has begun to use the element to communicate with others involved with the case and to build shared understanding and agreement among the parties; “it’s shared.”
  • Distinguished practice - There is evidence that the element is integrated into the worker’s practice as a way of doing business; “it’s integrated.”

Case review tool developed, piloted, and disseminated.

Return to top