SOP - Evaluation

OPTIONS AND STRATEGIES FOR EVALUATION
OF TRAINING AND IMPLEMENTATION

Purpose for SOP Evaluation

Successful training, coaching and use of Safety Organized Practice require the establishment of an overarching research and evaluation framework. The existing SOP research and evaluation strategies are aimed at assisting counties to create the systemic infrastructure necessary for realizing and sustaining Safety Organized Practice across all levels of the workforce. The information collected is analyzed and reported to ensure consistent practice and program fidelity.

[back to top]

Tools to Evaluate Safety Organized Practice

Practice Profiles

Click here to view/save the Practice Profile (PP)
Click here to view/save the PP FAQs

FOCUS
UC Davis continues to develop practical approaches to assessing whether staff members are implementing key elements of SOP or some form of safety-oriented practice. Presently we are working on finalizing the development of SOP fidelity of tools for Safety Mapping, Incorporating the Child’s Prospective, Harm and Danger Statements, Safety Planning, Safety Networks and Safety Goals. Following the development of the practice profiles of SOP tools, we will begin defining the critical components of SOP practices. These critical components of SOP consist of the broader themes that make up the core philosophy of SOP. Just as the SOP practice profiles break down developing skill to use a specific tool or method at the micro level, the critical components of SOP capture the various avenues in which a practitioner can enhance their ability in a particular area. The critical components are:

Advocacy/Communication: Such as recognizing individual or group needs, providing intervention on behalf of a client or client group, communicating with decision makers and initiating actions to secure or enhance a needed service, resource, or entitlement; in part accomplished by creating collaborative Safety Goals.

Cultural Humility: Interacting with families holding assumptions, respecting and learning from the unique characteristics and strengths of the family while acknowledging and honoring the diversity within and across cultures, and applying these skills to the partnership with the family and the options made available to them.

Engagement/Involvement: Effectively joining and partnering with the family to establish common goals concerning child safety, well-being, and permanency; could be accomplished by implementing Safety Mapping, Three Houses.

Critical Thinking (Assessment): Gathering information about reported concerns and family needs, evaluating the relevance of that information as well as identifying family strengths and community resources that may be applied to address those concerns and needs; could be accomplished by using Structured Decision Making and in formulating Harm and Danger statements, Three Houses, Safety Mapping.

Evaluation: Monitoring outcomes of service plans and system programs to determine if the desired goals are being achieved and if not, to use this information to reconsider goals and strategies developed in the planning phase or services and resources identified in the implementation stage.

PR0DUCTS
Fidelity measurement tool(s), research brief once the tools are piloted.

TIMELINE
Data collection design and measures:

• Practice profiles for tools: completing the last Ppractice Pprofile Safety Goals
• Once the final Safety Goal profile is complete we will begin piloting the tool with pPractice lLeaders and sSupervisors to start. The purpose this initial pilot is to obtain feedback on the definitions, scaling systems, and the overall ease of use.

Case File Review

FOCUS
This tool includes key elements of Safety Organized Practice in three areas: Engagement, Critical Thinking and Increased Safety. To complete the case review tool, both computerized case records (e.g. in automated child welfare information systems such as CWS/CMS) and paper files are examined. Before completing the case review tool, the reviewer talks with the caseworker about where information about Safety Organized Practice (SOP) may be found. Case information may be in more than one paper file depending on the case and the county. Different workers also may have different ways of documenting child, family, and safety network input in case notes. Scales are provided for each SOP element to define a progression of skill from a basic or “emergent” level through an advanced or “distinguished” level mirroring the framework used in California’s Practice Profiles. Scale points are defined generally as follows:

• Emergent practice - There is evidence in the case files that the worker has begun to use the practice or tool; “it’s there.”
• Accomplished practice - There is evidence that the worker has begun to use the element to communicate with others involved with the case and to build shared understanding and agreement among the parties; “it’s shared.”
• Distinguished practice - There is evidence that the element is integrated into the worker’s practice as a way of doing business; “it’s integrated.”

PRODUCTS
Case review tool developed, piloted, and disseminated.