Our approach to evaluation design is rooted in our overall organization mission of “helping the people who help people.” We base our approach to each project on the concepts of public justice, empowerment, and mutual respect while using an ecological framework. An ecological framework values input from all stakeholders from various settings such as students in classrooms, principals and other administrators, community leaders, and policy makers. It examines individuals and communities within their contexts to determine weaknesses and strengths regarding the ‘person-fit’ environment, and recognizes the multiple influences (e.g., political, economic, cultural) occurring within each setting. Our evaluation plan consists of the following general steps:
1. Stakeholder Input
2. Logic Model
3. Process Monitoring
4. Outcome Evaluation
5. Dissemination and Feedback
BSRI has experience conducting needs assessments for organizations and specific communities. We use multiple sources of data including both quantitative sources (archival data, survey data, county/local statistics) and qualitative data sources (key informant interviews, focus groups, case/record reviews). BSRI has expertise in focus group research, and can quickly plan, moderate, analyze, and interpret findings. Qualitative and mixed methods research can be done at all phases of the evaluation plan, and are helpful in learning about a population’s reaction to certain services and programs. Finally, survey administration and personal interviewing can also be used to determine needs or gaps in service, and to ensure all projects at hand are understood on a personal level. BSRI has the capability to administer surveys in person, on computers, and via web-based applications. Data administered electronically is easily transferred into statistical databases for storage and analysis.
Process and Outcome Evaluation
BSRI is trained and certified in the NIATx Quality Improvement Process which uses rapid cycle change techniques. NIATx, designed and supported through a collaborative sponsored by the University of Wisconsin School of Engineering, is a model of process improvement specifically for behavioral health care settings to improve access and retention in treatment.
Outcomes are often what funders and key stakeholders are most interested in when it comes to program performance. BSRI can help your agency specify program outcomes, link data to outcomes, and examine how those results measure up relative to other programs doing similar work. BSRI is always interested in working with agencies to gauge community-level impacts on population health.
In 2016, BSRI brainstormed, invested in and developed the “Data Outcome Evaluation System” (DOES). This data entry system is a fully secure and private web-based platform. All data can be viewed by the client and evaluation team in real time and is customizable based on client needs. BSRI can also analyze this data to produce data dashboards for clients based on priorities within evaluation.
Fidelity monitoring is particularly useful when your organization is implementing an evidence-based practice or intervention and refers to whether the program or strategy is being implemented in the way that developers intended. Measuring adaptations to programs and strategies, whether they are done purposefully (i.e. changing language) or in an unplanned way (lack of time for 90 minute sessions, e.g.) is necessary to make sure the strategies are still effective and that expected results occur without unintended negative consequences. BSRI has a great deal of experience in fidelity monitoring with evidence-based practices through record reviews, qualitative interviews, and measuring other process and outcome indicators of a program. We can help you tailor an intervention to best serve your community and help you measure and document changes along the way so if something is working, you know what it is and vice versa.