Measuring Success: Using the Indicators for Disability Grant Program Grantees March 7, 2019 Sandra Harrell, Associate Director Center on Victimization and Safety Vera Institute of Justice Overview of the Performance Indicators What are Indicators? • They allow users to evaluate performance and to develop strategic plans. • Indicators are easily measured details that are used to represent hard-to-measure, abstract concepts • They must be concrete (focused on one specific detail) and measureable (able to be observed, recorded, and scored). • Indicators must be collected over time. • Repeated collection over time allows for the users to track progress, regress, or lack of change. Indicator Fundamentals • No single indicator is good enough • Most access and safety issues are multi-faceted and require multiple measures • Groups of 4-6 complimentary indicators are used to measure one broad concept • No single data source is perfect • Data from multiple sources should be used to explore all aspects of the concept • Data sources are specified for each indicator History of the Indicators • Beginning in 2012, Vera developed a performance indicator management system with funding from OVW. • Released in 2015, the performance indicator management system measures a variety of aspects of different types of agencies’ ability to effectively serve survivors with disabilities. • In 2018, Vera developed a set of indicators to measure how well hearing agencies serve Deaf survivors. History of the Indicators (2) • There are 6 sets of performance indicators: Disability Service Organizations, Residential Domestic Violence Agencies, Non- Residential Domestic Violence Agencies, Rape Crisis Centers, Dual DV/SA Agencies, Deaf Indicators • Currently, 49 agencies use the indicators (not including 2018 grantees). TypeNumber Disability Orgs 20 Dual DV/SA 7 Rape Crisis Centers 7 Residential DV 5 Non-Residential DV 4 Deaf Indicator 6 TOTAL 49 The System The System 9 Elements of the System Commitment • Theme 1: Responsibility • Theme 2: Partnerships • Theme 3: Policies Capacity • Theme 1: Material Resources • Theme 2: Human Resources • Theme 3: Programmatic Activities Indicator Measures -Example Agency recognizes violence against people with disabilities and Deaf individuals as a priority by: disabilities in agency’s public outreach accessibility for people with disabilities in qHaving an internal committee or work qHaving a client non-discrimination policy that explicitly includes disability status. 1. Recognizes as Priority qSpecifically mentioning people with brochure or social media accounts. qIncluding efforts to increase agency’s agency’s strategic plan. group focused on enhancing the agency’s response to domestic and sexual violence against people with disabilities. Data Sources • Agency Documents • Data collection method: intake forms, policies, procedures, training materials, resource sheets • Agency’s physical and material resources • Data collection method: Agency Observations • The staff of your organization • Data collection method: Staff Interview Questions Measures and Data Sources Scoring • The indicators reflect your commitment to and capacity for serving survivors with disabilities. • Scores are based off of a point scale ranging from 0 to 100. Average Scores 15 Changes in Capacity Over Time 16 Changes in Capacity Over Time (2) 17 Online Indicator System • Vera is launching its online indicator system in 2018. • To score your agency, you will make use of internal documents, observations, and staff interview questions. • The performance indicators are scored every 6 months using this system. Online Indicator System (2) • The online system automatically generates your score once you have completed the indicator questions, and allows you to track trends over time, as well as compare your scores to agencies of the same type. • Upon registration, you will be able to select the name and location of your organization, and select your agency type. • You will also be able to add individuals to your organization and set their permissions. Getting Started Visit indicators.endabusepwd.org to get started. In the upper right hand corner of the screen you will see two options: log in and create an account. Click create an account to get started. Creating an Account Once you click “Create Account” a box will pop up. Enter your name, email, and a password to create an account. Creating an Account (2) Next you will fill out information about your organization. Here you can enter its name, where it is located, and its agency type. Successfully Creating an Account When you have successfully created an account you will see the following screen. Click on select to get started with the assessment process. Assessments Because you have not yet completed an assessment, there will be no data entered in the system. As such, it will show you a comparison of your organization and similar organizations. To enter data, click ”Create an Assessment.” Assessments (2) When you go to create an assessment, you will see the following system. This system is the online indicators. You will be required to check appropriate measures under each component to score your agency. Assessment (3) The panel on the left side of the screen shows you the source type needed to complete each section. As you successfully complete each section, a blue checkmark will appear next to each data source. You will make your way through the system by selecting answers to each indicator question under each data source, selecting the “Next” button at the bottom right hand corner of your screen to move forward through the assessment. Assessment (4) The system tracks your progress with the assessment on the bottom bar. When you have completed the assessment, you will have the option to view your results. Click the button at the bottom right to see them. Once you click that button, the above screen will appear. Click ”No, continue editing” to review or revise your assessment. Click “Yes, view results” to proceed to your score. Results Completed assessments will generate an snapshots of your results. You can view your overall score, or results for the commitment and capacity components. Results (2) You will be able to see which indicators for each component you have satisfied, and which indicators you have not. You may click view detailed results to see more intricate information about your results. Collecting Required Information Data Sources Required On the “My Assessments” page, you will have the option to view the above options. You may download the list of documents needed, the observation guide, and interview questions for staff. 31 Data Sources (2) 32 Repeat Every Six Months • Creates a built-in opportunity for self-reflection. • Allows you to identify changes that may not be obvious. • The process of using the indicators is faster. • Enhancing capacity takes time to plan, develop, and execute. Scores may not increase significantly in the short-term. • Continue to share results with your staff. • Celebrate successes. Questions? Please reach out to ldworkis@vera.org with any comments or concerns. Thank you! Please take a minute to complete our brief survey.