Research and Evaluation: What You Need to Know

A slew of surveys are in the works to gather information about Title IV-E grads as well as the state's child welfare workforce

CalSWEC Spotlight summarizes the latest developments in program initiatives and activities of interest to CalSWEC's community.

This month, Dr. Sandhya Rao Hermon, CalSWEC's Director of Research and Evaluation (R&E), gives us the latest news from the R&E team.

September 7, 2017

The Research and Evaluation (R&E) team at CalSWEC has embarked on a robust program evaluation plan for the Title IV-E Stipend Program in 2017. In past years, the emphasis has been mainly on the student perspective; but going forward, the program evaluation plan will include the student perspectives (from recruitment through completion of employment obligation), university partner perspective, and county agency partner perspective. 

Student perspectives will be captured in longitudinal fashion through the use of the following:

  • Baseline survey: A survey to be sent annually to all new Title IV-E students a month after acceptance into the Title IV-E program will include information on different factors, including: demographics (ethnicity, languages spoken and written), prior education, prior experience in public child welfare, application and selection process experiences. The information will be used to understand how these factors influence entry into the Title IV-E Stipend Program
  • New Graduate survey: The New Graduate Survey will be sent annually to all Title IV-E graduates six months after graduation. The survey includes questions on curriculum content and experiences, field training and experiences, field placement, job search, and job-related experiences. It will assess how well prepared students feel for their new jobs, their reflections of their educational experiences, their knowledge of the Title IV-E competencies, and some aspects of their new jobs.
  • Career Path surveys: Two Career Path surveys will be sent to Title IV-E graduates at two points in time: first, when they have recently completed their employment obligation, then two years later to assess their longer-term career experiences. The surveys include questions about educational preparedness, career path (transitions, promotions), agency culture and climate, and experiences in the field. Data from these surveys are used to examine the career paths of Title IV-E graduates, retention of Title IV-E Stipend Program graduates in their public child welfare positions, and the factors that influence retention.

University and county agency partners’ views and feedback on the Title IV-E Stipend Program will be captured at regular intervals using a variety of methods such as surveys, focus groups, and key information interviews.

  • The  27-Year Retrospective on all former Title IV-E stipend recipients. The survey will be sent to all Title IV-E graduates who have participated in the stipend program over the existence of CalSWEC and will be used to assess the career paths of Title IV-E graduates and the continued value and impact of the Title IV-E Stipend Program across California since its inception.
  • The County Workforce Study gathers information from child welfare staff employed at 13 (to date) California county agencies.The study seeks to understand ways to enhance the effectiveness of the child welfare workforce and to understand the specific needs and characteristics of each county’s workforce and  includes a number of attitudinal, organizational, and demographic factors (e.g., commitment to the agency and child welfare, perceptions of organizational culture and climate, respondent age, respondent gender, and Title IV-E status). Retention data are also examined to better understand how these factors influence long-term employment in the California child welfare workforce.

On the In-Service Training Coordination side, the development of items for 100-level classes across all blocks of Common Core 3.0 is complete; items for tests at the 200-level are currently being developed. Major emphasis in 2017 has been on the development of quarterly dashboards using Tableau. These dashboards have been designed to provide visually appealing, timely, and usable data to Regional Training Academies and CDSS.

The accomplishments of the R&E team would not be possible without the hard work and diligence of Elizabeth Wroughton, Esmirna Ramirez, Michael Biehl, Michael Sumner, and Tenia Davis.