LESSON 7:III – HOW CQI DRIVES THE INITIATIVE AND EACH PROJECT
Our entire 100% New Mexico initiative should work like a successful enterprise, with the most highly rated service in the county. By service, we mean that when elected leaders, stakeholders and the public hear of the projects being implemented to improve services, we hope the reaction is, “These people know how to make things happen.” For this reason, we are revisiting continuous quality improvement (CQI) so there’s no confusion about what we do and how we do it. While we love convening and discussing problems as much as anyone, this initiative creates ten action teams who spend their time turning data into action that’s measurable and meaningful to every family in the county.
As we have discussed throughout the course, CQI is the ongoing process of identifying, describing, and analyzing strengths and problems and then testing, implementing, learning from, and revising solutions.
CQI is an ongoing effort to improve products, services, or processes. These efforts can seek small or incremental improvement over time or, in some cases, lead to a huge “breakthrough” improvement all at once. All aspects of an organization’s processes, including co-workers collaboration and serving clients, are constantly evaluated and improved in the light of their user-friendliness, efficiency and flexibility. Within the public sectors, CQI is seen as a tool used to provide changes that are both measurable and meaningful to the public.
Many people have contributed to the field of quality improvement, notably Edward Deming who is best known for his work in Japan with the leaders of Japanese automobile industry in the 1950s.
There are a number of key actions of the CQI Process. The following seven are those we find especially important:
- Identify an issue using data or some other reliable source of information.
- Research ideas around the reasoning behind that issue and the current level of performance.
- Set a time-bound, measurable goal the team wishes to achieve after reviewing the issue.
- Develop action steps to address the issue.
- Action steps should include the person/people responsible and the target date for completion of the action step.
- Track and adjust the action steps to determine if planned interventions are working.
- Close the feedback loop by sharing the information learned with others.
CQI is our most favored model for quality improvement when working in the public sector with the goal of getting to results. Many people have invented various forms of quality improvement but if you scratch the service of most of them, you will see that improving a system or solving challenges comes down to four skills: assessing, planning, acting and evaluating.
CQI is the framework that will be guiding all action teams in the 100% Community Initiative. Every stakeholder involved in a particular sector should have at least a basic understanding of the CQI framework. With some projects the problem identified may be a lack of quality on the part of a particular agency. If this is the case, the action teams may propose to the agency leadership that CQI may be used to address agency’s challenges. Action teams may discover in the assessment process that it’s not the quality (or lack of quality) of an organization that’s the problem, instead it may be that there is not an organization to provide the service.
The key components of the CQI cycle that we use are assess, plan, act and evaluate.
Assess: Using data, a change agent or action team will identify the magnitude of a challenge, the capacity of local organizations to address a challenge.
Plan: After analyzing data, a change agent of action team will build a measurable plan. We recommend using a planning tool called a logic model that identifies the goal, inputs/partners needed, activities and measurable outcomes.
Act: Implement plan, working with strategic partnerships, with measurable short term, intermediate and long-term outcomes.
Evaluate: Monitor progress with all stakeholders.
Each of these four components, or phases, comes with a set of questions to ensure that the change agent or action team is using data to support the improvement process every step of the way.
CQI is a Team Process
CQI cannot operate in a vacuum. Objectives, goals, and implementation are shared responsibilities and activities. When the team shares an understanding of the process, the team can move forward as one. When an action team works together, CQI is fully supported.
Quality Data and the CQI process
We need quality data that is accurate and timely in order to assess a challenge. Data need to be current and analyzed with care to support the entire CQI process. Our action teams focused on the surviving and thriving services will be in contact with a wide variety of agencies providing specific services. Data will need to guide all attempts at improvement.
Who Wants CQI and Who Doesn’t
State and local stakeholders, including elected leaders, have a wide range of reactions to both CQI and a data-driven process. Data, used correctly, will show where systems aren’t working or don’t exist where they should. Many want this information in the fields of health, safety, education and economic development–and across the public sector. There are also those who prefer to use hunches or opinions to guide work, rather than data.
- Opens up all aspects of work to possible improvement.
- Frees up ways of thinking about work (we’ve never done it that way before).
- Reframes the idea of failure. Turns it into an experiment.
- Makes it a process of discovery and adaptation.
- Allows for growth and encourages growth.
- Helps to keep priorities upfront.
- Can change the culture of the office/organization.
- Improves organizational accountability.
- Refines service delivery process.
- Supports flexibility when meeting service needs change.
- Enhances information management, client tracking & documentation.
- Lends itself to design of new programs & program components.
- Allows creative/innovative solutions.
NEGATIVE REACTIONS OR DISRUPTIVE RESPONSES
- People may feel threatened by CQI and use of data to assess their work, leading to fears of being judged
- People feel a sense of loss as the old way of making decisions (by hunch, or idea of a higher up) is traded in for a data-driven process.
- It spotlights processes, services or products that aren’t working, and this may shine light on ineffective investments and investors.
- It may show how certain populations are experiencing social adversity, injustices and health, education and opportunity disparities.
We provide more information about the CQI process in Appendix G, including sample answers to all the CQI questions focused on assessing, planning, acting and evaluating. We use the example of an action team considering the development of a school-based behavioral health center to illustrate how CQI works.
CQI: questions for each phase with sample answers
The following Q+A is designed to provide 100% Community course participants and action team members with a sample of how to answer the key continuous quality improvement questions (CQI) related to developing a local innovation. In this example, an action team has reviewed the 100% Community survey data and seen that many parents report having a lack of access to behavioral health care in general, along with their children not having access to it at school.
Question: What is the challenge you identified?
Example: Our students at our middle school face emotional challenges due in part to adverse childhood experiences. Students at our school use alcohol and recreational drugs at early ages, as well as consider suicide.
Question: What data did you use to identify the challenge?
Example: In addition to the countywide 100% Community survey results where 46% of parents reported challenges to accessing behavioral health care in the community or at school, the Youth Risk and Behavior Survey (YRBS), specifically the data around initiation of alcohol use by age 12 and rates of suicidal thoughts by both middle and high school students. Data from the 2009 Public Health Survey of Adults that asked questions about ACEs. Data from the article, “The Prevalence of Confirmed Maltreatment Among US Children, 2004 to 2011” in JAMA (2014) by Wildman, Emanuel and Leventhal, concluded that maltreatment will be confirmed for 1 in 8 US children by 18 years of age. The authors reported that this was far greater than the 1 in 100 children whose maltreatment has been confirmed annually by child welfare systems across the nation. For black children, the cumulative prevalence is 1 in 5; for Native American children, 1 in 7.
Question: What do data tell you about the size of the challenge?
Example: According to the most recent Youth Risk Behavior Survey (YRBS), 1 out of 5 middle school students report suicidal thoughts. Slightly less for HS students in the county. Many students at our middle school did not take the YRBS, therefore the need may be greater than we thought.
Question: What do data tell you about which populations are being impacted by the challenge?
Example: Data from the article, “The Prevalence of Confirmed Maltreatment Among US Children, 2004 to 2011” in JAMA 2014 by Wildman, Emanuel, and Leventhal that concluded that maltreatment will be confirmed for 1 in 5 black children, and 1 in 7 Native American children. 1 in 5 children live in households where the income is below the federal poverty income. Our middle school is located in a zip code that has a mix of incomes and many students with different ethnicities, with a significant proportion being Hispanic.
Question: What do data tell you about the current capacity of your agency, community or county to address the challenge?
Example: The rates of underage substance use and suicide ideation have not changed significantly for many years indicating a lack of community capacity to address the challenge. Our school does not have staff to address substance use with our middle schoolers nor their parents. There is limited funding for our school to increase behavioral health services for our students.
Question: Which data can confirm that your challenge is a priority?
Binge drinking rates, from the YRBS, among MS and HS students indicate a challenge.
Rates of suicidal thoughts are present in middle school and continue to high school (from the Youth Risk Behavior Survey). Depression among adults and ACEs scores from the latest Behavioral Risk Factor Surveillance System (BRFSS) public health survey of adults.
Question: What data and/or research illustrates the root causes of the challenge?
Example: We are exploring root causes of suicidal thoughts by looking at academic literature and books on suicidal thoughts. There can be a host of reasons — including Adverse Childhood Experiences. Past research on ACEs shows that there is a link between adult behavioral health outcomes and early childhood experiences. In our state 60% of adults report having at least one ACEs according our state Public Health Adult Survey. In small samples, as many as three-quarters of high school students report having 3 or more ACEs. Some with 7 to 10 ACEs.
Q+A: Planning Phase
Question: Which component of the challenge will you address with a local project?
Example: There were many issues we identified (substance use, depression, ACEs and suicidal thoughts). We wish to focus on addressing depression in students, as well as adverse childhood experiences. We want to explore evidence-informed prevention processes.
Question: Describe your review of research focused on your challenge?
Example: More ACE research to come:
CDC’s Adverse Childhood Experiences (ACEs) Information: https://aae.how/219
Category Archives: Adverse childhood experiences: https://aae.how/220
Robert Wood Johnson Foundation on Adverse Childhood Experiences: https://aae.how/221
Youth Risk Behavior Surveillance System (Depression in Youth): https://aae.how/222
Question: What potential evidence-informed solutions exist in the research?
Example: Because ACEs represent a number of challenges (child abuse, neglect, growing up in households with violence or substance abuse, or with adults with mental health challenges, etc.), we want to first focus on identifying the challenges/ACEs endured by our middle school students. We understand that our students and their parents may have untreated trauma due to ACEs. In some households ACEs continue. We seek, at first, to be able to offer to students and their family members behavioral health care that is easily accessible and culturally appropriate.
We reviewed research on current evidence-informed solutions for ACEs. There are many challenges and approaches to addressing child abuse, neglect, growing up in households with violence or substance abuse, etc. We understand that families have many needs such as stable housing, secure food and access to medical care and transportation. We have been reviewing the literature on community schools which have funding for school-based behavioral health care among many other services. We are prioritizing school-based behavioral health care.
Since we mainly have influence at our middle school, we want to focus on first identifying/ACEs endured by our middle school students and helping our students and their families access behavioral health care. One approach we found is called Screening, Brief Intervention and Referral to Treatment (https://aae.how/223). We are still reviewing many forms of support that could be made available if our school had a behavioral health care center.
Question: What is the hypothesis that illustrates how your actions might solve the challenge? (As in, “if you do A, then B will happen.”)
Example: If we can increase our capacity to provide behavioral health care at our school, we can implement ACEs screening. With a team of school-based behavioral health care providers, we can help students and their families to get quality, culturally appropriate and easily accessible behavioral health. Behavioral health treatment can lead to a reduction in early substance misuse, depression and suicidal thoughts. We can help parents to address their problematic behaviors so that ACEs end in their household.
Question: What are your key steps, timelines, roles and responsibilities related to your project?
Example: Our course project requires creating awareness of the epidemic levels of ACEs and the relationship between ACEs and early substance misuse, depression and suicidal thoughts. This awareness is a short-term outcome.
Intermediate-term, we are looking into creating funding for more behavioral health care staff at the school.
Long-term, we want to fund the creation of a school-based health/wellness center with the capacity to offer behavioral health care to both students and their family members. This is a long-term project requiring, increased funding, and buy in from stakeholders on the school, district, city and county levels. We may also need support from state government and health care institutions.
Question: What unintended consequences might you encounter?
Example: Other school-based health centers we have looked at have had different reactions. Some have been very welcomed by the entire school community. Others have had backlash from parents who do not approve of behavioral health care or school-based health centers for fear that “personal family issues” may be revealed or that health centers may provide unwelcome health advice to youth.
We also may find that there are not enough appropriate referrals in our community to help families access vital services once we do identify problems related to ACEs.
Some behavioral health care agencies are limited to which services they can provide.
Q+A: Action Phase
Question: How will you secure buy in for the course project (innovation or change initiative) in your workplace and/or community?
Example: We will start with qualitative data. We hope to do informational interviews with local school management, school board members, parents, students, city hall, county government and the local hospital and behavioral health care providers. We will use their input to get buy-in. We need to build support among potential funders. We can then seek to gather data on the magnitude of ACEs, along with substance misuse and suicidal ideation, within our student and parent populations.
Question: Which, of all your proposed activities, are the most vital?
Example: Buy-in from school principal and school board for phase one: creating awareness of the challenges and potential solutions. We also need ongoing awareness of ACEs from our school staff and parents. For our long-term goals, we will need establishing funding to be one of our most vital activities. For all of our efforts, we will need data (qualitative and quantitative) as a vital component to educate and ensure buy-in along the way.
Question: How will you begin implementation of the course project?
Example: We will start by contacting the state Department of Health (DOH), Office of School and Adolescent Health and the state coalition for school-based health to learn the latest policies on school-based health centers. We will also explore how some school-based health centers were funded. We have identified fully-resourced community schools in the state and schools with fully-funded behavioral health care centers. From there we begin our process of information gathering through interviews with stakeholders.
Question: How will you record and monitor activities as your course project unfolds?
Example: All information interviews with stakeholders will be recorded, documented and filed on Google docs. We will also do surveys — via Google forms and record findings.
Question: What will your process be for making adjustments to your course project?
Example: We will meet monthly to assess progress. We will consider adjustments using the data we are collecting along the way as part of our process.
Question: How will you ensure data is collected along the way as your course project unfolds?
Example: We will have a monthly report at our meeting on all findings (qualitative and quantitative data, research articles on our topic areas) — and create a quarterly update sent out to all participants and stakeholders.
Q+A: Evaluation Phase
Question: How will you analyze and share all relevant data with those working on the experiment?
Example: Analyses will require our team working with a data specialist. We have identified one at our local college. We will create a database that all our members can access and a project management system called Freedcamp, to share all documents.
Question: How will you ensure that your data and evaluation are presented in an easy to understand manner?
Example: We will have a data expert on our team and run all data by a data committee. We will ask for support from the Office of School Health-DOH — in reviewing data. We will use data visualization (for example Tableau software) to help increase awareness.
Question: How will you determine the strengths and weaknesses of the course project?
Example: Strengths will be assessed by the aspects of the project that are going well and moving forward with support from stakeholders. Weaknesses will be identified by areas that have limited ability to make movement forward on our project.
Question: How will you determine the impact of the course project on your challenge?
Example: The overall impact for the project is long-term. Our initial phase is awareness which will be determined by follow-up surveys. Our intermediate phase will be determined by assessing the amount of funding we have been able to secure and number of providers we have been able to hire. Long-term outcomes can be monitored over the years. We can use measures from the YRBS on substance use, depression and suicidal thoughts, the number of ACEs screens we are able to complete at our middle school per year, and the number of students and families that are receiving services due to school ACEs screening.
Question: How will you identify the unintended consequences?
Example: We need to set up a process to track the parent’s reactions to school-based behavioral health. Surveys and informational interviews can be valuable tools to identify the unintended consequences.
Question: How will you measure if people (employees and/or residents impacted by the course project) are better off?
Example: We will use surveys and existing data to assess:
Short term: The amount of constructive dialogue between school staff, parents, students and district personal on the topic of ACEs and the need to address it
The percentage of awareness of school staff, parents and students of the local rates of ACEs, depression and substance misuse and potential evidence-based solutions
Usage of local behavioral health agencies by parents and youth
Usage of the counseling staff by students and family members
Amount of funding for school and community support for school-based behavioral health including financial support
Long term: significant changes in access to behavioral health care for our students and their family members, we should see a decrease in problems associated with youth depression, substance use and suicidal thoughts — as indicated by Youth Risk and Behavior Survey
Question: How will you ensure that you communicate with all stakeholders on the results of the experiment?
Example: We will use the internet and social networking tools to share our progress and successes — as well as challenges. We will make sure our monthly meetings include time to plan communication with stakeholders who are not able to attend meetings.
As you can see from this example of CQI question and answers, local community stakeholders can create a very informative and thoughtful strategic plan to take on a project. Please let the 100% Community course instructors or coaches know if you need assistance answering any question as you develop your project plan. We look forward to supporting the development of your course project/innovation. Make sure to review the course website’s links to research articles and other resources.