Hospitals lag decades behind other industries in the adoption of analytics into operational decision-making1
The impact of big data and analytics continues to be realized in healthcare but still has a long way to go. Brilliant minds continue to use optimization, machine learning, and other analytic methods to solve operational problems in the hospital, however, few make it to frontline workers for everyday decision making2.
Analytics can have sustained value when they serve the needs of the user
Many analytics projects fail because they are designed in silos and not trusted or understood by the end user3. Development is mostly focused on the technical process and data availability with little inclusion of the end-user4. Design thinking is a mindset that puts users at the core5. For analytic projects, it allows us to bring together what is needed by the end-user with what is technically feasible, desirable, and useful for the business4.
User-centered design helped us develop an analytic solution that currently supports daily operational decision-making
At Trillium Health Partners, a large community hospital in Canada, we believe analytics are a key enabler to quality and sustainable healthcare. However, we suffer from the same issues plagued by other organizations where models are built in silos, resulting in failed projects. Applying design thinking to analytic projects has shown to increase success of adoption and sustainability6. In a three phased approach, we used a set of core design principles to solve for a hospital capacity management problem.
Core design principles4:
- Empathy: Define and identify your end-users, stakeholders and subject matter experts early and continue to engage them throughout the process.
- Prototyping: Involves drawing and mapping out potential analyses and models to understand scope and feasibility (e.g. data availability, data understanding)
- Active and purposeful feedback: Gathering input and validation from your identified end-users, stakeholders and subject matter experts to ensure understanding of the problem and for development of a usable product.
- Build on the ideas of others: We are typically not the first to attempt to solve a problem. It is critical to understand how others have approached this problem (internally and externally to the organization) so we can learn from their failures and successes.
Three phased approach:
Use Case: Hospital Capacity Management
In Canada, hospitals often operate at over 100% capacity, which results in patients being treated in the hallways and poor quality of care. Hospital operational decision-makers need information about how many beds are occupied and empty in a timely manner in order to make decisions on whether to open or close additional surge spaces (e.g. auditorium). The following is a description of how we applied the design principles in the three phased approach.
Phase 1: Problem Identification and Ideation
In this initial phase, we spent a significant amount of time identifying the end-users (those who will be using the insights generated by the solution), stakeholders (those who may influence the success of the project (e.g. executive sponsorship), and subject matter experts (those who provide the knowledge in business or technical area).
We used a three-step user-centred design framework developed by our hospital embedded research institute, the Institute for Better Health, to support this phase. It employs the principles of empathy and validation through user feedback.
We met together regularly over two months to deeply understand the current process and agree on the problem we were attempting to solve.
Phase 2: Exploratory data analysis and model development
This phase is focused on understanding the data within the context of our problem and to explore the use of different analytic approaches. It is critical to guide this phase with process diagrams to identify areas of exploration and to communicate with team members and stakeholders.
We created a patient flow diagram illustrating bed locations in the hospital, which ended up being essential in our ability to understand and interpret the operational data (e.g., where is the admitted time stamp created on the patient flow diagram). These data definition conversations are typically best supported with process mapping documents, otherwise cross-functional teams typically talk in circles because they don’t understand where the process assumptions which led to the generation of the data timestamps. Using this diagram, the Project Lead and Data Scientist were able to work with the Subject Matter Expert (SME) to understand the context around patient flow and decision making for opening and closing spaces for beds.
We continued to plot the data in various ways and check back with the SME creating a continuous feedback loop in being able to understand the data within the context of hospital operations.
For example, we produced statistical control plotsto identify whether the census was stable. If stable, then we could apply certain statistical models. We did find situations where large transfers of patients occurred, and that typically led to poor results. Otherwise, if the census was within a couple standard deviations, we knew the process was stable.
Through a series of plots and statistical tests, we were able to evaluate a number of patient census prediction algorithms to inform tactical decisions about capacity expansion. This resulted in a model that was able to accurately predict patient census within 48 hours, therefore, meeting the needs of our stakeholders and end-users.
Phase 3: Implementation and operationalization
The goal of this phase is to implement our model in the real world. This means working with end-users to further understand how they want to receive the analytical insights (e.g. dashboard, static report) and how it integrates in their workflow (i.e. when do they need this information to make the decision).
We created user stories for our end-user which helped us understand that they wanted a static pdf report of the predicted census delivered by email every morning.
User story: As a Patient Flow & Registration Director looking to prepare the unit managers and program directors for the flow challenges likely over the next few days, I need good warning of what’s coming, and if it is extreme censuses, how long it will last. Census predictions indicate what the next few days hold;
- If I see extreme censuses on the forecast, I focus my time and staff on minimizing or altogether avoiding the extreme censuses.
- If I don’t, I know I can focus time and energy elsewhere.
Rapid prototyping was used to create the static pdf report. The end-user was engaged in numerous iterations until they were satisfied with the presentation of the report.
Figure 6: Example of the final static report
Currently, this report is in daily use by our hospital operational decision-makers who are now better able to make decisions around opening and closing bed spaces to better serve our patients.
Guided by core design principles, we have successfully conducted a hospital analytic project end to end. We had the following key learnings:
- Working together frequently with our end-users and subject matter experts allowed us to adopt similar terminology enabling us to better understand the data and therefore produce a better analytic solution
- Diagrams are your best friend. The patient flow diagram was a key communication tool in all of our meetings.
- Prototype. We built many prototypes for different models as well as our end state visualization.
- Drysdale E, Dolatabadi E, Chivers C, et al. Implementing AI in healthcare. Vector-SickKids Health AI Deployment Symposium. Toronto, Ontario, Canada, 2019.