Using metrics to assess quality management's efficacy is not a new concept. Within the last decade, a genuinely collaborative relationship has formed between the industry, academia and technology providers to design and operationalize a quality metrics program focused on proactive quality management. This work can help companies operate more efficiently and raise quality excellence beyond compliance.
Yet, quality metrics remain an enigma for many companies that react to changes and manually source data across multiple systems. To advance from reactive to proactive quality, companies can take steps to operationalize metrics while optimizing along the way. Knowing the regulatory requirements around metrics and how they have evolved is a critical first step.
Operationalizing quality metrics
When designing a metrics strategy, it is helpful first to understand how the regulatory and operational expectations of a quality metrics program have continued to change over time.
-
2013: FDA requested public input on using quality metrics data to evaluate product manufacturing quality to understand the underlying causes of drug shortages.
-
2014: ISPE launched the Quality Metrics Pilot to help the industry identify and define the metrics truly indicative of quality. ISPE believed that the "right" metrics could help promote positive behavior and instill a corporate culture of responsibility, covering challenges associated with how metrics are collected and interpreted.
-
2015: FDA issued a draft guidance, Request for Quality Metrics (80 FR 44973), proposing a mandatory program for product-based quality metrics reporting. Stakeholders shared feedback on the burden of collecting, formatting, and submitting data at a product level across multiple establishments, technical comments on the proposed metrics and definitions, and legal concerns regarding the proposed mandatory program.
-
2016: FDA published Submission of Quality Metrics Data (81 FR 85226), a revised draft guidance in response to the feedback received in 2015. This described an initial voluntary phase of the Quality Metrics Reporting Program, with participants reporting data either by product or establishment and through an FDA submission portal.
-
2016: FDA awarded a research grant to the University of St. Gallen, Switzerland, to help establish a scientific basis for relevant pharmaceutical manufacturing performance metrics, which may prove helpful in assessing the current state of quality and predicting the risks of quality failures and/or drug shortages.
-
2017: ISPE established a cross-functional sub-team ("Metrics Reporting and Analysis") under the Advancing Pharmaceutical Quality (APQ) core team to evaluate, summarize, and provide feedback on the FDA quality metrics portal.
-
2019: University of St. Gallen provided a comprehensive report on enhancing the pharmaceutical production system model to act as a true excellence model, the basis for an in-depth evaluation of a case for quality.
-
2022: FDA established a docket to solicit comments on changes to its previously proposed Quality Metrics Reporting program.
Today, quality metrics programs are shifting beyond focusing only on compliance. This approach reflects the changing needs of business, including operational factors like customer satisfaction, business performance, innovation, and sustainability. The next phase for quality metrics will focus on supporting the industry’s move from reactive to proactive quality using real-time data and analytics.
Changing the mindset from reactive to proactive
Simply meeting the minimum standard for a manufacturer — compliance with GMP requirements —does not indicate continuous improvement or promote a quality culture. GMP compliance is a given in the quality management maturity (QMM) model. Moving from reactive to proactive quality requires a change in mindset. The shift can renew the focus on integrating business and manufacturing with quality practices and drive technology advancements that can optimize product quality, enhance supply chain reliability, and deliver improvement.
Quality metrics contribute to developing an effective pharmaceutical quality system (PQS) and are a key element under the QMM umbrella. Metrics provide insights into performance indicators that can help detect problems before they occur and identify opportunities for innovation.
Metrics also contribute to informing the oversight of outsourced activities and material suppliers to minimize disruptions. Metrics remain an essential tool to monitor the overall health of a facility, and the FDA's Quality Metrics Initiative remains an active and vital aspect of quality maturity program development.
Real-time metrics refer to the continuous monitoring and reporting of data, providing up-to-the-moment insights and analysis. These metrics are updated and displayed continuously, improving decision-making. With access to real-time data, manufacturers can reduce the effort required for compliance reporting and increase the focus on data analysis and risk mitigation—areas that can significantly improve product quality.
Understanding the roadblocks
Metrics can help to transform quality management from a cost center to a revenue generator. Quality teams can make proactive decisions with real-time metrics, leading to better business outcomes. Since metrics can have a significant impact on the business, what are the roadblocks to achieving real-time quality metrics?
As companies grow, many establish quality systems that address specific challenges. Because of the adoption of point solutions, the technology landscape is very siloed with systems that don't speak to each other. It becomes difficult for companies to have a holistic view of their data or the state of their quality systems, causing reactive rather than predictive action.
Today, quality teams must search for information across various systems and spreadsheets to build reports. With this fragmented infrastructure comes a reliance on paper and manual methods to consolidate data for analysis and submission. The process introduces the risk of human error and lacks real-time, enterprise-wide visibility.
In a recent poll taken during a roundtable with biopharma quality leaders, it was clear that although most companies have moved beyond paper into a digital quality management system (QMS), most still consider themselves highly reactive and fragmented. Even basic metrics are still tough to aggregate - and none would say they have reached a proactive, unified quality state.
Only 15% of respondents have reached data-driven QMM supported by metrics and unified applications that can enable proactive quality management. The majority are still in a highly reactive model supported by fragmented systems.
Optimizing for real-time insights
With the correct design, your metrics program can evolve beyond standard compliance metrics. Including data on open and overdue quality events and actionable metrics such as real-time cycle time, recurrence/trending, CAPA effectiveness, annual product quality review (APQR), and quality management review (QMR) can help to significantly improve efficiency.
To successfully achieve real-time metrics:
- Standardize and harmonize quality processes as part of a transformation initiative to create standard data representation enterprise-wide.
- Identify a modern, cloud-based quality management system (QMS) that automates paper processes, connects data across the organization, and enables real-time reporting.
- Design and implement enterprise-wide quality metrics and key performance indicators (KPIs).
- Extend the quality ecosystem externally to third-party partners like suppliers or contract manufacturers.
- Build reports and dashboards that provide on-demand metrics, allowing users to identify trends or patterns and proactively act on critical pivot points.
Advancing quality operations in this way allows for strategic management of data assets from a quality and operational excellence perspective. It also enables building quality metrics around expected outcomes. Designing a metrics dashboard that includes KPIs can provide on-demand metrics to identify trends or patterns and act on signals that would have otherwise been a hindsight discovery.
In a proactive model, the QMS system provides internal and external connections with third parties like suppliers, contract manufacturers, and sponsors or customers, ensuring that established metrics deliver a complete picture of quality operations in real-time. This can drive continuous improvement as quality teams identify issues early on and develop new mitigations or processes to address them. Quality can become predictive and prevent adverse inspection outcomes with an infrastructure and metrics that provide transparency.
Gathering metrics typically requires manual effort to identify and gather data. The hardest metrics to attain are quality event recurrence and insights into staff re-training.
Modernizing for unified data and reporting
Technology is a critical element in successfully establishing real-time quality metrics. Companies can improve overall data quality with a QMS that easily integrates with other systems and enables role-based access to third-party partners. A modern QMS can also help by reducing or eliminating time-intensive manual data entry, delivering harmonized data sets across systems and geographies, and providing a single source of truth.
Identifying a system that offers pre-configured, commonly used, and trending metrics can help to set the framework for a quality metrics program quickly. Reports and dashboards for APQR and QMR provide a strong foundation while covering the key metrics required to maintain compliance.
Flexibility is also essential, so identifying an application that can be configured to meet unique needs will deliver more long-term value. This saves quality teams from creating metrics with every project or product and recreating them for future subsequent products.
By building an infrastructure that enables accurate and reliable data access, companies gain industry knowledge and real-time quality insights to support data-driven processes. The result is improved quality systems effectiveness, transparency across the entire quality ecosystem, and reliable trend predictions.
The ultimate goal is to build a quality metrics program and reach a state of predictive, proactive quality. To achieve this industry dream of proactive signaling, the first step is establishing foundational data and real-time reporting across a unified system.
