Can 13 Years of FDA Data Teach Us Anything?

Fact or fiction? Pharmaceutical plants that make prescription drugs have a greater likelihood of being inspected than those making non-prescriptions. FDA drug investigators with increased training find more acts of noncompliance at facilities they inspect. Sites that have changed ownership are more likely to have regulatory problems. If numbers don’t lie, these statements are true.Initial findings are finally coming forth from the landmark, number-crunching benchmarking effort undertaken by two researchers, Dr. Jeffrey T. Macher of Georgetown University and Dr. Jackson Nickerson of Washington University. The professors seek to provide statistical understanding of the relationship between FDA regulation and drug company performance, and pinpoint which products and types of manufacturers are likely to draw increased regulatory scrutiny.The pair’s work in progress is a comprehensive analysis of two massive data sets. One comprises facts and figures gathered from FDA inspection documentation over a 13-year period, from 1990 to 2003. Under a material transfer agreement with FDA, Macher and Nickerson have collected data from every single site inspection over that period—in total, more than 38,000 inspections of some 3,700 facilities conducted by 783 different drug investigators.The second data set is being assembled from statistics and information gathered from comprehensive Internet-based questionnaires completed by drug firms about their most intimate processes and procedures, and regulatory outcomes, covering 58 facilities. The researchers are working with “stragglers,” companies that have not yet provided completed questionnaires and statistics, to complete this set.Dr. Macher issued tantalizing tentative findings from the FDA research project at the recent Parenteral Drug Association (PDA) annual meeting in Chicago. He cautioned the audience not to read too much into the numbers just yet, but to begin taking stock of what they might mean for individual firms and the industry as a whole, as well as for FDA.Macher looked at the types of drugs being made at facilities and how they might attract the attention of regulators, and then later lead to compliance or noncompliance issues. Among the tidbits he offered:

• Plants manufacturing prescription drugs have a 13% greater chance of being inspected than those making non-prescription medicines. Once inspected, however, prescription drug facilities are less likely to be found noncompliant.
  
  
• Facilities manufacturing antibiotics have a 19% greater chance of being inspected than those making other drugs, while those making vitamins have an 11% greater chance. Antibiotic plants also have a much greater likelihood of being out of compliance, Macher noted.
  
  
• Facilities making biologics have a 7% lower chance of being inspected than the average facility.
  
  
• Plants that have received an OAI (Official Action Indicated) face a 13% greater likelihood of inspection.

Plant ownership changes can also signal regulatory difficulties, Macher stated. This happens not in the first year or two after ownership — “nobody’s done anything yet,” he hypothesized—but in the second or third year. Experienced employees (“gray hairs”) have been let go. New systems and processes have been instituted but not perfected. Noncompliance often follows, Macher believes, citing Chiron Corp.’s Liverpool plant as one possible example of this phenomenon.Macher also reached some initial conclusions on how the training of FDA investigators affects their ability to detect noncompliance:

• Investigators who complete FDA’s basic drug school have a 7% greater likelihood of detecting noncompliance than they did prior to the school. Completing higher degrees of training (such as courses on sterilization) also increases the likelihood investigators will find compliance issues at a plant. Macher’s conclusion: training matters.
  
  
• Macher also found that investigators who haven’t done many pharmaceutical plant inspections over the previous 12 months tend to forget their training, and find fewer acts of noncompliance upon returning to drug work. The suggestion, he says, is that investigators benefit from devoting their time wholly to drug investigations, rather than dividing their time between drug, food and device work.

No clear patterns have yet emerged from the data as to why FDA inspects a site, Macher noted, and that’s a good thing. No particular variables dominated the decision-making process, suggesting that FDA inspection decisions are made on a site-by-site basis and that the Agency cannot be “gamed” by firms hoping to avoid inspection.The two researchers’ work is nowhere near complete. They’re hoping to add more variables (e.g., those tied to specific manufacturing processes) to their statistical models, and to get more information from both industry and FDA (including CBER data). But it will take time, money (which is limited, Macher says) and manpower. “It’s just the two of us,” Macher reminded the crowd. “Plus we’ve got our day jobs, teaching.”Pharmaceutical Manufacturing will keep you updated on the researchers’ further findings. Will Macher and Nickerson make their database available for public consumption? Hardly. FDA and pharma firms have agreed to full participation under the condition that their most intimate information would not be shared. Firms themselves will be provided with confidential “scorecards” or regulatory profiles of sites as a reward for participation.

WARNING LETTER WATCH Concept Heidelberg, a GMP consulting and training firm based in Heidelberg, Germany, recently issued its annual analysis of FDA drug-related warning letters issued during the previous year. The report analyzes the 33 letters concerning 21 CFR Part 211, and offers some surprising insight into shifting FDA priorities. (For more information, visit www.concept-heidelberg.de) Among the findings: The total number of Part 211 warning letters decreased in 2004 to 33, from 46 in 2003 and 77 in 2002. Eighteen of the 33 letters cite deficiencies in 211.67, “Equipment cleaning and maintenance,” up from just eight in 2003. 211.113, or “Control of microbiological contamination,” was referenced 11 times in 2004, though just once in 2003. The following represent the total number of citations in 2004 for given paragraphs of 21 CFR, with 2003’s total in parentheses: CFR Paragraph Title Citations in 2004 (2003) 211.100 Written procedures; deviations 19 (25) 211.67 Equipment cleaning and maintenance 18 (8) 211.22 Responsibilities of quality control unit 17 (18) 211.166 Stability testing 17 (9) 211.192 Production record review 15 (18) 211.160 General requirements 13 (16) 211.165 Testing and release for distribution 13 (24) 211.113 Control of microbiological contamination 11 (1) 211.84 Testing and approval or rejection of components, drug product containers, and closures 10 (15) 211.188 Batch production and control records 9 (17) 211.186 Master production and control records 8 (11) 211.25 Personnel qualifications 7 (14) 211.42 Design and construction features 7 (3) Courtesy of Concept Heidelberg