Case Study 2: Environment for Regulatory Submission

This video provides an in-depth exploration of how biopharmaceutical companies and Contract Research Organizations (CROs) can modernize their statistical computing environments, particularly for regulatory submissions. Presented by TransCelerate BioPharma, a non-profit organization focused on accelerating and simplifying R&D, the discussion highlights the urgent need for cutting-edge technologies like machine learning, artificial intelligence, and natural language processing in regulated R&D. The core challenge addressed is the current limitation imposed by traditional software and narrow regulatory guidance, which struggles to accommodate the emergence of electronic and digital data sources and the advanced analytical capabilities required to process them.

The presentation introduces TransCelerate's "Framework for Modernization of Statistical Environment" (MSA environment). This framework outlines a set of principles designed to help organizations build and maintain a computing environment that health authorities will find reliable, ensuring that analytical outputs can be used with confidence to support regulatory filings. The foundational principles of this framework are Accuracy, Reproducibility, and Traceability (ART), which are emphasized as core tenets that must be cohesively integrated. The framework demonstrates that an MSA environment, whether utilizing traditional or non-traditional software, can produce accurate and reproducible results, supported by comprehensive documentation that traces the lineage of dependencies for each output. This end-to-end control, based on ART, is crucial for building health authority confidence in the efficacy and safety results of new drug candidates.

The video details the practical implementation of the MSA framework, outlining phases such as pre-planning, design, implementation, and production. It identifies key stakeholders essential for success, including statisticians, programmers, clinical quality teams, IT professionals, and engineers. A critical success factor highlighted is the early engagement and cohesive collaboration among these multi-functional teams (clinical, pharma, and IT) to co-design a feasible and aligned plan. A specific case study is presented, currently in the production phase, where a multi-server computational environment was built. This environment features two types of open-source installations: one for exploratory and general work, and a "frozen" installation specifically for submission-related work, with all packages verified and tested against the MSA framework's guidance.

The case study revealed significant benefits, such as demonstrating the validity of non-traditional tools through the framework's comprehensive guidance on testing and assessing accuracy, thereby mitigating risks related to trustworthiness. Challenges encountered included a lack of familiarity with the new environment among some stakeholders, necessitating clear communication between business and IT, and the substantial effort, time, and resources required for frequent updates before reaching equilibrium. To address these, dedicated data science teams were assigned, automated testing was maximized, and package validation was prioritized based on upcoming submission needs. The video concludes by emphasizing that the MSA framework is timely, enabling companies to proactively prepare for future health authority submissions and fostering a necessary cultural shift towards more malleable operations and modern computing solutions across the biopharmaceutical industry. This collective effort to build health authority confidence in advanced analytical software is seen as vital for accelerating drug development and bringing innovative therapies to market faster.

Key Takeaways:

• Urgent Need for Modernization: The biopharmaceutical industry faces an undeniable need for cutting-edge technologies to transform the analytical life cycle due to the rapid growth of electronic and digital data (e.g., wearables, EHR) and the limitations of traditional software.
• Regulatory Challenges with New Technologies: Current regulatory guidance narrowly defines reliable software, pushing companies to use older technologies and creating challenges for adopting advanced tools like machine learning, AI, and natural language processing in regulated R&D environments.
• TransCelerate's MSA Framework: The "Framework for Modernization of Statistical Environment" (MSA environment) provides a structured approach for building and maintaining a modern computing environment that health authorities will find reliable for regulatory filings.
• Core Principles: Accuracy, Reproducibility, Traceability (ART): The MSA framework is founded on the principles of Accuracy, Reproducibility, and Traceability, which must be cohesively integrated to ensure confidence in the reliability of efficacy and safety results for new drug candidates.
• Applicability to All Software: The MSA environment framework applies to both traditional and non-traditional software, demonstrating how to produce accurate and reproducible results with comprehensive documentation tracing the lineage of dependencies for each output.
• Phased Implementation Approach: Implementing the MSA framework involves distinct phases: pre-planning, design, implementation, and production, each with specific considerations, stakeholders, and potential challenges.
• Critical Stakeholder Collaboration: Effective collaboration among diverse roles—statisticians, programmers, clinical quality, IT, and engineering—is paramount for successfully advancing statistical computing environments and co-designing feasible plans.
• Case Study: Frozen Environments for Submission: A practical case study involved building a multi-server computational environment with separate installations for exploratory work and "frozen" installations for submission-related work, with packages verified using the MSA framework.
• Benefits of the Framework: The framework offers excellent guidance for testing and assessing accuracy, demonstrating the validity of non-traditional tools and mitigating potential risks around their trustworthiness for regulatory use.
• Communication is Key for Stakeholder Alignment: A significant challenge is ensuring all stakeholders, particularly business and IT, are familiar with new environments and work closely together to bridge communication gaps.
• Resource Allocation for Updates: Organizations must plan for substantial effort, time, and resources for frequent updates and adjustments to the environment to meet evolving demands and company focus.
• Strategies for Addressing Challenges: Solutions include assigning dedicated data science teams, leveraging automated testing as much as possible, and prioritizing package validation based on upcoming submission requirements.
• Cultural Shift Towards Malleability: A broader cultural shift is needed within organizations to adopt more malleable approaches to operations, embracing modern computing solutions and frameworks to accelerate industry transformation.
• Opportunity Cost of Delay: Delaying the adoption of innovative software capabilities carries an opportunity cost, ultimately impacting the industry's ability to bring drugs to market faster.
• Building Health Authority Confidence: The framework provides an industry-wide approach for demonstrating the reliability of analytical software, including emerging tools, thereby building health authority confidence and facilitating the modernization of statistical tools.

Key Concepts:

• Modernized Statistical Analytical (MSA) Environment: A computing environment designed and maintained according to specific principles (ART) to ensure its reliability and trustworthiness for supporting regulatory filings with health authorities.
• Accuracy, Reproducibility, Traceability (ART): The three core tenets of the MSA framework. Accuracy refers to the correctness of results, Reproducibility ensures that the same results can be obtained under the same conditions, and Traceability means that results can be linked back to their original data, code, and processes.
• Non-traditional Analytical Tools: Advanced computational methods and software beyond conventional statistical packages, including Machine Learning (ML), Artificial Intelligence (AI), and Natural Language Processing (NLP).
• Frozen Installations: A specific, controlled version of a software environment (including libraries and dependencies) that is kept static and unchanged for submission-related work to guarantee consistency, reproducibility, and compliance.

Examples/Case Studies:

• Case Study 2: Environment for Regulatory Submission: This case study details the implementation of a multi-server computational environment. It features two distinct types of open-source installations: one for general exploratory work and another, "frozen" installation specifically designed for submission-related work, with its packages verified and tested according to the MSA framework's guidance.