R Package Quality & Validation:

Current Landscape

Key Info

Posit Validation Info

Create R version & Package plan:

Define process of OS implementation within an organization such as R versions & process for adding packages

3 Main Areas

1. Posit Team Software

2. R

3. Packages

Pharmas

  • Roche
  • “Clinical trial reporting is at a pivotal moment today transitioning away from propriety languages. At Roche this has meant freezing our legacy macros library, and we can now celebrate having R as the default backbone for evidence generation in late-stage (with space for more OS languages!).” James Black
  • OS NDAs

Do we have to Validate?

A popular question is if regulatory bodies require “validated” packages. The R FDA Submission Pilot used as-is CRAN packages as outlined below. This submission was accepted successfully by the FDA. https://github.com/RConsortium/submissions-pilot1/blob/main/renv.lock https://github.com/RConsortium/submissions-pilot1

Part 1 Posit Team

Part 1

Validation Info

Part 2

R

Part 2

A Guidance Document for the Use of R in Regulated Clinical Trial Environments

Part 3

Part 3 A (Public Packages)

There are 2 key things to focus one when it comes to packages in a clinical environment:

    1. Stable repositories inside the organization
    1. Users documenting the packages used for clinical reporting

Posit Packages

RStudio in 2020, in collaboration with the community and pharma, released validation guidance documents covering tidyverse, tidymodels, r-lib, gt, shiny and rmarkdown. The main page and direct links are below:

Open Source Package Development & Maintenance

For R packages, maintenance is open and in public view. This can be very helpful for many Quality Teams as it makes an abundance of information available that is often hidden for commercial software. Below we will review this for the ggplot2 package available here: https://github.com/tidyverse/ggplot2

  • It is important for Quality Professionals to have a high-level understanding of version control (git etc.) and the maintenance of public open source code via tools like github.

CRAN Checks

Getting a package added to CRAN is no small feat (pass R CMD CHECK & survive the review process). CRAN maintainers test packages across a matrix of R versions & operating systems: https://cran.r-project.org/web/packages/policies.html & https://cran.r-project.org/web/packages/submission_checklist.html

ggplot2

Code Example

Author Tests

Additional Tests

Many package authors have added their own tests which help ensure that the package’s functions work as intended. Most often, these tests are built into the package in a specific way so that whenever the automatic CRAN checks are run the custom tests also run. These additional tests are part of the package source and for more information please review: http://r-pkgs.had.co.nz/tests.html#test-cran & https://testthat.r-lib.org/

Validation Hub’s Risk-Based Approach guidance

In addition to the Posit Packages (ggplot2 https://posit.co/products/open-source/rpackages/ etc.), pharmaceutical organizations will also add other packages to the baseline of packages to be used by statistical programmers. Organizations will often select other packages to be included based on the R Validation Hub’s Risk-Based Approach guidance here: https://www.pharmar.org/white-paper/

riskassessment demo

https://rinpharma.shinyapps.io/riskassessment/

Regulatory R Package Repository

Community Work is underway by the R Validation Hub to help bring availability of validated packages for clinical trials. This includes support for the test and validation of public packages commonly used. You can follow this effort here: https://pharmar.github.io/regulatory-r-repo-wg/

Pharmaverse

Internal Repository of Packages for Clinical Trials

The documents above, and the implemented risk-based approach, allow organizations to establish a baseline of existing packages that are deemed validated and controlled for internal use. Organizations can use Posit Package Manager mentioned above to create an internal repository of packages selected specifically by Quality and IT for clinical trials. Steps to do this are outlined here: https://github.com/philbowsher/Creating-an-Internal-Repository-of-Validated-R-Packages

renv for Statistical Programmers

To help document the packages used and for collaboration with colleagues, statistical programers will use a R package called renv: https://rstudio.github.io/renv/articles/renv.html renv is a companion package used by users in conjunction with the IT best practices used for managing packages. renv creates a file called renv.lock that describes the state of the users project’s library at some point in time and is created with renv::init(). renv.lock records repositories, package versions and can be versioned with git! Here is the R FDA Pilot Submission renv.lock!: https://github.com/RConsortium/submissions-pilot1/blob/main/renv.lock

Part B (Internal Packages)

Testing R Functions:

Test and Reproducibility for Shiny

Alternative Approach - Buying Validated Packages

Some organizations prefer to buy validation documentation for existing open source packages. There are various vendors selling validation tests and documents as well as groups selling services to support validation. One such group is Atorus that provides OpenVal which is a subscription-based repository of nearly 200 validated R packages and growing.

Thank you!

@philbowsher

@philbowsher@linkedin.com

youtube.com/rinpharma

rinpharma.com

PharmaSug Conf