Agenda

1. Why: Why worry about policy robustness?

• Colorectal cancer (CRC) screening widely recommended, guidelines use modeling modeling.
• But consensus cannot be taken for granted: Nordicc trial and conflicting guidelines reveal disagreement and renewed skepticism towards screening.

2. What we do: A stress-test of CRC screening strategies

• Bayesian inference of natural history parameters

• Robustness analysis of policy recommendation

Conflicts of Interest

• No conflicts of interest to report

Funding statement

• This research was supported by grant U01-CA253913 from the National Cancer Institute (NCI) as part of the Cancer Intervention and Surveillance Modeling Network (CISNET).

• This research used resources of the Argonne Leadership Computing Facility, which is a DOE Office of Science User Facility supported under Contract DE-AC0206CH11357. This research was completed with resources provided by the Laboratory Computing Resource Center at Argonne National Laboratory.

• The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health, the Argonne National Laboratory, the Fred Hutchinson Cancer Center or the RAND Corporation.

Modeling informs cancer screening policy

• CISNET simulation models used by the United States Preventive Services Task Force (USPSTF) to assess screening strategies

• in 2021, age to start colorectal cancer (CRC) screening reduced from 50 to 45

• ~ 15 million colonoscopies performed in the US every year

It seemed there was clear messaging and widespread consensus around CRC screening guidelines, until ...

2022: Cancer screening makes headlines

Nordicc trial 10-year results released.

• To no one's surprise: telling people to get a colonoscopy doesn't prevent cancer (quintessential non-compliance problem for interventions that are not "pleasing")

• But getting a colonoscopy reduced mortality by ~ 50%

• Outcomes somewhat line with model predictions (see van den Berg 2023), yet causes of low ITT effects remain explained

2023: Cancer screening makes headlines (again)

American College of Physicians releases new CRC screening guidelines based on an evidence review. Contrary to current USPSTF recommendations:

• discourages screening from age 45 to 49

• And discourages annual FIT

• By our calculations, that could erase ~ 1/5 of the benefit of screening

• We and many others responded to the guidance statement, how ACP guidelines will change screening practice is unclear.

The need to address early onset of colorectal cancer

• Colorectal cancer (CRC) incidence has increased among adults younger than 50 years

• Adults born around 1990 have double the risk of colon cancer and quadruple the risk of rectal cancer compared to those born in 1950 (Siegel, 2017)

• Age-Period-Cohort modeling identifies a birth cohort effect underlying the increase in incidence

• Screening policy needs to be responsive to that

How CRC models work

• Cancer screening models simulate the natural history of colorectal cancer, assuming all cancers come from the adenoma pathway.

Estimation of Benefits, Burden, and Harms of Colorectal Cancer Screening Strategies: Modeling Study for the US Preventive Services Task Force.” JAMA - Journal of the American Medical Association 315(23): 2595–2609.

We use Incremental Mixture Bayesian Computation (IMABC) for calibration

Given:

A set of $y$ calibration targets.

A vector of $\theta$ parameters and their prior distributions $\pi \left(\theta \right)$

A model $M\left(\theta \right)$ that maps parameters to observable targets $\hat{y}$.

Do the following:

• Use the prior distribution $\pi \left(\theta \right)$ and simulate the model iteratively $M\left({\theta }_i\right)$, preserving predictions within error bounds $\hat{y}\in y\pm {ϵ}_y$.

• The resulting set of parameters approximate the posterior distribution of $f\left(\theta \right)$.

IMABC provides an algorithm to perform ABC in a batch-sequential fashion. This process is computationally intensive and requires HPC for microsim models.

How to address early onset in CRCSPIN?

We concurrently investigate two mechanisms that could explain higher CRC incidence among young adults:

1. Adenomas might initiate before age 20 ("young onset")

• Calibrate separate model specifications separately.

2. Recent birth cohorts might have increased risk of adenoma initiation (birth cohorts)

• Add birth cohort effects to the model, and add calibration targets to inform them.

Adenoma risk model now includes birth cohort effects

Adenoma risk posteriors vary by model

Strong birth cohort effects after 1940

Stress-testing CRC screening guidelines

• We now have a posterior distribution for natural history parameters (including birth cohort effects) for two specifications of CRC-SPIN (BC-10 and BC-20).

• They capture uncertainty about the disease process and recent changes in risk.

• In an appendix in the paper, I demonstrate the plausibility of four colonoscopy sensitivity scenarios (Very Low, Low, Baseline, High).

• How robust are our recommendations to those uncertainties?

Experimental design

The entire experimental design combines:

• The 26 colonoscopy screening strategies assessed in the 2021 USPSTF analysis

• Two model specifications (BC-20 and BC-10, sampling 500 points from each posterior distribution).

• Four colonoscopy sensitivity Scenarios (Very Low, Low, Baseline, High).

• Each run simulated for 2 million individuals.

• Report prediction intervals for each strategy conditional on each scenario for Benefit (LYG), Burden (N colonoscopies), and Incremental Cost Effectiveness Ratio (ICER).

LYG estimates depend crucially on assumptions

Computing the cost-effectiveness frontier over models' posterior distributions

• Selected strategies for the cost-effectiveness frontier separately for each model specification and sensitivity combination using posterior mean*.

• To be selected for the Pareto frontier, a strategy had to be non-dominated and not extended dominated by other strategies (i.e., also no worse than a linear combination of pareto-efficient strategies).

• After selecting non-dominated strategies in expectation, for each scenario, compute LYG, # of colonoscopies, and ICER for the strategies defined as part of the Pareto frontier for each parameter set in the posterior distribution.

• Report prediction intervals for LYG, # of colonoscopies, and ICER.

Incremental effectiveness ratios more stable

Takeaways

• Cost-effectiveness measures varied substantially within and across scenarios.

• But the Pareto frontier seems robust across scenarios and models.

• Uncertainty from the natural history parameters seems as relevant as colonoscopy sensitivity assumptions.

• Models represent mutually exclusive assumptions about adenoma initiation, but those differences did not change cancer screening recommendations.

• Paper reports bayesian prediction intervals for effectiveness, burden, and efficiency ratios accounting for natural history uncertainty.

Calibration targets date back to the 1800s

Estimating birth cohort effects

• Ideally, we would want to estimate ${\alpha }_y$ for each birth-cohort year within the set of years $I=[1880,1881,...,1975]$.

• But this would result in a substantial expansion of the parameter space.

Instead:

• Define a set of years at which the birth cohort effect will be estimated: 1980, 1910, 1940, 1955, 1970, and 1975.

• 1940 is a reference point at which the effect is zero.

• The minimum and maximum dates are chosen based on the available calibration targets.

• The intermediate knots were chosen to be evenly spaced, with a finer resolution after 1940.

• Produce a smooth, monotonic interpolation between these points to define birth cohort effects between the years used as knots, using a method based on piecewise radial functions (Stineman 1980, Jhannesson 2018).

• More details: See our pre-print

Colonoscopy sensitivity assumptions

Stress-testing CRC strategies requires high-performance computing

• Experimental design defined to fit a ~ 200,000 core-hour budget.

• Combining 2 model specifications, 500 natural history parameter sets for each model, 26 screening strategies, 1 “No-Screening” scenario used as the comparator, and four colonoscopy sensitivity scenarios, resulting in 105,000 unique model runs.

• Simulating 2 million people for each run requires simulating 210 billion life histories and over 0.63 trillion adenomas.

• Experiment conducted on the Theta Supercomputer using a couple 1000s of concurrent nodes.