rocacu

workflowr

• Summary
• Checks
• Past versions

Last updated: 2023-11-15

Checks: 5 2

Knit directory: ~/multistate2/

This reproducible R Markdown analysis was created with workflowr (version 1.7.0). The Checks tab describes the reproducibility checks that were applied when the results were created. The Past versions tab lists the development history.

---

R Markdown file: uncommitted changes

The R Markdown file has unstaged changes. To know which version of the R Markdown file created these results, you’ll want to first commit it to the Git repo. If you’re still working on the analysis, you can ignore this warning. When you’re finished, you can run wflow_publish to commit the R Markdown file and build the HTML.

Environment: empty

Great job! The global environment was empty. Objects defined in the global environment can affect the analysis in your R Markdown file in unknown ways. For reproduciblity it’s best to always run the code in an empty environment.

Seed: set.seed(20230211)

The command set.seed(20230211) was run prior to running the code in the R Markdown file. Setting a seed ensures that any results that rely on randomness, e.g. subsampling or permutations, are reproducible.

Session information: recorded

Great job! Recording the operating system, R version, and package versions is critical for reproducibility.

Cache: none

Nice! There were no cached chunks for this analysis, so you can be confident that you successfully produced the results during this run.

File paths: absolute

Using absolute paths to the files within your workflowr project makes it difficult for you and others to run your code on a different machine. Change the absolute path(s) below to the suggested relative path(s) to make your code more reproducible.

absolute	relative
~/multistate2//code/smoothtest.R	code/smoothtest.R
~/multistate2//code/newsmooth.R	code/newsmooth.R
~/multistate2/code/fitarray.R	code/fitarray.R
~/multistate2/code/arrayindicate.R	code/arrayindicate.R
~/multistate2/code/frs30_URBUT/fun.frs_30ynew.R	code/frs30_URBUT/fun.frs_30ynew.R
~/multistate2//output/dfascvd_newbp.rds	output/dfascvd_newbp.rds
~/multistate2/code/frs30_URBUT/fun.frs_30y.R	code/frs30_URBUT/fun.frs_30y.R

Repository version: 988af83

Great! You are using Git for version control. Tracking code development and connecting the code version to the results is critical for reproducibility.

The results in this page were generated with repository version 988af83. See the Past versions tab to see a history of the changes made to the R Markdown and HTML files.

Note that you need to be careful to ensure that all relevant files for the analysis have been committed to Git prior to generating the results (you can use wflow_publish or wflow_git_commit). workflowr only checks the R Markdown file, but you know if there are other scripts or data files that it depends on. Below is the status of the Git repository when the results were generated:

Ignored files:
    Ignored:    .DS_Store
    Ignored:    .Rproj.user/
    Ignored:    analysis/.DS_Store
    Ignored:    code/.DS_Store
    Ignored:    data/
    Ignored:    lesliepics/.DS_Store
    Ignored:    output/
    Ignored:    plots/.DS_Store

Unstaged changes:
    Modified:   analysis/NRI.Rmd
    Modified:   analysis/index.Rmd
    Modified:   analysis/rocplots.Rmd

Note that any generated files, e.g. HTML, png, CSS, etc., are not included in this status report because it is ok for generated content to have uncommitted changes.

---

These are the previous versions of the repository in which changes were made to the R Markdown (analysis/rocplots.Rmd) and HTML (docs/rocplots.html) files. If you’ve configured a remote Git repository (see ?wflow_git_remote), click on the hyperlinks in the table below to view the files as they were in that past version.

File	Version	Author	Date	Message
Rmd	273d024	Sarah Urbut	2023-09-22	Update
Rmd	ff43074	Sarah Urbut	2023-09-21	Update
Rmd	c14de32	Sarah Urbut	2023-09-20	Update
Rmd	4d7d65c	Sarah Urbut	2023-09-17	Update

---

R Markdown

This is an R Markdown document. Markdown is a simple formatting syntax for authoring HTML, PDF, and MS Word documents. For more details on using R Markdown see http://rmarkdown.rstudio.com.

When you click the Knit button a document will be generated that includes both content as well as the output of any embedded R code chunks within the document. You can embed an R code chunk like this:

NOw ROC

source("~/dynamichr/code/utils.R")
source("~/multistate2//code/smoothtest.R")
source("~/multistate2//code/newsmooth.R")
source("~/multistate2/code/fitarray.R")
library("reshape2")
source("~/multistate2/code/arrayindicate.R")
source("~/multistate2/code/frs30_URBUT/fun.frs_30ynew.R")

load("~/Library/CloudStorage/Dropbox-Personal///pheno_dir/output/merged_pheno_censor_final_withdrugs_smoke.rds")
dfh$cad.prs.lec=cut(dfh$cad.prs,breaks = c(-5,-0.84,0.84,5),labels = c("low","mid","high"))
dfh$int=interaction(dfh$f.31.0.0,dfh$cad.prs.lec)
# Relabel the levels of the interaction variable
levels(dfh$int) <- c(1,2,3,4,5,6)


train=dfh[1:(nrow(dfh)*0.80),]

dfascvd=readRDS("~/multistate2//output/dfascvd_newbp.rds")
test=dfh[!(dfh$identifier%in%train$identifier),]
test=merge(test,dfascvd[,-which(names(dfascvd)%in%c("age","anylipidmed0","bp_med2","smoke"))],by.x="identifier",by.y="sample_id")
test$ascvd_10y_accaha=test$as2
test$phenos.enrollment=test$f.21003.0.0
test=data.table(test)
source("~/multistate2/code/frs30_URBUT/fun.frs_30y.R")
library(CVrisk)
ages=c(40:80)
nstates=c("Health", "Ht","HyperLip","Dm","Cad","death","Ht&HyperLip","HyperLip&Dm","Ht&Dm","Ht&HyperLip&Dm")
modelfit=fitfunc2(data.table(train),ages = ages,nstates = nstates,mode = "binomial",covariates ="cad.prs+f.31.0.0+smoke+antihtn_now")

set.seed(456)
enrollments = c(41:70)
aucmat = matrix(NA, nrow = length(enrollments), ncol = 4)
prcmat = matrix(NA, nrow = length(enrollments), ncol = 4)
semat = matrix(NA, nrow = length(enrollments), ncol = 4)
pmat = matrix(NA, nrow = length(enrollments), ncol = 2)

ages = 40:80
enrollments = c(41:79)
## return a matrix of coefficients over all ages for a given state to state transition
b=coefplotsmooth2(ages = ages,start = "Health",stop = "Cad",modelfit = modelfit,window_width = 20,span = 0.75,degree = 2)
ggplotly(b$plot)
coefs=b$custom_smooth



for (z in 1:length(enrollments)) {
  age = enrollments[z]
  start = age
  stop = 80
  
  df_frame = test
  atrisk = df_frame[age < Cad_0_censor_age &
                      age < Ht_0_censor_age &
                      age < HyperLip_0_censor_age &
                      age < Dm_0_censor_age & smoke == 0 , ]
  
  
  df_updated = data.frame(atrisk)
  ar = data.frame(
    "intercept" = 1,
    "cad.prs" = atrisk$cad.prs,
    "sex" = atrisk$f.31.0.0,
    "smoke" = atrisk$smoke,
    "antihtn_now" = ifelse(atrisk$antihtn == 1 &
                             atrisk$htn_age < age, 1, 0)
  )

    # ar = data.frame(
    #   "intercept" = 1,
    #   "cad.prs" = atrisk$cad.prs,
    #   "sex" = atrisk$f.31.0.0,
    #   "smoke" = atrisk$smoke,
    #   "antihtn_now" = ifelse(atrisk$antihtn == 1 &
    #                           atrisk$htn_age < age, 1, 0),
    #   "statin_now" = ifelse(atrisk$statin == 1 &
    #                            atrisk$statin_age < age, 1, 0)
    #   )
  dat = data.frame(
    "id" = atrisk$identifier,
    "mysex" = as.factor(atrisk$sex),
    "myage" = rep(age, nrow(atrisk)),
    "mysbp" = atrisk$sbp,
    "mytreat" = ifelse(atrisk$antihtn == 1 &
                         atrisk$htn_age < age, 1, 0),
    "mysmoking" = atrisk$smoke,
    "mydiabetes" = ifelse(atrisk$Dm_0_Any == 2 &
                            atrisk$Dm_0_censor_age < age, 1, 0),
    "mytotalchol" = atrisk$choladj,
    "myhdl" = atrisk$hdladj,
    "Race" = atrisk$Race,
    "mystatnow" = ifelse(atrisk$statin == 1 &
                           atrisk$statin_age < age, 1, 0)
  )
  
  mso = compute_prediction_product_matrix(
    coefmat = coefs,
    atrisk = ar,
    agepredinterval = c(start:stop)
  )
  
  df_updated$ms = mso$PredictedIntervalrisk
  f = fun.frs_30y(
    dat,
    id = "id",
    sex = "mysex",
    age = "myage",
    sbp = "mysbp",
    treat = "mytreat",
    smoking = "mysmoking",
    diabetes = "mydiabetes",
    totalchol = "mytotalchol",
    hdl = "myhdl"
  )
  df_updated$frs.30y = f$frs_orig
  df_updated$frs.recal = f$frs_recali
  ### return matrix of smoothed coefficeints
  #library(purrr)
  
  rm(atrisk)
  
  #require(pROC)
  df_updated$outcome = ifelse(df_updated$Cad_0_Any == 2 &
                                df_updated$Cad_0_censor_age < stop,
                              1,
                              0)
  #d = df_updated[round(phenos.enrollment, 0) == age, ]
  d = df_updated
  
  #d=d[!is.na(d$ascvd_10y_accaha),]
  aucmat[z, 1] = roc(d$outcome ~ d$ms)$auc
  semat[z, 1] = sqrt(var(roc(d$outcome ~ d$ms)))
  aucmat[z, 2] = roc(d$outcome ~ d$frs.30y)$auc
  semat[z, 2] = sqrt(var(roc(d$outcome ~ d$frs.30y)))
  aucmat[z, 3] = roc(d$outcome ~ d$frs.recal)$auc
  semat[z, 3] = sqrt(var(roc(d$outcome ~ d$frs.recal)))
  aucmat[z, 4] = roc(d$outcome ~ d$cad.prs)$auc
  semat[z, 4] = sqrt(var(roc(d$outcome ~ d$cad.prs)))
  
  
  roc1 <- roc(d$outcome, d$ms)
  roc2 <- roc(d$outcome,d$frs.recal)
  roc3 <- roc(d$outcome,d$cad.prs)
  pmat[z, 1] = roc.test(roc1,roc2)$p
  pmat[z, 2] = roc.test(roc1,roc3)$p
 
  
  require(PRROC)
  fg <- d$ms[d$outcome == 1]
  bg <- d$ms[d$outcome == 0]
  
  prcmat[z, 1] = pr.curve(scores.class0 = fg, scores.class1 = bg)$auc.integral
  
  #semat[i,1]=roc(d$outcome~d$ms)$se
  
  #require(PRROC)
  fg <- na.omit(d$frs.30y[d$outcome == 1])
  bg <- na.omit(d$frs.30y[d$outcome == 0])
  prcmat[z, 2] = pr.curve(scores.class0 = fg, scores.class1 = bg)$auc.integral
  
  
  fg <- na.omit(d$frs.recal[d$outcome == 1])
  bg <- na.omit(d$frs.recal[d$outcome == 0])
  prcmat[z, 3] = pr.curve(scores.class0 = fg, scores.class1 = bg)$auc.integral
  
  fg <- na.omit(d$cad.prs[d$outcome == 1])
  bg <- na.omit(d$cad.prs[d$outcome == 0])
  prcmat[z, 4] = pr.curve(scores.class0 = fg, scores.class1 = bg)$auc.integral
  
 
  
  print(paste0("Completedforage", age))
  
  
  
}

saveRDS(aucmat,file = "../output/auc_30year.rds")
saveRDS(prcmat,file = "../output/prc_30year.rds")

rownames(aucmat) = enrollments
colnames(aucmat)=c("MSLT","FRS30","FRSRC","PRS")
m = melt(aucmat, id.vars = "Age")
names(m) = c("Age", "Model", "AUC")
m$Model = as.factor(m$Model)

levels(m$Model)[1] = "MSGene"
levels(m$Model)[2] = "FRS30y"
levels(m$Model)[3] = "FRS30recal"
levels(m$Model)[4] = "PRS only"
m$se=as.vector(semat)

aucplot <-
  ggplot(m, 
    aes(x = Age, y = AUC, color = Model, ymin = AUC + se, ymax = AUC - se))+ geom_point() +
  geom_line(aes(group = Model, color = Model), linewidth = 3) +
  geom_pointrange() +
  ylim(0.5, 0.8) +
 theme_classic() + ggtitle(paste0("Lifetime risk prediction, ROC AUC"))

library(pROC)
rownames(prcmat) = enrollments
colnames(prcmat)=c("MSGene","FRS30y","FRS30.recal","PRSonly")
prcmat=prcmat[,c("MSGene","FRS30y","FRS30.recal","PRSonly")]
m = melt(prcmat, id.vars = "Age")

names(m) = c("Age", "Model", "AUC")
m$Model = as.factor(m$Model)

levels(m$Model)[1] = "MSGene"
levels(m$Model)[2] = "FRS30y"
levels(m$Model)[3] = "FRS30.recal"
levels(m$Model)[4] = "PRSonly"
m$se=as.vector(semat[,c(1,2,3,4)])/10
#m = m[m$Model %in% c("MSGene", "FRS30y", "PRS only"), ]
prplot <-
  ggplot(m, 
    aes(x = Age, y = AUC, color = Model, ymin = AUC + se, ymax = AUC - se))+ geom_point() +
  geom_line(aes(group = Model, color = Model), linewidth = 3) +
  geom_pointrange() +
 
 theme_classic() + ggtitle(paste0("Lifetime risk prediction, Precision Recall"))


ggarrange(aucplot,prplot,common.legend = T,legend = "right")


figure <- ggarrange(aucplot + rremove("ylab") + rremove("xlab"), prplot + rremove("ylab") + rremove("xlab"), # remove axis labels from plots
labels = NULL,
ncol = 2, nrow = 1,
common.legend = TRUE, legend = "top",
align = "hv",
font.label = list(size = 10, color = "black", face = "bold", family = NULL, position = "top"))
fig=annotate_figure(figure, left = textGrob("AUC", rot = 90, vjust = 1, gp = gpar(cex = 1.3)),
bottom = textGrob("Age at Calculation", gp = gpar(cex = 1.3)))

ggsave(fig,file="../output/jointaucprc_30.pdf",dpi=600,width=10,height=8,units = "in")

enrollments = c(41:70)
aucmat = matrix(NA, nrow = length(enrollments), ncol = 3)
prcmat = matrix(NA, nrow = length(enrollments), ncol = 3)
semat = matrix(NA, nrow = length(enrollments), ncol = 3)


## return a matrix of coefficients over all ages for a given state to state transition
b=coefplotsmooth2(ages = ages,start = "Health",stop = "Cad",modelfit = modelfit,window_width = 20,span = 0.75,degree = 2)
ggplotly(b$plot)
coefs=b$custom_smooth



for (z in 1:length(enrollments)) {
  age = enrollments[z]
  start = age
  stop = min(age+10,80)
  #stop=80
  df_frame = test
  atrisk = df_frame[age < Cad_0_censor_age &
                      age < Ht_0_censor_age &
                      age < HyperLip_0_censor_age &
                      age < Dm_0_censor_age & smoke == 0 , ]
  
  
  df_updated = data.frame(atrisk)
  ar = data.frame(
    "intercept" = 1,
    "cad.prs" = atrisk$cad.prs,
    "sex" = atrisk$f.31.0.0,
    "smoke" = atrisk$smoke,
    "antihtn_now" = ifelse(atrisk$antihtn == 1 &
                             atrisk$htn_age < age, 1, 0)
  )

    # ar = data.frame(
    #   "intercept" = 1,
    #   "cad.prs" = atrisk$cad.prs,
    #   "sex" = atrisk$f.31.0.0,
    #   "smoke" = atrisk$smoke,
    #   "antihtn_now" = ifelse(atrisk$antihtn == 1 &
    #                           atrisk$htn_age < age, 1, 0),
    #   "statin_now" = ifelse(atrisk$statin == 1 &
    #                            atrisk$statin_age < age, 1, 0)
    #   )
  dat = data.frame(
    "id" = atrisk$identifier,
    "mysex" = as.factor(atrisk$sex),
    "myage" = rep(age, nrow(atrisk)),
    "mysbp" = atrisk$sbp,
    "mytreat" = ifelse(atrisk$antihtn == 1 &
                         atrisk$htn_age < age, 1, 0),
    "mysmoking" = atrisk$smoke,
    "mydiabetes" = ifelse(atrisk$Dm_0_Any == 2 &
                            atrisk$Dm_0_censor_age < age, 1, 0),
    "mytotalchol" = atrisk$choladj,
    "myhdl" = atrisk$hdladj,
    "Race" = atrisk$Race,
    "mystatnow" = ifelse(atrisk$statin == 1 &
                           atrisk$statin_age < age, 1, 0)
  )
  
  mso = compute_prediction_product_matrix(
    coefmat = coefs,
    atrisk = ar,
    agepredinterval = c(start:stop)
  )
  
  df_updated$ms = mso$PredictedIntervalrisk
  
  
pce.reverse.tenyear = 
      compute_CVrisk2(
        df = dat,
        scores = "as2",
        age = "myage",
        gender = "mysex",
        race = "Race",
        totchol = "mytotalchol",
        sbp = "mysbp",
        hdl = "myhdl",
        bp_med = "mytreat",
        diabetes = "mydiabetes",
        smoker = "mysmoking",
        lipid_med = "mystatnow"
      )$as2
    

  df_updated$pceten = pce.reverse.tenyear
 
  rm(atrisk)
  
  #require(pROC)
  df_updated$outcome = ifelse(df_updated$Cad_0_Any == 2 &
                                df_updated$Cad_0_censor_age < stop,
                              1,
                              0)
  #d = df_updated[round(phenos.enrollment, 0) == age, ]
  d = df_updated
  
  #d=d[!is.na(d$ascvd_10y_accaha),]
  aucmat[z, 1] = roc(d$outcome ~ d$ms)$auc
  semat[z, 1] = sqrt(var(roc(d$outcome ~ d$ms)))
  aucmat[z, 2] = roc(d$outcome ~ d$pceten)$auc
  semat[z, 2] = sqrt(var(roc(d$outcome ~ d$pceten)))
  aucmat[z, 3] = roc(d$outcome ~ d$cad.prs)$auc
  semat[z, 3] = sqrt(var(roc(d$outcome ~ d$cad.prs)))

  
  pmat[z, 1] = roc(d$outcome ~ d$ms)$auc
  

  
  
  require(PRROC)
  fg <- d$ms[d$outcome == 1]
  bg <- d$ms[d$outcome == 0]
  
  prcmat[z, 1] = pr.curve(scores.class0 = fg, scores.class1 = bg)$auc.integral
  
  #semat[i,1]=roc(d$outcome~d$ms)$se
  
  #require(PRROC)
  fg <- na.omit(d$pceten[d$outcome == 1])
  bg <- na.omit(d$pceten[d$outcome == 0])
  prcmat[z, 2] = pr.curve(scores.class0 = fg, scores.class1 = bg)$auc.integral
  
 
  fg <- na.omit(d$cad.prs[d$outcome == 1])
  bg <- na.omit(d$cad.prs[d$outcome == 0])
  prcmat[z, 3] = pr.curve(scores.class0 = fg, scores.class1 = bg)$auc.integral
  
 
  
  print(paste0("Completedforage", age))
  
  
  
}

saveRDS(aucmat,file = "../output/aucten.rds")
saveRDS(prcmat,file = "../output/prcten.rds")

rownames(aucmat) = enrollments
m = melt(aucmat, id.vars = "Age")
names(m) = c("Age", "Model", "AUC")
m$Model = as.factor(m$Model)

levels(m$Model)[1] = "MSGene Ten"
levels(m$Model)[2] = "PCE Ten"
levels(m$Model)[3] = "PRS only"
m$Age=enrollments
m$se=as.vector(semat)
#m = m[m$Model %in% c("MSGene", "FRS30y", "PRS only"), ]
aucplot <-
  ggplot(m, 
    aes(x = Age, y = AUC, color = Model, ymin = AUC + se, ymax = AUC - se))+ geom_point() +
  geom_line(aes(group = Model, color = Model), linewidth = 3) +
  geom_pointrange() +
  ylim(0.5, 0.8) +
 theme_classic() + ggtitle(paste0("10-year risk prediction, ROC AUC"))

library(pROC)
rownames(prcmat) = enrollments
colnames(prcmat)=c("MSGene 10 year","PCE","PRSonly")

m = melt(prcmat, id.vars = "Age")

names(m) = c("Age", "Model", "AUC")
m$Model = as.factor(m$Model)

levels(m$Model)[1] = "MSGene 10year"
levels(m$Model)[2] = "PCE 10y"
levels(m$Model)[3] = "PRSonly"
m$se=as.vector(semat[,c(1,2,3)])/2

prplot <-
  ggplot(m, 
    aes(x = Age, y = AUC, color = Model, ymin = AUC + se, ymax = AUC - se))+ geom_point() +
  geom_line(aes(group = Model, color = Model), linewidth = 3) +
  geom_pointrange() +
 
 theme_classic() + ggtitle(paste0("10-year risk prediction, Precision Recall"))


ggarrange(aucplot,prplot,common.legend = T,legend = "right")


figure <- ggarrange(aucplot + rremove("ylab") + rremove("xlab"), prplot + rremove("ylab") + rremove("xlab"), # remove axis labels from plots
labels = NULL,
ncol = 2, nrow = 1,
common.legend = TRUE, legend = "top",
align = "hv",
font.label = list(size = 10, color = "black", face = "bold", family = NULL, position = "top"))
fig=annotate_figure(figure, left = textGrob("AUC", rot = 90, vjust = 1, gp = gpar(cex = 1.3)),
bottom = textGrob("Age at Calculation", gp = gpar(cex = 1.3)))
library(ggpubr)
library(gridExtra)
library(grid)
ggsave(fig,file="../output/auctenyear.pdf",dpi=600,width = 10,height=8)

ggarrange(aucplot,ga,nrow = 2,labels=c("A"))

Session information

sessionInfo()