Imputed Dataset from explore copula model

[TomRow90]

Hi Donald,

I’m using the Copula model in explore to estimate CD & CI graphs, with impute set to TRUE. I’m then using the posterior_predict function to draw samples from the PPD, & the predicted_probability function to estimate conditional probabilities (& 95% intervals) of the presence of a health condition given scores on items that were conditionally dependently associated in the copula GGM.

I’m also using the PPD samples to perform qualitative Post. Predictive Checks (PPC) by using the bayes plot package (ppc_bars) to compare the replicated data to the empirical, as well as looking at how well model simulated data reproduces item skewness.

ppc_bars however only accepts datasets of the same length, so what I’ve done at the moment is remove the entries corresponding to rows with missing data & compared the empirical to PPD samples with missing data rows removed - but I can't find a lot of information on how to best approach posterior predictive checks when data has been imputed, & as I'm not a statistician I'm not convinced this current approach is remotely legitimate.

I was therefore wondering if there is a way to get the imputed dataset from the explore object so that the complete data set (observed + imputed) can be compared to the replicate data sets sampled from the PPD?

Many thanks in advance.

Tom

[donaldRwilliams]

Hi
The reason that the data sets are not stored, is that there would be one for each iteration. This can result in a huge file size.

If I am hearing correctly, you want to see how the ppd looks when considering the imputed data as well ?

[TomRow90]

Hi Donald,

Thanks for your reply. I see, I did wonder if there was imputation for each iteration under the hood & that was why none of the imputed data sets seemed to be stored.

And yes, from what I understand the idea of posterior predictive checks is that if the model is a reasonable 'fit' we should be able to use it to generate data that is similar to the observed data set. If I have used imputation though, the datasets generated by posterior_predict are complete data sets without any NAs, so my complete empirical data set (Y) is a different length to the complete replicated data sets (Yrep).

For example, bayesplots ppc_bars requires Y and Yrep to be the same length without any missing entries. So I get an error if I try to use ppc_bars to plot Yrep against Y ("Error: NAs not allowed in 'y'").

At present, I have plotted Y (with rows containing NAs removed) against Yrep (with corresponding rows removed which were removed in Y) so they are the same length without any NAs, i.e. (excuse my code if this isn't a very efficient way of doing this):

posteriorPredictions <- posterior_predict(mod, iter=5000, progress=TRUE)

ind <- which(is.na(Y), arr.ind=TRUE)
ind <- ind[,1]
ind <- unique(ind)
Y2 <- Y[-ind, ]
yrep <- posteriorPredictions[-ind, , ]

I guess I could get round the ppc_bars requirement by just making the plots manually by plotting the histograms of the empirical and then overlaying the median and specified quantiles from the posterior_predict object, but I assumed there was a reason ppc_bars only accepts data of the same length without NAs, & that I'm missing something important about PPCs with imputed data.

I realise this is probably moving away from BGGM now and is more of a general question about Posterior Predictive Checks when you've imputed data, so I understand if this isn't the place to answer it, but if you do have any recommended reading or blogs/tutorials/links etc. that cover this I'd be grateful if you could send it my way.

[donaldRwilliams]

All good ! it is a great question. Honestly, I think removing the NAs would be fine in most cases. Certainly not ideal, but unless you have a ton of NA's, I do not see it compromising the predictive check. Note that you are still getting the check for the non-nas when impute = TRUE.

also note that if you did have 500 data sets, then you would have 500 checks to visualize and look at.

How many NAs do you have ?

[TomRow90]

Ok thanks Donald, at least I haven't done anything that sounds like too much of a 'statistical sin' then.

My dataset is p=20, n=924. Total number of NAs is 354 / 18840 ~ 0.02, and no individual variable has > 5% missing, but if I performed listwise deletion this would reduce n to 752.

[donaldRwilliams]

Well, sometimes we have to make it work..lol.. Now, ideally, we could look at the predictive distribution for a missing value if we wanted to. Typically, however, we have one outcome and here we have many, essentially. So as I mentioned, it resulted in huge object.

That is not a lot of missing data !

In a paper, perhaps just state what you said to me. Seems reasonable.

[TomRow90]

Ok great, thanks for your replies.

[donaldRwilliams]

No problem. Let me think about adding an option to store the imputed datasets.