Post

Very interesting talk by @mendelrandom.bsky.social on the apparent effect of shingles vaccines on dementia. Clear discontinuity from shingles vaccines on shingles, but not on dementia, in this much larger follow up in England, using similar analytic choices as the Wales study that showed a benefit.

Worth noting that many (not all) of the observational studies so far have been funded by GSK, which sells the shingles vaccine. Vaccines are great and the shingles vaccine is also great (for shingles), but people have got to be a little more skeptical of observational studies funded by pharma IMHO.

TBH it just shows how hard this stuff is to do, and the many traps that are easy to fall into when working with these types observational data - where you have no control over the data generating process. This seems to be “kicking the tyres” on SAIL findings, which is how it should be.

People also need to think about how the extremely expensive access fees for SAIL and related systems mean only pharma has access to the relevant data, with all but the best funded public researchers shut out.

Can you say more about the data sources, Thomas? I was curious about this!

P-hacking is a greater risk when data have already been collected. Pharma has led the way in the clinical trials arena to avoid p-hacking (pre-specification in protocols, SAPs, now estimands) for prospectively collected data. P-hacking using retrospective data is a much wider problem than Pharma.

I'm not disparaging pharma, but they (like academics) need to be within an incentive framework that encourages good behaviour - and I worry that the trusted data environments for observational work are not that.

Academic studies on observational data also tend to suffer from false positives of course but it's tens of thousands of pounds to run even the simplest analysis so they won't be able to run lots of associations, it will be more forking paths type problems (I conjecture).