An SR of AEs 2021

Inspired by Bäumler  et al 2021.[1]

SR – systematic review
AE(s) – adverse event(s)
SAE(s) – serious adverse event(s)
RoB – risk of bias

key to acronyms

I was a bit surprised to see this big AE review full of forest plots in BMJ Open last week. This is the first review of adverse events that has included a form of meta-analysis of prospective studies, and the first with an RoB assessment.

Prospective research of AEs was begun with the SAFA study in the 90s.[2] I had just started out as a medical acupuncturist in private practice in Plymouth. I had taken over a practice from Adrian White, who was first author of the study, so naturally I joined in with 77 other colleagues from the BMAS and the AACP and reported all my AEs each month.

A very similar study was conducted within the traditional acupuncture community at about the same time.[3] Adrian had invited them to join SAFA, but it was deemed safer to do a separate study, and both were published together in the BMJ, with an accompanying editorial by Charles Vincent in 2001.[4]

There was something subtly different about either the way practitioners were asked to report AEs, or a systematic difference in assessment of what constitutes an AE between the two different communities of practitioners. This resulted in the rate of AEs reported being more than twice the rate in one study than the other. Now imagine what effect that would have when the data is pooled in meta-analysis…

Well, this review included not only these two studies of course, but 19 others, so there were plenty of opportunities for differences to arise. Consequently, the data pooling was crippled by statistical heterogeneity, not only for the total AE estimates, but also for the SAEs. You might have thought the latter would be consistently low and rather similar, but no!

Adrian wrote his MD thesis on safety, and I remember him remarking about the different rates of reporting depending on whether you ask the practitioners or the patients to report the AEs. The former tended to under report relative to the latter, who in turn may be more prone to attribution errors.

The figure we are most interested in is that for SAEs, principally because we must discuss this when obtaining consent from new patients. Since the huge Taiwanese retrospective cohort study on pneumothorax,[5] which I highlighted on this blog a while ago, I have tended to use the figure 1.75 per million (treatments in at risk areas) when discussing the risk of pneumothorax – the most common of the preventable SAEs.

So, I was a little alarmed to see a figure of 1.01 per 10 000 patients in the abstract of this review as the reported estimate of SAEs. This would mean that the consent form we published in 2001 was at risk of having its longevity curtailed,[6] as it includes the line:

Serious side effects are very rare – less than one per 10,000 treatments

I needed to take a closer look, and fortunately the paper is very comprehensive in reporting the data, so it did not take long to establish that most of the SAEs included were either completely unrelated or very unlikely to be the result of acupuncture.

Even the figure for SAEs related to acupuncture was a little high, coming in at either 2.57 or 5.45 per million. There were still SAEs included here that in my estimation were either unlikely to be caused by acupuncture or misclassified as serious in the first place.

For example, one very large study reported all serious medical events that occurred in patients during the 6-month period of the research irrespective of any causal link to acupuncture.[7] As far as I can see, none of the SAEs in this study were clearly caused by acupuncture.

A paper with a high rate of SAEs included patient reported AEs that they considered serious.[8] One of these was a car crash without injury 2 days after acupuncture in a patient who had stayed up very late the night before the crash talking to her boyfriend, who had not previously known about the condition (alopecia) for which she was being treated.

In summary, whilst this is a very comprehensive review, I am unconvinced that the pooled estimates are useful, and on the face of it could be quite misleading to the casual reader who is not prepared to scratch beneath the surface of this rather novel review.

References

1          Bäumler P, Zhang W, Stübinger T, et al. Acupuncture-related adverse events: systematic review and meta-analyses of prospective clinical studies. BMJ Open 2021;11:e045961. doi:10.1136/bmjopen-2020-045961

2          White A, Hayhoe S, Hart A, et al. Survey of Adverse Events following Acupuncture (Safa): A Prospective Study of 32,000 Consultations. Acupunct Med 2001;19:84–92. doi:10.1136/aim.19.2.84

3          MacPherson H, Thomas K, Walters S, et al. The York acupuncture safety study: prospective survey of 34 000 treatments by traditional acupuncturists. BMJ 2001;323:486–7. doi:10.1136/bmj.323.7311.486

4          Vincent C. The safety of acupuncture. BMJ 2001;323:467–8. doi:10.1136/bmj.323.7311.467

5          Lin S-K, Liu J-M, Hsu R-J, et al. Incidence of iatrogenic pneumothorax following acupuncture treatments in Taiwan. Acupunct Med 2019;37:332–9. doi:10.1136/acupmed-2018-011697

6          White A, Cummings M, Hopwood V, et al. Informed consent for acupuncture–an information leaflet developed by consensus. Acupunct Med 2001;19:123–9.

7          Endres HG, Molsberger A, Lungenhausen M, et al. An internal standard for verifying the accuracy of serious adverse event reporting: the example of an acupuncture study of 190,924 patients. Eur J Med Res 2004;9:545–51.

8          Macpherson H, Scullion A, Thomas KJ, et al. Patient reports of adverse events associated with acupuncture treatment: a prospective national survey. Qual Saf Health Care 2004;13:349–55. doi:10.1136/qhc.13.5.349

---

Declaration of interests MC