Inspired by He et al 2022 and Gang et al 2022.[1,2]
RCTs – randomised controlled trialskey to acronyms
SR – systematic review
CAM – complementary and alternative medicine
CAIM – complementary alternative and integrative medicine
LBP – low back pain
I rarely refer to protocols on this blog. One exception was a blog titled: Should I read a protocol? But that was a protocol for one of the largest comparative RCTs of acupuncture to date with a rather surprising comparison of TCM and segmental approaches within it. This by contrast is a protocol for a guideline aimed at improving the quality of clinical trials of acupuncture. It sounds a bit dry, doesn’t it?
Well, the reason it caught my interest is because I have been involved in this debate for over 20 years and have been actively trying to influence research in China for the last couple. The later activity led to me being invited to join a group based in Beijing and Toronto but including all the big names in acupuncture research globally. I participated in 3 rounds of a Delphi process to identify the key methodological issues in RCTs of acupuncture, and in commenting on the text of the first paper to emerge from that activity. Despite having some big names on the paper, it has just been rejected by the BMJ. I saw the notification on the same day as these two papers appeared on PubMed. It seemed like a spooky coincidence; hence this blog is devoted to those aspects of the story that are either already in the public domain or are not confidential. For the juicier stuff you’ll have to join the blog webinar on Wednesday.
I guess the story starts for me around 25 years ago when I was working on my first SR with the help and guidance of Adrian White. We were aware of a rather vocal group from Oxford who regularly published negative comments on acupuncture research in a magazine called Bandolier. The publication was overall very good, but there seemed to be a strong inherent bias against CAM or CAIM approaches in general and in favour of the orthodoxy. Although I should note that the label CAIM had not been invented!
When this group published a highly critical SR of acupuncture RCTs in neck and back pain in the journal Pain, I was outraged. Not least because my SR had just been rejected from the same journal after several months with a one-liner from the EiC telling me that they did not publish SRs. Clearly there was something black in the dahl!
I decided to respond by writing a long critique of the Smith review and used almost the same title to get it noticed. I don’t think it did get noticed, but I had my day to respond in a public forum. This has already been detailed in a blog titled: The problem with sham.
It wasn’t long after this that Jacky Filshie organised a consensus meeting of prominent researchers from around the world to discuss the problems in acupuncture research. This resulted in us publishing the first discussion of the topic with recommendations. This paper has been cited consistently over the years, with 12 citations in 2021 – a full 20 years after first publication.
So, 20 years later we see a protocol for development of a definitive guideline coming from Chengdu. I don’t recognise any of the names involved. It seems a shame they were not included in the project that started almost 2 years ago.
So that brings me to the other paper published on the same day. This one has an intriguing title, that includes the phrase: comparing China-based vs Western-based studies. Unfortunately, the main text of the paper is in Chinese, so I am restricted to the abstract. Two of the authors come from the China Academy of Chinese Medical Sciences in Beijing, but the other appears to be based in the US and is affiliated to the American Academy of Acupuncture and Oriental Medicine (AAAOM). In 2014 he was the president of AAAOM, but now I cannot find him on their directory at all. His most recent prior publication appears to be an expert consensus discussion of the infamous JAMA Internal Medicine paper on acupuncture for chronic stable angina.[7,8] I covered this paper on the blog in July 2019. It was a multicenter trial based in Chengdu.
I used it as an example of what not to do in various presentations over the last 2 years, including one I recorded for a big research event in Beijing organized by Professor Jing, following which she told me:
Some authors related to your presentation attended the Conference. Maybe they were thinking about your comments on their papers. The methodology needs improving in the future…
So, what did Gang, Gong & Jing conclude about their comparison of studies from China and the West? There is a hint in the abstract:
We propose the following suggestions to remediate the acupuncture clinical research paradigm: First, acupuncture clinical research should be carried out step-by-step according to universal, agreed-upon research protocols. Second, norms for reporting outcome metrics need to be standardized for each stage of a study.
The comment on outcome metrics is not particularly contentious, but probably difficult to achieve globally. Universally agreed-upon protocols seems like rather a utopian ideal, but I am all for trying to achieve it.
1 He Y, Li J, Li Y, et al. Strengthening the quality of clinical trials of acupuncture: a guideline protocol. BMJ Open 2022;12:e053312. doi:10.1136/bmjopen-2021-053312
2 Gang W-J, Gong C-Z, Jing X-H. [Acupuncture randomized controlled trials: comparing China-based vs. Western-based studies]. Zhongguo Zhen Jiu Chin Acupunct Moxibustion 2022;42:3–7. doi:10.13703/j.0255-2930.20201224-k0001
3 Cummings TM, White AR. Needling therapies in the management of myofascial trigger point pain: A systematic review. Arch Phys Med Rehabil 2001;82:986–92. doi:10.1053/apmr.2001.24023
4 Smith LA, Oldman AD, McQuay HJ, et al. Teasing apart quality and validity in systematic reviews: an example from acupuncture trials in chronic neck and back pain. Pain 2000;86:119–32.
5 Cummings M. Teasing Apart the Quality and Validity in Systematic Reviews of Acupuncture. Acupunct Med 2000;18:104–7. doi:10.1136/aim.18.2.104
6 White AR, Filshie J, Cummings TM. Clinical trials of acupuncture: consensus recommendations for optimal treatment, sham controls and blinding. Complement Ther Med 2001;9:237–45. doi:10.1054/ctim.2001.0489
7 Gong C-Z, Liang F-R, Li C-H, et al. [Discussion on Acupuncture as adjunctive therapy for chronic stable angina: a randomized clinical trial published in JAMA Internal Medicine]. Zhongguo Zhen Jiu Chin Acupunct Moxibustion 2021;41:359–64. doi:10.13703/j.0255-2930.20190929-k0003
8 Zhao L, Li D, Zheng H, et al. Acupuncture as Adjunctive Therapy for Chronic Stable Angina: A Randomized Clinical Trial. JAMA Intern Med Published Online First: July 2019. doi:10.1001/jamainternmed.2019.2407