Stimulated by Tao et al 2023.[1]
TTH – tension-type headache
key to acronyms
CTTH – chronic tension-type headache
SR – systematic review
TSA – trial sequential analysis
RCT – randomised controlled trial
SMD – standardised mean difference
RR50 – responder rate based on a 50% reduction in days with headache
RIS – required information size
I picked out this paper on TTH before I wrote last week’s blog, and it just so happens that the last author on this paper is the first author from last week’s key paper. Hui Zheng was also last author on the TSA paper mentioned on the last blog and one of the reasons I chose this paper was because it has some nice clear results that I can use to explain what TSA is all about.
Hui Zheng has popped up before on the blog. The post titled Acupuncture for CTTH from July 2022 featured another of his papers – one that is included in this SR.
14 trials were included and the results are not a surprise since we have been relatively confident about the efficacy of acupuncture in TTH since the Cochrane review led by Klaus Linde in 2016.[2] But the point of a TSA is to tell us if we have done enough data to be sure,[3] and we may not have as yet for all the different outcomes.
Technically a TSA is a method for reducing the risk of type I and type II statistical errors in SRs, that is, false positive and false negative results.
A type I error occurs when a paper gives a statistically significant result by chance rather than because there is a real difference between groups. When using a cut off of p<0.05 the chance of a type I error is 1 in 20. The problem often arises when there are multiple throws of this 20-sided dice within the same analysis. This may apply to a single trial with lots of tests for statistical significance (in this case a statistical correction should be applied) or repeated SRs each adding a new paper to the pooled data and performing a statistical test each time. The latter example is where TSA comes in to alter the p value based on the amount of data available up to that point of testing.
In acupuncture research we are more used to the type II error. This occurs because of a lack of statistical power to detect the real difference, usually between one form of needling and another eg real vs sham acupuncture. A lack of statistical power relates to the size of the trial and the size of the difference between interventions. In a TSA, an estimate is made of what would be an adequate information size and this is used to determine whether the result in your SR has reached a definitive conclusion in terms of a positive effect or in terms of futility ie no point doing any more research on this because it will never be likely to show an effect.
The SR comes from Chengdu in Sichuan, which is just to the right of Tibet in the centre of China, and just above Yibin, where the Yangtze River is formed from the Jinsha Jiang and the Min Jiang according to the names on Google Maps.
Whilst the SR was conducted in China, only one of the 14 RCTs included derives from China. This attracted me somewhat, and I was pleased that I recognised most of the trials listed, in particular the larger ones from Germany.
The first pooled result in the paper includes data on TTH frequency from 8 trials of acupuncture against sham and records an SMD of -0.80. That is a large effect size, but the forest pot shows significant heterogeneity, and the TSA plot does not confirm that we have sufficient data as yet to be sure of the result.
The second pooled result considers the data in terms of responder rates (RR50) from 5 trials and records a risk ratio in favour of acupuncture over sham of 1.28. This result did not show significant heterogeneity and the Z curve of the TSA got closer to the line of definitive positive effect but did not quite reach it.
The third pooled result was for AEs, which showed no difference on the forest plot including data from 7 trials, and the Z curve on the TSA just reached the line of futility, meaning that no more data is needed to confirm that acupuncture is not statistically riskier than sham acupuncture in TTH.
In the supplement to this paper, I found the data on acupuncture compared with no acupuncture. Most of this comes from the huge ARC headache study.[4] Interestingly the Z curve for the data on TTH frequency easily crosses the RIS line showing a definitive effect on this measure. By contrast, the data on responder rate is not so clearly positive on either the forest the TSA plots.
So, what is the bottom line here? Well, to my surprise, whilst we can be fairly sure that real acupuncture is not riskier than sham in TTH, we have not yet reached the RIS to be sure that acupuncture is better than sham. Acupuncture is clearly better than no acupuncture in terms of TTH frequency, but a quirk of the statistics means we cannot say the same when looking at responder rates.
References
1 Tao Q-F, Wang X-Y, Feng S-J, et al. Efficacy of acupuncture for tension-type headache prophylaxis: systematic review and meta-analysis with trial sequential analysis. J Neurol Published Online First: 5 April 2023. doi:10.1007/s00415-023-11695-1
2 Linde K, Allais G, Brinkhaus B, et al. Acupuncture for the prevention of tension-type headache. Cochrane Database Syst Rev Published Online First: April 2016. doi:10.1002/14651858.CD007587.pub2
3 Wetterslev J, Thorlund K, Brok J, et al. Trial sequential analysis may establish when firm evidence is reached in cumulative meta-analysis. J Clin Epidemiol 2008;61:64–75. doi:10.1016/j.jclinepi.2007.03.013
4 Jena S, Witt C, Brinkhaus B, et al. Acupuncture in Patients With Headache. Cephalalgia 2008;28:969–79. doi:10.1111/j.1468-2982.2008.01640.x
The BMAS Blog 
One thought on “Acupuncture for TTH 2023”
Comments are closed.