======================================================================== HSEM 3010 Spring 2007 Clinical Trials Assignment 2 Due: February 7, 2007 Answer the following for the paper of Shlay, Chaloner et al. on Acupuncture and Amitriptyline for HIV-Related Peripheral Neuropathy (J. Amer Med Assoc 1998: v. 280, pp 1590-1595). 1. What was the medical problem that was being addressed by this study ? Peripheral neuropathy - essentially, pain in the feet and lower legs that is a complication of AIDS (also of diabetes mellitus). 2. Were the investigators trying to study two different treatments at the same time? What are advantages of doing this? Disadvantages? Yes - they wanted to study both acupuncture and amitriptyline as treatments for peripheral neuropathy. The design was essential a factorial design, with most patients being randomized on both factors. The advantage of doing this is that you may obtain information from one clinical trial that might be equivalent to what you would get from two clinical trials of the same size. This is also likely to save time and other expenses. You may also get information about how the two interventions work together. A disadvantage is that the two interventions could act against each other, weakening your ability to see the effects of either one. This is the problem of interaction. 3. Was there good reason to think that either of the two treatments might be effective? There were certainly proponents of both treatments. There have been many reports of the use of acupuncture for pain in a wide variety of conditions. Amitriptyline is "frequently prescribed for neuropathic pain" associated with diabetes, hereditary neuropathies, and toxic neuropathies. 4. The acupuncture aspect of this trial was 'placebo-controlled'. Was it double-blind? Single blind? What are some of the issues about this? Two kinds of acupuncture were used: the 'active' acupuncture was given at needle-insertion sites which were believed to be effective in relieving foot and leg pain. The 'placebo' sites were chosen as needle-insertion sites which were not believed to be effective. The patient was not told whether the sites were 'active' or 'placebo'. The acupuncturists knew what the treatment assignment was. I would describe this as single-blind, though that is not completely accurate, because the pain questionnaire was administered by someone who did not know the treatment assignment. One issue is that the the person doing the acupuncture knew which kind of treatment was being given. He/she could have conveyed that knowledge overtly, inadvertently or subconsciously to the patient. The patients might also have done a little research on their own and found out whether the sites they were assigned to would be expected to be effective. Another issue is that the patients knew that they might be assigned to placebo sites, which would not be expected to relieve their pain. This does not emulate ordinary practice, where patients are NOT given sham procedures or sham drugs. Another issue is that the sites might have been chosen "incorrectly" - see also below. 5. Some people are already convinced that acupuncture works. Others are convinced that it does not. What do you think? If you were already convinced one way or the other, would it be ethical for you to conduct this clinical trial? I don't know if it works. I have never tried it. The underlying theory does not make sense to me in the same way that standard descriptions of nerve pathways and pain transmission make sense. I think it might be ethical in this case to conduct a clinical trial even if you think you know the answer. The reason is, there may not be other effective treatments. Acupuncture is relatively benign - doesn't cause a lot of pain in itself and at least some doctors would say it doesn't do permanent damage - [note: this is debatable - ] so even if it doesn't work, you may not have done harm to the patient. If it does work, you will have benefitted the patient who would not have received this benefit if the trial had not been done, and you benefit future patients also. The people in the placebo group are presumably no worse off than they would have been before. So I think it would be ethical. Using amitriptyline is a different matter. It is addictive. If it doesn't work, you might be subjecting some people needlessly to amitriptyline addiction. So if you were convinced that it doesn't work, you would be exposing some people to a drug addiction. Plus like any drug, it has side effects. 6. How do you interpret the confidence intervals given in the 'Results' section of the Abstract (p. 1590) ? Is the evidence for an effect stronger for acupuncture or for amitriptyline? At 6 weeks, the confidence interval for acupuncture was (-.11, +.12), centered almost exactly on 0 (no effect). The confidence interval for amitriptyline was (-.22, .08), somewhat more lopsided toward the amitriptyline-is-good direction. But the p-value for amitriptyline was p = 0.38, which indicates very weak evidence. At 14 weeks the situation was essentially the reverse, tending to favor acupuncture, but still not close to the magic p < 0.05). So to me it looks like a wash. It is worth noting that this clinical trial, like any other, does not actually "prove" anything. The conclusion is essentially a probability statement: in this case, the conclusion is, the investigators do not reject the null hypothesis. The probability that, if the alternative hypothesis were true, they would have seen the results that they actually got, is low. 7. The investigators randomized 250 patients. What was their basis for doing this? They had a target sample size of 260 patients, based on a pain-score difference of 0.20 favoring the treatment, a 90% power, and a signif. level of alpha = 0.05. 8. At the end of the Statistical Analysis section, the text says "in February 1997, the monitoring board recommended closing the study because it concluded that the results were definitive for both acupuncture and amitriptyline." What does that mean, "definitive"? Evidently they thought it was clear that the study was going to turn out negative for both interventions. That is, they did not expect to see a significant difference of the kind originally hypothesized in the alternative hypothesis (see above). The word "definitive" was maybe not the best choice - I think what they meant was, it was clear that if the study went the whole planned length, it was very unlikely, perhaps impossible, to reject the null hypothesis. 9. How did the investigators choose the points at which the acupuncture needles were to be inserted? Not entirely clear. I think they chose some points that they thought were effective (for the 'active' group) and some others that they thought were not effective (for the placebo group) , and then they had these reviewed by a panel of 8 acupuncturists. And apparently the panel agreed with what was proposed. But they don't describe the expertise and qualifications of the 8 consultants. 10. Why isn't amitriptyline sold over the counter? It is addictive. It is an antidepressant and may have side-effects in some people. 11. State the main conclusions of this trial in your own words. There is not convincing evidence that neither acupuncture nor amitriptyline, used as described in this study, is effective in relieving pain associated with the peripheral neuropathy of the lower extremity that is seen in some AIDS patients. 12. What do you think the consequences of this trial were for patients and for medical practice? Patients and their physicians will need to find other ways to relieve pain. They might try acupuncture and see if it works. If it does, fine. If it doesn't, they can abandon it and try something else. As for amitriptyline: this is a prescription drug, widely used, so patients cannot get it in the U.S. without a prescription. They probably could get it in Mexico. It is less likely to be prescribed than previously, though the study may not have made much difference. 13. See the Letters to the Editor and the authors' responses. What issues are raised by the letter writers? Do you agree with what they say? Some of the letter writers thought that the wrong acupuncture points were selected for the 'active' therapy. Others thought that it might have worked better with electrical stimulation. One writer thought the amitriptyline dose was too low. I don't know if these people might be right. 14. How might the investigators have decided on a trial design which would have addressed the letter writers' concerns? They might have consulted a wider panel of recognized experts in acupuncture. They might have added a third kind of acupuncture, with electrical stimulation. However any addition of other treatments or doses implies that the sample size and probably the length of the study would increase. 15. Do you think the letter writers would have said the same things about the acupuncture points if the results had turned out in favor of acupuncture? No! They would have either said nothing or they would have praised the trial for proving the efficacy of acupuncture! That's my guess, anyway. 16. The 239 patients in the acupuncture part of the study all knew they were getting some kind of acupuncture. That is, among them there was really no 'pure control' group of people who didn't get any kind of acupuncture. In actual medical practice, patients get either acupuncture treatment which is believed to be effective, or none at all - they do not get an acupuncture treatment which is NOT believed to be effective. Do you think the investigators should have included a 'pure control' group? Maybe. But by its very nature, the outcome of this trial was a subjective assessment. There could have been a strong 'placebo' effect which would have made the pure no-treatment group look worse. On balance, I think they were right to try to blind participants to the treatment. 17. Some people have done studies of acupuncture to relieve pain in dogs and horses. Is it possible to carry out the evaluations so as to rule out the possibility that the investigator's bias can affect the results? You could have the pain assessment done by someone who was not allowed to see what treatment was used. This might be difficult. First, because in a small clinic, people might talk about the treatment. Second, because presumably the pain relief is greatest while the treatment is being given, so the evaluator would see the treatment being given. Third, it is not clear how dogs and horses convey their impressions of pain, since they cannot talk. You might also try to somehow monitor the nerve impulses in the brain in regions which are related to the site of the pain. 18. If you were suffering from peripheral neuropathy, would you want to try acupuncture? In light of this study, should your insurance company or HMO have to pay for it? Same questions for amitriptyline. I might try it briefly. I don't see much rationale for thinking it has a long-acting effect (after the needles are removed), so if it didn't have an immediate effect, I would probably not continue it. Unless there are other carefully done objective clinical trials showing a benefit, I think an insurance company or an HMO would be justified in deciding not to pay for it. 19. Do you think there are other alternatives to the two treatments tested in this trial? Yes: ordinary painkillers - aspirin, ibuprofen, Alleve, tylenol, lidocaine[?], barbiturates, morphine and related drugs (oxycodon). At least some of these may not be effective. At least some of them are more addictive than amitriptyline. 20. Do you have any other comments on this clinical trial? It was an admirable example of an attempt to assess acupuncture somewhat objectively. However overall, in my view, it is not all that convincing. Other forms of acupuncture might work. Amitriptyline at a higher dose might work. ========================================================================