CLA Honors Seminar 3010: Clinical Trials Spring 2007 Final Exam : Scores and Answer Key --------------------------------------------- Final Exam Scores: 72 84 86 86 88 89 89 89 89 91 92 93 96 97 98 Final exams may be picked up in the Biostatistics Main Office, A-460 Mayo Building. Final Exam Answer Key: Below May 8, 2007 Page 1 of 5 HSEM 3010 - Spring 2007 - Final Exam Name:_________________________________ =============================================================================== This exam is based on the paper 'Corticosteroids for the Prevention of Atrial Fibrillation After Cardiac Surgery', J. Halonen, P. Halonen, et al., JAMA 297: 1562-1567, 2007. Note: some of the questions may not be answerable from the information in the paper. You should comment on this when you respond to Question 20. 1. What medical question was being addressed by this study? Prevention of atrial fibrillation (AF) after first cardiac surgery. 2. What were the eligibility requirements? Patients undergoing first cardiac surgery (CABG, aortic valve replacement) with no previous AF or flutter, cardiac bypass surgery. Age, 30-85. Various medical exclusions. 3. What was the primary endpoint? Why is it an important endpoint? Occurrence of AF within 84 hours after surgery. It is important because it is associated with stroke and increased length of hospitalization and increased costs of medical care. 4. What was the null hypothesis? Both groups (steroid vs. placebo) would have the same probability of having AF within 84 hours post-surgery. 5. What was the alternative hypothesis? Steroid group: 15% would have AF Placebo group: 30% would have AF May 8, 2007 Page 2 of 5 HSEM 3010 - Spring 2007 - Final Exam Name:_________________________________ =============================================================================== 6. What is a p-value? Explain in your own words. It is the probability of observing an outcome as extreme as, or more extreme, than that which actually was observed in a given study, assuming that the null hypothesis is true. Or: It is the probability of observing results as extreme as those which actually were observed, or more so, assuming that the results are strictly due to chance and the two groups being compared have the same true distribution of values. 7. The paper (p. 1564) says that this clinical trial was planned to have 80% power. What does that mean? Explain in your own words, not from the definition in the notes - pretend you are trying to explain it to a very bright high-school student. Power is the probability, assuming the alternative hypothesis (30% versus 15%, for example) of observing a statistically significant difference between the groups, when the study is completed. 8. Was this study stopped early? If so, why? If not, should it have been ? Not stopped early - they randomized 241 people, and were planning to enter 240. Should it have been? Probably not, though the final p-value (p = .003) and the difference in the rates of AF (48% in the placebo group, 30% in the steroid group) indicated a fairly strong difference in favor of the steroid group. The study was not of long duration, and not many serious side effects were reported (one death in each group). 9. Examine Table 1. What is the meaning of the "Univariate P Value" column? That is the probability that the two groups would be as different (or more so) than the observed difference, assuming the "true" distribution of the specified variable is not different between the two groups. 10. If one of the p-values in Table 1 had been very small - for example, if the p-value for age had been p < 0.03, with the placebo group having older mean age - would that mean that the results of the clinical trial were not valid? Would it mean that randomization did not work? Note that there are 14 characteristics being compared in Table 1. Make a guess about how likely it is that one or more of the characteristics in Table 1 would have a p-value less than 0.03. p < 0.03: This would not invalidate the study, but it should be taken into account in a secondary analysis in which age is entered as a covariate. How likely? Don't know - maybe 30-40%. May 8, 2007 Page 3 of 5 HSEM 3010 - Spring 2007 - Final Exam Name:_________________________________ =============================================================================== 11. What was the primary method of statistical analysis? Kaplan-Meier time-to-event analysis using a chi-square test probably based on either the log-rank test (although the latter is not explicitly stated, and there is another chi-square test that might have been used). 12. What were the main findings of this clinical trial? The Steroid group did better: 30% had AF, whereas in the Placebo group, 48% had AF (within 84 hours); p = 0.003. May 8, 2007 Page 4 of 5 HSEM 3010 - Spring 2007 - Final Exam Name:_________________________________ =============================================================================== 13. What does Figure 2 tell you? It indicates a strong and consistent difference between the two groups in the incidence of AF, with the placebo group having a higher incidence of AF than the steroid group. 14. What does Figure 3 tell you? It indicates that 3 different studies have found very similar results with regard to steroids preventing AF in cardio-surgery patients. 15. On page 1567, the authors say, "Although we found that intravenous administration of hydrocortisone was well tolerated, our study was underpowered to assess the safety of corticosteroid therapy." What does this mean? Since safety of a therapy is a key question, and they carried out a study which was not large enough to answer that question, was it ethical to carry out this study? It means that there might be a clinically significant difference between the two groups in some serious side effect - death, for example - and their study was too small to "prove" it one way or the other. They showed that steroid treatment reduces atrial fibrillation. Atrial fibrillation in itself is basically an annoying symptom, but it is associated with stroke and mortality and increased length of hospital stay. In that sense it is a surrogate for more serious events. It may be that there is some rare but very serious side effect of steroids such that one occurrence of that side effect would outweigh, say, 20 occurrences of AF. This study did not have sufficient power to show whether there might be a difference between the groups in such a side effect. Was it ethical to do the study? At this point it is just speculation that there might be a serious but uncommon side effect of the treatment. We may never know, because, given the results of this study, it may be regarded as unethical to do another (larger) one like it. On balance, I think it was ethical to do this study as planned, and given the p-value at the end (0.003), it probably would not have seemed ethical to continue it up to a large enough sample size to rule out serious-but-rare side effects. May 8, 2007 Page 5 of 5 HSEM 3010 - Spring 2007 - Final Exam Name:_________________________________ =============================================================================== 16. Was there a data and safety monitoring board? If not, should there have been one? As far as I can tell, there was no DSMB. There very definitely should have been one. 17. Was this clinical trial stopped before the planned time of completion? If so, why? And if not, should it have been? See also Q. 8. No, it looks like they continued until they achieved the target sample size. Should it have been stopped earlier? A very difficult question. They should have (but did not) established monitoring boundaries to provide a guide for their (nonexistent) DSMB. If the results crossed the monitoring boundaries, the DSMB would have considered earlier termination, and some patients would not have been subjected to the inferior treatment. We do not know now what the interim data might have shown. 18. Is AF (atrial fibrillation) a surrogate endpoint? If so, a surrogate for what? Yes, as noted above - it may be a relatively minor problem in itself, but it is "associated with" stroke and probably other problems, including sudden cardiac death. 19. What do you think will happen as a result of this clinical trial? What SHOULD happen? I think use of steroids with cardiac surgery will be more widely accepted, and that is what SHOULD happen. However it is also possible that someone would argue that a larger definitive study should be done, since AF is a surrogate. 20. What do you see as some reasons for criticizing this study? There was apparently no DSMB and no guidelines for early termination of the study. The main endpoint could be regarded as a surrogate for more serious problems. The authors do not provide justification for restricting the occurrence of the endpoint (AF) to the first 84 hours after surgery.