Adaptive trial design is a hot issue in the drug development community. Adaptive conferences and
web seminars abound, and suddenly, every consultant or vendor to the industry has become an
adaptive expert. The FDA and EMEA are also much more receptive to adaptive trials than they
were a few years ago.
However, the picture is a shade different from what the industry had expected. Originally, much
of the focus was on adaptive phase III trials as well as on seamless phase II/III trials. Those are,
indeed, successfully being implemented today. But the real action is in phase II dose-finding trials
and even in Phase I trials for safety.
In this issue of Peer Perspectives in Oncology, Medelis talks with Ranganath Nayak, Ph.D., the
CEO of Cytel, Inc., about the benefits, opportunities, and challenges presented by adaptive trial
design in phase I and phase II studies.
Ranganath Nayak is the Chief Executive Officer of Cytel, Inc., a leading provider of clinical trial services and specialized statistical software for the biopharmaceutical, medical device, academic, and government research markets. Cytel has designed more adaptive trials for industry sponsors than any other service provider and in virtually every therapeutic area (including medical devices), helping pharmaceutical and biotech companies raise trial success rates while reducing development time and costs.
The methods developed at Cytel are validated and published, with their authors regularly invited to speak on a range of adaptive topics including advances in Bayesian and Frequentist methods, adaptive trial modeling and simulation, key operational areas such as flexible randomization, and drug supply forecast and management. Cytel experts also periodically provide training to FDA biostatisticians on the use of its software.
Prior to joining Cytel, Dr. Nayak was Vice President of The Boston Consulting Group working with US clients such as Gillette and Ford Motor Company. He previously spent two decades at Arthur D. Little, Inc., the technology and management consulting firm, where his consulting had a particular focus on innovation, new product development, and organizational learning. He also oversaw the Arthur D. Little School of Management.
Dr. Nayak is the co-author of “Product Juggernauts: How Companies Mobilize to Generate a Stream of Market Winners,” a Harvard Business School book on managing product development, and principal author of “Breakthroughs!,” a best-selling study of hugely successful commercial innovations such as Tagamet, the microwave oven, Federal Express, the videocassette recorder, the Compact Disc, Post It notes, the CAT scanner, and the Toyota production system.
He received a Ph.D. in Mechanical Engineering from the Massachusetts Institute of Technology
Medelis: When the industry started seriously talking about adaptive trial design a few years back, the focus was on phase III and seamless phase II/III trials. Yet it turns out that the real action today is in phase II dosefinding trials and even in phase I trials for safety. Why?
Dr. Nayak: There are several reasons adaptive trial design is gaining so much momentum in early stage studies. First, adaptive trials are most helpful when you do not know enough at the beginning of a trial to design it with confidence. This is much more likely to be true in phases I and II than in phase III.
Second, the best way to make expensive phase III trials more successful is to do more thorough work in Phases I and II. This is clear from the study commissioned by PhRMA some years ago of why drugs fail at various stages of clinical development (see table one). As David Brennan, the CEO of AstraZeneca, is reported to say repeatedly within his company, “Once in my life, I hope we will get the dose right.” Getting the dose right through well-designed phase I and phase II trials is the best way to maximize success in phase III, which then leads to a higher rate of NDAs.
In addition, the FDA’s concern about the potential for bias resulting from the unblinding of data on patient response is much less with these trials than with phase III registration trials. When it comes to phase I and II adaptive trials, regulatory acceptance is much less of a hurdle than in phase III confirmatory studies. Our experience is that you can “sell” adaptive to regulators as long as you can show that there is a really good reason for using an adaptive design, and you thoroughly evaluate several options, including a non-adaptive one, and then submit the details of your statistical work as part of your protocol submission.
Medelis: What are the most significant benefits sponsors can realize by using an adaptive design in an early stage trial?
Dr. Nayak: The benefits fall into three categories: commercial, ethical, and budgetary. The commercial benefit is that much better information is obtained on the drug and its dosage, resulting in a much greater chance that the phase III confirmatory trials will be successful, with a reduced probability that either the phase III dose will be toxic or show inadequate efficacy. This is a deferred benefit, but it is huge.
Another commercial benefit is that the time to market for a drug can be reduced by combining proof-of-concept trials (to show that the drug works) with dose-finding trials (to pick the right dose) or by combining dose-finding trials with confirmatory trials. This eliminates the “lost” time between the end of one trial and the startup of the next.
The ethical benefit is that in both the early-stage and late-stage trials, fewer patients will receive doses that do not work. This benefit is both immediate (early stage) and deferred (late stage).
The budgetary benefit is that, frequently, the number of patients required in a phase II trial can be reduced.
Medelis: Are sponsors comfortable accepting an independent decision-making body when required in adaptive trials?
Dr. Nayak: This is not an issue for phase I and II adaptive trials. Instead, the FDA’s concern about unblinding during a trial – and the resulting potential for bias – is mainly for confirmatory phase 3 trials. Right now, yes, we are seeing sponsors accept the independent recommendations, although total reliance on external decision-making bodies has not yet taken hold as standard practice. The industry does appear to be moving in that direction.
Medelis: What about site investigators and Institutional Review Boards – how are they accepting adaptive designs?
Dr. Nayak: It’s hard to generalize, but we’ve found that when you discuss this with them early and figure out what their concerns are, they typically will accept an adaptive design. For phase 1 trials, various forms of adaptive dosing have already been widely practiced, so there is not a lot that is new to investigators and IRBs. In phase II trials, it is already the practice to have multiple doses, so increasing the number is not a particular concern. The idea that some doses are dropped as the trial proceeds has a convincing value; patients are more likely to get a dose that works than in a conventional phase II trial.
Medelis: Is patient consent easy to get when they know the trial is adaptive?
Dr. Nayak: In our experience, patient consent is not difficult. The main point to make is that in an adaptive trial, the patient’s risk of getting a dosage that either does not have adequate efficacy or has an unacceptable toxic effect is significantly reduced when compared to a conventional
trial.
Medelis: Can you share some best practices?
Dr. Nayak: First, always consider the adaptive alternative, whether in phases III, II or I. In particular, use adaptive methods for getting the dose right in phase II trials. Also, consider the more classical group sequential design in any phase III trial.
Second, compare the costs of the adaptive and conventional alternatives, but also compare the probability of success.
If you choose to do adaptive, do a thorough analysis of the statistics and be clear about how the adaptive change is going to be made, and by whom. Make this thorough analysis a part of your protocol submission to the FDA.
Medelis: Do we know how to deal with the statistical issues?
Dr. Nayak: The answer is an unequivocal “yes”. The key issue is whether you have controlled the Type 1 or “false positive” error – the probability that when the drug does not work, the trial shows by chance that it does. The FDA requires that you prove that you have controlled this so that the error is less than 5%. This is easy to do for a conventional trial because the math is relatively simple.
For adaptive trials, the math gets very complex, and in most instances, custom software needs to be developed to prove through simulation that the error has been controlled. The FDA asks that you submit this simulation software along with your trial design. Cytel is particularly adept at this because we have expertise in both the statistics of adaptive trials and in the rapid development of validated software.
In dose-finding trials, you also have to have the software that allows you to look at data on a small set of patients and decide rapidly which dose to drop. We have developed this software in collaboration with Merck.
Medelis: Can sponsors get the data needed for the adaptive decision in a timely manner?
Dr. Nayak: That depends on two things. One, can the patient end point be observed reasonably quickly, and two, are there adequate EDC and data management systems in place so that data for the adaptive decision are available? If it takes a long time to observe the end point, then adaptive trials will only work if a validated surrogate end point, such as a biomarker, is available. Phase II trials of modest size can be run without sophisticated EDC and data management, but that will add to the cost. For phase III trials, EDC and first-class data management are essential.
Medelis: Is an adaptive change to a trial easy to implement at all sites?
Dr. Nayak: That is a multifaceted question. The first issue: Can you drop doses easily? If you have centralized randomization, the answer is “yes;” you just assign a probability of zero to the dose in the randomization system, and the dose is no longer allocated.
The second facet: Can you enrich subpopulations? The answer is “maybe,” depending on whether the sites have the capacity to do this or not. Careful selection of the sites will be necessary.
Third, what will the investigator need to be paid if the number of patients to be recruited is indeterminate until the adaptive change is made? Again, there is no general answer. You will need to negotiate the solution and possibly see some of the economic savings from adaptive trials evaporate. When this happens, it’s important to remind yourself that the primary motivation for doing adaptive trials is not to save money, but to heighten the chances of eventual market success
and to get there faster. The economic value of that far outweighs any cost savings or increases.
Medelis: Are there safety issues to be considered?
Dr. Nayak: Safety issues can never be dismissed, but it is worth noting that in most adaptive designs, the number of patients on doses that go to market is almost never reduced, and is often increased. That means that the amount of relevant safety information almost never diminishes.
Medelis: How should a sponsor budget for an adaptive trial?
Dr. Nayak: Good question! Imagine that the trial design team says, “The number of patients will be between 500 and 750, with an expected value of 600.” What number do you budget for? Practice will evolve here, and as experience is gained, statistical tools will be developed to guide budgeting. Quite likely, sponsors will need to maintain a central pool of funds into which savings from adaptive trials flow and from which monies go to adaptive trials that incur added cost beyond the original
estimate.
Medelis: Are there any other planning issues to be aware of?
Dr. Nayak: Yes. Adaptive trials require very thorough planning. Get a cross-functional team together that includes representatives of medical supply, manufacturing, packaging and labeling, biostatistics, project management, and the therapeutic areas, and charge them with doing the
planning together. Do not leave the logistical and project management issues as an afterthought. With adaptive designs, the design is not complete until you have figured out how to supply the trial.
In particular, simulate the trial ahead of time—accruals, randomization, response, and medical supply.
Medelis: What about medical supply logistics?
Dr. Nayak: The concern here is overages and stockouts. Overages have generally been considered unimportant, but managing supplies for an adaptive trial in the conventional manner could lead to overages of as much as 500%. At this time, the only way to keep the probability of a stockout
low without having massive overages is to simulate the supply system ahead of time and use the simulations to optimize the system of manufacturing campaigns, inventories, re-order points, re-order quantities, and shipping methods. Good simulation tools are now available, although their ability to optimize is still low.
In the future, it is likely that supply optimization tools will be developed, and this will be a boon to the industry’s ability to systematically conduct complex adaptive trials with ease. It is also likely that some of the vendors who supply clinical trials will create technological innovations in manufacturing, packaging, and labeling. These are the ones that will prosper. This has happened in other industries with complex product development processes, such as automotive, white goods, and consumer electronics.
Medelis: Any last advice?
Dr. Nayak: One thing is clear: If you are planning a clinical study without considering an adaptive design as one of your options, you are not getting good advice. It’s not good enough anymore to talk about the difficulties as though they were reasons for not doing adaptive trials. It’s time to figure out how to overcome those difficulties. It’s time to move, to learn by doing. Those biopharmaceutical companies that do this right will be the winners because their return on investment in clinical drug development will outstrip the industry.
Relationships are important to us. Reach out to learn more about how our approach, experience, investigator relationships, bid transparency and flexibility deliver success for our clients.