Feedback from the EBF – Focus Workshop: bringing assay validation and analysis of biomarkers into practice

The European Bioanalysis Forum's (EBF) interest to contribute to the intensive discussions in the international bioanalytical community on biomarker (BM) assay validation is well known. Already in 2010, internal EBF discussions started, resulting in a recommendation paper in 2012 [1]. The EBF contributions to the BM discussions continues to be based upon that recommendation, and as such hopes to inspire many scientists to take a step back and reflect on the four pillars referred to in the recommendation paper prior to starting the actual laboratory work.

As the discussion moved onto the next level after the release of the US FDA draft Guidance for Bioanalytical Method Validation [2] and a consecutive Crystal City V (CC-V) [3] and VI (CC-VI) meeting [4], the EBF committed to keep the finger on the pulse and continued to provide opportunities for the community to come together and share visions, strategies and experience on (BM) assay validation. At the same time, and talking to scientists around the world, the EBF observed a need to give additional opportunities to discuss on how to bring their recommendation paper into daily practice, especially on the aspects of the newly introduced fifth pillar: communication and the interactions with bioanalytical stakeholders or end users of the data. In that spirit, the EBF organized a focus workshop in Lisbon, Portugal [5]. The aim of the workshop was to share current practices in drug R&D related to BM assay validation and sample analysis. The workshop provided the delegates and, the wider industry a better understanding into the day-to-day challenges we face when setting up BM assays and performing sample analysis. At the same time, it connected the bioanalytical strategies used in many laboratories, including the vast experience from experts in clinical diagnostics, with the recent CC-VI and EBF recommendation on BMs. The workshop covered the different challenges with BMs in the different drug development stages experienced at a CRO as well as in pharma companies. Examples discussed included the bioanalysis of an exploratory BM in a discovery stage, a safety BM on a GLP study, the integration of an exploratory BM in an early clinical stage as well as diagnostic BMs in support of assessing DDI and an efficacy BM affecting dose setting in an early clinical study. The meeting was attended by 92 experts representing more than 50 different organizations. This paper summarizes the discussions and conclusions from the workshop.

A developing regulated environment for BM assay validation

The workshop kicked off with a session reflecting on the developing regulated environment for BM Assay Validation from a broad perspective. As plenary keynote presenter, John Allinson, presenting on behalf of the EBF, gave a comprehensive review of past and present publications relating to BM assay validation and compared and contrasted those used in drug development with those used in the clinical diagnostic arena. From a drug development perspective, he also included comparison of publications from regulators and other papers [1,6,7]. The major focus was:

• What elements were in consensus throughout the publications;

• Where potential harmonization was lacking.

He reviewed the major issues from a purely scientific and clinical perspective. It was pointed out that all of the regulatory guidance documents allow for flexibility in the validation process to enable the exercise to fit the clinical utility of the BM(s) being measured with statements such as: ‘approach used for PK assays should be the starting point’, ‘some characteristics may not apply’ or ‘different considerations may need to be addressed’, but unfortunately it was often the case that these flexibilities were not used and laboratories continued to validate BM assays as they do PK assays. Having spent his career (over 40 years) being involved in clinical laboratories in some way, John demonstrated the structure and development of very detailed assay validation guidelines used in those laboratories. He stressed that the assays were not only used for diagnosis, but also drug effect, efficacy and other clinical utilities – very similar to how we (plan to) use them in drug development. Therefore, as emphasized at CC-VI, John repeated the strong plea that a lot can be learned from clinical laboratories and scientists on how to validate BM assays appropriately.

John's introduction paved the way for a presentation on recent industry discussions on the subject. Marianne Scheel Fjording, presenting on behalf of the EBF, continued the session by reflecting on the essential message of the aforementioned EBF recommendation paper [1]. In her presentation she focused on the importance to consider the different aspects at the start of a project. When starting up an assay development for a BM assay, it is important to ask how the data will be used and align this with what the assay can actually perform. It is recommended to prepare a bioanalytical plan in which the drug development phase of the BM to be measured is taken into account as there can be differences in the regulatory requirement for an early phase compared with a late-phase BM data submission. In that a priori reflection, and at all times, both the observed or anticipated BM level changes and the agreement if the BMs are for exploratory or safety decision should be discussed. The BM assay should be reliable in order to take the right decisions. The stakeholders who handle the BM data should understand that the BM data cannot be treated as PK data, as in most cases the data are relatively quantitative and may additionally reflect biological variation, which in many cases are unknown. At the CC-VI meeting, there was a consensus that BM assays are not the same as PK assays. The following are only a few examples to illustrate the complexity of a BM assay and its differences with PK assays:

• Reference material is often recombinant protein, which does not resemble the endogenous BM in contradiction to PK assays, where the standard and the drug is of the same material, immediately impacting the possibility to document accuracy;

• The sample matrix often contains the endogenous BM, and therefore most BM assays use a surrogate matrix for standards whereas PK assays use sample matrix;

• The spiking of reference material into matrix contributes to the difficulty to perform stability experiments in comparison to PK assays. The latter providing support to reflect on putting more emphasis on incurred sample stability.

These are just a few scientific considerations, and the message that BM assays are not PK assays should be repeated until the bioanalytical community, which most often are trained as PK scientists in the US FDA or EMA BMV guidance/guidelines, becomes more comfortable in using alternative validation approaches dedicated for BMs. In addition, the BA community should learn to ask clarifying questions and understand the biology of the BM.

Philip Timmerman, also presenting on behalf of the EBF, continued the session. Building further on the contribution of Marianne, his contribution took a deeper dive on the importance of communication between the bioanalyst and the stakeholders, and he suggested some handles to overcome challenges. In his introduction, he anecdotally referred to ‘the lack of/poor communication’ being the Achilles’ heel of any success story and quoted from GB Shaw that “the single biggest problem in communication is the illusion that it has taken place.” Philip described the essence of an essential two-way communication: the bioanalyst giving insight into the assay requirements (performance, limitations, alternatives and regulatory landscape for bioanalysis) and stakeholders having a clear view on the data requirements from their perspective (i.e., biology or pharmacology of the BM, decisions taken and regulatory landscape for the use of the data). He illustrated his talk with a number of today's challenges in the communication, not necessarily unique to bioanalysis:

• Impossibility for face-to-face interactions;

• Time zone differences;

• BA Scientist (CRO or sponsor) not part of project teams;

• (Scientific) language and cultural barriers;

• IP boundaries;

• Trust.

In his conclusion, Philip presented his views on the aforementioned essential communications needed to enable success [8], and referred to responsibilities of the three stakeholders involved in making BM bioanalysis a success:

• The bioanalyst;

• The end users of the data;

• The regulators.

This session ended with a panel discussion moderated by John Smeraglia and Begoña Barossa, during which it became evident that, although everybody accepts that BM assays are not PK assays, there are still many questions about how to deal with the differences between them, especially in terms of assay validation.

During the panel discussion, the delegates connected with earlier EBF recommendation papers, not only about how to deal with BMs, but also stressing the importance of scientific validation to get meaningful data. However, although the principles of the EBF recommendation paper are becoming widely accepted, scientists recognize the hurdles to get it implemented. An important question from the group was to provide more tools to take the recommendation further into practice in many innovator companies and CROs, BM assays are run in a range of laboratories including laboratories that comply with CAP/CLIA regulations and regulatory bioanalytical laboratories supporting PK, making it difficult to avoid using PK assay criteria or other criteria that may not be appropriate for BM assays. Moreover, there is often a disconnect between how the data are utilized by translational medicine scientists, clinicians and other end-users of the data across the development program and the bioanalytical experts developing the assays. It is critical to ensure the most appropriate assay or technology is utilized to ensure the analytical methodology is fit-for-purpose, which requires a great deal of scientific and technical discussions with CROs, who are often performing the analysis. It was identified that this level of interaction is often lacking.

Breakout sessions

The workshop continued with smaller breakout sessions, where the presenters shared user cases or specific challenges for different types of BM questions in different stages of development.

Liz Hickford presented a session challenging the perception that dilution linearity experiments are necessary for the fit-for-purpose validation of BM immunoassays. Some commonly held beliefs surrounding linearity experiments were outlined, including:

• The view that additional information can be provided from the data;

• That reassurance can be given in the absence of parallelism data (e.g., due to lack of incurred sample availability);

• That there is an expectation from regulatory agencies to see linearity experiments performed.

The parallelism experiment is considered one of the key components of a BM immunoassay validation. As well as demonstrating parallelism of the endogenous analyte it can provide a range of additional information including relative accuracy, selectivity, minimum required dilution, sensitivity and assay range [9]. As such, linearity experiments may be considered irrelevant – especially in relative quantitative assays where the recombinant calibrator may not be wholly representative of the endogenous molecule. Not surprisingly, this notion triggered a heated debate during the ensuing panel discussion. It also highlighted possible strategies when there are no suitable samples available to assess parallelism. In these circumstances, there are several options that can be considered, all of which have their disadvantages (as discussed in [9]), and these will often be limited by what material is available.

Sidath Katugampola highlighted some of the challenges in demonstrating proof-of-mechanism and proof-of-principle BM assays in early oncology phase trials. Due to limited numbers of patients per dose cohort, trials are often being conducted in multiple tumor types, combined with tumor heterogeneity in an advanced disease stage. This often poses significant challenges to obtain reliable, robust data for decision making. This is particularly challenging for first-in-class agents. Sidath continued to share examples of some of the limitations in developing tumor-based BM assays and the continuous evolvement of the fit-for-purpose strategy. Discussions also focused on the kind of strategies that can be adopted to establish a patient enrichment BM assay to be confident early on in trials, that the right drug is being tested on the right patient group. Some of the challenges in developing such assays for immune targets, compared with tumor-associated target antigens, were also discussed. Bioanalysis requirements for each assay, depending on how the assay would be used in the context of the trial, were also a topic of discussion and debate. The take home message from this session was that it is pivotal to demonstrate proof-of-mechanism and concept in early phase oncology trials. However, a pragmatic approach, with clear communication between sponsor and investigator is crucial when presenting such BM data in the context of the entire trial and its end points.

Ulf Diczfalusy, Hamza Kandoussi and Cecilia Arfvidsson gave an industry/academia joint session on 4β-hydroxycholesterol (4β-HC) to illustrate how a scientific cross-functional collaboration has led the way to establish this BM as an emerging BM for CYP3A4 activity. Ulf presented an overview from his 20+ years of research on 4β-HC and some of the oxysterols’ physiology important to understand when setting up an assay [10–13]. Hamza then followed with the assay characteristics and criteria to take into consideration if the data generated should be used for interpretation of CYP3A4 activity and shared the recently published bioanalytical recommendations [13] on the development of a reliable assay, suggesting that:

• Plasma should be used instead of serum;

• Addition of antioxidant only required if samples to be stored long term (>1 year);

• Simultaneous quantification of 4α-hydroxycholesterol should be used to confirm sample quality only in long efficacy studies at multiple sites.

Finally, Cecilia shared data from a clinical development project to illustrate how the use of clear 4β-HC data within the pharma industry had been used to facilitate decisions in distinct projects at an early stage, but also how the assay criteria may be different if looking at induction versus inhibition or if you are building BM knowledge and insight versus having a specific project claim.

During the following discussion the group was challenged on any possible future adaptation of the 4β-HC assay. Being a well-defined and established BM, the assay may be adapted in terms of range and criteria in order to be more cost effective and available for all. The day ended with two plenary sessions. The first session focused on the development and use of BM assays within discovery and the considerations for transition with projects into the later stages of development. Adrian Freeman discussed a number of considerations based upon a real-life example of a multiplex thyroid hormone assay that was developed to support a wide range of projects, mainly as a safety BM but also as a mechanistic BM. The initial scope was to develop an assay for adult rat samples, but as the project progressed it became a requirement to measure for thyroid samples within pups and this changed the sensitivity requirement of the assay. The initial multiplex assay that was developed for adult samples was not sensitive enough for measurement of pup samples. This, therefore, meant a technology change for the project and a break in data continuity for assessment. The latter part of the presentation therefore focused on setting up BM assays for a project and discussing the need to consider biology, the technologies available, assessment of project need and the level of validation required to ensure the assay is fit-for-purpose. Overall the discussion focused on the balance of risk and the factors to consider, but that the consideration of the biology of the analytes being measured is paramount when assessing the requirements of the assay.

In the closing session of the first day, Ulrich Kunz presented a case study about an exploratory BM that has been chosen to support early clinical drug development as a pharmacodynamic BM. The focus of the presentation was a Pharma company approach to identify the BM and develop the bioanalytical assay for the BM. The first step in the bioanalytical BM process is gaining sufficient information about the BM, the intended use of the BM data, the evaluation strategy, planned studies and the optional future use of the assay. In order to foster this important discussion between stakeholder and bioanalyst, he presented how this can set up as a formal assay request process with a comprehensive questionnaire that asks for all necessary information. It was discussed whether such a questionnaire could be a general approach for other companies too and which information would be mandatory. The next step is the translation of the information into an optimal bioanalytical strategy. Optimal means selection of assay platform, selection of reagents and what is the workload for the intended use. An example was given in which several ELISA kits from various vendors were tested. Some aspects were discussed within the audience. Several participants confirmed to focus on ELISA kits from reliable vendors only, that reporting post- to predose ratios instead of concentrations may be an option in case of lack of parallelism and that an in-study validation concept based on retrospective quality control performance evaluation without prospective set quality control acceptance criteria would be acceptable for low level validation measurements. Another point of discussion has been the assignment of different validation levels to assays at different stages of drug development. This is handled very different at different companies. The main reason for a higher validation level should be the higher risk for internal decisions and of course if the data are used for patient relevant decisions. Assays do not necessarily have to be revalidated at a higher level when going from, for example, Phase I–II clinical development, even if there are major changes in the intended use of the BM data.

Day 2 started with a session on the verification and validation of an efficacy BM during early clinical development. Robert Nelson immediately took the workshop delegates on an interactive breakout session focusing on the project information required prior to implementation of a bioanalytical strategy. He used the measurement of a BM in a pivotal trial as an example for the group conversations based upon the questionnaire by Ulrich the day before. The delegate groups gave feedback that information about the expected BM level and changes upon a therapeutic intervention is needed in order to set up the assay. In addition, information was needed in order to know what the data would be used for; exploratory, critical decision, safety or efficacy. The discussions also, and again, highlighted the importance of input from the bioanalytical scientist into the sample collection and processing procedures. Robert then presented a case study describing how a fit-for-purpose approach to assay development and validation was used during the identification and subsequent verification of BMs of target engagement and disease activity during the early clinical phases of development of a mAb therapeutic. The importance of BM parallelism assessment throughout method establishment, and continuing through the sample analysis phase was the core message.

Having made the introductory presentation to the workshop, John Allinson was privileged to deliver the closing presentation (representing LGC) in front of the closing panel discussion on the workshop itself. Here, John presented a large number of case studies of BM projects covering a wide range of analytical, scientific and clinical issues. He demonstrated common problems as well as presenting projects with both positive and negative outcomes of differing degrees, and the challenges that need to be understood when using new technology platforms.

Key to the science presented was an understanding of physiology and the clinical utilities of the BMs being studied. Also how to underpin studies where information may be absent by using statistically relevant acceptance criteria so that the variability of the results data was at least understood and could be passed on to statisticians who assist in the data interpretation. Two case studies also proved that both analytical and physiological variability could have a very big impact upon data interpretation and suggested ways that the information could be used to improve outcomes. Overall the focus was on good clinical science, statistically driven acceptance criteria and a clinically knowledgeable BM team as being key to the best outcome for both drug developers and patients, and therefore health outcomes as a whole.

Closing panel discussion

The workshop ended with a closing panel discussion where the delegates were invited to give their thoughts on following two questions:

• What is missing to bring an EBF recommendation paper into practice, and reflect on internal and external BA communication?

• What is missing to allow the PK bioanalyst to become more informed to understand the BM questions required?

Although, the answers to the questions were prepared in three separate breakout discussions, the outcome and proposals were very similar and there was a lot of overlap in the suggested action points.

The majority of the responses to the first question focused on making sure the right tools for communication are available. A lot of valuable suggestions were made and there was support for the EBF to develop a questionnaire to help the bioanalyst in retrieving the right information about biology and assay requirements (e.g., LLOQ) from the requester and stakeholders. The questionnaire is intended to be used within Pharma companies but can also be used for communication to the CRO as well.

Related to the internal communication, the group highlighted the increasing need for in-depth biology knowledge of the bioanalysts. Although, many have a thorough biology background as part of their education or experience, and this certainly hold true for the LBA experts, BM science is developing so fast that continued and focused training will be very beneficial to allow the bioanalyst a better focus. In addition, there was discussion that many bioanalysts have a background as a PK bioanalyst, and we tend to fit the PK assay requirement to the BM assay, which as discussed earlier is not always good practice. The PK bioanalyst needs to rethink many of the assay validation parameters from the FDA guidance and EMA BMV guidelines.

With respect to external communication, there was agreement on the value in connecting bioanalytical experts more closely and prior to bioanalytical requests/decisions are made with the project teams. At the same time, the challenges of achieving this were recognized: time and resource constraints, CRO-Pharma boundaries on IP, difficulties in organizing global companies/projects.

We also discussed the growing ambiguity on the requirements for GLP standards for BMs, especially in labs with a strong preclinical focus. The terms ‘GLP validation’ and ‘validate a BM assay according to GLP standards’ seem to often be misinterpreted.

Finally, there was an overall theme relating to the fear/experience on reported concentration data being (mis)used outside of their intended scope. With different levels of bioanalytical quality being applied to answer different types of questions, in the end, the bioanalyst loses control of the data once reported. Hence, if the end user is not aware of boundaries within the data with acceptable reliability, they may end up in supporting decisions for which the assay was not validated for. Although, some valuable suggestions were made at the meeting, no conclusion was reached, and hence this may certainly be an area of increased discussion. Indeed, any inappropriate misuse of the data has led and may lead to the bioanalytical community drifting back to setting up ‘PK data’ or GLP-validated data using the highest regulatory standards for BMs.

Conclusions & action items from the meeting

From the meeting, three tangible action items were agreed for the EBF to focus on in upcoming meetings or further interaction:

• The continued strong message that BM assays are not PK assays has been brought out loud and clear by many and now it is time to start filling in what BM assays are and what makes them different from the PK assays with regards to assay validation. As a bioanalytical community, we should try to keep this message visible and provide clear scientific examples to support this message;

• EBF plans to develop a questionnaire to facilitate the communication with stakeholders. This questionnaire can be used to ensure all the aspects of BM assay validations are considered prior to the start of a study. The questionnaire should develop into a strong handshake between the laboratory generating the data and the end users of those data. By having intensive a priori communication, the bioanalyst and the clinical pharmacologist can work together to incorporate the background biology into the assay;

• As for other areas in regulated bioanalysis, the BM community remains unclear on how GLP requirements (should) impact BM analysis. There is a risk of overinterpreting the GLP rules for BM sample analysis, and the EBF plans to include this challenge for BMs in the already ongoing discussions at international meetings. In continuation, EBF plans a discussion paper on the subject for publication in 2018.

Conclusion & future perspective

The discussion on method validation and the bioanalysis of BMs will be an important discussion area in our scientific community for many years to come. The growing array of BM assays, the increased importance of BM data in critical decision making (in all phases of drug development) and the rapidly increasing technology improvements are only a few drivers to continue to build a community of experts which can provide guidance for the challenges of the future. A few of these challenges are real today: what is the required level of (regulatory) bioanalytical validation required? And how can we ensure timely and relevant communication on the assay requirements between the laboratory and end users of the data? The EBF remains committed to be part of a global scientific community to promote the discussion on the aforementioned challenges.