Performance evaluation of thermal cyclers for PCR in a rapid cycling condition

Introduction

Since the development of polymerase chain reaction (PCR) in the 1980s, it has become one of the most popular laboratory techniques for DNA technology. PCR is designed to selectively amplify targeted DNA sequences to several billion-fold, and its remarkable selectivity and sensitivity of DNA amplification has led to its widespread application. Having been subjected to continuous improvements and modifications (1–3), PCR is now routinely applied in most bioanalytical and clinical laboratories, as well as in basic research fields (3–9).

In spite of its power and versatility, PCR must be cautiously designed and applied only after appropriate validation. The efficiency of and results from DNA amplification are easily affected by subtle variations in reaction conditions, which may lead to significant variation in results due to the exponentially amplifying nature of PCR (10,11). Therefore, major factors affecting the reproducibility and comparability of PCR must be considered and controlled to establish reliable PCR-based assays. These factors include quality and quantity of components such as template DNA, Taq polymerase, and various chemical components, as well as the performance of deployed thermal cyclers.

The reliability of PCR-based assays has been addressed from multiple viewpoints. Neumaier et al. (12) thoroughly discussed the fundamentals for quality assurance of the PCR method in clinical diagnostics. Several interlaboratory studies have been performed to assess the factors affecting the reproducibility and comparability of PCR as a molecular detection technique (13–16). Various thermostable DNA polymerase species for PCR have been compared in terms of amplification efficiency and fidelity (17). Saunders et al. (15) studied the effects of temperature variations and the importance of calibration of thermal cyclers on the reproducibility of PCR performances among laboratories. A more substantial work examining the in-tube temperature profiles of six different commercial thermal cyclers was also recently reported (18). In this study, two of the six tested thermal cyclers presented temperature profiles that severely deviated from their random amplified polymorphic DNA (RAPD) protocol. In addition, two instruments showed unmistakable over- and undershooting phases in their temperature profiles, which could also harm the comparability of PCR-based assays. Therefore, a substantial portion of commercial thermal cyclers has been clearly demonstrated to insufficiently perform for rapid PCR protocols, which supports the speculation of failure of interlaboratory comparability trials partly due to the use of inappropriate thermal cyclers (15).

In spite of revealing the hampered performances of some commercial instruments with physical data, the report by Schoder et al. (18) did not directly connect the poor performances of thermal cyclers with the probable and subsequent undesirable outcomes of PCR. Furthermore, well-to-well variations, another important concern regarding the performance of thermal cyclers, were not described. We have previously developed a temperature-sensitive multiplex PCR kit and a multiple channel temperature monitoring system specifically designed for evaluating the performance of dynamic temperature control of thermal cyclers (19). We applied the diagnostic PCR kit and temperature monitoring system in parallel to several thermal cyclers to establish the correlation between instrument performance and PCR results. The recent trend of PCR protocols—rapid cycling with reduced reaction volumes—presents a more challenging demand in the performance of a thermal cycler, which will result in more failures in PCR-based assays if not carefully designed. Therefore, this trend was implemented in the testing protocol of the study presented here. Inner and outer wells of a thermal block were specifically tested in parallel to assess the degree of well-to-well variations. Our study not only supports the findings of Schoder et al. (18) with results from a larger set of test instruments, but also further establishes the direct connection between poorly performing instruments and undesirable PCR results. Variations between wells were substantial enough to significantly affect the results of some sensitive PCR protocols.

Materials and methods

Temperature Measurement

For preparation of the temperature probes, type T micro-thermocouples (RS Components GmbH, Mörfelden-Walldorf, Germany) were used as temperature sensors. The thermocouples were implemented in 0.2 mL PCR tubes (Microamp; Applied Biosystems, Foster City, CA, USA). The lead wires of a sensor were arranged to pass the tube wall through a fine hole provided by a guide, which helped air-tight sealing against expanding pressure of boiling water. Seams around the guide were sealed with epoxy glue, and satisfactory sealing was confirmed if no significant loss of test solution was detected after leaving the test solution in the closed temperature probes for more than 30 min at 95°C. A typical temperature probe is shown in Figure 1, A and B.

To assess the degree of variation of temperature among wells, eight temperature probes were placed on a thermal block, as shown in Figure 1, C and D. The probes were connected to a 34970A data acquisition/switching system (Agilent, Palo Alto, CA, USA), which acquired temperature data from the probes at 1 s intervals. Temperature data from the acquisition system were transferred to a personal computer and processed with Agilent BenchLink Data Logger (Agilent) software in real time.

Nineteen commercial thermal cyclers of eight different models were tested in the current study. The selected instruments, including both old and new models, are: four sets of GeneAmp 2400, one set of GeneAmp 9600 (both from Perkin Elmer), four sets of GeneAmp 2700, four sets of GeneAmp 9700 (both from Applied Biosystems), one set of PTC-220, two sets of PTC-225 (both from MJ Research), two sets of Mastercycler (Eppendorf), and one set of Primus96plus (MWG-Biotech). More detailed information on the tested instruments is given in Table 1. Each thermal cycler was programmed to perform a specific protocol of thermal cycling: 30 cycles of 94°C for 20 s, 60°C for 20 s, and 72°C for 30 s, where the maximum ramp rate available was applied. The lid temperature was set to 105°C. Temperature data acquired by 1 s intervals were averaged for 20 s in annealing and denaturation steps, and for 30 s in an elongation step to indicate the “effective” temperature of each reaction step. The effective temperatures of eight probes were then averaged to calculate the effective temperature of each thermal cycler. The standard deviation (sd) in the temperature measurement table (Table 2) indicates homogeneity of temperature measurement among eight probes, which is a more meaningful indication of the homogeneity of temperature on a thermal block. Before averaging data from the eight probes, we averaged data from each probe for three consecutive cycles to cover variations during multiple cycles. Temperatures at the start and end of each reaction step were indicated as the “initial” and “terminal” temperatures of each reaction step, respectively. Effective step-lengths were determined from the lengths of time when the actual temperatures were within ±0.5°C of the programmed temperatures for each reaction step.

Table 1.  Information on the Assessed Thermal Cyclers

Table 2.  Measured Dynamic Temperatures and Step-length of Three-step Temperature Protocols (Annealing at 60°C for 20 s, Elongation at 72°C for 30 s, and Denaturation at 94°C for 20 s) Using In-tube Temperature Probes

The temperature measurement system was calibrated at the Korea Research Institute of Standards and Science (KRISS; Daejeon, Korea), a national measurement and calibration authority, in accordance with specifications established for legal metrology. The calibration was carried out at 60°C, 72°C, and 95°C.

Multiplex PCR for Performance Evaluation

Preparation and use of the temperature sensitive multiplex PCR kit for performance evaluation of thermal cyclers was previously described in detail (19). Briefly, the multiplex PCR was designed to amplify four DNA bands of 200, 300, 400, and 480 bp from human placental genomic DNA template (D-4642; Sigma, St. Louis, MO, USA); the reaction generates four DNA bands in a temperature-dependent manner as detected by gel electrophoresis. Composition of the multiplex PCR kit was slightly modified to increase reproducibility of the kit. The new PCR mixture (unit reaction volume: 20 µL) consisted of 1.6 U of HotstarTaq polymerase (Qiagen, Hilden, Germany), 1.5× of PCR buffer (Qiagen), 0.375 mM dNTPs (Sigma), 75 ng of sheared human placental DNA (Sigma), and 4 primer pairs (Bioneer, Daejeon, Korea). The sequences and final concentrations of the primers are as follows:

193f: CACACTTCATATTTACCCAT −2.19 µM

193r: TTGTTTAATAGAGACGAAGG −2.19 µM

300f: ATGGACATTTACGGTAGTGG −0.31 µM

300r: AAGTATTTCAATGCCGGTAG −0.31 µM

400f: GCTAGCTGTAACTGGAGCCG −0.47 µM

400r: GTCTGCTGAAACTGCCAACA −0.47 µM

482f: GGCCTGCTGAAAATGACTGA −1.41 µM

482r: TAGAAGGCATCATCAACACC −1.41 µM

To ensure the precision of the quantity of each component of the PCR mixture, 800 reaction units (20 mL) were prepared as a single batch, which was gently rocked for more than 1 h to homogeneity. After thorough mixing, the whole batch was aliquoted to individual reaction tubes either for immediate use or for storage. Homogeneity of the aliquots was confirmed by comparing the agarose gel banding pattern of the PCR products of 24 tubes using thermal cycler O in Table 1. The homogeneity test was repeated three times at 1 day intervals to ensure the stability of the PCR mixture. The mixture was stored at 4°C.

Eight tubes of the PCR mixture were placed in each thermal cycler. Immediately after temperature measurement, the PCR reaction was carried out (the protocol is below), and the reaction products produced at various locations (where thermal cyclers existed) were brought to the main laboratory for performing gel electrophoresis at one location under the same conditions. PCR products were separated through 2% NuSieve 3:1 agarose (Lonza, Basel, Switzerland) gel electrophoresis.

The PCR protocol is as follows:

1. Activation of polymerase: 95°C, 15 min

2. Thermal cycling: 30 cycles

   • Denaturation: 94°C, 20 s

   • Annealing: 60°C, 20 s

   • Elongation: 72°C, 30 s

3. Extension: 72°C, 10 min

4. Storage: 4°C, as necessary

Results and discussion

In-tube Temperature Monitoring and Temperature-sensitive Multiplex PCR

For in-tube temperature measurement in a small reaction volume and judgment of rapidity in reaching a thermal equilibrium, the temperature sensor itself should not significantly contribute to the overall heat capacity of the PCR tube. Otherwise, the speed of reaching a thermal equilibrium would be substantially affected by the presence of the sensor in the PCR tube. The micro-thermocouple used in this work was lightweight, and the portion in the tube weighed approximately 0.8 mg, or approximately 4% of the weight of the 20 µL test solution, allowing increase in heat capacity by the tip of the micro-thermocouple to be disregarded. This assumption was supported by the observations of some spiky temperature profiles, as they indicate that the temperature sensors responded to rapid changes in temperature instead of smoothing the transient changes by heat absorption or release. In addition, the temperature probes at elevated temperatures were sealed air-tight to avoid any adverse effect of thermodynamic pressure-volume work on the speed in reaching thermal equilibrium. Well-sealed tubes showed no observable loss of in-tube solutions even after 30 min of heating at 95°C.

The usefulness of a temperature-sensitive multiplex PCR kit in determining the accuracy of actual annealing temperature has been previously demonstrated (19). Reproducible production of the multiplex PCR kit, however, was not guaranteed, as the kit was prepared from a commercial pre-mixed form. In the current study, we prepared the PCR mixtures using well-specified component reagents. We changed the polymerase to the HotStarTaq, thus several other reaction components had to be modified to some degree; final concentrations of primers were finely tuned to generate the desired optimized temperature-dependent gel banding pattern as shown in Figure 2A. Intensities of the 200 bp and 480 bp bands gradually weakened as the annealing temperature was increased. Upon careful comparison of neighboring banding patterns, the effect of differences in annealing temperature of 0.8°C could be distinguished. Results of the multiplex PCR carried out simultaneously at 60°C are also presented in Figure 2B. A slight variation in the intensity of the 200 bp band was observed, found to be due to slight differences in annealing temperatures between wells of the given thermal cycler. Homogeneity of the PCR mixture was acceptable within this degree of variation.

Overview of Results

For the given cycling speed, most thermal cyclers presented some type of distortion in their temperature profiles. Ideally, the reaction temperature should change in a linear fashion until it reaches a specific step in the PCR and remain constant during the step. Several thermal cyclers, however, showed slow changes in temperature at the beginning of a PCR step, resulting in “curved” temperature profiles, which substantially shortened the duration of the temperature plateau (i.e., more than 20% of the programmed duration). In contrast, other instruments maintained the rapid speed of temperature changes, even at the beginning of a new PCR step, but showed a momentary fluctuation of temperature before stabilizing. At the phases in the PCR reaction that required the temperature to ascend, this fluctuation resulted in a momentary increase beyond the designated step temperature (“overshooting”). In the same manner, temperature was significantly below the designated step temperature when conditions required the temperature to descend (“undershooting”). Only 2 of the 19 tested instruments showed nearly ideal behavior (Figure 3A). The observed distortions are classified, then, as “curved” (Figure 3B), “overshooting” (Figure 3C), or "undershooting (Figure 3D) temperature profiles. The thermal cyclers analyzed in this study are labeled in Table 2 based on their characteristic temperature profiles.

These characteristic trends in temperature profiles are closely associated with the models of thermal cyclers. Curved temperature profiles were prominently observed in the models with relatively slow ramping rates (GeneAmp 2400 and 2700 in Table 1), whereas the models of fast ramping rates (GeneAmp 9700, PTC-220, and PTC-225) tended to produce overshooting and/or undershooting. Therefore, the observed temperature profiles are likely the outcomes of the designs of thermal cyclers. We also noticed substantial amounts (up to 5°C) of well-to-well variation in temperature that differed among the same model instruments, which could be explained either by loose quality control in manufacturing or by weathering of the instrument from lack of proper maintenance. Consistent with this latter possibility, greater well-to-well variation was observed in older instruments. None of the tested instruments except thermal cycler O had proper maintenance or calibration records.

The observation described above is not only in accordance with the report by Schoder et al. (18), but also illustrates a general issue in dynamic temperature control. It is critical to know that a significant subset of the tested instruments was not capable of precisely executing a specified programmed cycling sequence, particularly for short step-lengths. The distortions in temperature profiles were not negligible, as they resulted in failures in sensitive PCR reactions, the multiplex PCR in this work. The performances of the tested instruments during the annealing, elongation, and denaturation steps were numerically expressed and are summarized in Table 2. For each reaction step, the mean temperature (the average temperature of the duration of the step indicated as the effective step-temperature) is presented, and deviation from the programmed temperature is a reasonable parameter of the fidelity of each instrument. Along with the effective step-temperatures, effective step-length data are also given, which were defined as the period of time when the actual temperature was maintained within ±0.5°C of the programmed step-temperature. Consideration of both effective step-temperature and effective step-length provides an idea of the rapidity of temperature settlement as well as accuracy. For example, for two different instruments with the same effective step-temperature, the instrument with the longer effective step-length would show rapid settlement at the beginning of the step. Regarding the annealing step, the effective annealing temperature of 11 instruments deviated by at least 0.5°C from the programmed temperature (Table 2). More severe deviations in certain wells can be predicted from the sd (0.3° to 1.3°C) of the mean temperatures. In the annealing step, 11 instruments had effective step-lengths of more than 10 s shorter than the programmed step-length. Deviations from the programmed temperature profile correspondingly resulted in poor outcomes of the multiplex PCR. Examples of the remarkably close relationship between temperature profiles and PCR outcomes are shown in Figure 4; these examples are described in detail in the following sections.

Curved Temperature Profiles by Slow Response

Based on the step-length data in Table 2, it is apparent that the majority of the tested thermal cyclers were unable to precisely execute the programmed temperature profile. The effective step-lengths were substantially shorter than the programmed value at the rapid cycling mode. Thus, the large variations in the effective step-length among tested thermal cyclers would result in substantial variations in the results of PCR-based assays if different instruments were used. Among the instruments, the temperature profiles of the cyclers C, G, Q, and S were highly curved, and their effective annealing temperatures were 1.2°, 0.8°, 1.0°, and 1.6°C higher than the programmed annealing temperature (Table 2) with effective step-lengths of the annealing step of 1.1, 8.7, 3.5, and 1.1 s, respectively. Consequently, the PCR results with these instruments were relatively poor (Figure 4A). In the case of cyclers G and Q, well-to-well variation was as high as 5°C, and several wells (5, 6, and 8 of cycler G, and 7 and 8 of cycler Q) never reached the designated annealing temperature. The extreme case was cycler S, which never reached the denaturation temperature (Figure 4B). The average effective denaturation temperature was only 87.8°C, and the effective step-length for the denaturation step was 0 s. Consequently, no observable DNA band was produced in the multiplex PCR.

A recent trend in PCR techniques to speed up PCR-based procedures is the use of rapid cycling. Although PCR formulations, including Taq polymerase, meet the speed of 20 s step-length, a significant portion of thermal cyclers may not be compatible with the speed as demonstrated in this study, which would lead to an increased discrepancy in the results of sensitive PCRs.

Overshooting and Undershooting in Temperature Profiles

Another critical issue in the performance of thermal cyclers is the fluctuation of temperature when initiating a specific step in the PCR, resulting in overshooting or undershooting. Overshooting that occurs when temperature is increasing for the denaturation step or elongation step is not problematic, because mildly elevated temperatures in those reaction steps do not create significant problems for the given PCR protocol. Among the tested instruments, overshooting was observed with four thermal cyclers (J, N, O, and P) that were classified as fast-acting instruments. For fast responses, damping of the temperature change is set to weak. As a result, rapidly responding thermal cyclers tend to result in momentary temperature fluctuation. As predicted above, however, PCR results with these instruments were normal, except for one instrument that was also prone to undershooting (cycler J).

In contrast, undershooting could be problematic, specifically during the annealing step, because undershooting would result in the annealing temperature being lower than predicted for a certain period of time. The specificity of a PCR reaction is largely governed from creating the precise conditions that facilitate specific binding between the given primers to the specific sequences of the template DNA (“annealing”). Weaker binding of primers to interfering sequence homologies (“mis-annealing”) is avoided by thermal agitation. Thus, for maximizing the thermal agitation, the highest temperature that still allows the designated annealing is chosen as the optimal annealing temperature. Use of an annealing temperature lower than the optimum temperature may allow mis-annealing and is likely to result in PCR amplification of unwanted DNA sequences. As primers bind template rapidly, even a short exposure of several seconds to a lower annealing temperature could substantially harm the specificity of the PCR. Two thermal cyclers (E, J) showed a distinctive pattern of undershooting at the beginning of the annealing step. The magnitude of undershooting differed among the tested wells, and the PCR results clearly showed the effect of undershooting. The dips in annealing temperature caused by undershooting were in the range of 0.9°C and 1.3°C less than the set temperature for cycler E (Figure 4C), and the 200 bp band of the lowest annealing temperature, placed as a model interfering PCR band in the multiplex PCR, is clearly noticeable in the PCR products from the wells associated with undershooting. In spite of the short duration, the impact of undershooting on PCR results was substantial. This is a critical aspect that should be carefully taken into consideration for PCR protocols, particularly if they are prone to nonspecific amplifications.

Well-to-Well Variations

The variation in temperature between individual wells within one thermal block is another important performance parameter of a thermal cycler, as this may greatly influence the homogeneity of the results of temperature-sensitive PCR assays. In this study, temperatures of the inner-side wells and outer-side wells were comparatively measured (Figure 1, C and D). Probes 1, 2, 3, and 4 measured temperatures in the wells of the external column in the thermal block, whereas probes 5, 6, 7, and 8 measured temperatures of the wells in the middle column of the block. In 14 of the 19 tested thermal cyclers, the temperature profiles of the probes along the edges (probes 1, 2, 3, 4) were grossly clustered together and noticeably different from the cluster of the profiles of the probes in the center (probes 5, 6, 7, 8). One prominent case (thermal cycler N) is presented in Figure 4D, in which not only the temperature profiles but also the results of the multiplex PCR were noticeably distinct between the edge group and the center group. In one case (thermal cycler M), we observed the temperature profiles of probes 1, 4, 5, and 8 as clustering together, while the remaining probes 2, 3, 6, and 7 were clustered in a separate group. In this case, the division of the center and the edge of the thermal block is determined by the closeness to the center of the block along the vertical axis rather than the horizontal axis (refer to Figure 1D). Thus, the detected temperature profile variation is most likely associated with the relative positions of temperature sensors on a thermal block that feed temperature signals to the temperature control unit. Nevertheless, current instrumentation techniques applied to some thermal cyclers do not sufficiently exclude well-to-well variations of PCR results due to inhomogeneous temperature on a thermal block.

In addition to systematic well-to-well variations of temperature, irregular temperature deviations at certain wells were also observed. These variations were more likely due to improper thermal contact provided to the given wells, and this problem may be easily corrected by carefully cleaning the troublesome wells.

Effects of Extended Step-lengths and Increased Reaction Volumes

Thermal cycler S generated no PCR products for the temperature-sensitive multiplex PCR, as it operated with the most sluggish temperature control speed (Figure 4B). When it was tested with elongated step-times (50 s for each step), however, strong PCR product bands were obtained (Figure 4E). Under the longer cycling conditions, the temperature of cycler S eventually reached plateaus for each step. Although denaturation temperatures for inner wells were still noticeably lower than those of outer wells (the same as the designated denaturation temperature), PCR results of inner wells were still robust. This particular example strongly supports the fact that sluggish thermal cyclers can still be functional by applying relatively slow cycling to allow sufficient time to reach the programmed temperatures.

As can be expected, increased reaction volume resulted in smoother temperature profiles. Spiked temperature profiles became substantially smoother with the increase in reaction volume from 20 µL to 50 µL (Figure 5). Although the degree of the effect of increasing the volume varied among different models, the trend was common and obvious.

Therefore, the reliability of PCR-based assays could be significantly improved by applying slow cycling or using increased reaction volumes to reflect the dynamic temperature control performances of individual thermal cyclers. PCR protocols of rapid cycling with small reaction volumes should be cautiously applied unless the performance of the employed thermal cyclers has been validated and confirmed.

In conclusion, this work reveals remarkable imperfections of various representative commercial thermal cyclers in rapid thermal cycling, using both in-tube temperature monitoring and temperature-sensitive multiplex PCR. Considering the current trend toward rapid cycling with reduced reaction volumes in PCR protocols and techniques, it is strongly recommended that manufacturers of thermal cyclers pursue substantial improvements in temperature control performances under these conditions. At the minimum, such performance parameters should be provided to researchers so they can make proper selections and adjustments regarding their specific experimental approaches. In the meantime, users of thermal cyclers can apply the findings of this work to assist in the design of sensitive PCR assays that require high levels of reliability.