Performance of BioFire array or QuickVue influenza A + B test versus a validation qPCR assay for detection of influenza A during a volunteer A/California/2009/H1N1 challenge study

Background Influenza places a significant burden on global health and economics. Individual case management and public health efforts to mitigate the spread of influenza are both strongly impacted by our ability to accurately and efficiently detect influenza viruses in clinical samples. Therefore, it is important to understand the performance characteristics of available assays to detect influenza in a variety of settings. We provide the first report of relative performance between two products marketed to streamline detection of influenza virus in the context of a highly controlled volunteer influenza challenge study. Methods Nasopharyngeal swab samples were collected during a controlled A/California/2009/H1N1 influenza challenge study and analyzed for detection of virus shedding using a validated qRT-PCR (qPCR) assay, a sample-to-answer qRT-PCR device (BioMerieux BioFire FilmArray RP), and an immunoassay based rapid test kit (Quidel QuickVue Influenza A + B Test). Results Relative to qPCR, the sensitivity and specificity of the BioFire assay was 72.1% [63.7–79.5%, 95% confidence interval (CI)] and 93.5% (89.3–96.4%, 95% CI) respectively. For the QuickVue rapid test the sensitivity was 8.5% (4.8–13.7%, 95% CI) and specificity was 99.2% (95.6–100%, 95% CI). Conclusion Relative to qPCR, the BioFire assay had superior performance compared to rapid test in the context of a controlled influenza challenge study. Supplementary Information The online version contains supplementary material available at 10.1186/s12985-021-01516-0.


Background
In uenza remains a major global health concern with signi cant morbidity and mortality from seasonal infections and poses the potential for catastrophic pandemics (1). In addition to the human cost, in uenza infection also results in a tremendous economic burden with more than 20 million days of lost annual productivity and an estimated $11.2 billion in annual direct or indirect costs in the United States alone (2).
Distinguishing in uenza infection from other acute respiratory conditions based on symptoms alone is di cult (3), but timely and accurate detection of in uenza infection is a key component of both global disease surveillance monitoring, individual clinical case management, and clinical studies. A goldstandard for de nitive diagnosis of in uenza infections is a quantitative real-time polymerase chain reaction (qPCR) assay where the copies of viral nucleic acid in clinical samples are quanti ed and compared to a standard curve produced using the same strain (4,5). However, standard qPCR approaches are time consuming and require trained operators in a laboratory setting (5). Other options for more rapid diagnosis are designed to overcome some of these limitations, including point-of-care colorimetric immunoassay based rapid in uenza tests, and sample-to-answer PCR-based systems (6).
However, performance characteristics of each assay should always be a major consideration when choosing an in uenza test.
Controlled human infection studies where healthy volunteers are challenged with in uenza virus are important tools for evaluation of novel in uenza therapeutics (7)(8)(9). Such studies also provide a rare opportunity to examine the performance characteristics of in uenza diagnostics in a highly disciplined setting. While multiple studies have reported the relative performance of different in uenza detection methods (10), such assays are rarely (if ever) compared using clinical samples obtained from a cohort of individuals all exposed to a genetically identical virus. Samples obtained in a volunteer A/California/2009/H1N1 in uenza challenge study were separately compared to test outcomes of a validated qPCR assay versus a sample-to-answer qPCR device (BioMerieux BioFire FilmArray RP) and an immunoassay based rapid test kit (Quidel QuickVue In uenza A + B Test).

Methods
In uenza challenge study design and sample collection   (Table S1). To evaluate the quality of the NP swab samples, a separate PCR reaction was performed to detect the Human RNPase P gene. Detection of this gene con rms that the swab sample is of su cient quality that cell associated virus can be detected and quanti ed and acts as an internal control for any possible PCR inhibitors in the swab sample. A one-step quantitative RT-probe Hydrolysis kit, Ambion AgPath-ID™ One-Step kit (Thermo Fisher, Waltham, MA) was used in the PCR reaction following manufacturer's instructions. Final concentration of primers was 0.8 µM and 0.2 µM for the probe. 5 µL of the extracted material was used in each reaction. PCR conditions using an Applied Biosystems ABI 7500 PCR system (Thermo Fisher, Waltham, MA) were as follows: 50.0 °C for 30 minutes; 95.0 °C for 10 minutes; 45 cycles of 95 °C, 15 seconds followed by 55 °C for 34 seconds.
To develop a standard curve for quantitation, the HA gene sequence was obtained from GenBank (KU933485.1) for A/California/07/2009. A forward primer at positions 1-25 (ATGAAGGCAATACTAGTAGTTCTGC) with a 5' T7 promoter and a reverse primer at positions 1702 − 1673 (TTAAATACATATTCTACACTGTAGAGACCC) were used to generate a transcript of 1702 base pairs in a one-step RT PCR reaction. The product was run on a 1% gel and the band was puri ed with the Zymoclean Gel DNA Recovery Kit (Zymo Research, Irvine, CA). A Megascript T7 transcription kit (Thermo Fisher, Waltham, MA) was used to generate an RNA transcript. The transcript was cleaned up using the Qiagen RNeasy Mini Kit (Qiagen, Hilden, Germany), run on a 1% agarose gel to con rm the size, and then quanti ed by multiple readings on a Nanodrop. The concentration and copy number were calculated from the OD readings. Standard curves were generated by freshly diluting transcripts tenfold from 4.0 × 10 6 to 4.0 copies/µL (2.0 × 10 7 to 20.0 copies/reaction) prior to each run. Standard curves were shown to have an average e ciency of 100% based on the slope of the curves. A positive control of extracted A/California/04/2009 H1N1 virus was run in the reaction over 20 times by two technicians over a 5-week period to obtain data to set an acceptance range based on 2 standard deviations of the average quantity obtained in the assay.
To con rm speci city towards A/California H1N1, eight different in uenza A and B viruses (Table S2) were tested in the assay. Only A/California H1N1 speci c isolates were detected in the assay. Additionally, to demonstrate speci city of the primers and probe, the PCR product of the positive virus control was run on a 2% agarose gel and assessed for a band at 177 bp to con rm that only the targeted portion of the gene was ampli ed ( Figure S1). Intra-assay precision and intermediate precision was determined to have a coe cient of variance (CV) of 11% and 25%, respectively.
The limit of detection (LOD) for the assay was determined from running the standard reference in twofold dilutions surrounding the lower end of the standard curve in replicates of 20. The LOD was then calculated as the concentration where 95% of the reference standard dilutions gave a positive response (Ct ≤ 40). The LOD was calculated to be 16 copies/reaction. For purposes of comparison with BioFire or rapid test, qPCR samples above the LOD were considered positive, samples below the LOD were considered to be negative.
BioFire FilmArray NP swab samples were loaded into the Bio re FilmArray respiratory panel cassette according to manufacturer's instructions and analyzed using the BioFire FilmArray Multiplex PCR System (BioMerieux, Marcy-l'Étoile, France). This device uses a fully automated procedure for nucleic acid puri cation, ampli cation, multiplexed PCR and melting analysis, and generates a report with binary outcomes for various respiratory pathogens. Samples positive for in uenza A in this report were considered positive, all other samples were considered negative. Samples with positive for any other targets were excluded from analysis.

Results
During a recent volunteer in uenza challenge study, 143 healthy volunteers were challenged intranasally with A/California/2009/H1N1 virus. Nasopharyngeal (NP) swabs were routinely collected and tested for presence of in uenza virus as part of the clinical conduct of this study. Duplicate NP swabs were collected and used for analysis with either the BioMerieux BioFire FilmArray RP (BioFire) or Quidel QuickVue In uenza A + B Test (rapid test) in addition to the validated qPCR assay, thus providing an opportunity to compare the relative performance of these tests.
To facilitate the analysis described here, the gold-standard qPCR assay was considered to be diagnostically accurate for detection of in uenza A virus in all samples. All values above the LOD by qPCR were recorded as positive, and all samples below the LOD were recorded as negative.   Fig. 1). As such, the sensitivity and speci city of the BioFire assay relative to qPCR was 72.1% (63.7%-79.4%, 95% Con dence Interval (CI)) and 93.5% (89.3%-96.4%, 95% CI), respectively (  Fig. 2). This resulted in a sensitivity of 8.5% (4.8%-13.7%, 95% CI) and speci city of 99.2% (95.6%-100%, 95% CI) for rapid test relative to qPCR ( Table 2).

Discussion
Rapid and simpli ed methods for in uenza virus detection are important tools, but it is important to understand how these tests compare to other assays in different settings. Here the performance of BioFire, a simpli ed sample-to-answer qPCR-based detection method, and QuickVue In uenza A + B Test rapid test, a colorimetric immunoassay, were assessed by comparing their diagnostic results to a validated qPCR assay using paired samples from a A/California/2009/H1N1 volunteer challenge study. Based on this analysis, BioFire was largely consistent with qPCR, while the rapid test suffered from a large degree of false negatives.
BioFire is an arrayed multiplexed qPCR-based device delivering binary results (positive or negative) for targets included in its assay cassettes. Although BioFire had a high speci city of 93.5%, sensitivity of BioFire was lower at 72.1%. This lower sensitivity may re ect a higher LOD by the BioFire assay for the A/California/2009/H1N1 in uenza challenge strain compared to qPCR in the context of this study. A higher LOD could result from differences in sample prep, primer sets, instrumentation, or programmed reporting threshold. Our calculated sensitivity of 72.1% for BioFire was only slightly lower than the 73-89% sensitivity range reported in other studies (13)(14)(15). Taken together, while such assays are simpler to use and offer a quicker turnaround compared to standard qPCR, this may come at the expense of slightly decreased sensitivity.
Rapid in uenza diagnostic tests are well known to have decreased sensitivity compared to qPCR (16). While some studies have reported sensitivities as high as 63-71% (17,18), others report much lower values in the range of 26-33% (19,20). In our analysis the rapid test sensitivity was very low, at only 8.5% relative to qPCR. This wide range of sensitivities reported across studies may be attributed to differences in virus strain or magnitude of antigen load, at least in part, by the day post-infection in which samples are collected. Additionally, as a colorimetric test, the qualitative results are susceptible to greater variation due to biases in operator readings compared to the quantitative driven results of qPCR and BioFire. Nevertheless, the speci city of the rapid test is very high. Our study found a speci city of 99.2% for this test, in line with other reports of 96-100% speci city (17)(18)(19)(20). Therefore, despite poor sensitivity, when present, a positive rapid test is a strong indicator that a sample will also be positive by qPCR.
For operational reasons, this study was limited to examining samples starting at 4 days post-viral challenge, corresponding to approximately three days post-symptom onset in challenge models (7). In other studies examining performance of the in uenza detection assays during community acquired infection (13,(17)(18)(19)(20), NP samples were collected immediately when patients met a set of subjective symptom criteria. Symptom-based selection in those studies may bias for individuals with higher magnitudes of viral titer and antigen load compared to a controlled challenge study in which all study participants are assessed. These differences may explain the reduced overall performance of the rapid test in this study compared to prior studies. Nevertheless, the data reported here is important for our broader understanding of the performance of this diagnostic tool, especially for those considering their use in future in uenza challenge studies. If available from future in uenza challenge studies, an examination of samples taken over the entire course of virus shedding period, including from earlier timepoints post-inoculation would be worthwhile to determine the relationship between day postinfection, viral load, and performance of both rapid tests and sample-to-answer molecular tests for in uenza A.

Conclusion
In summary, we have compared performance of two simpli ed in uenza A detection methods to a validated qPCR assay in a controlled A/California/2009/H1N1 challenge study. In this setting, BioFire closely re ected virus shedding detected by qPCR, while the rapid test did not. While not without limitations, integration of sample-to-answer in uenza tests such as BioFire into environments where standard qPCR assays are impractical stands to greatly improve detection of in uenza over antigenbased rapid tests alone.

Additional Files
Additional File 1 (.xlsx): Table S1 Additional File 3 (.pdf): Figure S1 Figure S1: Con rmatory PCR product from positive virus control. An expected single band of ~177bp ampli ed by PCR demonstrates speci city of the primers to amplify the targeted region of the HA gene. Figure 1 Venn diagram of BioFire performance vs qPCR. Number of samples positive by qPCR (blue), positive by BioFire (green), or positive by both qPCR and BioFire (yellow). Samples negative for both qPCR and BioFire (grey). Size of circles is proportional to n.

Figure 2
Venn diagram of rapid test performance vs. qPCR. Number of samples positive by qPCR (blue), positive by rapid test (green), or positive by both qPCR and rapid test (yellow). Samples negative for both qPCR and rapid test (grey). Size of circles is proportional to n.

Supplementary Files
This is a list of supplementary les associated with this preprint. Click to download. AdditionalFile1TableS1.xlsx AdditionalFile2TableS2.xlsx AdditionalFile3FigureS1.pdf