Planning a Method Verification Study in Clinical Microbiology Labs

Method verification studies are standard practices in any clinical laboratory. These studies are required by the Clinical Laboratory Improvement Amendments (CLIA) (42 CFR 493.1253) for non-waived systems before reporting patient results. Non-waived systems, which include tests of moderate or high complexity, refers to the skills, reagents and steps needed to perform the assay and require documentation that they can be reliably used in the operator’s environment. This should be done for any new assay or equipment and when there are major changes in procedures or instrument re-location. However, the process can be confusing and verifications for microbiological methods don’t always fit the parameters for analytical assays. So how do you begin to confirm that your new test panel is ready to use?

Determine the Purpose of the Study

Is it a Verification or a Validation?

The terms validation and verification are sometimes used interchangeably. However, they are different. A validation is a process meant to establish that an assay works as intended. This applies to non-FDA cleared tests (e.g., laboratory developed methods) and modified FDA-approved tests. Modifications are changes to the assay not specified as acceptable by the manufacturer and can include using different specimen types, sample dilutions or test parameters such as changing incubation times. These changes could affect the performance of the assay and would need to be validated before implementing. A verification is for unmodified FDA-approved or cleared tests. It is a one-time study meant to demonstrate that a test performs in line with previously established performance characteristics when used as intended by the manufacturer.

A Qualitative or Quantitative Assay?

It is important to know the type of assay being implemented since it can influence how the CLIA standards are met. Testing methods can be divided into 3 main categories based on the results that will be reported:

Qualitative and semi-quantitative are the more common assay types for a microbiology lab.

Establish the Study Design

Once you know the purpose of the study, you can determine what criteria needs to be tested. For an unmodified FDA-approved test, laboratories are required to verify the characteristics listed below:

Below are suggestions to meet the verification criteria for qualitative/semi-quantitative assays:

Accuracy

For the number of samples, use a minimum of 20 clinically relevant isolates. For qualitative assays, use a combination of positive and negatives samples and for semi-quantitative assays, use a range of samples with high to low values. The acceptable specimens can come from standards or controls, reference materials, proficiency tests, de-identified clinical samples, if tested previously or in parallel with a validated method or consider including different sample matrices, if applicable. For the calculations, use the number of results in agreement over total number of results multiplied by 100. The acceptable percentage of accuracy should meet the stated claims of the manufacturer or what the CLIA director determines.

Precision

For the number of samples, use a minimum of 2 positive and 2 negatives tested in triplicate for 5 days by 2 operators. If system is fully automated, user variance is not needed. For qualitative assays, use a combination of positive and negatives samples and for semi-quantitative assays, use a combination of samples with high to low values. The acceptable specimens can come from controls or de-identified clinical samples. For the calculations, use the number of results in agreement over total number of results multiplied by 100. The acceptable percentage of precision should meet the stated claims of the manufacturer or what the CLIA director determines.

Reportable Range

For the number of samples, verify using a minimum of 3 samples. For qualitative assays, use known samples positive for the detected analyte and for semi-quantitative assays, use a range of positive samples near the upper and lower ends of the manufacturer determined cutoff values. To evaluate, the reportable range for a qualitative or semi-quantitative assay will be defined as what the laboratory establishes as a reportable result (e.g., Detected, Not detected, Ct value cutoff), verified by testing samples that fall within the reportable range.

Reference Range

For the number of samples, verify using a minimum of 20 isolates. For qualitative and semi-quantitative assays use de-identified clinical samples or reference samples with a result known to be standard for the laboratory’s patient population. This can be provided by the manufacturer, such as, samples negative for methicillin resistant-Staphylococcus aureus (MRSA) for an assay detecting MRSA. The reference range for a qualitative or semi-quantitative assay will be defined as what the laboratory establishes as an expected result for a typical sample, verified by testing samples representative to the laboratory’s patient population. If the manufacturer’s reference range does not represent the laboratory’s typical patient population, additional samples from the laboratory’s population should be screened and the reference range re-defined.

Once you have the above areas outlined, write a verification plan for what needs to be done before starting the study.

Create a Verification Plan

If a written verification plan is required by your lab, it will need to be reviewed and signed off by the lab director. The verification plan should include: