Required Steps for the Validation of a Laboratory Information Management System

Elizabeth Turner, Laboratory Chief Jojean Bolton, Quality Control Officer
USACE Washington Aqueduct Washington, D.C.

The task of managing laboratory data is not a new one. Over the past two decades, the use of Laboratory Information Management Systems (LIMS) has revolutionized how laboratories manage their data. A LIMS is more than software; it has become the workhorse of the laboratory encompassing laboratory workflow combined with user input, data collection, instrument integration, data analysis, user notification, and delivery of information and reporting. Types of organizations that utilize LIMS vary greatly from research laboratories to manufacturing laboratories to environmental testing laboratories.

Commercially-available LIMS have been around since the 1980's. In addition, many laboratories have designed, implemented, and maintained in-house LIMS. The heart of any LIMS is the software. Like other laboratory systems, the LIMS software is subject to quality control and quality assurance checks. In regulatory environments, this associated QA/QC is referred to as "system validation." The primary purpose of system validation is to ensure that the software is performing in a manner for which it was designed. For example, the system acceptance criteria should be established and tested against quantifiable tasks to determine if the desired outcome has been achieved. LIMS features, such as autoreporting, reproducibility, throughput, and accuracy, must be quantifiable and verifiable. System validation ensures that the entire system has been properly tested, incorporates required controls, and maintains and will continue to maintain data integrity. Laboratories must establish protocols and standards for the validation process and associated documentation. Although vendors of commercial LIMS perform initial internal system validations, the system must be revalidated whenever the end user, vendor or third party adds modifications or customizations to the LIMS.

Currently, detailed guidance regarding system validation of LIMS is not available to the user. The issue is addressed in Good Automated Laboratory Practices (GALP) and National Environmental Laboratory Accreditation Conference (NELAC) documents which indicate specific requirements or recommendations for operational checks and periodic testing, however, it is up to the laboratory to determine suitable methods to accomplish these tasks. Proper validation of a LIMS will allow a laboratory to comply with regulations and also provide comprehensive documentation on the system that is necessary to troubleshoot future problems.

The validation of a Laboratory Information Management System (LIMS) is becoming an increasingly important issue for many laboratories. Over the past several years, guidenance and regulations addressing data system validation have been developed and applied. For the most part, the guidance documents such Good Automated Lab Practices (GALP) and USFDA Good Manufacturing Practices, 21 CFR part 11 (Electronic Signatures) and Quality System apply primarily to the food and pharmaceutical industries. In August 2001, the US Environmental Protection Agency (USEPA) proposed the Cross Media Reporting and Record-Keeping Rule for USEPA environmental programs and laboratories. Validation is a pre-requisite for all systems that handle regulatory data and is a fundamental requirement for compliance with 21 CFR part 11 and similar protocols. The objective of the validation process is to ensure that a system does what it purports to do and will continue to do so. Validation not only satisfies regulatory requirements but is also an excellent tool for organizations to use so that they can be confident that the LIMS performs the way it is expected to perform.

There is no single, standard way to plan and implement a validation process. The one universal truth in the validation process is that validation activities need to be conducted throughout the entire LIMS life-cycle. The validation process starts with the functional requirements development phase in the purchase of a LIMS and continues through specification, testing, implementation, operation and retirement of a system.

Principles of Software Validation
There are ten general validation principles that are applicable to a LIMS.

1. Timing: Validation is not a one time event. It should begin when planning and input for a system begin. Validation does not end until the product is no longer used.
2. Management: Proper validation of a LIMS includes the planning, execution, analysis and documentation of appropriate validation activities and tasks.
3. Plans: Established design and development plans should include a specific plan or how the software validation process will be controlled and executed.
4. Procedures: Validation procedures should be developed and documented. The validation process should be conducted according to the established procedures.
5. Requirements: To validate a LIMS there must be predetermined and documented requirements. If a request for proposal (RFP) was thoroughly developed, it will contain the requirements necessary for validation.
6. Testing: Verification includes static and dynamic techniques. Static techniques include paper / document reviews, dynamic techniques include physical testing - demonstrating the system's run time behavior in response to selected inputs and conditions. Dynamic analysis alone may not provide sufficient information to determine if the system is fully functional and free of avoidable defects. Static techniques are used to offset limitations of dynamic analysis. Inspections, nalyses, walk-throughs and design reviews may be more effective in finding, correcting and preventing problems at an earlier stage of the development process.
7. Partial validation: A system cannot be partially validated. When a change is made to the system, the validation status of the entire system should be evaluated.
8. Amount of effort: The magnitude of the validation effort should be commensurate
   with the risk associated with dependence on critical function. The larger the project and staff involved, the greater the need for formal communication, more extensive written procedures and management control of the process.
9. Independence: Where possible, the validation activities should be conducted using the basic quality assurance concept of "independence of review".
10. Real world. It is fully recognized that LIMS are designed, developed, validated and regulated in a real world environment. Environments and risks cover a wide spectrum and each time a validation principle is used, the implementation may be different.

Life Cycle Activities
Activities in a typical LIMS life-cycle include:

• Management / Project Initiation Phase
• Requirements Phase
• Design Phase
• Implementation Phase
• Installation and Test Phase
• Operation and Support Phase

The process of validation needs to start at the beginning of the LIMS lifecycle. Performing validation at the end of the process would add three to five months to a project and would preclude use of the LIMS during validation. For each of the life-cycle phases, certain validation tasks are performed.

Management / Project Initiation Phase
During design and development planning, a validation plan is developed to identify required validation tasks, procedures for reporting anomalies and their resolution, andresources needed for validation and management review requirements. The validation plan should include:

• Specific validation tasks for each life-cycle activity
• Methods and procedures for each validation task
• Acceptance criteria for each validation task
• Inputs for each validation task
• Outputs from each validation task
• Criteria for defining and documenting outputs
• Roles, resources and responsibilities for each validation task
• Risks and assumptions

21 CFR 820 requires that management identify and provide the appropriate validation environment and resources. Each validation task will require personnel as well as physical resources. The validation plan should identify the personnel, facility and equipment resources for each validation task. Procedures should be created for the review and approval of validation results including the responsible organizational elements for such reviews and approvals.

Requirements Definition Phase
A LIMS requirement specification document should be created with a written definition of the software functions to be performed. It is not possible to validate a LIMS without predetermined and documented requirements. Typical requirements specify the following:

• All inputs the system will receive such as sample collection data
• All outputs the system will produce such as reports
• All functions the system will perform such as calculations and trend analysis
• All performance requirements that the system will meet (data throughput, reliability, timing, etc.)
• The definition of all internal, external and user interfaces (instruments, other
  software)
• What constitutes errors and how errors should be handled
  The internal operating environment for the software (hardware platform, operating
  system, etc.)
• All ranges, limits, defaults and specific values the software will accept

Design Phase
The design phase of the LIMS life-cycle is performed by the LIMS vendor. In order to ensure that the vendor performs the required validation activities, the laboratory may request a vendor audit and specify the validation requirements in the request for proposal. The primary goal of the audit is to ensure that the vendor's software development and management procedures are consistent with accepted practices and are traceable to a reference point. Often a vendor will be certified by an independent auditor as complying with ISO 9000 standards. A copy of the LIMS vendor's ISO 9000 certificate may be sufficient to meet the design phase validation requirement.

Implementation Phase
Implementation is the activity where detailed design specifications are implemented as source code. For commercial LIMS products, the implementation phase is carried out by the vendors' programming departments.

Installation and Test Phase
Installation testing is an essential part of validation for a LIMS. Terms such as beta test, site validation, user acceptance test and installation verification have all been used to describe installation testing. Installation testing is any testing that is conducted at a user's site with the actual hardware and other software that will be part of the installed LIMS configuration. The testing is accomplished through either actual or simulated use of the software being tested within the environment in which it is intended to function. If the computers for the LIMS are placed on a network, a validation procedure for proper network installation and connection will need to be established. All modems, printers, fax machines and instruments for data acquisition will need to be tested for proper integration.

Test plans should be created prior to the system installation. They should identify the test schedules, test environments, resources (people, tools, etc.), methodologies, cases (inputs, procedures, outputs, expected results), documentation and reporting criteria. Individual test cases should be associative with particular specification elements and each test case should include a predetermined, explicit and measurable expected result derived from specification documents in order to identify objective success / failure criteria. Installation testing should follow the predefined plan with a formal summary of testing and a record of formal acceptance. There should be retention of documented evidence all testing procedures, test input data and test results. There should be evidence that hardware and software are installed and configured as specified. The testing phase should continue for a sufficient amount of time to allow the system to encounter a wide spectrum of conditions and events in an effort to detect any latent faults which are not apparent during normal activities.

The number of test plans will depend on the complexity of the LIMS and the level of detail required to adequately test the key features. Each externally visible function and each internal function should be tested at least once. Detailed written procedures and checklists are often used to facilitate consistent application of intended testing activities. The methodologies used to identify test cases should allow for a thorough examination of the LIMS application.

In addition to an evaluation of the ability of the LIMS to properly perform its intended functions, there should be an evaluation of the user's ability to interface with it. Users should be able to perform the intended operations and respond in an appropriate and timely manner to all alarms, warnings, errors, etc.

There are several areas that require special attention during a LIMS validation study. All procedures for data entry and receipt of data, either electronic or manual, must be formal and implemented to ensure consistent execution. Data ranges should be developed as well as system edits activated to limit the introduction of erroneous data. Each input field should be identified and the allowable input defined. Testing should be conducted to demonstrate accurate processing of valid or "acceptable" data and the rejection of invalid or "unacceptable" data. If the LIMS performs calculations, evidence of the reliability of the formula / algorithm must be documented. Documentation often takes the form of published algorithms. Most hard coded calculations are preexisting formulae and can be traced to a published source.

All reports generated by a LIMS must conform to be established criteria for format, content and accuracy. Pre-established criteria may be part of a LIMS RFP. The specification document for each system output should describe the contents and format of the report. For a user-defined report, the report details should be defined and configuration management documented.

Operation, Maintenance and Support Phase
After a system is validated and becomes operational, changes, which may impact validation status, ill occur during its operational lifetime. Any change to the LIMS should trigger consideration of revalidation of the system. There may be instances where no revalidation would be necessary after a change. One way to evaluate the need for revalidation is to review the impact the change would make to data accuracy, security and integrity.

Examples of changes to a system include hardware maintenance and upgrade, upgrade of the operating system and evolution of the LIMS application overtime. Configuration management is a set of procedures to insure adequate identification, control, visibility and security of any changes made to hardware, firmware, network, program source code or any specialized equipment associated with the LIMS.

The process of configuration management is quite simple. The initial system  configuration is thoroughly documented. A listing of all components of the system should be compiled: includes all the release numbers, serial numbers of the application software and the operating system. The components comprising the hardware such as disks, memory, type of CPU and any peripherals as well as any documentation that is used with the system should be listed as well. As modifications to the system configuration are made the information should be recorded in the configuration log.

Change control defines responsibilities and documents the process of change. Change can include the resolution of system bugs and errors. The impact of the change to the  LIMS needs to be evaluated. Items to consider include: time required to make the change, cost of the change, resources to make the change and benefits of the change. The effects of the change should be documented. Operational logs need to be updated to reflect any changes.

Validation activities are conducted throughout the entire LIMS lifespan. It starts with the requirements phase and continues through the operational phase. Even when a system is retired, it ust be noted in the operational logs for future reference. Proper validation of a LIMS system will allow a laboratory to comply with regulations and also provide thorough documentation on the ystem that will be needed to help troubleshoot the system.

References
ASTM. Guide E2066-00 Standard Guide for Validation of Laboratory Information Management Systems. 2001

EPA. Good Automated Laboratory Practices, Recommendations for Ensuring Data Integrity in Automated Laboratory Operations. Washington, D.C. August, 1995.

FDA. Quality System Regulation. 21 Code of Federal Regulations part 820, 1996.

McDowell RD. Operational Measures to Ensure Continued Validation of Computerized Systems in Regulated or Accredited Laboratories. Lab Automat Info Manage 31, 1995.

National Institute for Science and Technology. Software Verification and Validation: Its Role in Computer Assurance and Its Relationship with Software Project Management
Standards. 1996.

Paszko, C. and Turner, E. Laboratory Information Management Systems, Second Edition, Revised and Expanded. Marcel Dekker. 2001.