Operational T&E

The key step of an effective T&E effort, which drives product deployment, is the conduct of Operational Test and Evaluation (OT&E). OT&E is a formal test and analysis activity, performed in addition to and largely independent of the DT&E conducted by the development organization. OT&E brings a sharp focus on the probable success of a development article (software, hardware, complex systems) in terms of performing its intended mission once it is fielded. Probable success is evaluated primarily in terms of the “operational effectiveness” and “operational suitability” of the system in question. Operational effectiveness is a quantification of the contribution of the system to mission accomplishment under the intended and actual conditions of employment. Operational suitability is a quantification of system reliability and maintainability, the effort and level of training required to maintain, support and operate the system, and any unique logistics requirements of the system.

While DT&E comprehensively tests to the formal program requirements, OT&E concentrates on assessing the Critical Operational Issues (COI) identified for each program. Measures of Effectiveness (MOEs) are defined (ideally, early in the program life cycle) to support quantitative assessment of the COI. An MOE may reflect test results for one or several key requirements, while some (secondary) requirements may not map into any MOE. Similarly, the OT&E team uses Measures of Suitability (MOSs) to quantify development product performance against the “ilities” relevant to the particular development product. While the DT&E team may give little attention to evaluation of MOEs and MOSs, the OT&E team uses these technical measures extensively to focus their test planning and as a standardized and compact vehicle for communicating their findings to responsible decision authorities.

An OT&E campaign conventionally has two phases: an initial study and preparation phase, followed by a highly structured formal testing phase. Significant OT&E planning, assessment and preparation efforts occur throughout the first phase, which occurs in parallel to the development effort. Independent reports and recommendations are also made to the acquisition authority during this phase. The second phase, in DOD parlance known as Initial OT&E (IOT&E), follows final developer delivery of the target product, but precedes operational deployment. IOT&E is a series of scripted tests conducted on operational hardware, using developer-qualified deliveries, and under test conditions as representative as practical of the expected operational environment. If testing results meet pre-established criteria, IOT&E culminates with a recommendation to certify the development article for operational deployment. Once past the Full Rate Production milestone, Follow-on OT&E (FOT&E) of the development article may occur to verify the operational effectiveness and suitability of the production system, determine whether deficiencies identified during IOT&E have been corrected, and evaluate areas not tested during IOT&E due to system limitations. Additional FOT&E may be conducted over the life of the system to refine doctrine, tactics, techniques, and training programs and to evaluate future increments, modifications, and upgrades.
OT&E independence from the development program (including the program manager and immediate program sponsors) is a key attribute distinguishing it from DT&E. However, use of a common Test and Evaluation Master Plan (TEMP) is typical, and well controlled integrated DT/OT testing (integrated testing) is encouraged.

How GreenDart can help you: We are proven experts in designing and executing operational T&E programs for all hardware and software developmental program efforts. Please contact us.

Please provide any comments you might have to this post.

Written by Bedjanian