Remote Testing

OVERVIEW

Given the current COVID-19 environment and the pressure that puts on the mobility of test teams to complete Operational Test and Evaluation requirements on key programs, much more emphasis is now placed on integrating various aspects of remote testing into test event planning, execution and reporting. The three main test planning, execution and reporting efforts augmented by a more aggressive remote testing requirement are: pre-event information capture from non-traditional operational test sources; operational observation; event execution data collection and in-process event early assessments coordination.

PRE-EVENT DATA/INFORMATION/CAPABILITY CAPTURE

Techniques: Developmental test results analysis; customer, contractor, user community coordination for suitable data sources.

Tools: Developer/community test environments.

For those test events where local representation and data collection will be limited, the use of non-traditional data sources for operational system assessments may be employed. Specifically, product developer test results data and general user community operational information can be exploited and used as part of operational system performance assessments. Additionally, the use of developer/ integrator test facilities, equipment and staff may be considered, assuming the geography promotes this, to create additional operational test data. These pre-event data captures must come from sources with customer-credible pedigree and the pre-event collected data must be controlled as though captured at the specific operational test event to sustain independence. The strategy to use this type of data should be assessed against the need to verify critical functionalities of the operational system. This type of data collection is useful for non-critical validation or for reducing the time spent in OT once restrictions are lifted.

EVENT OBSERVATION AUGMENTATION

Techniques: Remote observation opportunity assessments

Tools: Non-intrusive real time collaboration environments such as Zoom, Skype, and others and non-real time operator interaction recording that can be analyzed at a later time.

Since a large part of operational assessments and the broader operational test and evaluation events require visual inspection of operator-system interactions remote testing strategies may include a remote capability to visually capture operational performance. Typically, these techniques include real-time initiatives such as screen-sharing and recording using remote collaboration tools such as Zoom and Skype. In other situations, these operator-system interactions are captured by organic system recording and logging functions. In general, any recording technology inserted into the operational test environment cannot impact the system under test. Therefore, the insertion of a remote observation/local recording capability must be justified and must be verified to have minimal impact to normal operations performance.

EVENT EXECUTION/DATA COLLECTION

Techniques: Creation of remote data capture strategies; coordination with operational site coordinators.

Tools: Remote data capture enabler technologies such as cloud access/storage mechanisms.

Remote testing relies heavily on close coordination and cooperation with the local operational communities and others that are  official witnesses to the event.  This is very true for remote data collection. Remote data collection requires a well-defined test script actions and close pre-test, execution, and post-test coordination with local data capture enablers.

Other critical considerations: adequate training of remote operational test team in system-under-test operation, data collection system function, and awareness and intent of operational test. Ensuring these three items will ensure adequate data volume, resolution, and quality of test observations exist to support robust evaluation development.

How GreenDart can help you: We are proven experts in designing and executing operational T&E programs for all hardware and software developmental program efforts. Please contact us.

Authors: Stramese, Turnbaugh, Bedjanian