T 2.29 Software testing with production data

Frequently it happens that software tests are being performed with production data. The main reasons given for this are that the only way to make a definitive assessment of the functions and performance of the product is to compare it directly with existing, operating data. Additional reasons for doing this are inadequate security awareness, exaggerated confidence in the software under test, and ignorance of potential damage.

Testing with production data may result in the following problems:

• Software is tested with copies of production data in an isolated test environment:

If new software is tested with data which has not been made anonymous, unauthorised employees or third parties who have been put in charge of testing the software may gain access to files carrying information which are confidential.

• Software is tested with production data in actual operation:

Software which malfunctions under test may, as in the before-mentioned case, lead not only to impaired confidentiality but also to a loss of integrity and availability of production data.

Because different programs may be incompatible, side effects can arise which may lead to significant impairments in other system components. In the case of networks this may range from loss of performance through to a crashing of the network.

If software under test performs incorrectly or operating errors are made, production data may be inadvertently modified. It is possible that such a modification may not be able to be identified. To avoid redundancy, databases are increasingly shared by different programs, so that these errors potentially have an effect on other IT applications as well. When damage occurs there are not only costs involved in reconstructing the data but, existing working data must also be checked for integrity.