Test Execution and Analysis (41)
Description:
Test execution and analysis involves testing the actual RUC system by conducting the execution test scripts (see Test Plan and Scripts building block), recording test results, correcting discovered issues, and retesting after issues are corrected.
Details:
The vendor executes unit tests, and the state validates the vendor’s unit test results based on the test script and specification documents. When integration tests are needed, the developers of both systems participate, and the state validates the test results. The state designs and oversees system acceptance testing, or end-to-end testing, with support from the vendors. Small-scale operational trials can be carried out when system corrections based on end-to-end test results are complete.
Primary Use:
Check system behavior and performance and correct issues discovered.
Best Practices/Lessons Learned:
- Manage the project schedule closely during testing.
- Make a formal record of and follow up on any issues discovered as soon as possible. Record issues that must be accepted without fixing.
- Small-scale operational trial system behavior is identical to live system operations and may transition to live system operations when ready.
- End-to-end testers are generally “friendly” to the system and informed/skilled in system testing. Small-scale operational trial testers are normal system users.
State Government Context and Assumptions:
This task should be completed by the state and technology vendors together. Unit testing begins as soon as software units are developed. Integration testing begins when both system interfaces are complete. End-to-end testing begins after unit testing and system integration testing are complete, and the small-scale operational trial begins after issues discovered in end-to-end testing have been corrected. In an open market system, certification processes can replace this entire building block of RUC system testing.