Challenges

Having products and technologies which allow consumer electronic device manufacturers and TV operators to successfully enable end-users to enjoy Pay-TV content, this company wanted to incorporate automatic test suite during development phase of their products in order to improve quality and speed up the testing of wide range of STBs they produce.

Their aim was to use this environment in order to quickly pass acceptance tests and check for regressions. Also they wanted to cover scenarios which are otherwise difficult to validate manually, e.g. requirements related to product robustness and performance.

Summary

REQUIREMENTS

  • Acceptance/Regression functional testing of STB
  • Robustness/Performance testing of STB
  • Automation of customer’s test cases
  • Low maintenance

ACHIEVMENTS

  • 200+ automatic tests at the end of the each project
  • All tests fully applicable on different platforms with minimum effort
  • Reduced human effort and influence during test execution
  • Improved robustness of final product
  • Improved efficiency of test engineers

Solution

Solution based on basic test system equipped with RT-AV100 grabber device and RT-Executor application was deployed. RT-Executor is used to control:

  • the grabber device
  • versatile devices in the test environment (signal modulators, power switches, RC emulators, etc.)
  • different algorithms for audio/video analysis from different STB outputs (SCART, HDMI, S/PDIF, etc.)

As a starting point the fact that most of customer STBs share common features is used. A test suite containing around 150  tests which covers those features is developed, significantly reducing the time needed by test engineers to customize automatic test environment to a new platform. In early stage of development this customization usually means covering the basic set of STBs features, making the test suite ideal to speed up the acceptance phase.

As project grows, so the number of test cases executed grows. For each new finished feature test case is added which covers this feature resulting in test suite which contained several dozens of test cases in beginning to several hundred test cases at the end of the project. The fact that automatic test environment is doing the job, enabled running of full set of long run robustness/performance tests for each release. In parallel testers are free to work on other tasks.

Quick customization and executing of already implemented features (minimum changes once the script is written) is ideal when you need a set a small team of testers, which still need to be able to run significant number of tests and ensure quality of the product.

qa-procedures
QA Procedures