About BBT Testing System Go to top

BBT platform is based on the Black Box Testing methodology (Figure 1). This type of testing ignores internal mechanisms of the system or component being observed and focuses specifically on the outputs generated as the system response to specific inputs and conditions of test execution. Test designer selects valid input and determines the correct output.

BBT platform is a framework for automated testing of multimedia devices. BBT platform consists of software and hardware components offering a wide range of possibilities for testing of integrated DTV systems, digital satellite and terrestrial receivers (STB), DVD, Blu-ray players etc. Platform utilizes Python scripting language for test automation, providing ways to control all components of a test system through simple Python API.

Black Box Testing methodology
Figure 1. Black Box Testing methodology
 

Typical BBT Test setup Go to top

Considering general Black Box Testing ideas shown in Figure 1, we will present you with a typical test-setup based on our BBT platform. As you can see in Figure 2, each general item from Figure 1 has its representation in our system, with addition of BBT test station which controls all components of the test system. Test station executes automatic test scripts which are written in Python scripting language. Python test scripts have access to each hardware or software component of the system through platform API functions. Central software component of the test station is RT-Executor, application through which test scripts are loaded and executed and which provides drivers needed to establish control of physical hardware from Python script.

BBT Platform
Figure 2. BBT Platform

BBT test station may control various signal sources in order to provide known inputs to DUT. In example use-case, we may use stream modulator such as Dektec DTU-215 to provide known test input to STB being tested. The list of supported signal sources is provided in supported devices section.

As  Black Box Testing methodology assumes that we control DUT by using only its external controls, we follow the same approach here. Typical STB/DTV device is controlled via remote controller and therefore BBT platform provides several programmable remote control units, such as  RT-IR001 and  RT-IR016U (see section Remote Controller device). Issuing RCU commands to DUT is fully supported in Python, with the possibility to create complex navigation patterns using macros.

Having DUT set into expected state and providing known input, the next step is capturing DUT output. For DUT such as STB, typical output is HDMI. RT-RK provides its own audio/video grabber device RT-AV100, which is a perfect match for testing purposes. Furthermore, BBT platform provides support for 3rd party devices such as Black Magic grabber device.

The outcome of a test is determined by analyzing captured DUT output. For this purpose, BBT platform provides various audio/video/picture analysis algorithms (treated as Logical Devices) which may be used to analyze various aspects of captured audio/video/picture. For example, OCR device may be used to recognize text on part of the captured image, Picture Compare device may be used to match a part of captured image with the referent image, Audio Presence Algorithm may be used to detect presence of audio in a captured audio content.

 

BBT test execution procedure Go to top

Flow chart of test execution procedure used in BBT Test System is illustrated on Figure 1. As figure shows, it is not as simple as loading and executing single test case, due to the fact that each test case requires proper environment in which it is executed. This environment assumes devices needed for test which are properly initialized.

Figure 3: BBT test execution procedure

Loading of test plan is actually a preparation phase for the test case execution. This operation includes several important operations:

  • Obtaining the list of test cases from the server
  • Loading the list of devices used in a test plan
  • Initialization of loaded devices

At the end of this step, test execution environment if fully initialized and ready for the test execution.

Load Test Case is a phase where main test script file and all other files and configurations relevant for testing (Audio/Video referent files, region masks etc.) are loaded from the server and stored locally to the test station.

Execute Test Case is a main part of a test procedure where main script file is loaded and executed. Script file is written in Python programming language as a series of steps which define test scenario. More details how to write Python script file can be found in Test Scripting section.

Store Results To Database stage is executed after previous test case script has been successfully executed. In this stage, main test case result (Pass, Fail or Inconclusive) will be stored to the central database together with all intermediate results obtained during test case execution.

Generate Reports stage is performed after all test cases from loaded test plan are executed and their results written to the database. It’s important to mention that all the results are stored in the centralized database so user can create different kinds of reports, depending on which reports should be presented.

 

Get started! Go to top

Now that you are introduced to the basic principles of the Black Box Testing and corresponding components of the BBT platform, it is time to setup an environment needed to run your first test script using the BBT platform. Please follow these steps:

  1. Software set-up (Installation of suitable version of RT-Executor on your test station)
  2. Hardware installation (Typical STB test system usually includes RT-AV100 device)
  3. Creating and running your first test script