KD Executor
A Quick Tour of KD Executor 2.0

KD Executor 2.0 is a technology and market leader in automated testing of Qt based applications. By recording actual user input, KD Executor quickly captures new tests. There are no complicated test scripts to write or debug, making it easy for users of all skill levels to quickly become proficient in its use. At any time during the capture process, you can probe your application to record key properties. The values of these properties become test benchmarks. KD Executor's sophisticated test management subsystem allows you to replay these tests in any combination and automatically validate your application's execution against these test benchmarks.

The purpose of this document is to provide a Quick Tour of KD Executor. After you read this document, you will be able to start testing your own applications. This tour deliberately ignores many of the more powerful and sophisticated features of KD Executor. Once you have successfully recorded and replayed your own tests, you should browse some of the Advanced Tours to better understand the more sophisticated features of KD Executor. These tours cover such topics as:

If you used the first generation of KD Executor, you are probably familiar with its command line approach. To record/replay a test, you would simply type:

     kdexecutor your-application

at your system's command prompt. Some users continue using KD Executor in this mode because it best fits their existing testing infrastructure. However, many users simply use the new graphical user interface provided with KD Executor 2.0.

You can start this visual environment by simply typing kdxshell at your command prompt. However, we suggest that you create an icon or a shortcut to the KD Executor shell executable in the bin directory of KD Executor. For example, creating a shortcut to kdxshell.exe on Windows-based systems will result in the following icon on your desktop:

When you start KD Executor for the first time, you will see an empty main window as in the screen shot below. The main window includes:

 
The KD Executor main window after first startup

Creating a New Test Suite

With KD Executor now running, the first step is to create a new Test Suite. A Test Suite is made up of one or more Tests that you typically execute together. A Test Suite may also include other Test Suites. This allows you the flexibility to organize your Test Suites to match your formal test plan.

You can create a new Test Suite by either selecting New Test Suite from the Test menu or by right clicking in the Test Management Area and selecting New Test Suite from the popup menu. In the screen shot below, the new Test Suite was created using the main menu.

 
Creating a new test suite is the first step in using KD Executor

Defining a New Test Suite

To create a new Test Suite, you need to define the following four groups of configuration options in the Edit Test Suite Dialog:

If your application needs any parameters to run, or should run with a specific working directory (e.g., perhaps it needs to load some icons at runtime), you can define these configuration parameters in the AUT section of the Edit Test Suite Dialog. You can also change the name of the Test Suite displayed in the Test Management Area. By default, it is the name of the AUT. However, if you are creating multiple Test Suites for the same application, you might want to change the name of the Test Suite to better identify the purpose of the suite.

The Edit Test Suite Dialog also provides direct access to configuring the runtime record/playback of KD Executor. These are advanced options that most users will not need and so will not be covered on this tour.

 
The edit test suite dialog is used to identify the AUT, its runtime environment and any pre/post processing

The next set of configuration options establish the global test environment for all the tests in the suite. The first two options in this section allow you to specify commands to be executed prior to the start of the AUT and after it is completed. This way you can set up a test environment by staging fresh copies of data files. After the test is run, you can check the results into your configuration management system and delete directories, files and databases that are no longer needed.

A Post-Test check command can also be defined here. This command will be executed after every test. Typically, you would use this to validate a database or some other data that is not visible at the user interface level. KD Executor interprets a non-zero exit from the Post-Test command as a failure.

The final section of the Edit Test Suite dialog deals with the handling of error codes generated by the AUT. By default, these non-zero error codes are ignored. However, you can request that KD Executor recognize these error codes and fail a test when the error code falls outside the range of successful error codes.

Creating a New Test Case

A Test Suite is an empty container; you have to define one or more Test Cases. The screen shot below shows the creation of a new Test Case by right clicking on a Test Suite and selecting New Test Case in the popup menu that appears. Alternatively, you could have selected the Test Suite and then selected New Test Case under the Test menu.

 
Test cases can be created using the main menu or by right clicking on a test suite

The creation of a Test Case requires the completion of the Create A New Test dialog. This dialog should seem familiar because it shares a number of similar configuration options with the Edit Test Suite dialog. The following four groups of options need to be filled in:

You are also required to provide a location and name for the KD Executor Test Script File. This file will be used to store the test script generated when you start your application in record mode.

The next group of options deal with the record and validation of your application's execution. There are two types of values you can record: Qt properties (i.e., values of text entry fields, states of checkboxes, etc.) and bitmapped screen shots. By default, KD Executor is configured to allow you to record both Qt properties and screen shots. The location of these files and their names are based on the script file provided for the test case itself. If you uncheck either of these items, the corresponding file will not be created.

 
Define the test script location and validation options, then hit Record to create your first test!

If you have screen shots, you can choose what happens when the comparison fails. Since screen shots can fail for both major changes as well as for irrelevant changes (screen size changes or increases/decreases in color depth), you have the option to review failed screen shots manually after the test is run. During this review, you can identify tests as pass/fail. KD Executor provides several mechanisms to identify where a screen shot fails quickly to match a baseline. More details are provided in the specialized tour on validating screen shots.

The Advanced Record/Playback Options section provides you with the ability to override the generic settings of the Test Suite. Like the Test Suite dialog, you can provide the names of commands or shell scripts that are executed before the test starts, validate data after the test run, and then clean up after the test.

Now that you have your Test Case defined, all you need to do is to hit the Record button (see the bottom of the screen shot) and start recording your test!

Executing the Test Case

Your application starts and the KD Executor Sidebar appears after you press the Record button. This Sidebar provides you access to a number of features including screen/widget snapshots, inclusion of pre-recorded test scripts, execution of external commands (for application validation), establishment of a synchronization point and access to the Property Picker. In this tour, we are just going to use the Property Picker. The rest of the options are covered in the other tours of KD Executor.

 
The KD Executor property picker appears as the AUT starts execution

Using the Property Picker

The Property Picker is both easy to use and powerful. You place the mouse pointer over any widget in your application and hit the left mouse button. A popup menu appears that includes the most common properties you might want to record at the top, and at the bottom, there is an "Advanced" menu that allows you full access to the widget hierarchy.

In the screen shot below, we are trying to capture the value of the text field that contains the characters "This is a Test". The property "text" contains these characters, so we highlight it and then release the mouse button.

 
Picking properties at runtime

Once you have picked a property, it is added to the Property Picker. The top part of the Property Picker lists the properties that are currently selected. The bottom of the Property Picker dialog provides you with the opportunity to include a corresponding description of the selected property. It is strongly encouraged that you add a description because if a comparison ever fails, this description is included in the test report. This will allow you to quickly isolate the test case and specific test that failed so that you can determine the appropriate corrective actions.

Note the checkbox in the lower left corner of the Property Picker. This allows you to save the set of properties you have picked so that you can reuse them later. As you create a set of tests, you will quickly notice that you tend to check the same set of properties repeatedly during the testing of your application. By saving your property selections, you can reuse them later and avoid having to manually select them each time. There is an additional advantage for collecting a common set of picked properties: if your user interface ever changes, you only have to make the changes in one place, and the change is propagated to all scripts that use this common set of picked properties.

 
Although descriptions are optional, it is highly recommended that you add them in the Property Picker

Establishing a Test Baseline

After recording your test and terminating your application, you will be returned to the main window of KD Executor. In the example below, you can see that the test case "Quick Tour" has been added to the current Test Suite. The status is marked as being Never Run.

 
New tests are shown as recorded and never run

If you click the checkbox next to the Test Suite name, all the tests of the suite are automatically checked. By selecting Run All from the Run menu, KD Executor will start the AUT and run all the tests. You also have the option to select only a couple of the tests in a Test Suite and run them using the Run Selected menu option.

 
Running all tests

Examining the Results of a Test

After you run a Test Suite, you should check the following for each test executed:

 
The test management area is updated to reflect the running of a test

The screen shot also shows that KD Executor has provided more details in the Test Results section. Here the various categories of validation tests are recorded and their success indicated by the word "OK". If a test failed, it would be indicated by the word "Failed".

Determining the Failure

If a test failed because it generated different values for the properties that were picked, you can use the KD Executor Property Editor to easily determine where the failure occurred. To bring up the Property Editor, right click on the specific test run and select Property Editor from the popup menu.

 
Examining test results using the Property Editor

The top half of the Property Editor shows a list of properties picked, with their description and values (both test and baseline). By clicking a specific row, you can load the baseline and test results into the lower half of the Property Editor. Although it will often be obvious by looking at the baseline and actual results in the top half what went wrong, in some cases you need the extra space provided in the lower half of the Property Editor to examine long text fields.

In some cases, you will find that you have changed your application, and the value produced by the test is now the correct result. In those cases, simply click the button Copy Actual Values to Expected Values at the bottom of the screen. The actual values will become the new baseline, and your test will be stamped as "OK" (assuming this is the last failure point).

KD Executor allows you to generate a html format test report after running a Test Suite. To generate this report, first click the Results tab in the Test Management Area. Then right click the timestamp for the Test Suite and select the Export to HTML option (see the screen shot below).

 
KD Executor can generate HTML formatted test reports

To review the actual report generated by this Quick Tour, click Here.

Before you exit KD Executor, you should save your Test Setup. A Test Setup is your personal collection of Test Suites, Tests and Results. It is effectively what you see in the Test Management Area. The Test Setup under KD Executor is a key concept. It allows multiple testers to share a common set of test scripts, while organizing them in a manner that make sense for their job. For example, many companies create a Test Setup for their nightly regression tests. This Test Setup is run overnight by either UNIX/Linux cron jobs or a Windows or MacOS scheduling facility.

Saving a Test Setup is easy. Just select Save Test Setup in the file menu. See the screen shot below.

 
Saving your test setup

After you have successfully saved your Test Setup, you will notice the KD Executor window title change to point to the location of your saved Test Setup. See the title bar in the window below.

 
The window title changes to reflect the name of the test setup

What's Next?

This tour is just the beginning! KD Executor is a sophisticated tool that provides the facilities needed to test even the most demanding Qt-based application. The next step is to try KD Executor on a couple of your applications and gain some experience with the tool. Afterwards, there are all the other tours to take to get to know KD Executor in depth!