Test Run

A Test Run is the process of executing your defined tests. It involves running a selected Test Plan, which includes a collection of configured Test Cases, on a designated Bench and against the target. This action puts your test strategy into motion to verify the DUT’s behavior.

Executing a Test Run

1. Sign-in to vsec.blockharbor.io

2. Navigate to Test on the sidebar menu to enter the VSEC Test application and then select the Test Runs submenu.

3. Click the Add test run button to open the Create New Test Run dialog.

4. Fill out the required details to Create a Test Run.

   Field	Requirement	Description
   Bench *	Required	Select the Bench to execute the testing through. This can be a physical Bench or a Simulation Bench.
   Test plan *	Required	Select the Test Plan to execute.
   Test name *	Required	A unique, short descriptor (e.g., MY26 CAN Protocol Evaluation).
   Description	Optional	Internal notes or context regarding the purpose of this Test Run.

5. Click the Create button to execute / queue the Test Run.

If the plan contains Test Cases for an interface type the selected Bench does not have, the dialog warns you that those Test Cases will not be executed.

You can also start a Test Run from a Test Plan: click the play icon on its row in the Test Plans list, or the Run test plan button on its details page.

Running Tests on a Simulation Bench

Test Runs can be executed against a Simulation Bench instead of a physical Bench. Once selected, the Test Run executes the same way as on a physical Bench.

1. Click the Add test run button to open the Create New Test Run dialog.
2. In the Bench select box, choose the desired Simulation (e.g., UDS and XCP Starter Challenge).
3. Select the Test plan to execute.
4. Specify a Test name.
5. Click the Create button to execute / queue the Test Run.

Stopping a Test Run

While a Test Run is in progress, click the stop icon on its row in the Test Runs list and confirm with Stop run. Test Cases still waiting are cancelled, but the Test Case currently executing on the Bench cannot be aborted remotely and runs to completion.

Exporting Test Run Results

1. Navigate to the Test Runs submenu.
2. Select the desired Test Run by clicking the view icon ( ) on the corresponding row.
3. Details and Test Case results will be displayed with a passed / failed / error status.
4. Click one of these buttons at the top of the Test Run details to export. They are disabled until the Test Run has finished.
   • Download log - A zip file showing details relating to any errors on the Test Cases (e.g., vehicle-my-2026_run.zip).
   • Download report - A report-style PDF with Pass / Fail stats and descriptions (e.g., vehicle-my-2026_report_3.pdf).

Viewing Test Case Parameters

In a Test Run’s details, the Test Cases table includes a Parameters column listing the parameter name/value pairs recorded for that execution (or No parameters recorded if none were captured).

Comparing Test Runs

1. Open a Test Run’s details as described above.
2. Click the Compare button next to the download buttons.
3. Choose the Test Run to compare against from the Compare against… dropdown.
4. The Test Cases table switches to a comparison view with columns for Case, Interface, Status, Execution time, Results, and Parameter changes. Rows where the status differs between the two runs are highlighted, and any changed value is shown alongside the other run’s value. A Test Case that only ran in one of the two runs is flagged as only present in that run.
5. Click Exit comparison (shown as Cancel comparison before a target run is chosen) to return to the normal results view.

Deleting Test Run Results

1. Navigate to the Test Runs submenu.
2. Click the trash can icon ( ) to delete the Test Run.
3. Your browser asks you to confirm the deletion. Confirm to delete the Test Run.