Posts tagged with: exploratory


Exploratory testing with Testlab

In this article we introduce you to the recently released new features enabling more streamlined workflow for exploratory testing.

Exploratory testing is an approach to testing where the tester or team of testers ‘explores’ the system under test and during the testing generates and documents good test cases to be run. In a more academic way, it is an approach to software testing that is concisely described as simultaneous learning, test design and test execution.

Compared to scripted testing – where test cases and scenarios are pre-planned before execution – exploratory testing is often seen as more free and flexible. Each of these methodologies have their own benefits and drawbacks and in reality, all testing is usually something in between of these two. We won’t go into methodological detail in this article as we focus on how to do the actual test execution in explorative way with Testlab. We can conclude though, that exploratory testing is particularly suitable if requirements for the system under test are incomplete, or there is lack of time.


Pre-planning exploratory test cases

As said, all testing approaches can be usually placed in between a fully pre-scripted and fully exploratory approach. It is often recommended to consider if pre-planning the test cases in some way would be beneficial. If the system under test is not a total black box meaning there are some knowledge or even specifications available it might be a wise idea to add so called stubs for your test cases in pre-planning phase. Pre-planning test case stubs might give you better insight in testing coverage as in pre-planning, you have an option to bind the test cases to requirements. We’ll discuss using requirements in exploratory testing in more detail later.

For example, one approach might be that you could just add the test cases you think you might need to cover some or all areas of the system without the actual execution steps. The actual execution steps, preconditions and expected results would be filled out in exploratory fashion during the testing. Alternatively, you might be able to plan the preconditions and expected results and just leave the actual execution steps for the testers.

Keep in mind, that pre-planning test cases does not and should not prevent your testers from adding totally new test cases during the testing. Additionally, you should consider if pre-planning testing might affect your testers’ way of testing. Sometimes this is not desirable and you should take into account the experience level of your testers and how the different pre-planning models fit into your testing approach and workflow in overall. 


Exploratory testing session

Exploratory testing is not an exact testing methodology per se. In reality, there are many testing methods such as Session-based testing or pair testing which are exploratory in a way. As Testlab is methodology agnostic and can be used with various different testing methods, in this article we combine all these methods by just establishing the fact that the testing must be done in a testing session. The testing method itself can be any method you wish to use but the actual test execution must be done in a session which optionally specifies

  • the system or target under testing (such as a software version),
  • environment in which the testing is executed in (such as production environment, staging environment, …) and
  • sets a timeframe in which the testing must be executed in (a starting date and/or a deadline).

To execute test cases in a testing session add a new test run to your project in Testlab. Go to the Test runs view and click the Add test run… button to add a new blank testing session.

New testing session as a test run

When the needed fields are set you have an option to just Save the test run for later execution or, to Save and Start… the added test run immediately. The test run is added to the project as blank meaning it does not have any test cases bound to it yet. We want to start testing right away so we click the Save and Start… button.


Executing tests

The set of functionality while executing is a good match for an exploratory testing session. As said, test execution in Testlab enables you to

  • pick a test case for execution,
  • record testing results for test cases and their steps,
  • add issues such as found defects,
  • add comments to test cases and for executed test case steps,
  • add existing test cases to be executed to the run,
  • remove test cases from the run,
  • re-order test cases in the run to a desired order,
  • add new test cases to the test case hierarchy and pick them for execution,
  • edit existing test cases and their steps and
  • cut and copy test cases around in your test case hierarchy.

The actual user interface while executing looks like this:


The left hand side of the window has the same test case hierarchy tree that is used to manipulate the hierarchy of test cases in test planning view. It enables you to add new categories and test cases and move them around in your hierarchy. The hierarchy tree may be hidden – you can show (and hide) it by clicking the resizing bar of the tree panel. The top right shows you the basic details of the test run you are executing and the list below it shows the test cases picked for execution in this testing session.

The panel below the list of test cases holds the details of a single test case. When no test cases for execution are available, the panel disables itself (like shown in the shot above) and lists all issues from the project. This is especially handy when testing for regression or re-testing – it makes it easy to reopen closed findings and retest resolved issues.

The bottom toolbar of buttons enable you to edit the current test case, add an issue and record results for test cases. The “Finish”, “Abort” and “Stop” buttons should be used to end the current testing session. Keep in mind, that finishing, aborting and stopping testing have their own meaning which we will come to later in this article.


Adding and editing test cases during a session

When exploring, it is essential to be able to document the steps for later execution if an issue is found. This way scripted testing for regression is easier later on. Also, if your testing approach aims to document the test cases for later use by exploring the system you must be able to easily add them during execution.

If you have existing test cases added which you would like to pick for execution during a session you can drag the test cases from the test hierarchy tree on to the list of test cases. Additionally, you can add new test cases by selecting New > Test case for the test category you want to add the new test case to. Picking test cases and adding new via inline editing is demonstrated in the following video:




Editing existing test cases is similar to the adding. You just press the Edit button at the bottom bar to switch the view to editing mode. The edits are made in identical fashion compared to adding.


Ending the session

When you have executed the tests you want you have three options:

Finish, abort or stop

It is important to understand the difference which comes from the fact that each executed session is always a part of a test run. If you wish to continue executing tests in the same test run at the later time you must Stop the session. This way the test run can be continued later on normally.

If you conclude that the test run is ready and you wish not to continue it anymore you should Finish it. When done so the test run is marked as finished and no testing sessions can be started on it anymore. It should be noted though, that if you discard a result of a test case from the run later on the run is reset back to Started-state and is again executable.

Aborted test runs are considered discarded and cannot be continued later on. So, if for some reason you think that the test run is not valid anymore and should be discarded you can press the Abort run button.


Asset workflows and user roles in exploratory testing

Requirements, test cases and issues have an asset workflow tied to them via project’s workflow setting. This means that each asset has states that they can be in (In design, Ready for review, Ready, …) and actions which can be executed on them (Approve, Reject, …). In exploratory testing having a complex workflow for project’s test cases is usually not desirable. For example, having a workflow which requires review of test cases from another party makes no sense when testers should be able to add, edit and execute test cases inline during testing.

That said, if using default workflows it recommended to use the “No review” workflow for your projects. 


No review workflow


If executing test cases which are not yet approved as ready Testlab tries to automatically approve them on behalf of the user. This means, that if the test case’s workflow allows it (and the user has the needed permissions to do so) the test case is automatically marked as approved during the session. This way using more complex workflows in a project with an exploratory testing approach might work if the transitions between test case’s states are suitable. That said, as the testers must be able to add and edit test cases during executing having a review based workflow is useless.

The asset workflows’ actions are also tied to user roles. For the testers to be able to document test cases during execution the tester users should also be granted a TESTDESIGNER role. This ensures that the users should have the needed permissions to add and edit test cases they need.


Using requirements in exploratory testing approach

When designing test cases in exploratory testing sessions the test cases are added without any links to requirements. In Testlab, testing coverage is reported against the system’s requirements and in testing parlance, a test case verifies a requirement it is linked to when the test case is executed and passes.

It is often recommended to bind the added test cases to requirements at the later stage. This way you can easily report what actually has been covered by testing. It should be noted, that the requirements we talk here don’t have to be fully documented business requirements for this to work. For example, if you would just like to know which parts of the system have been covered you might want to add the system’s parts as project’s requirements and bind the appropriate test cases to them. This way a glance at Testlab’s coverage view should give you insight which parts of the system have been tested successfully.

Better yet, if you did pre-plan your test cases in some form (see above) you might consider adding a requirement hierarchy too and linking your test case stubs to these requirements. This would give you insight into your testing coverage straight away when the testing starts.



In this article we talked about the new test execution features of Testlab enabling you to execute your tests using exploratory testing approaches. We went through the advantages of pre-planning some of your testing, using requirements in your exploratory testing, talked about testing sessions and noted how Testlab’s workflows and user roles should be used in exploratory testing.


Tags for this post: example exploratory features testing usage 


New features: Enhanced reporting, exploratory testing and easy screenshots

A new quarterly feature release of Testlab (codenamed Last Resort) is out! This update gives you enhanced reporting, better support for exploratory testing, easier screen and video capturing and a bunch of other enhancements. Read on.



Enhanced reporting

Reporting in Testlab is now much better than before. You can now preconfigure report parameters and publish your reports so that they can be launched with a single click. Reports can be saved to three different scopes: to a project, to all projects or to personal use only.

  • New report view added for adding and launching reports
  • Preconfigure report parameters and publish your reports with a descriptive name (for example “Critical open issues of BETA” or “Last month testing activity”)
  • Save reports to your project, to all your projects or as private report just for you.
  • Launch your reports with a single click
  • Time bound reports such as progress reports can be configured with relative dates. For example, you can configure a testing progress report to always be reported from start of the current month to current day. Generating a monthly progress report this way is just a click away.
  • New report type added: Testing activity. This reports a summary of all testing activities performed during a time frame or for a set of versions including executed tests and updated assets such as issues, requirements or test cases.
  • Listing reports now include all configured custom fields and comments.
  • All reports now take into account the project’s time zone and user’s locale for date and time formatting.
  • Testlab’s search function also looks up published reports.



Exploratory testing

Pre-planning your testing and test cases is not always possible or desirable. Exploratory testing is an approach to testing where the tester or team of testers ‘explores’ the system under test and during the testing generates and documents good test cases to be run.

This type of testing is now well supported in Testlab. It is possible to add, remove and edit test cases during execution. It is also possible to add notes or comments to test cases during execution. This can enhance communication between testers and even between different testing sessions. 

  • Add test cases to the test run during a testing session.
  • Remove test cases during a testing session.
  • Edit existing test cases and their steps during execution.
  • Comment test cases during execution and have access to comments entered for test cases’ steps on previous runs.
  • Execute test cases in any order by jumping between sessions’ test cases.

All new functions are implemented in a way that they available to all testing approaches.



Capturing screenshots and video

Testlab integrates with Monosnap, a powerful screenshot tool, allowing you to easily capture and annotate screenshots. You can also record a screencast or video. Testlab offers an interface to Monosnap so that you can add the content to Testlab with a click of a Upload button. Alternatively, you can just drag-and-drop captured content from Monosnap to attach it easily to Testlab’s asset.

Captured content is uploaded in a secure way directly to Testlab and stored as file attachment to your project asset.

  • Capture screenshots and annotate them – great way to highlight any issues.
  • Easy and fast.
  • Record video and upload MP4 encoded screencast to Testlab.
  • Integration supports the desktop client of Monosnap for Windows and Mac OS X.


Other enhancements

In addition to major features described above, the version includes enhancements such as:

  • Test cases can be easily executed directly from the test hierarchy tree context menu. In addition to this, the issue view enables you to easily run the test case the issue is related to – handy, when you just want to re-test and verify if the issue is really fixed or not.
  • Attached images are shown directly in UI by hovering on them with your mouse cursor.
  • When adding a new test run the dialog now features “Save and start …” button to start executing with the new test run immediately.
  • Major performance enhancements in situation when the project has a huge number of executed tests or test runs.


Sincerely yours,

Meliora team

letterOn every British nuclear submarine, there is a safe. Inside that safe is another safe. And inside that safe is a handwritten letter from the British Prime Minister, to be opened only if the country has been decimated by nuclear war.” (This American Life, episode 399).

United Kingdom’s nuclear doctrine is unique in a way that each Prime Minister can decide the orders for the nuclear submarines in the event that the British government has been incapacitated by a nuclear strike. These orders are handwritten by the Prime Minister, placed in a safe on each submarine and are destroyed unopened after the Prime Minister leaves office. What action each Prime Minister would have ordered is only ever known to the outgoing Prime Minister. If the orders were ever to be carried out, the action taken would be the last official act of Her Majesty’s Government.



Tags for this post: announce exploratory features release reporting screenshots 

Best-of-class cross-browser hosted SaaS quality and test management, testing and issue management tools to improve your quality. Site information.