In this post, we will discuss a bit about our upcoming release schedule and offer a glimpse to the features in upcoming releases.
Challenges with test automation
Business challenges related to interpreting results of automated tests are obvious. More often, tests are created by developers with various different motives, the number of tests is often numerous and traceability to the specification is often lacking. Features are needed to make sense of these results and gain the business benefits outside the scope of regression or smoke testing – which in practice, often tends to be the use for automated tests.
If you are able to map and scope the tests efficiently, you
- have a better understanding of what is currently passing or failing in your system under testing,
- have your team collaborate better,
- ensure that key parties in your project can make educated decisions instead of being reactive and
- have a tool to really transition from manual to automated testing as existing test plans can be easily automated by mapping automated tests to existing test cases.
Automation in Testlab
In Testlab, results from automated tests can be mapped to test cases in your project. This way of working with your automated tests has benefits as the results can be tracked and reported in exactly the same way as the manual tests in your current test plan. What happens is that when results for automated tests are received (for example from a Jenkins’ job), a test run is added with the mapped test cases holding the appropriate results. The logic how the mapping works is currently fixed so that the test cases can be mapped with pre-defined fixed logic: If the ID of the automated test starts with an ID bound to a test case, it will receive results of this automated test.
Upcoming Automation workbench
Future versions of Testlab will include a new Test automation view with a workbench aimed at managing the import and mapping of your automated tests. A screenshot of the current prototype can be seen below (click for a peek):
- allows you to define “sources” for your tests: a source is a new concept which allows you to configure how the incoming results from such source are handled. You might want to think the source as your “Jenkins job” or your Jenkins installation as a whole if your jobs follow the same semantics and practices. A source is defined with “rules”, which define how the automated tests are mapped. The workbench
- features a rule engine, which will allow comprehensive mapping rules to be defined with
- ignores (which tests are ignored),
- creation rules (how test cases for tests are automatically created) and
- mapping rules (such as “starts-with”, “ends-with”, regular expressions – defines how the results for the tests are mapped to your test plan) and
- the workbench allows you to
- easily upload new results by dragging and dropping,
- execute any rules as a dry run to see how the results would be mapped before the actual import,
- create any test cases by hand – if preferred over the creation rules – and
- use pre-built wizards which help you to easily create the needed rules by suggesting appropriate rules for your source.
We feel this is an unique concept in industry to help you get better visibility in your automation efforts.
The prototype shown is work-in-progress and is due to be changed before release.
As our roadmap implies, The new major automation related features are planned to be released in Q3 of our 2019 release cycle. The team will be hard working with these new features which means that the Q2 release will be a bug-fix release only. That said, we will be releasing maintenance releases in between, but the next feature release will be the version with automation features in Q3/2019.
Meliora Testlab Team