Smoke Testing is a very crucial step in the testing cycle. Whenever a new build is received from the development team, Smoke Testing ensures whether the new build is healthy or not. It is also known as “Day-0” testing.
This guide discusses Smoke Testing, its types, how it differs from other types of Testing, and how to perform Smoke Testing.
Table of ContentsWhat is Smoke Testing?
Smoke Testing is a software testing technique that basically determines whether a new build delivered by the Development team is bug-free or not. It gives the go-ahead to the QA team to move further with their testing rounds.
It is also known as Confidence Testing or Build Verification / Build Promotion Testing and is made up of a collection of tests that are performed on each build received during the software development life cycle to determine its health.
In simple words, Smoke tests verify whether the most important features work as expected and that there are no showstopper issues in the build that can potentially lead to blocking the entire testing team. It helps in deciding if the build is flawed or not and hence, prevents the entire team from wasting time or resources.
Why is Smoke Testing important?
Smoke Testing is important because it:
- Ensures that the product is testable and doesn’t let the QA teams get blocked.
- Uncovers major issues at the beginning of the sprint itself. This ensures that both Development and QA teams have enough time to fix and test the issue further.
- Makes sure that the critical or acute functionalities of the product are working fine.
- Helps determine the system or the application’s stability.
Types of Smoke Testing
There are three different types of Smoke Testing:
Advantages of Smoke Testing
Some of the major advantages of Smoke Testing are:
- It helps in finding critical issues at an early stage
- It reduces the time and resources of the team, which eventually decreases the risk.
- It makes the application robust
- It reduces integration issues.
- It gives enough time to rectify major issues if any.
Smoke Test Cases: Example
The below table illustrates a scenario of an E-Commerce website and some sample Smoke Tests for the same.
When to perform Smoke Testing?
Now you must be wondering when we need to actually perform Smoke Testing. The answer is simple; Smoke Testing needs to be performed whenever there are some new features or capabilities added to the software, and a new build is delivered. In this new build, testers need to ensure that neither of the major key functionalities that were already existing is getting hampered, nor the new functionality is blocked due to any issue.
Once a new build has been delivered by the dev team, some of the QA team members can start with Smoke Tests. They can prepare a list of cases that ensures all key features of the functionality are not affected and perform the tests. If the tests are successful, the build is promoted in the QA environment, and the rest of the QA team can further move on with the Functional Testing of the new feature as well as existing features. If the tests fail, the build is rejected, and the QA environment keeps running with the old stable build unless a new build is further delivered by the Dev team, which has all the issues addressed. This cycle continues till the end of Code Freeze.
For Example – In the example shared above, once a build is received which has the fix for Checkout functionality, Smoke Tests should ensure that the existing features of Login and Addition of Items to the Cart are working seamlessly, along with the Checkout functionality bug fix.
What happens if Smoke Testing is not performed?
If Smoke Tests are not performed at the early stages, and critical issues are found towards the later stages of the Software Development Life Cycle, it can incur a heavy cost on the entire team with respect to time, resources as well as money. These issues can also affect the overall release cadence and pose a major threat to the product image overall.
Differences between Smoke and Sanity Testing
Let us now take a look at some of the major differences between Smoke and Sanity Testing.
| Smoke Testing | Sanity Testing |
|---|---|
| This is done to ensure that the critical functionalities of the product are working fine | This is done to ensure that existing functionalities in detail are working fine, along with the bug fixes that have been delivered in the latest build. |
| This is considered a subset of Acceptance Testing | This is considered a subset of Regression Testing |
| This can be done by either Developers or Testers | This is mostly done by Testers |
| This is documented | This is not documented |
| This can or can not be stable. | This is stable |
| This is scripted | This is not scripted |
| This is done whenever a new build is out. | This is done even when a new build is out or can be done daily as well to determine the environment’s health. |
Differences between Smoke Testing and Regression Testing
Next, let us take a look at some of the major differences between Smoke and Regression Testing.
| Smoke Testing | Regression Testing |
| This is known as Surface Level Testing and ensures build and environment stability. | This is known as Deep Level Testing, and it ensures the overall product’s functionality. |
| This can be done by either Developers or Testers | This is mostly done by Testers |
| This incurs very less cost | Regression Testing is usually costly. |
| This is completed quickly and takes lesser manpower | This takes time to complete and needs more manpower. |
| This is scripted | This is not scripted |
| This is done whenever a new build is out. | This is also done when a new build is out, as well as it’s done over the entire development lifecycle. |
| This is performed on all builds and determines whether the build can be accepted or rejected | This is performed only on a stable build. |
Automation and Smoke Testing
Automated Smoke Tests can drastically reduce the overall time spent, as we don’t need to perform Smoke tests only once. These need to be run again and again whenever there is a new build delivered.
Now, depending on the size and scale of the application, the no of test cases can vary. For a very mature product, the total number of test cases will usually be on the higher side. That can take even one full day for a tester to test manually. Imagine in that case, one person will have to spend one full day after every 2-3 days only for Smoke Tests. That can be terrible with respect to cost as well as with testers, as they are going to be burdened with the same task again and again. Plus if you report issues only after a day or two, half of the team already wasted some time in order to wait for the new build.
Thus, Automated Smoke Testing is the way ahead. It can bring the overall time spent on it to some minutes and, as a result, improve the overall product development cycle. Testers can report issues faster and consume the latest builds faster. As a result, they can deliver more features in much lesser time.
There are a lot of tools available for automating Smoke Tests. For example, Selenium, Rest Assured, TestNG, Cucumber, and PhantomJS are some of the most widely used tools today for automating smoke tests. Here, Selenium can be used for UI Automation, whereas Rest Assured can be used for Backend Automation. Using Cucumber or TestNG, one can maintain test cases, and PhantomJS is an option for integration of the tests with Continuous Integration tools such as Jenkins. One can select any tool or framework which meets the requirements best of their teams.
How to Perform Automated Smoke Testing?
Now that we know about the benefits of Automated Smoke Tests let us look at the steps for doing the same.
Step 1 Select a Testing Framework
The first step here is to choose a framework that meets all the requirements of the project. All the frameworks come with their own set of pros and cons, one needs to evaluate well because changing the framework at a later stage can prove to be very expensive. Some of the majorly used tools or frameworks today are
Run Automated Tests on Real Devices for Free
Step 2 Select a Test Runner (Optional)
In case your framework doesn’t come with a Test Runner already, you need to select one, ensuring that it works smoothly when integrated with chosen framework.
Some of the widely used Test Runners are:
Step 3 Write the code
Once the Framework and Test Runner are selected and built, the coding part begins, where we need to build our utilities and write test cases using the same. Once the tests are automated, we can create a Test Suite like Smoke, Sanity, Regression, etc.
Step 4 Integrate the Tests With a CI Platform
Next step is to integrate these tests with a CI Platform in order to run your tests. Where you can also schedule your tests or integrate it with the build development flow so that your smoke tests get triggered automatically once a new build is out, or your Sanity tests are triggered automatically every day at 12 pm regardless of the build. Some of the options available are
Step 5 Schedule or Integrate your tests to run frequently
As mentioned in the previous point, CI tools can help in running the tests without any human intervention. So, once the tool is selected and the tests are integrated, we can either schedule them to run every day at an nth time, or we can integrate them with the Development team’s pipeline that will trigger the tests whenever a new build is out. This way, we can reap most of the benefits of a CI tool.
Conclusion
Now, we know how Smoke Testing plays a very crucial role in the overall Software Development lifecycle. However, there are still a lot of places where this is usually being ignored due to time or resource crunch, but one must understand that dedicating time and energy towards Smoke Testing or its Automation is definitely going to reap benefits over the longer run. Plus, as the product grows, the need for Smoke Testing intensifies more.
In this growing era of automation and cut-throat competition, QA teams must attempt to place a Smoke Testing solution through automation in place and enjoy the benefits of time and resource-management along with a very stable and robust product.
It is always recommended to run tests on real devices to ensure all the real user conditions are taken into consideration while testing for more accurate test results. BrowserStack allows access to 3000+ real devices and browsers for a comprehensive testing experience.
ncG1vNJzZmivp6x7o77OsKqeqqOprqS3jZympmeXqralsY6spKijlWLBpr%2FToqWg