In DevOps, it is often the mentality that everything has to happen as quickly as possible. This can mean rushing things out the door with little or no testing. Legacy patterns of multi-day manual testing and multi-week performance test scripting are often cited as the reasons for choosing to “test in production” in place of proactive testing. The myth is that you can’t have both speed and quality, but the reality is that equipping dev teams to write their own tests with their own tools changes the game. Committing tests along with code makes near-instantaneous feedback for every build, every deployment candidate possible.
A common misconception about load testing is that it can only take place at the end of the lifecycle, as traditionally developers had to wait for performance tests to be developed and run by someone else. Waiting until the end of the process to test for performance adds unnecessary drama and surprises. Modern open-source tools and domain specific languages have made developer-driven performance testing possible. Developers who are already taking responsibility for functional tests are beginning to use low-volume performance tests to examine functionality and performance in a single pass, especially with API endpoints. These endpoint tests then become building blocks for automated integration, deployment and post-production tests.
Another common false idea about performance testing is that a test suite must be run sequentially. This goes back to the “nightly build” pattern in which several hours of tests were the norm. It is more efficient to run multiple smaller tests in parallel because the combined test will only take as long as the longest piece takes to run. For example, if you are running 12 tests of three minutes each and three that are four minutes long, running them sequentially will take at least 48 minutes, but running them in parallel will take just four minutes.
Whether developing a mobile or web app, open-source tools are becoming the norm for developers. Developers tend to have a favorite open-source tool that they use, and they are wary about using other tools that others impose on them. This doesn’t need to be a problem, as frameworks are emerging that “play nice” with a wide variety of tools. The key is to have a unified orchestration framework that individual tools can snap into. No matter what tool you use, the process of load testing can be part of a single unified whole without imposing restrictions on tool choice.
In today’s enterprise, the lines continue to blur between the roles and responsibilities on developer teams; this is especially true as DevOps becomes the norm. In some cases, the testing function can fall to a select few “engineer in test” developers, which limits the ability for tests to be run at all stages in the lifestyle.