One of the biggest challenges to implementing an end-to-end Continuous Delivery pipeline is making sure adequate test automation is in place. However, even if you have automated across your entire Test Suite, there is a second challenge: How do you manage Test infrastructure and environments without breaking the bank?
If you want to move towards Continuous Delivery, you need to execute a majority of your tests for each code change in a pristine environment that is as close to your production environment as possible. This ensures that code defects are identified immediately and every code change is therefore 'shippable'. However, creating these environments and updating them with each new application version or every time the config changes adds a lot of overhead. If you're testing an application with many tiers or microservices, the complexity increases since each tier might need to be tested indepedently in its own environment against specific versions of other tiers.
The utopia of Test Automation is the following:
Environment definitions are represented by infrastructure-as-code tools like Ansible, Terraform, Puppet, or Chef. The provisioning scripts are committed to source control and versioned, so you can go back to an earlier state if needed.
All (or at least a good majority) of your tests are automated and either committed to source control or hosted on services such as Nouvola, Sauce, etc.
You have a completely automated deployment pipeline that automatically spins up a production-like Test environment for every code change, triggers your automation, and if all tests succeed, destroys the environment. If tests fail, the right folks are notified and the environment is kept live until someone can debug the failures.
The first step is already happening in most organizations. The DevOps movement encouraged Ops teams to start writing scripts to provision and manage environments and infrastructure, and multiple vendors support this effort quite effectively. The second step is still a challenge in many organizations, but this is really something that needs executive buy-in and a commitment to automation, even if it slows down product development for a while.
This whitepaper presents a method of implementing the third step - spinning up test environments on-demand and destroying them automatically after the automated test suite is executed.
Read the full blog here