Reduce Infrastructure Costs Guided by Performance Testing

When we talk about performance tests, we usually do it with the goal of making our applications faster. However, we can also focus on the reduction of resources we need to deliver the same performance. If we do not want to blindly meddle with our infrastructure, we need to run our performance tests to guide the optimisation. Let us explore our options.

Right-sizing the infrastructure

The challenge with right-sizing the infrastructure is to figure out what the right size is. This sounds like a banality, but it is surprising how often this point is neglected.

Before we can decide on the right size, we need to clarify our needs. Is the current performance sufficient, do we need more throughput, or can we settle for a slightly slower application? Depending on our answers, we know if we need to optimise our application or if we can focus on reducing the infrastructure.

Test on identical infrastructure

If we want to use performance tests to guide our infrastructure optimisations, we must have an identical set-up to the one we want to optimise. For this goal we cannot interpolate as we did with the performance tests so far, here we focus on the infrastructure and that must match. If it does not, we are unable to get meaningful measurements and we can stop the exercise.

If we use Terraform or another tool to define our infrastructure as code, we are in a great spot and can use the same configuration to run our test on identical infrastructure. Without such a tool it is more work, but we should take the time to start with an identical set-up.

We can register our measurements as we usually do and experiment with more or less complex load tests. Instead of diving into the application to find the hot path, we go to the configuration file and reduce the services for our infrastructure. Then we rerun our performance tests and see if the smaller infrastructure had an impact on the response times.

We expect a degradation when we reduce the power of our machines, but that may not be distributed equally. Maybe we can go one level back in our web service without noticing anything, while the same step back on the database service may be too much.

Finding an optimal balance between the different parts of our infrastructure needs a lot of testing, but in the end, it may save us a lot of money.

Optimise the application if needed

Especially when we run our application in the cloud, we may find an optimised infrastructure set-up that has a too large impact on our application. In this case it may be worthwhile to go for an extra round and optimise the application. It is important to time-box the activity, then we gain nothing if we save $100 per month in infrastructure code and spend $10000 on optimising the application.

Be aware of surprising prices in the cloud

Using less resources should be cheaper, right? While that sounds obvious, it is not always the case. In February we had a strange pricing curve in Azure to host an App Service in Switzerland North. The Basic package offers less storage than the Standard package and is usually cheaper. But in February that changed, and you got 50GB of storage cheaper than 10 GB:

The Basic plan comes with 10GB and costs $59.86.

IMAGE: The Standard plan comes with 50GB and costs $48.18.

Using the larger Standard plan was $11.68 cheaper even when you only needed the Basic plan. Be aware of such dents in the pricing curve. Reserving more resources may cost you less.

Optimizing applications to reduce resource usage, rather than to serve more customers, is not typically our first thought when we think about performance testing. However, when the business aims to cut costs, this approach can be a viable strategy. But make sure that smaller machines indeed cost less – especially in the cloud.

Next week we finish this series on performance testing with some parting thoughts.