Differences Between Pre & Post Cloud Performance Testing

This blog was originally posted on 08/19/2016. Updated on 05/03/2021.

The last part of our blog posting series, “The Need for Load and Performance Testing in the Cloud” will give an answer to the question: what is the difference in all this to pre-cloud times? The article is based upon a talk I gave at the AWS PopUp Loft in Berlin.

Is there a difference in post-cloud performance testing?

After my overview on the types of performance testing, I’d like to raise the question: Is this actually any different from what it used to be pre-cloud?

I would clearly have to answer: Yes and no. The testing needs and methods have not changed a lot. The needs have maybe increased a bit because our environments tend to be more complex overall.

The main difference is that multiple, scalable, performance testing environments used to be very expensive and very hard to manage. Not many could actually do that for real projects.

This is where the automation capability of the cloud can really make a difference. If you have your entire infrastructure, services, servers and code automated, spinning up testing environments suddenly becomes viable. You can create entire infrastructures and environments fully automated, run a series of performance tests, gather all relevant data and shut it down again within a couple of hours, and for very little money.

One important thing to keep in mind when aiming for such a testing environment: Do not forget about state. With state, I am referring to databases, caches that are warm, etc. Having a reproducible environment, including test data, is one of the bigger challenges. Frankly, I haven’t seen a good and sound solution for that yet.

Conclusion

In a nut shell, you can state:

• Scaling resources is not scaling application
• Understanding is very important to make your system efficient and scalable
• Complexity is still around and will become apparent again in non-trivial ways when it comes to performance
• You should utilize the cloud to make testing simpler

Nowadays infrastructure and software architecture in general is ever changing and evolving. Getting solid data on how your system is performing is not only relevant for your application in particular, but also an important piece when it comes to Continuous Architecture. Your architecture is ever evolving to run tests and validate it!

Want to know more? Check out the previous posts in this series:

• The Need for Load and Performance Testing in the Cloud
• Types of Performance Testing

Related Presentation

“Last- und Performancetests in der Cloud” (Talk) & Expert @ Developer Week 2017, Nürnberg

On June 26th Sebastian Cohnen attended the “Ask the expert” event of Developer Week Nürnberg.

Besides that he was interviewed at Nighthacking Interviews @Developer Week:[/vc_column_text]

The next day he gave a talk about load and performance testing in the cloud.

Urs Enzler‏ (@ursenzler) made this awesome sketchnote of the talk. Thanks once again!