Introduction #
Cloud waste is a significant industry problem, with over $17B spent each year on unused or idle cloud resources. Cloud waste is also a significant contributor to global climate change, with greenhouse gas emissions from data centers being roughly equal to emissions generated by the airline industry.
To learn more about cloud waste perceptions and experiences at the enterprise level, we surveyed 105 IT professionals with knowledge of their organization’s cloud spend. We found that cloud waste is recognized as a significant issue with serious impacts, and it’s a high priority for organizations to address.
Key Findings Include:
- Respondents know that their organization is wasting significant cloud resources, estimating that 48% of cloud spend is wasted on average.
- Reducing cloud waste is a high priority for most organizations (76%).
- The biggest causes of cloud waste are cloud complexity and over-provisioning to ensure application performance.
- The most significant impacts of cloud waste for enterprises include reduced profitability and damaging IT’s reputation to the rest of the organization.
Cloud Spend on the Rise #
Our survey reached a broad range of organizations in terms of cloud spend, with monthly cloud spend ranging from less than $1K for some, and over $1M for others. Common across all of those ranges though is the expectation that cloud spend will increase over the next 12 months, with 32% of respondents saying they expect their organization’s cloud spend to increase by a lot and another 44% expecting it to increase somewhat.
Predictability of monthly cloud spend is not an issue, with 94% saying they know, at least roughly, what their cloud spend will be each month.
What is your company’s average monthly cloud spend?
Do you expect your company’s cloud spend to increase, decrease, or stay the same over the next 12 months?
How predictable is your company’s cloud spend?
Half of Cloud Spend is Wasted #
Respondents estimated that nearly half of their cloud spend is wasted on unused or idle resources. They also indicated that they feel confident in knowing how much of their cloud spend is wasted, with 35% feeling extremely confident in their estimate and another 49% feeling somewhat confident.
What percentage of your organization’s cloud spend do you believe is wasted, i.e. spent on unused or idle resources?
How confident are you that your organization knows with certainty how much of your cloud spend is wasted?
Reducing Cloud Waste is a High Priority #
Cloud waste has a significant impact on the organizations we surveyed, with reduced profitability and damage to IT’s reputation being the top two impacts cited. Not surprisingly, respondents say that reducing cloud waste is a priority for their organization, with 33% saying it’s a very high priority and another 43% saying that, while not the highest priority, it is still important.
What would you say is the impact of your organization’s cloud waste? Select all that apply.
Is reducing cloud waste a high priority for your organization?
Cloud and Kubernetes Complexity, Over-Provisioning are to Blame #
The two most significant causes of cloud waste according to respondents are cloud complexity, which makes it hard to estimate the resources that are actually needed, and also intentional over-provisioning used as a safety measure to ensure application performance. Kubernetes is a significant contributor to the cloud complexity issue, with 62% agreeing that it is a major or contributing factor.
What are the biggest causes of cloud waste for your organization, in your opinion? Select all that apply.
If you are currently using Kubernetes, do you believe the complexity of Kubernetes contributes to your organization’s cloud waste?
Kubernetes Resource Choices are Not Optimized #
Given the significant contribution of Kubernetes complexity to the cloud waste issue, we wanted to find out how those decisions are made. In most organizations (55%) IT Ops or Cloud Ops teams are responsible for making those decisions, while dev and engineering teams are responsible in 29% of companies. Only 27% are making resource allocation decisions based on optimization recommendations from machine learning.
In your organization, who makes decisions about Kubernetes resource allocation (e.g. memory, CPU, replicas)?
How are Kubernetes resource allocation decisions generally made in your organization?
Why is Cloud Waste Still a Problem? #
If you consider the key results from our survey:
- Nearly half of cloud resources paid for by enterprises are wasted – unused or idle.
- This wasted spend has a major impact on profitability, perception of IT, and competitiveness.
- It’s a high priority for most organizations to fix the problem.
It begs the question – why is cloud waste still a problem?
The answer lies in the causes of cloud waste. It’s primarily a result of cloud and Kubernetes complexity. IT teams have no way of being certain of the number of resources needed to ensure their applications perform well, so they err on the side of heavy over-provisioning.
The Answer? Machine Learning Powered Optimization. #
When deploying an app on Kubernetes, you have to make many decisions on resource allocation including memory requests and limits, CPU requests and limits, and replicas. Add to that app-specific parameter settings like JVM heap size and garbage collection, and multiply by the number of containers, and you quickly have a highly complex, multi-dimensional optimization problem that impacts the cost of running the app, the performance of the app, and its reliability.
This level of complexity is beyond human ability to effectively manage. That’s why most organizations either rely on default settings or they attempt to tune their applications through a process of trial and error, neither of which are effective.
Application optimization solutions like StormForge use machine learning to explore this complex, multi-dimensional parameter space. This process of rapid experimentation can identify the configurations that best balance the trade-offs between performance and cost, resulting in cloud cost savings of 30 to 60 percent for most organizations and performance improvements in the same range.
Survey Demographics #
We surveyed 105 IT professionals across North America who are familiar with their organization’s cloud spend and usage. Key demographics of respondents are shown below.
What is your company’s annual revenue?
Which of these most closely matches your job function?
What is your job level?
Is your organization using Kubernetes?
How many Kubernetes clusters does your organization have in production?
