The cloud is everywhere, and it’s continuing to grow.
According to a 2019 Cloud Report by Flexera, 90 percent of companies practice cloud computing.
As efficient and cost-effective as the cloud may be, managing cloud spending and cloud governance is proving to be a challenge across companies and workloads. Cloud costs are slowly exceeding original set budgets and cloud compliance is harder to achieve as companies look to scale or increase in complexity.
As easy as it may be to underestimate these issues, a local startup in Fulton, Maryland, cloudtamer.io, seeks to solve this problem in an effective and simple way for all who use the cloud.
We spoke with David Konigsberg, VP of Software Engineering, who gave us insight into how they achieve cloud governance, what their tech stack is, and how they maintain relevant services in an era of rapid growth in cloud computing.
The conversation below has been edited for length and content.
What do you think are the shortcomings of cloud providers and services?
For a given service, you may find loyalists to one cloud or another. But I think the biggest shortcoming is how different they are. If you’ve been using AWS for a long time, it’s difficult to get up and running in Azure and vice versa. It’s not so much a shortcoming as a technical challenge.
Amazon and Microsoft have some interesting tools for managing compliance and governance, but they’re very different. They’re hard to use simultaneously if you’re not an expert in each of those clouds. That’s the biggest shortcoming.
A good example of where cloud native struggles is getting into financials in the cloud. Both Amazon and Microsoft have their own tools. In the case of AWS, it’s Cost Explorer, and in the case of Microsoft, it’s a few different things, including Cloudyn.
With Cost Explorer, Amazon only reports spending in AWS and not Azure or other clouds. Azure will report on AWS spend, but this reporting is limited and comes at a premium. Even if you’re just using Amazon and their native tools, they do not give you the level of granularity and ability to manipulate the data you would want as a user.
Cost explorer can only report within a single AWS organization, so if you have a more complex organizational structure you can’t see it all in one place. You’re confined to the options they give you in their tool. It’s just not well designed for the users that are using the cloud. While it can be useful for some large-scale visibility, the lack of resource-level filtering can make it less useful at the developer/end user level.
We work with some customers that have a large, distributed cloud presence within their company, and they need to be able to budget specific projects against specific funding streams. They need to be able to account for how much is going to be spent in the future. There are some tools in Amazon to help. There are budgets, but they’re very simplistic. There are these new savings plans which are neat, but they don’t really address this customer need.
This is where we’ve decided to innovate and take the raw data that Amazon provides and do our own processing of it. Even though Cost Explorer has an API, it’s not very comprehensive. We take the raw data they give us, adjust it to our own system, and go from there.
cloudtamer.io aside, that whole cloud agnostic thing is a big effort by a lot of organizations and certainly the government. That’s a big challenge for customers and users.
How do you leverage cloud services that bring people to your product?
A big part of our philosophy at cloudtamer.io is not to reinvent the wheel. Amazon spends lots of time and money building these services. When there’s a service that exists and does what you need to do, there’s really no reason not to use it.
Our product has really focused on making it easier for all users to take advantage of the cloud and have better insight on what they’re doing, while streamlining the process. They can benefit from cloud best practices without having to have a big cloud management team that’s manually fiddling with all the bits and pieces.
Our biggest differentiators would be our multi-cloud support and centralizing everything into a single pane of glass to deliver governance that helps customers manage accounts, stay on budget, and remain compliant.
Obviously, the cloud providers want your data to be safe and for you to follow best practices. But a lot of that is on you and they leave it up to you to make sure you are using the services safely. It’s a challenge and a lot of the onus is on you.
We just recently released a whole new compliance dashboard that leverages the open-source Cloud Custodian solution and several built-in policies that come with our product, to help you identify where the compliance issues are throughout your organization.
Organizational structure is a big part of cloudtamer.io. Microsoft and Amazon have organizational structures, but they’re limited in how big and interconnected they can be. Obviously, you can’t have Azure projects inside of an AWS organization unit (OU) structure.
By bringing the clouds together and enabling your resources to comingle, it makes it a lot easier, at an enterprise scale, to:
- See how the different parts of your organization are spending money
- Understand from a compliance standpoint, which areas of your organization are potentially less compliant than others
In Amazon, you can have service control policies that apply to organization units. There are limitations to how many layers deep you can have those OUs, and those service control policies are going to apply to everything below where they are applied. If you have a very big organization with a lot of depth, or you constantly have exceptions to the rule, it becomes very complex.
cloudtamer.io helps with that because of our flexible org structure. It’s all user defined. It includes anything you can build out using OUs. You can have these nested OUs, which contain projects, which contain accounts, which could be from any cloud, and then see those in groupings and manage them within the scope that you choose.
If you have a department in your organization that needs to be HIPAA compliant, because you’re dealing with patient data, you could apply that policy just to that part of your organization. That makes life easier for someone in a completely different department who needs to be able to use services that are not necessarily HIPAA compliant.
What tech stack did you all use and what was the motivation in choosing it?
Our database layer is MySQL, which was chosen mostly because of its proliferation and ease of use. As we’ve gone on, we’ve realized we may want to break into alternate types of databases that are differently structured for certain components of the application.
But for now, it’s MySQL, which when deployed in AWS we swap out for RDS Aurora.
The backend is all written in Go. When we were first working on the product, we were deciding between Go, Python, and PHP as different options for building a web-based tool. We went with Go for several reasons, but one of the main ones is that it’s easy to learn.
If you know Python, it’s easy to pick up Go. But Go has some nice strict rules built into it to prevent you from doing anything potentially dangerous.
When you’re bringing on a new team of people to build a new product where code conventions don’t yet exist, having a language that helps enforce some of that really makes a difference.
We had a team of 4-5 people initially, and they all came from very different backgrounds. Go is a great language to learn. I really love interacting with it. I feel like every time I learn a new feature of the language, I discover new ways to improve my code. It makes my code better going forward and provides a fun challenge.
On the front end we’re using Angular, a JavaScript framework. One of our principal engineers, Eric, does all our data visualization and he uses D3 for that. It’s amazing to see some of the things he’s been able to achieve.
How does your team actively adapt over time given the growing nature of the cloud?
Cloud providers are always coming out with new stuff. Every time we create a new feature, we can assume that Amazon is going to offer something similar eventually. At the same time, when they add the feature natively, we can just start using that and leverage it to make our own application even faster.
A good example of this is around financials. Amazon financials can be very difficult. They contain some “gotchas.” If you don’t really understand them and you don’t know why they’re there, you won’t understand why certain things don’t add up in the numbers that you’re seeing.
I know Amazon is going to change it eventually. In the meantime, we offer services that make that easier. But as soon as they fix it, we’ll happily use whatever improvements they have and extend it. It benefits us and then we can innovate on top of it. It’s fun.
