Are you also noticing that Postgres is becoming a default database for AI workloads? Do you think this trend will continue or things will evolve over time?
I feel like these days everyone embraced the fact that SQL is here to stay and it's going to become a standard even for modern AI-driven infrastructures. That's why Postgres is booming and new ecosystem is being created around it. WDYT?
I guess the abstraction levels over sql might change now with natural language processing capabilities, but I guess there will be sql somehwere down the stack always :D
there is a middle ground where you can get NVMe-optimized networked storage with solutions like simplyblock. It gives you local-like performance but separates compute from storage and gives more options for backup & DR.
What do you think of storage space in the context of AI and cloud repatriation? Much happening with upcoming players like VAST, DDN, simplyblock or Hammerspace..
From business perspective it's often hard to see these long-term lock-ins and potential effects of those. And hyperscalers focus in on creating narration where moving to the cloud "accelerates" your business. Sure, you don't need to deal with data center things, but eventually you still have people operating the cloud and it's just different kind of talent - but they are still FTEs you need to manage. So the business question remains on the ROI of the investment and currently data center investments bring very high ROI for those who dare.
Significant reductions for "fast S3", however the base capacity pricing is still very high at $110/TB/month. That is as expensive as EBS io2 which is much much faster in terms of latency.
Yup. I plugged the numbers in just to check it out and for our very busy service, the storage costs eat up the savings from the reduced GET and PUT pricing. For us, express is still 8x more expensive.
We're pursuing Fsx NetApp ONTAP to store the objects in the short term in our processing pipeline. Unfortunately, we may need to build our own lifecycle management component if we go that route.
I'll have to look into that. We heavily bias towards AWS offered managed services. We have a massive enterprise support contract and we use it. We're almost always going to look for something AWS can provide directly as long as it meets the requirements.
actually, the S3 Express One zone - as the name indicates - is not global HA, but single zone. So questioning it over fast block storage like io2 or simplyblock is actually valid. It is just matter of s3 interface over a block interface.
aws and other hyperscalers will keep growing, no doubt. Public cloud adoption is at around 20%. So the new companies that migrate into the cloud will keep the growth going. That doesn't deny the fact that some might be repatriating though. Especially ones that couldn't get the benefits out of the cloud.
One thing I've seen in every startup I've been in over the last decade is that cloud asset management is relatively poor. Now I'm not certain that enterprise is better or worse, but ultimately when I think back 10+ years ago resources were finite. With that limitation came self-imposed policing of utilization.
Looking at cloud infrastructure today it is very easy for organizations to lose sight on production vs frivolous workloads. I happen to work for an automation company that has cloud infrastructure monitoring deployed such that we get notified about the resources we've deployed and can terminate workloads via ChatOps. Even though I know that everyone in the org is continuously nagged about these workloads I still see tons of resources deployed that I know are doing nothing or could be commingled on an individual instance. But, since the cloud makes it easy to deploy we seem to gravitate towards creating a separation of work efforts by just deploying more.
This is/was rampant in every organization I've been a part of for the last decade with respect to cloud. The percentage of actual required, production workloads in a lot of these types of accounts is, I'd gather, less than 50% in many cases. And so I really do wonder how many organizations are just paying the bill. I would gather the Big cloud providers know this based on utilization metrics and I wonder how much cloud growth is actually stagnant workloads piling up.
The major corps I've worked in that did cloud migrations spent so much time on self-sabotage.
Asset management is always poor, but thats half because control over assets ends up being wrestled away from folks by "DevOps" or "SREs" making K8S operators that just completely fuck up the process.
The other half is because they also want "security controls" and ensure that all the devs can't see any billing information. How can I improve costs if I can't tell you the deltas between this month and last?
reply