Sysadmin

Hyper-v will not survive when it is dragging customers away from the Azure cash cow

Pretty sure that's why they made Azure Stack HCI, it's hyper-v, but doesn't work without an up to date azure subscription and charges you monthly fees to run vms on hardware you own.

It's great, the worst of both worlds..... Fucking thing doesn't even report on disk provisioned, only utilization, so get fucked it you want to capacity plan without writing your own report script.

Proxmox has those features. Also I personally wouldn't mix CPU archs but you should be able to as it is all KVM. Maybe there is a different memory layout or something

The problem isn't necessarily the GUI. The problem is there are a lot of admins who don't understand what is happening under the hood. I've talking to people of all ages who have no understanding of how basic things like networking work. I've also talked to to people of all ages that have a deep understanding of the system.

The biggest takeaway is that to be a good admin you need to understand the details. Don't be afraid to dig in. Either dig in, move to management or both

I’ve been using Proxmox professionally for years now, and not once did i have s problem i could not fix myself.

And how many of the environments you have left behind became an unmanageable mess when the company couldn't hire someone with your skillset? One of the downsides to this sort of DIY infrastructure is that it creates a major dependency on a specific skillset. That isn't always bad, but it does create a risk which business continuity planning must take into account. This is why things like OpenShift or even VMWare tend to exist (and be expensive). If your wunderkind admin leaves for greener pastures, your infrastructure isn't at risk if you cannot hire another one. The major, paid for, options tend to have support you can reach out to and you are more likely to find admins who can maintain them. It sucks, because it means that the big products stay big, because they are big. But, the reality of a business is that continuity in the face of staff turnover is worth the licensing costs.

This line, from the OP's post, is kind of telling as to why many businesses choose not to run Proxmox in production:

It is just KVM libvirt/qemu and corosync along with some other stuff like ZFS.

Sure, none of those technologies are magic; but, when one of them decides to fuck off for the day, if your admin isn't really knowledgeable about all of them and how they interact, the business is looking at serious downtime. Hell, my current employer is facing this right now with a Graylog infrastructure. Someone set it up, and it worked quite well, a lot of years ago. That person left the company and no one else had the knowledge, skills or time to maintain it. Now that my team (Security) is actually asking questions about the logs its supposed to provide, we realize that the neglect is causing problems and no one knows what to do with it. Our solution? Ya, we're moving all of that logging into Splunk. And boy howdy is that going to cost a lot. But, it means that we actually have the logs we need, when we need them (Security tends to be pissy about that sort of thing). And we're not reliant on always having someone with Graylog knowledge. Sure, we always need someone with Splunk knowledge. But, that's a much easier ask. Splunk admins are much more common and probably cheaper. We're also a large enough customer that we have a dedicated rep from Splunk whom we can email with a "halp, it fell over and we can't get it up" and have Splunk engineers on the line in short order. That alone is worth the cost.

It's not that I don't think that Proxmox or Open Source Software (OSS) has a place in an enterprise environment. One of my current projects is all about Linux on the desktop (IT is so not getting the test laptop back. It's mine now, this is what I'm going to use for work.). But, using OSS often carries special risks which the business needs to take into account. And when faced with those risks, the RoI may just not be there for using OSS. Because, when the numbers get run, having software which can be maintained by those Windows admins who are "used to click their way though things" might just be cheaper in the long run.

So ya, I agree with the OP. Proxmox is a cool option. And for some businesses, it will make financial sense to take on the risks of running a special snowflake infrastructure for VMs. But, for a lot of businesses, the risks of being very reliant on that one person who "not once [had a] problem i could not fix myself", just isn't going to be worth taking.

God damn Linux admins are pretentious.

I suppose thats the difference between Windows admins and Linux admins. Windows admins are used to click their way though things with fancy gui's and wizards :)

That hasn't been a thing for a decade. Any time you run into an issue or need to do something on mass you fire up PowerShell

What I am saying is that it depends. Proxmox is rock solid if you know what you are doing. It is a massive train wreck of you do the set it and forget it approach. Deploying Proxmox requires planning and a understanding on the system.

So yes it is good for production as long as you understand how it works.

Good catch

It has a much higher barrier to entry. If they can make it easy to get on a makeshift network made of old hardware then I might look more into it. I want to use it for personal use before I even think of prod. Proxmox has been in homelabs for a while which helps quite a bit to mature the product.

I assume you are using libvirt? Do you setup VM fail over?

Are you Azure focused?

If you are an Azure shop go with Azure Stack HCI. I haven't use it personally but I see a lost of Reddit comments about it.

Nutanix is not especially cheap, in my opinion/experience, nor is it particularly easy to manage and maintain

I personally don't see much reason to use LXC. Sure it theoretically is faster but it creates lots of headaches. I do use LXC but for only for simple things like jump boxes.

Containers run on top of VMs.

Promox is not immutable. Immutable Linux uses a read only filesystem. They normally have automatic role backs and updates usually take the form of deltas. (Not always)

The benefit is that the system is highly predicable and updates only apply on a reboot. If there is a bad update it just fails the self test and then rolls back. This is way more dynamic than stock Debian.

I think you might be confusing stability with immutably