I'd like to encourage you to take another look at Authentik, it sounds like their Proxy Provider is exactly what you're looking for: https://docs.goauthentik.io/docs/providers/proxy/
Authentik can certainly get complex, but only if you want/need it to. It is by far the most user-friendly IDP solution I've found, especially for what it offers. Their docs also have step-by-step guides for how to integrate a lot of popular self-hosted apps.
Only takes a couple mins to spin up a test environment using their Docker compose file: https://docs.goauthentik.io/docs/installation/docker-compose
For sure! If you do end up taking it for a spin, feel free to ping me with any questions.