I do this with traefik as the reverse proxy. To host something new, all I need to do is add a label to the new container for traefik to recognize. It's neat with a wildcard cert that traefik automatically renews. I've also heard good things about caddy, a similar alternative.
Yeah, I've heard that these new reverse proxies are great like that. I have to run certbot (which I do) and I should have created wildcard certs but I didn't. I use traefik on k3s and it's good there for other stuff.
