I had the same issue, it took a while but I managed to resolve by passing my own certs to the phpldapadmin image. You can see instructions to do this here: GitHub - osixia/docker-phpLDAPadmin: phpLDAPadmin container image 🐳🌴 (modify the ldap.env config file).
Additionally I found it useful to add the phpldapadmin container to the reverse proxy and avoid having to use the port altogether. This was a bit more involved, but in the end I got it working with web-ui at my-server.com/web-ui and phpldapadmin at my-server.com/ldap, both served on https with a certificate automatically generated and renewed via certbot.
I’ve documented an outline of the steps involved here: Install ODK-X sync endpoint with https via certbot and reverse-proxy for phpldapadmin · GitHub
Would be great if anyone else tries this and has feedback.