It is definitely a problem with Windows, because this works fine in macOS and Cyberduck. But I would like to find a workaround. One of the answers in that post made me think the problem could be that the certificate is issued to the local hostname, but I am connecting over the internet by IP address, as there is no FQDN. I guess I have to figure out how to force Windows to accept this certificate.