Dear community,
the service provider figured out that SAP ByD does not support SNI+SSL. After a few changes in the configuration everything worked fine.
Apparently SNI+SSL is supported by most apps and browsers, but not by SAP...
Thanks for your help and best regards,
Georg