Rtorrent does not accept (self signed) server certificates, where the certificate hostname does not match the site hostname.
The following error will probably be shown:
[SSL peer certificate or SSH remote key was not OK]
rtorrent currently has no option to disable the host checking of certificates, so a small line of code has to be added to the curl_get.cc file (which can be found in /rtorrent/src/core folder)...
After the line
curl_easy_setopt(m_handle, CURLOPT_MAXREDIRS, (long)5);
add the following code
curl_easy_setopt(m_handle, CURLOPT_SSL_VERIFYHOST, (long)0);
and recompile rtorrent as usual.
The certificate has still to be in your CA-folder (usually /etc/ssl/certs) and hashed, but the hostname does not have to match the url...
However, you should know what you are doing, since this has some security implications...
Any server which uses the certificate you have added to the CA-folder will be accepted by rtorrent.