SSL option documentation

[Hrvoje Niksic]

i agree with you, hrvoje. we should fix the ssl options before the
1.10 release or we will have much bigger problems later.

OK. Thanks for your support. The SSL options have been submitted by
an external contributor, and I consider it my fault that I have not
reviewed them more carefully. I will try to rectify that oversight.

--sslcerttype=0/1 to --sslcerttype=PEM/ASN1
--sslcheckcert=1/0 to --no-sslcheckcert/--sslcheckcert
--sslprotocol=0-3 to --no-ssl/--ssl=SSLv2/SSLv3/TLSv1

The name could (and IMHO should) be made even more readable,
e.g. --ssl-cert-type or even --ssl-certificate-type. It might make
sense to drop the "ssl" prefix altogether because those options also
apply to TLS. The option would then be --certificate-type, which is
shorter and nicer. I believe curl has done that.

Since --sslprotocol can specify TLS protocol, it might be more
accurate to name it --secure-protocol (--protocol is too general),
with the accepted values "auto" (default), "sslv2", "sslv3", and
"tlsv1", all case-insensitive. (Note that the current --sslprotocol=0
does *not* correspond to --no-ssl; it means choose automatically. The
fact that it confused you is further proof of the brokenness of
current option names!)

the other options seem fine to me, although i prefer names like
--ssl_cert_file than --sslcertfile.

Sure, except it should be --ssl-cert-file; Wget (and GNU software in
general) doesn't use underscores in option names.

[Hrvoje Niksic]

I just grep'd again through the openssl distribution, and there is
no mention of the environment variables in any of the documentation,
just in the code itself.

If they are completely undocumented, is it wise to even consider them
part of the public API? curl's documentation apparently doesn't
mention them either.

Maybe someone should ask on the OpenSSL mailing lists about this.

[Doug Kaufman]

Post by Hrvoje Niksic

I just grep'd again through the openssl distribution, and there is
no mention of the environment variables in any of the documentation,
just in the code itself.

If they are completely undocumented, is it wise to even consider them
part of the public API? curl's documentation apparently doesn't
mention them either.
Maybe someone should ask on the OpenSSL mailing lists about this.

I am on the openssl developers list and will ask about
documentation for this. It should be in the man page for
"SSL_CTX_set_default_paths", but no one has written the man page for
that function yet. Documentation has been low priority for the openssl
project. I have never written documentation in .pod format myself.

All this made me look once again at the code for default certificate
locations in the openssl code and in the wget code. I think I need
to withdraw my suggestion for documentation of SSL_CERT_FILE and
SSL_CERT_DIR in the wget documentation, since a careful look at
gen_sslfunc.c shows that we aren't using them. The openssl function
"SSL_CTX_set_default_paths" sets the paths for the trusted X509
certificates to the file "cert.pem" in the OPENSSL directory (as
defined when openssl was compiled) and to the directory "certs" under
the OPENSSL directory. It also allows the two environnment variables
above to override those defaults. For wget to use them, it would
have to call the "SSL_CTX_set_default_paths" function. It doesn't do
that. As I understand from looking at the code in gen_sslfunc.c, wget
doesn't do any verification at all unless called with the sslcheckcert
option set to a non-null value. If sslcheckcert is called, wget looks
only in a location specified by the sslcafile or sslcadir options,
and is not using any default locations. Please let me know if I am
misinterpreting the current behavior.

I am certainly not an encryption specialist, but I would favor
different defaults for this. I would think that verifying the cert
for a "secure" site should be the default, or wget may be giving a
false sense of security when it retrieves the files. I would also
favor using the openssl defaults, allowing them to be overridden by
wget command-line options. This would probably mean making changes in
gen_sslfunc.c to call "SSL_CTX_set_default_paths" just before calling
"SSL_CTX_load_verify_locations", getting rid of "can_verify", and
setting "verify" to "SSL_VERIFY_PEER" unless "sslcheckcert" is set to
0 (or equivalent renamed option is used).

If we make changes similar to this, it would make installation
easier, since most unix machines would already have the bundle of
trusted certificates installed in the openssl default location
(where it could be used by all the programs linked to the openssl
library). Installation would not have to involve installing another
bundle, and the user wouldn't have to know or remember the location
each time wget is invoked. On platforms where an executable file is
built on one machine and used on another (e.g., DOS, Windows), the
environment variables can be set one time to point to the cert bundle
on the user's machine, taking care of this for all the openssl-linked
programs.

Curl does not use the openssl default values. I think that it was
about 2 years ago, that I posted about this on the curl list. The lynx
browser does use the default values (see the file in the lynx source
distribution "www/Library/Implementation/HTTP.c"). Lynx also documents
the environment variables in "docs/README.sslcerts".

Doug

[Hrvoje Niksic]

Post by Doug Kaufman
All this made me look once again at the code for default certificate
locations in the openssl code and in the wget code. I think I need
to withdraw my suggestion for documentation of SSL_CERT_FILE and
SSL_CERT_DIR in the wget documentation, since a careful look at
gen_sslfunc.c shows that we aren't using them.

Except that, as you note later, maybe we *should* be using them.

Post by Doug Kaufman
doesn't do that. As I understand from looking at the code in
gen_sslfunc.c, wget doesn't do any verification at all unless called
with the sslcheckcert option set to a non-null value.

That's right. And it seems like a very lousy default.

Post by Doug Kaufman
I am certainly not an encryption specialist, but I would favor
different defaults for this. I would think that verifying the cert
for a "secure" site should be the default, or wget may be giving a
false sense of security when it retrieves the files. I would also
favor using the openssl defaults, allowing them to be overridden by
wget command-line options. This would probably mean making changes
in gen_sslfunc.c to call "SSL_CTX_set_default_paths" just before
calling "SSL_CTX_load_verify_locations", getting rid of
"can_verify", and setting "verify" to "SSL_VERIFY_PEER" unless
"sslcheckcert" is set to 0 (or equivalent renamed option is used).

That sounds like a good plan. I'll try to make such a change. If we
do call SSL_CTX_set_default_paths, should we document SSL_CERT_* env
variables as you originally suggested?

Since you seem to be knowledgable about SSL implementation(s), what do
you think about GNU TLS? Is its development active? How hard would
it be to use it in Wget?

[Daniel Stenberg]

Since you seem to be knowledgable about SSL implementation(s), what do you
think about GNU TLS? Is its development active? How hard would it be to
use it in Wget?

I'm not Doug, but since I just recently made libcurl capable of using GnuTLS
(as an option instead of OpenSSL) I think can fill in some info:

GnuTLS is alive and it is working. It is even documented somewhat better than
OpenSSL (which isn't saying a lot, I know).

Converting an OpenSSL-using program into using GnuTLS instead isn't very hard.

[Doug Kaufman]

Post by Hrvoje Niksic

Post by Doug Kaufman
I am certainly not an encryption specialist, but I would favor
different defaults for this. I would think that verifying the cert
for a "secure" site should be the default, or wget may be giving a
false sense of security when it retrieves the files. I would also
favor using the openssl defaults, allowing them to be overridden by
wget command-line options. This would probably mean making changes
in gen_sslfunc.c to call "SSL_CTX_set_default_paths" just before
calling "SSL_CTX_load_verify_locations", getting rid of
"can_verify", and setting "verify" to "SSL_VERIFY_PEER" unless
"sslcheckcert" is set to 0 (or equivalent renamed option is used).

That sounds like a good plan. I'll try to make such a change. If we
do call SSL_CTX_set_default_paths, should we document SSL_CERT_* env
variables as you originally suggested?

I think so. I did send a message to the openssl-dev list about this.
Let's wait to see what the openssl developers say.

Post by Hrvoje Niksic
Since you seem to be knowledgable about SSL implementation(s), what do
you think about GNU TLS? Is its development active? How hard would
it be to use it in Wget?

I have never really followed GNU TLS. My impression was that it was a
less mature implementation. For routine use, I would use either. When
security was important, I would tend to favor the implementation which
had been tested more thoroughly, which I believe is Openssl. Openssl
can be FIPS certified; I don't know about GNU TLS. I think that there
are two separate questions. One is whether the encryption code can be
easily integrated with the application. The other is how secure the
implementation of the encryption library is. I don't know the answer to
either.
Doug

[Hrvoje Niksic]

Post by Doug Kaufman

Post by Hrvoje Niksic
That sounds like a good plan. I'll try to make such a change. If
we do call SSL_CTX_set_default_paths, should we document SSL_CERT_*
env variables as you originally suggested?

I think so. I did send a message to the openssl-dev list about this.
Let's wait to see what the openssl developers say.

Any news on this?

A side-effect of this development is that wget-1.10-beta1 refuses to
download from any SSL server if the certificate authorities aren't
locally configured. Since OpenSSL doesn't come with a preinstalled CA
certificate bundle and Wget doesn't come with a preinstalled bundle
either, where is the user to get a bundle from?

(On my Debian installation the certificates come with the
"ca-certificates" package and are apparently assembled from different
sources, the most significant being Mozilla. On SuSE 9.2 the CA
certificates come with the "openssl" package.)

The users will complain about this, and I'd like to know what to tell
them other than "use --no-check-certificate".

[Daniel Stenberg]

(On my Debian installation the certificates come with the "ca-certificates"
package and are apparently assembled from different sources, the most
significant being Mozilla. On SuSE 9.2 the CA certificates come with the
"openssl" package.)

There are no license restrictions that prevent you from using/bundling/include
the Mozilla one (if you want to). I have a little service up and running for
those who wants the latest Mozilla ca cert bundle in PEM format:

http://curl.haxx.se/docs/caextract.html

The Debian wget packager will of coure be encouraged to make wget use the
already installed cacert file.

[Hrvoje Niksic]

Post by Daniel Stenberg
There are no license restrictions that prevent you from
using/bundling/include the Mozilla one (if you want to). I have a
little service up and running for those who wants the latest Mozilla
http://curl.haxx.se/docs/caextract.html

Thanks for pointing this out. I now tested Wget on an OpenSSL
installation without a "site bundle" and the extracted bundle works
flawlessly.

Post by Daniel Stenberg
The Debian wget packager will of coure be encouraged to make wget
use the already installed cacert file.

Yes, wget should "suggest" (or possibly even "recommend") the package
with the CA certificates.

[Doug Kaufman]

Post by Hrvoje Niksic

Post by Doug Kaufman

Post by Hrvoje Niksic
That sounds like a good plan. I'll try to make such a change. If
we do call SSL_CTX_set_default_paths, should we document SSL_CERT_*
env variables as you originally suggested?

I think so. I did send a message to the openssl-dev list about this.
Let's wait to see what the openssl developers say.

Any news on this?

Nothing yet, but it isn't unusual for it to take weeks to get a comment
or reply.

Post by Hrvoje Niksic
A side-effect of this development is that wget-1.10-beta1 refuses to
download from any SSL server if the certificate authorities aren't
locally configured. Since OpenSSL doesn't come with a preinstalled CA
certificate bundle and Wget doesn't come with a preinstalled bundle
either, where is the user to get a bundle from?

This is the problem with having real security. It should be obtained
from a "trusted" source. I extracted my certificates from Microsoft'
Internet Explorer. Various packages have cert bundles distributed with
them, but the user doesn't have an easy way to know that they are
legitimate.

Post by Hrvoje Niksic
The users will complain about this, and I'd like to know what to tell
them other than "use --no-check-certificate".

I am not sure that there is an easy answer. The more secure the
certificates, the more trouble they are to obtain.
Doug

[Mauro Tortonesi]

Post by Hrvoje Niksic

--sslcerttype=0/1 to --sslcerttype=PEM/ASN1
--sslcheckcert=1/0 to --no-sslcheckcert/--sslcheckcert
--sslprotocol=0-3 to --no-ssl/--ssl=SSLv2/SSLv3/TLSv1

The name could (and IMHO should) be made even more readable,
e.g. --ssl-cert-type or even --ssl-certificate-type. It might make
sense to drop the "ssl" prefix altogether because those options also
apply to TLS. The option would then be --certificate-type, which is
shorter and nicer. I believe curl has done that.
Since --sslprotocol can specify TLS protocol, it might be more
accurate to name it --secure-protocol (--protocol is too general),
with the accepted values "auto" (default), "sslv2", "sslv3", and
"tlsv1", all case-insensitive. (Note that the current --sslprotocol=0
does *not* correspond to --no-ssl; it means choose automatically. The
fact that it confused you is further proof of the brokenness of
current option names!)

--certificate-type and --secure-protocol seem fine for me.

Post by Hrvoje Niksic

the other options seem fine to me, although i prefer names like
--ssl_cert_file than --sslcertfile.

Sure, except it should be --ssl-cert-file; Wget (and GNU software in
general) doesn't use underscores in option names.

right.
--
Aequam memento rebus in arduis servare mentem...

Mauro Tortonesi http://www.tortonesi.com

University of Ferrara - Dept. of Eng. http://www.ing.unife.it
Institute of Human & Machine Cognition http://www.ihmc.us
GNU Wget - HTTP/FTP file retrieval tool http://www.gnu.org/software/wget
Deep Space 6 - IPv6 for Linux http://www.deepspace6.net
Ferrara Linux User Group http://www.ferrara.linux.it