[Bug-wget] Header line removal behavior

[Gregory R Fellow]

Hi. Is it the intended behavior for wget to allow sending custom header
lines with no value?

The following clears previous user-defined headers as described in the
documentation:
--header=

The following both send a header with no value:
--header="Accept-Encoding:"
--header="Accept-Encoding: "

This gets an "Invalid header" error from wget:
--header="Accept-Encoding"

I noticed this behavior while trying to disable some of wget's
automatically generated headers (which apparently isn't possible except
for User-Agent via --user-agent=).

Thanks in advance for your help.

Greg

[Tim Rühsen]

Post by Gregory R Fellow
Hi. Is it the intended behavior for wget to allow sending custom header
lines with no value?
The following clears previous user-defined headers as described in the
--header=
--header="Accept-Encoding:"
--header="Accept-Encoding: "

Well, --header is kind of an "expert" option. Possible use is to test
web servers. A lenient handling is IMO better here than a too strict one.

Post by Gregory R Fellow
--header="Accept-Encoding"

One of the few checks applied :-)

Post by Gregory R Fellow
I noticed this behavior while trying to disable some of wget's
automatically generated headers (which apparently isn't possible except
for User-Agent via --user-agent=).

If you need that, please open an issue at
https://savannah.gnu.org/bugs/?func=additem&group=wget with your
detailed ideas.

Regards, Tim

[Darshit Shah]

Hi Gregory,

What you've mentioned is indeed the intended behaviour.

Post by Gregory R Fellow
Hi. Is it the intended behavior for wget to allow sending custom header
lines with no value?
The following clears previous user-defined headers as described in the
--header=

Yes, as you've correctly mentioned, this case is mentioned in the documentation
as a way to remove any previously defined headers.

Post by Gregory R Fellow
--header="Accept-Encoding:"
--header="Accept-Encoding: "

These are both accepted by Wget and they will send the exact same HTTP request.
This is because, both of them are considered equivalent _and_ valid according
to the HTTP/1.1 Spec in RFC 7230. The spec clearly mentions that a leading
whitespace is okay and so is an empty Header value.

Post by Gregory R Fellow
--header="Accept-Encoding"

Of course. Because it is an invalid header according to the HTTP spec :). Per
RFC 7230, a header _must_ have the form: "<header-name>: <header-value>". The
colon being immediately after the header-name is a hard requirement.

Post by Gregory R Fellow
I noticed this behavior while trying to disable some of wget's automatically
generated headers (which apparently isn't possible except for User-Agent via
--user-agent=).

In Wget, you can't disable the headers, but you can indeed overwrite them to
something else or even make them empty. The real question here is, why do you
want to disable some headers? Since Wget only generated headers that are
absolutely necessary for parsing the request correctly.

Post by Gregory R Fellow
Thanks in advance for your help.
Greg