wget ssl certificate problem

[James Mcphee]

Yeah, take a look at the makefile for wget and you can get an idea of how
complicated these kinds of general use programs are.  you can make a
relatively simple http client in code, but trying to get it to handle all
the corner cases of the web, it's just easier to depend on something that
already does all the heavy lifting.  For scripting, it's usually either
wget or curl.  Full languages will tend to have their own http libs and
don't have to reach outside, though they will tend to depend on SSL/TLS
from openssl or gnu_tls on the OS to avoid having the implement that whole
stack in native code.  Tend, not required.  There is a native ssl
implementation in java for example.

Interesting about wget2.  The distros I tend to use are so ancient I wasn't
aware it had been released.  Finally support for some of the more modern
http options, which has always been a weakness of wget.  Thanks for that!

Regarding the certificate trust issue, if you want to continue poking,
check to see if you have
/etc/ssl/certs/USERTrust_RSA_Certification_Authority.pem
check to see if that's in /etc/ssl/certs/ca-certificates.crt

If it is, try wget with --ca-certificate or --ca-directory options and see
if that helps.

Based on the error, ERROR: cannot verify www.gutenberg.org's
<http://www.gutenberg.org%27s/> certificate, issued by ‘CN=Network
Solutions OV Server CA 2 ,O=Network Solutions
L.L.C.,L=Herndon,ST=VA,C=US’:  that
should follow the chain to CN = USERTrust RSA Certification Authority.
Since it's not, that would be where I would look.  "sudo
update-ca-certificates -f" if you need to clean up /etc/ssl/certs from old
links.

On Sun, Sep 18, 2022 at 9:26 AM Jim via PLUG-discuss <
plug-discuss at lists.phxlinux.org> wrote:

> I was looking in muon and found wget2.  In the description it says: GNU
> Wget2 is the successor of GNU Wget.  So I installed wget2 and tested it to
> find it works.  Do any other apps use wget?  If so, could I replace
> /usr/bin/wget with a symbolic link to /usr/bin/wget2?  I ask because I
> thought about using muon to purge wget, but it warned me that a bunch of
> stuff would also be removed, so I clicked cancel.
> On 9/17/22 15:08, James Mcphee via PLUG-discuss wrote:
>
> wget, curl, etc are compiled with gnu_tls or openssl or libressl, or
> whatever.  usually when adding those config options, you'll have some vars
> for distro-specific settings.  anyway.  in ubuntu, ca-certificates is the
> pkg that holds your normal trust stuff.  update-ca-certificates is the
> command you'd use to do the update.  So, if you think you broke your trust
> store, you could try update-ca-certificates, and if that didn't work, a
> reinstall of ca-certificates.  specifically, what update-ca-certificates
> does is takes the list from /etc/ca-certificates.conf from /etc/ssl/certs
> and updates the various ca bundles like the java cacerts and the
> ca-certificates.txt, and anything else if the distro decided to use that in
> its TLS/SSL config.
>
> On Sat, Sep 17, 2022 at 11:46 AM Michael Butash via PLUG-discuss <
> plug-discuss at lists.phxlinux.org> wrote:
>
>> Some quick searching as I don't often use wget, it looks like it doesn't
>> use local system certs, and has no inherent trust to certs at all.  If you
>> search "wget ssl certificates" like I just did, you see others posting how
>> to skip the check and trust anyways, and various discussions wtf this is
>> even a thing still.  Weird software caveat I'd say it doesn't just
>> reference system cert trusts, or just hasn't felt the need to be updated in
>> 20 years because you know, security is meh.
>>
>> -mb
>>
>>
>>
>> On Sat, Sep 17, 2022 at 10:40 AM Jim via PLUG-discuss <
>> plug-discuss at lists.phxlinux.org> wrote:
>>
>>> It's not just ww.gutenberg.org. That's an example of what happens no
>>> matter what site I try to use wget on.  About the truststore, how do I add
>>> to or update it?  I decided to ask for help after trying to install
>>> openwebrx following the instructions here.
>>> https://www.openwebrx.de/download/ubuntu.php  Also I found out today
>>> that something similar happens with youtube-dl.  I tried to use it today
>>> and this is what happened.   Youtube-dl works if I use the
>>> --no-check-certificate option.
>>>
>>> $ youtube-dl https://www.youtube.com/watch?v=VW3XQDDGhA4
>>> [youtube] VW3XQDDGhA4: Downloading webpage
>>> WARNING: Unable to download webpage: <urlopen error [SSL:
>>> CERTIFICATE_VERIFY_FAILED] certificate ver
>>> ify failed: unable to get local issuer certificate (_ssl.c:1131)>
>>> [youtube] VW3XQDDGhA4: Downloading API JSON
>>> ERROR: Unable to download API page: <urlopen error [SSL:
>>> CERTIFICATE_VERIFY_FAILED] certificate veri
>>> fy failed: unable to get local issuer certificate (_ssl.c:1131)> (caused
>>> by URLError(SSLCertVerifica
>>> tionError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify
>>> failed: unable to get local issuer
>>> certificate (_ssl.c:1131)')))
>>>
>>>
>>>
>>> On 9/16/22 17:33, James Mcphee via PLUG-discuss wrote:
>>>
>>> check out the verification of the cert chain.  it works for me with a
>>> new build of 20.04, so it might be that you need to add or update your
>>> truststore.
>>> openssl s_client -connect www.gutenberg.org:443 < /dev/null | openssl
>>> x509 -text -noout
>>>
>>> up there at the top, this is what it looks like when it works
>>> depth=2 C = US, ST = New Jersey, L = Jersey City, O = The USERTRUST
>>> Network, CN = USERTrust RSA Certification Authority
>>> verify return:1
>>> depth=1 C = US, ST = VA, L = Herndon, O = Network Solutions L.L.C., CN =
>>> Network Solutions OV Server CA 2
>>> verify return:1
>>> depth=0 C = US, ST = Utah, L = Salt Lake City, O = Project Gutenberg
>>> Literary Archive Foundation, CN = *.gutenberg.org
>>> verify return:1
>>> DONE
>>>
>>> I can see that i have that usertrust network cert in /etc/ssl/certs, so
>>> all is good.  if i had to add one i'd have then run update-ca-certicates.
>>>
>>> On Fri, Sep 16, 2022 at 2:17 PM Jim via PLUG-discuss <
>>> plug-discuss at lists.phxlinux.org> wrote:
>>>
>>>> This has been bugging me for a while, but today it's annoying me to the
>>>> point I want to fix it.  Wget gives me an error whenever I try to use it.
>>>> I have no problem getting files using a web browser.  Here's an example.
>>>> Using firefox I was able to download the file, but this can be a pain in
>>>> the butt when I'm trying to add a repository.  I have Ubuntu 20.04
>>>> installed.
>>>>
>>>>
>>>> $ wget https://www.gutenberg.org/ebooks/68992.epub.images
>>>> --2022-09-16 14:08:02--
>>>> https://www.gutenberg.org/ebooks/68992.epub.images
>>>> Resolving www.gutenberg.org (www.gutenberg.org)... 152.19.134.47,
>>>> 2610:28:3090:3000:0:bad:cafe:47
>>>> Connecting to www.gutenberg.org (www.gutenberg.org)|152.19.134.47|:443...
>>>> connected.
>>>> ERROR: cannot verify www.gutenberg.org's certificate, issued by
>>>> ‘CN=Network Solutions OV Server CA 2
>>>> ,O=Network Solutions L.L.C.,L=Herndon,ST=VA,C=US’:
>>>>  Self-signed certificate encountered.
>>>> To connect to www.gutenberg.org insecurely, use
>>>> `--no-check-certificate'.
>>>>
>>>> Any idea how to fix this?  thanks
>>>>
>>>>
>>>> ---------------------------------------------------
>>>> PLUG-discuss mailing list: PLUG-discuss at lists.phxlinux.org
>>>> To subscribe, unsubscribe, or to change your mail settings:
>>>> https://lists.phxlinux.org/mailman/listinfo/plug-discuss
>>>
>>>
>>>
>>> --
>>> James McPhee
>>> jmcphe at gmail.com
>>>
>>> ---------------------------------------------------
>>> PLUG-discuss mailing list: PLUG-discuss at lists.phxlinux.org
>>> To subscribe, unsubscribe, or to change your mail settings:https://lists.phxlinux.org/mailman/listinfo/plug-discuss
>>>
>>> ---------------------------------------------------
>>> PLUG-discuss mailing list: PLUG-discuss at lists.phxlinux.org
>>> To subscribe, unsubscribe, or to change your mail settings:
>>> https://lists.phxlinux.org/mailman/listinfo/plug-discuss
>>
>> ---------------------------------------------------
>> PLUG-discuss mailing list: PLUG-discuss at lists.phxlinux.org
>> To subscribe, unsubscribe, or to change your mail settings:
>> https://lists.phxlinux.org/mailman/listinfo/plug-discuss
>
>
>
> --
> James McPhee
> jmcphe at gmail.com
>
> ---------------------------------------------------
> PLUG-discuss mailing list: PLUG-discuss at lists.phxlinux.org
> To subscribe, unsubscribe, or to change your mail settings:https://lists.phxlinux.org/mailman/listinfo/plug-discuss
>
> ---------------------------------------------------
> PLUG-discuss mailing list: PLUG-discuss at lists.phxlinux.org
> To subscribe, unsubscribe, or to change your mail settings:
> https://lists.phxlinux.org/mailman/listinfo/plug-discuss



-- 
James McPhee
jmcphe at gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.phxlinux.org/pipermail/plug-discuss/attachments/20220919/d9680123/attachment.html>