Mailing List Archive

Infinite loop?
opensuse LEAP 15.0
linux 4.12.14-lp150.12.64-default x86_64

gpg2 seems to enter an infinite loop. The system recently updated; gnupg
does appear in the update log. That is all I know.

I executed the command
$ gpg2 --debug-all --list-keys
to get a debug output. The resulting file is really large: 8.9MB. I have
provided a Dropbox link to the debug file below.

https://www.dropbox.com/s/xc2jtntmzhr10lg/gpg2-debug.txt?dl=0


$ gpg2 --version
gpg (GnuPG) 2.2.5
libgcrypt 1.8.2
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<https://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Home: /home/jmoe/.gnupg
Supported algorithms:
Pubkey: RSA, ELG, DSA, ECDH, ECDSA, EDDSA
Cipher: IDEA, 3DES, CAST5, BLOWFISH, AES, AES192, AES256, TWOFISH,
CAMELLIA128, CAMELLIA192, CAMELLIA256
Hash: SHA1, RIPEMD160, SHA256, SHA384, SHA512, SHA224
Compression: Uncompressed, ZIP, ZLIB, BZIP2


--
James Moe
moe dot james at sohnen-moe dot com
520.743.3936
Think.

_______________________________________________
Gnupg-users mailing list
Gnupg-users@gnupg.org
http://lists.gnupg.org/mailman/listinfo/gnupg-users
Re: Infinite loop? [ In reply to ]
On 23/06/2019 11.53 AM, James Moe via Gnupg-users wrote:

> gnupg does appear in the update log
>
Sigh. Typo.
gnupg does NOT appear in the update log. Nor does libscrypt.

The repetitive part of the log starts at line 475.

--
James Moe
moe dot james at sohnen-moe dot com
520.743.3936
Think.

_______________________________________________
Gnupg-users mailing list
Gnupg-users@gnupg.org
http://lists.gnupg.org/mailman/listinfo/gnupg-users
Re: Infinite loop? [ In reply to ]
On Sun 2019-06-23 15:00:40 -0700, James Moe via Gnupg-users wrote:
> On 23/06/2019 11.53 AM, James Moe via Gnupg-users wrote:
>
>> gnupg does appear in the update log
>>
> Sigh. Typo.
> gnupg does NOT appear in the update log. Nor does libscrypt.

Without having access to your pubring.gpg, it's hard to say what is
triggering this loop.

I also note that gpg version 2.2.5 is fairly old -- is it possible to
try to upgrade to a newer version?

> The repetitive part of the log starts at line 475.

The repetitions at 475 actually end at 8144, and different things appear
to be happening in that debug log up until you sent it an interrupt to
cancel.

Is it possible that your pubring.gpg is corrupt?

Can you replicate this with a minimal GNUPGHOME?

Can you try migrating from pubring.gpg to pubring.kbx and see if you
still have the same problem? A script for doing that migration is here
if you want to try:

https://salsa.debian.org/debian/gnupg2/blob/debian/master/debian/migrate-pubring-from-classic-gpg

(you might want to back up your ~/.gnupg/ before trying)

All the best,

--dkg
Re: Infinite loop? [ In reply to ]
On 25/06/2019 8.30 AM, Daniel Kahn Gillmor wrote:

> Is it possible that your pubring.gpg is corrupt?
>
As it happens, yes.
The size of pubring.gpg was 20MB; the backup copy was 1.3MB. After
restoring from backup, gpg2 performs normally.
Thank you.

--
James Moe
moe dot james at sohnen-moe dot com
520.743.3936
Think.

_______________________________________________
Gnupg-users mailing list
Gnupg-users@gnupg.org
http://lists.gnupg.org/mailman/listinfo/gnupg-users
Re: Infinite loop? [ In reply to ]
On Tue 2019-06-25 12:02:13 -0700, James Moe via Gnupg-users wrote:
> On 25/06/2019 8.30 AM, Daniel Kahn Gillmor wrote:
>
>> Is it possible that your pubring.gpg is corrupt?
>
> As it happens, yes.
> The size of pubring.gpg was 20MB; the backup copy was 1.3MB. After
> restoring from backup, gpg2 performs normally.

Interesting! my pubring.kbx is 147MiB, but GnuPG still should not run
forever when doing --list-keys. It takes 17s to complete the listing of
my pubring.kbx, as measured by "time gpg --list-keys > /dev/null"

gpg should not take forever to list a 20MB keyring.

If you still have a copy of the corrupt 20M pubring.gpg, it might be
interesting to see it as an example, because it sounds like it's
tickling a bug.

however, sharing your keyring might be sensitive, and 20MB certainly
won't fit in a message on this mailing list. If you are willing to
share it privately so this can be tracked down, feel free to reply to me
privately and we can figure out the details.

--dkg
Re: Infinite loop? [ In reply to ]
On 2019-06-25 at 18:47 -0400, Daniel Kahn Gillmor via Gnupg-users wrote:
> Interesting! my pubring.kbx is 147MiB, but GnuPG still should not run
> forever when doing --list-keys. It takes 17s to complete the listing of
> my pubring.kbx, as measured by "time gpg --list-keys > /dev/null"

With GnuPG 2.2.16 :

% ls -ldh ~/.gnupg/pubring.kbx
-rw-r--r-- 1 pdp pdp 241M Jun 22 22:16 /home/pdp/.gnupg/pubring.kbx
% time gpg --list-keys >/dev/null
[...]
gpg --list-keys > /dev/null 1473.99s user 1965.72s system 99% cpu 57:19.85 total
% kbxutil --stats .gnupg/pubring.kbx
Total number of blobs: 5640
header: 1
empty: 0
openpgp: 5638
x509: 1
non flagged: 5638
secret flagged: 0
ephemeral flagged: 1

This is an "Intel(R) Atom(TM) CPU D2500 @ 1.86GHz" and is where I've
long had my high-security keys. One bright side to this box and its
speed: it's immune to speculative prediction attacks. None of that
newfangled nonsense. ;)

I've long been resigned to this being normal. An unthinking import of a
fuller keyring (probably this one) to my recent new work laptop
(Thinkpad X1 Carbon, running Ubuntu) led to confusion as I re-acclimated
to a Linux desktop after years of macOS usage, because core parts of
system preferences appeared to just hang and do nothing. Until I
finally realized the problem and nuked the keyring down to a dozen keys
which most mattered here. I hadn't realize that my GnuPG keyring was
being exposed in my view of the preferences.

In fact, I got so used to seahorse just dying that I adjusted my login
scripts to ignore it and fire up my own ssh-agent so that I wouldn't
lose the ability to log into other machines. I made that conditional
upon the socket being dead and grumpily chalked it up to Linux
flakiness, but I see now that this hasn't been getting triggered
recently.

The X1 Carbon is 8 claimed cores of "Intel(R) Core(TM) i7-8650U CPU @
1.90GHz" and 16GiB RAM. It was definitely not happy at a keyring which
lets me comfortably verify software releases from signers in the strong
set.

> If you still have a copy of the corrupt 20M pubring.gpg, it might be
> interesting to see it as an example, because it sounds like it's
> tickling a bug.

If you're interested, I can share mine; there are no "secret" keys in it
and I'll trust you not to leak the communications graph of which
software I care about verifying :) or the public signatures from the
strong set showing where I've been over the years or the local
signatures for "yeah, I grabbed these fingerprints from a web-page, I'll
trust them locally but won't attest to them publicly".

-Phil
Re: Infinite loop? [ In reply to ]
On Tue 2019-06-25 23:03:18 -0400, Phil Pennock wrote:
> With GnuPG 2.2.16 :
>
> % ls -ldh ~/.gnupg/pubring.kbx
> -rw-r--r-- 1 pdp pdp 241M Jun 22 22:16 /home/pdp/.gnupg/pubring.kbx
> % time gpg --list-keys >/dev/null
> [...]
> gpg --list-keys > /dev/null 1473.99s user 1965.72s system 99% cpu 57:19.85 total
> % kbxutil --stats .gnupg/pubring.kbx
> Total number of blobs: 5640
> header: 1
> empty: 0
> openpgp: 5638
> x509: 1
> non flagged: 5638
> secret flagged: 0
> ephemeral flagged: 1
>
> This is an "Intel(R) Atom(TM) CPU D2500 @ 1.86GHz" and is where I've
> long had my high-security keys. One bright side to this box and its
> speed: it's immune to speculative prediction attacks. None of that
> newfangled nonsense. ;)

i'm glad you have a sense of humor about it, but this sounds
unacceptably slow to me. --list-keys isn't even doing any cryptographic
processing, right?

> I've long been resigned to this being normal.

the timing you show above suggests that --list-keys operates at ~100
keys per minute, or not even 2 keys per second.

And why on earth is so much time spent in the kernel? for my own run,
the breakdown is:

real 0m17.555s
user 0m17.367s
sys 0m0.184s

But yours appears to have > 50% of its time spent in "sys". Can you use
strace -T or some other profiler to see what system calls it's making
that makes it spend so much time in the kernel?

> In fact, I got so used to seahorse just dying that I adjusted my login
> scripts to ignore it and fire up my own ssh-agent so that I wouldn't
> lose the ability to log into other machines. I made that conditional
> upon the socket being dead and grumpily chalked it up to Linux
> flakiness, but I see now that this hasn't been getting triggered
> recently.
>
> The X1 Carbon is 8 claimed cores of "Intel(R) Core(TM) i7-8650U CPU @
> 1.90GHz" and 16GiB RAM. It was definitely not happy at a keyring which
> lets me comfortably verify software releases from signers in the strong
> set.

numbers like 5K keys and 241MiB are not large for a machine of this
caliber. my own kbxutil --stats looks like:

Total number of blobs: 4166
header: 1
empty: 0
openpgp: 3787
x509: 378
non flagged: 4165
secret flagged: 0
ephemeral flagged: 0

If you import your larger keyring on the X1 Carbon, what is the output
of "time gpg --list-keys > /dev/null" there?

> If you're interested, I can share mine; there are no "secret" keys in it
> and I'll trust you not to leak the communications graph of which
> software I care about verifying :) or the public signatures from the
> strong set showing where I've been over the years or the local
> signatures for "yeah, I grabbed these fingerprints from a web-page, I'll
> trust them locally but won't attest to them publicly".

If the issue is just a large keyring, i can generate that myself pretty
easily, thanks. I was concerned by the OP that there was an acual
infinite loop somewhere. But if the run is just taking a long time
because of unoptimized code, we should try to address that as a separate
issue.

--dkg

ps fwiw, i think even 17s is too long for my own 4K keys, esp. with a
hot fileysystem cache. that's still only ~220 per second, but it's
managable, and i've had enough other things i want to get fixed in
GnuPG that i haven't dug in too deeply to see where the problem is or
how it could be sped up. But it's nothing near the order of
magnitude that you're describing.