gpg doesn't fail on target file existing when decrypting

[Andrew Flerchinger]

I'm in windows trying to run gpg (GnuPG) 1.4.9 (Gpg4win 1.1.4) in batch
mode, completely non-interactive. I can encrypt a file like this:
gpg --passphrase **PASS** --trust-model always --batch --output
"test.txt.pgp" --sign --recipient **RECIP** --encrypt "test.txt"

and it runs fine. If I do it a second time and the output file exists, it
exits with a non-zero error code. Passing in --yes overwrites the file and
success is returned. So far, so good.

When I decrypt a file, I use a very similar command:

gpg --passphrase **PASS** --trust-model always --batch --output "test.txt"
--decrypt "test.txt.pgp"

It mostly works fine. If the output file doesn't exist, it creates it
properly. If it does exist and I add --yes, it overwrites it properly. My
problem is when I don't tell it to overwrite and the target exists, it looks
like it properly decrypted the file, except it does nothing. The return code
is still zero and the output looks exactly the same as when the file doesn't
exist and it creates it.

I'm trying to figure out if I'm doing something wrong, it's a bug, or if the
intended behavior is not the same between encryption and decryption. If I
remove --batch from either encrypt or decrypt, both prompt me for a
replacement file name. I was assuming both would fail given the option. Can
anyone shed some light on this for me?

Thanks.
Andrew
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/attachments/20090311/1143f254/attachment-0001.htm>

[vedaal]

Andrew Flerchinger icrf.ml at gmail.com

My problem is when I don't tell it to overwrite
and the target exists, it looks like it
properly decrypted the file,
except it does nothing
I'm trying to figure out if I'm doing something wrong,

no, you're doing everything correctly

it's a bug,

no,
it's just not telling you that it overwrote it


to test this,

save "test.txt" as "originaltest.txt"

change the text of "test.txt" after you encrypted it, and save it
as "test.text"
and also as "changedtest.ext"

now use your commands to decrypt

you will see that when you now open "test.txt"
it will be the same as your original (i.e. "originaltest.txt")

if gnupg 'did nothing'
it would be the same as "changedtest.txt"


vedaal

any ads or links below this message are added by hushmail without
my endorsement or awareness of the nature of the link

--
Click for quality replacement window deals and save!
http://tagline.hushmail.com/fc/BLSrjkqfaqWn87hYaT2noI7UHq15QMdzE7mClayZwCCoOuxIoFTX3NsWTEM/

[Andrew Flerchinger]

Post by vedaal
Andrew Flerchinger icrf.ml at gmail.com

My problem is when I don't tell it to overwrite
and the target exists, it looks like it
properly decrypted the file,
except it does nothing
I'm trying to figure out if I'm doing something wrong,

no, you're doing everything correctly

it's a bug,

no,
it's just not telling you that it overwrote it
to test this,
save "test.txt" as "originaltest.txt"
change the text of "test.txt" after you encrypted it, and save it
as "test.text"
and also as "changedtest.ext"
now use your commands to decrypt
you will see that when you now open "test.txt"
it will be the same as your original (i.e. "originaltest.txt")
if gnupg 'did nothing'
it would be the same as "changedtest.txt"
vedaal

But it is. If I pass in --yes, it does indeed overwrite as I'd expect. If I
don't, it does NOT overwrite the file. The data in the file stays the same
and the altered date on the file does not change. It doesn't overwrite,
which is as expected, it's just not telling me there was a problem with
decryption like it does when I'm encrypting something.
Are you seeing that behavior? GPG is always overwriting on decryption, even
without --yes?

Thanks for the reply (and sorry, vedaal, if you got this twice, my first
didn't include the mailing list).
Andrew
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/attachments/20090316/b7aa7f12/attachment.htm>

[vedaal]

Andrew Flerchinger icrf.ml at gmail.com

Post by Andrew Flerchinger
If I pass in --yes, it does indeed overwrite as I'd
If I don't, it does NOT overwrite the file.
it's just not telling me there was a problem with
decryption like it does when I'm encrypting something.

there isn't a problem with decrypting,

gnupg asks if you want to overwrite,
and if you answer no (N)
then it asks you where you want the file to be written to

here is what i get when i try it (on windows) without the --yes
option :


c:\gnupg>gpg --passphrase aaaa1 --output c:\q.txt --decrypt
c:\q.txt.gpg
:pubkey enc packet: version 3, algo 1, keyid 7DC4274AF9015496
data: [2047 bits]
gpg: public key is F9015496

You need a passphrase to unlock the secret key for
user: "aaaa1 <aaaa1 at key.test>"
2048-bit RSA key, ID F9015496, created 2005-12-01

gpg: encrypted with 2048-bit RSA key, ID F9015496, created 2005-12-
01
"aaaa1 <aaaa1 at key.test>"
gpg: TWOFISH encrypted data
:compressed packet: algo=1
:literal data packet:
mode b (62), created 1236869352, name="q.txt",
raw data: 3 bytes
gpg: original file name='q.txt'
File `c:\q.txt' exists. Overwrite? (y/N) n
Enter new filename:
Enter new filename: c:\q2.txt
gpg: decryption okay
gpg: session key:
`10:6EB46AC795C6CCB418116E50DDFDC8CBD3D345761C2759DD5223E8D5D30923DC
'

n.b.
i use the options of 'verbose verbose',
so gnupg gives a lot more information than what you might be used
to seeing

here it is again without the 'verbose' options:

c:\gnupg>gpg --passphrase aaaa1 --output c:\q.txt --decrypt
c:\q.txt.gpg

You need a passphrase to unlock the secret key for
user: "aaaa1 <aaaa1 at key.test>"
2048-bit RSA key, ID F9015496, created 2005-12-01

gpg: encrypted with 2048-bit RSA key, ID F9015496, created 2005-12-
01
"aaaa1 <aaaa1 at key.test>"
File `c:\q.txt' exists. Overwrite? (y/N) y
gpg: session key:
`10:6EB46AC795C6CCB418116E50DDFDC8CBD3D345761C2759DD5223E8D5D30923DC
'

c:\gnupg>


so,
does gnupg prompt you to 'overwrite' if you don't use the --yes
option ?


vedaal

any ads or links below this message are added by hushmail without
my endorsement or awareness of the nature of the link

--
Click to replace your roof - modern technology.
http://tagline.hushmail.com/fc/BLSrjkqfVeHljhxoDr3jzvpPFptCL3JglXsrLIhZsdQXvB48wr7WwuSpB5W/

[Andrew Flerchinger]

Post by vedaal
Andrew Flerchinger icrf.ml at gmail.com

Post by Andrew Flerchinger
If I pass in --yes, it does indeed overwrite as I'd
If I don't, it does NOT overwrite the file.
it's just not telling me there was a problem with
decryption like it does when I'm encrypting something.

there isn't a problem with decrypting,
gnupg asks if you want to overwrite,
and if you answer no (N)
then it asks you where you want the file to be written to
here is what i get when i try it (on windows) without the --yes
c:\gnupg>gpg --passphrase aaaa1 --output c:\q.txt --decrypt
c:\q.txt.gpg
:pubkey enc packet: version 3, algo 1, keyid 7DC4274AF9015496
data: [2047 bits]
gpg: public key is F9015496
You need a passphrase to unlock the secret key for
user: "aaaa1 <aaaa1 at key.test>"
2048-bit RSA key, ID F9015496, created 2005-12-01
gpg: encrypted with 2048-bit RSA key, ID F9015496, created 2005-12-
01
"aaaa1 <aaaa1 at key.test>"
gpg: TWOFISH encrypted data
:compressed packet: algo=1
mode b (62), created 1236869352, name="q.txt",
raw data: 3 bytes
gpg: original file name='q.txt'
File `c:\q.txt' exists. Overwrite? (y/N) n
Enter new filename: c:\q2.txt
gpg: decryption okay
`10:6EB46AC795C6CCB418116E50DDFDC8CBD3D345761C2759DD5223E8D5D30923DC
'
n.b.
i use the options of 'verbose verbose',
so gnupg gives a lot more information than what you might be used
to seeing
c:\gnupg>gpg --passphrase aaaa1 --output c:\q.txt --decrypt
c:\q.txt.gpg
You need a passphrase to unlock the secret key for
user: "aaaa1 <aaaa1 at key.test>"
2048-bit RSA key, ID F9015496, created 2005-12-01
gpg: encrypted with 2048-bit RSA key, ID F9015496, created 2005-12-
01
"aaaa1 <aaaa1 at key.test>"
File `c:\q.txt' exists. Overwrite? (y/N) y
`10:6EB46AC795C6CCB418116E50DDFDC8CBD3D345761C2759DD5223E8D5D30923DC
'
c:\gnupg>
so,
does gnupg prompt you to 'overwrite' if you don't use the --yes
option ?
vedaal

Yes, I do see that behavior. The primary difference is that I never want it
to prompt me for anything, since I'm writing a headless wrapper. Try
including the --batch parameter, which suppresses any interaction. I can't
find any indication that it failed to overwrite the destination by return
code or program output.

It works just fine when encrypting, though, which is what has me confused. I
can work around it (mostly, the operation won't be atomic, but it's probably
close enough for me), but the inconsistency between encrypt/decrypt gave me
pause.

Thanks.
Andrew
-------------- next part --------------
An HTML attachment was scrubbed...
URL: </pipermail/attachments/20090316/1c98a01f/attachment-0001.htm>

[Doug Barton]

Post by Andrew Flerchinger
Yes, I do see that behavior. The primary difference is that I never want
it to prompt me for anything, since I'm writing a headless wrapper.

What you're suggesting isn't "safe" in any case. What I would do in
your situation is the following:

1. Use mktemp to safely create a new, unique file
2. Send the decryption output to that file
3. Test if the "real" file exists, and if so unlink it
4. mv $newfile $realfilename


hth,

Doug

[Andrew Flerchinger]

Post by Doug Barton

Post by Andrew Flerchinger
Yes, I do see that behavior. The primary difference is that I never want
it to prompt me for anything, since I'm writing a headless wrapper.

What you're suggesting isn't "safe" in any case. What I would do in
1. Use mktemp to safely create a new, unique file
2. Send the decryption output to that file
3. Test if the "real" file exists, and if so unlink it
4. mv $newfile $realfilename
hth,
Doug

You're right, I could do that to make my work-around act atomic. Or
are you saying that even the functional encrypt behavior isn't "safe?"
I'm assuming that's essentially what gpg is already doing.

I guess I'm still trying to determine the reason for the inconsistent
behavior between encryption and decryption functions. If it's a bug,
I'd like to report it. If it's a design decision, I'd like to know the
rationale behind why. If it's something else, I'd love to be
surprised.

Thanks.
Andrew

[Doug Barton]

Post by Andrew Flerchinger

Post by Doug Barton

Post by Andrew Flerchinger
Yes, I do see that behavior. The primary difference is that I never want
it to prompt me for anything, since I'm writing a headless wrapper.

What you're suggesting isn't "safe" in any case. What I would do in
1. Use mktemp to safely create a new, unique file
2. Send the decryption output to that file
3. Test if the "real" file exists, and if so unlink it
4. mv $newfile $realfilename
hth,
Doug

You're right, I could do that to make my work-around act atomic. Or
are you saying that even the functional encrypt behavior isn't "safe?"

For typical command line use by a real person I think it's just fine.
If I were doing a script that used gpg in an unattended fashion I'd do
the same thing for encrypt as I suggested for decrypt above.

Post by Andrew Flerchinger
I'm assuming that's essentially what gpg is already doing.

Don't assume. Test. Depending on what you're using it for (and in what
environment) the bar is much, much higher for programs (including
scripts) that run in an automated environment than for those that run
with real human interaction. Not that humans always make the right
choices by any stretch.

Post by Andrew Flerchinger
I guess I'm still trying to determine the reason for the inconsistent
behavior between encryption and decryption functions.

You're approaching this problem from the standpoint of unattended
usage, which is not how the current command line behavior was intended.


Doug

[Andrew Flerchinger]

Post by Doug Barton
You're approaching this problem from the standpoint of unattended
usage, which is not how the current command line behavior was intended.
Doug

Okay, I can work around it in a satisfactory fashion. My personal
problem is solved.

Now, assuming that --batch is supposed to make gpg run in an
unattended fashion, as documentation indicates, and behavior differs
in this case between encrypt and decrypt, is there any reason this
isn't a bug (albeit very minor) that should be reported? If so, should
I start with the gnupg-devel mailing list, or create an account with
the bug tracking system and create there?

Thank you all for your help.
Andrew

[Sven Radde]

Hi!

Post by Andrew Flerchinger

Post by Doug Barton
1. Use mktemp to safely create a new, unique file
2. Send the decryption output to that file
3. Test if the "real" file exists, and if so unlink it
4. mv $newfile $realfilename

You're right, I could do that to make my work-around act atomic.

Be careful, this is not necessarily atomic. You're assuming transactions
where no such thing exists.
If the system crashes in the wrong moment, you would have the real file
unlinked and the renaming has not yet taken place. It might even be the
case that the tempfile is not even persisted to disc.

I am not making this up, see for example the current discussion about
the EXT4 data loss issue:
<http://www.h-online.com/open/Possible-data-loss-in-Ext4--/news/112821>

cu, Sven

[Doug Barton]

Post by Sven Radde
Hi!

Post by Andrew Flerchinger

Post by Doug Barton
1. Use mktemp to safely create a new, unique file
2. Send the decryption output to that file
3. Test if the "real" file exists, and if so unlink it
4. mv $newfile $realfilename

You're right, I could do that to make my work-around act atomic.

Be careful, this is not necessarily atomic. You're assuming transactions
where no such thing exists.

I noticed that you were replying to Andrew, but FWIW I very carefully
avoided using the word "atomic" in my post, for good reason.

Post by Sven Radde
If the system crashes in the wrong moment, you would have the real file
unlinked and the renaming has not yet taken place. It might even be the
case that the tempfile is not even persisted to disc.

True, although you're talking millisecond timing it could happen.
OTOH, if untrusted users are allowed on the system the old "symlink a
harmless file to an important file so that the latter can be
splattered by an automated process" is a long-known vulnerability that
can easily be defended against.

On the _other_ other hand, I left out the whole discussion about "What
is your threat model?" and, "What is the value of the data relative to
the amount of work you're willing to spend protect it?"

It's also worth noting that even if the new decrypted file is lost
before the old decrypted file is replaced, it's likely that you still
have the clear original to work from so we're not talking about a
catastrophic data loss. If we were concerned about destroying the
clear original immediately after the new encrypted version is created
then you would at minimum want to get the new version onto the same
file system as the old one before you started removing things.

Post by Sven Radde
I am not making this up ...

I believe you. :) My purpose was more to get the OP thinking in terms
of better procedures for an automated process, and to try to point out
that the current behavior of gnupg doesn't seem to be a bug.


hth,

Doug

[Andrew Flerchinger]

Post by Doug Barton
My purpose was more to get the OP thinking in terms
of better procedures for an automated process, and to try to point out
that the current behavior of gnupg doesn't seem to be a bug.
hth,
Doug

That's still what I'm confused about. What is a reason for, under the
same conditions, intentionally making encrypt throw an error but
making decrypt pass on as successful? Why wouldn't it be consistent?

Andrew