|
|
|
|
|
|
|
| Bifroest | Mr. Tines | CTC Home | CTClib | MacCTC | CTCjava | Manual |
This appendix describes the file formats used externally by Pretty Good Privacy (PGP), the RSA public key cryptography application. The intended audience includes software engineers trying to port PGP to other hardware environments or trying to implement other PGP- compatible cryptography products, or anyone else who is curious.
[To be included: a description of ASCII armor. And ASCII armored file is just like a binary file described here, but with an extra layer of encoding added, framing lines, and a checksum at the end.]
All integer data used by PGP is externally stored most significant byte (MSB) first, regardless of the byte order used internally by the host CPU architecture. This is for cross-compatibility of messages and keys between hosts. This covers multiprecision RSA integers, bit count prefix fields, byte count prefix fields, checksums, key IDs, and timestamps.
The MSB-first byte order for external packet representation was chosen only because many other crypto standards use it.
RSA arithmetic involves a lot of multiprecision integers, often having hundreds of bits of precision. PGP externally stores a multiprecision integer (MPI) with a 16-bit prefix that gives the number of significant bits in the integer that follows. The integer that follows this bitcount field is stored in the usual byte order, with the MSB padded with zero bits if the bitcount is not a multiple of 8. The bitcount always specifies the exact number of significant bits. For example, the integer value 5 would be stored as these three bytes:
00 03 05
An MPI with a value of zero is simply stored with the 16-bit bitcount prefix field containing a 0, with no value bytes following it.
Some packets use a "key ID" field. The key ID is the least significant 64 bits of the RSA public modulus that was involved in creating the packet. For all practical purposes it unique to each RSA public key.
Some packets contain a "user ID", which is an ASCII string that contains the user's name. Unlike a C string, the user ID has a length byte at the beginning that has a byte count of the rest of the string. This length byte does not include itself in the count.
Some packets contain a timestamp, which is a 32-bit unsigned integer of the number of seconds elapsed since 1970 Jan 1 00:00:00 GMT. This is the standard format used by Unix timestamps. It spans 136 years.
Many of these data structures begin with a Cipher Type Byte (CTB),
which specifies the type of data structure that follows it. The CTB
bit fields have the following meaning (bit 0 is the LSB, bit 7 is the
MSB):
Other CTB packet types are unimplemented.
The 8-, 16-, or 32-bit packet length field after the CTB gives the length in bytes of the rest of the packet, not counting the CTB and the packet length field.
| Offset | Length | Meaning |
| 0 | 1 | CTB for RSA public-key-encrypted packet |
| 1 | 2 | 16-bit (or maybe 8-bit) length of packet |
| 3 | 1 | Version byte (=2). May affect rest of fields that follow. |
| 4 | 8 | 64-bit Key ID |
| 12 | 1 | Algorithm byte for RSA (=1 for RSA). -Algorithm byte affects field definitions that follow. |
| 13 | ? | RSA-encrypted integer, encrypted conventional key packet. (MPI with bitcount prefix) |
The conventionally-encrypted ciphertext packet begins right after the
RSA public-key-encrypted packet that contains the conventional key.
| Offset | Length | Meaning |
| 0 | 1 | CTB for secret-key-encrypted (signed) packet |
| 1 | 2 | 16-bit (or maybe 8-bit) length of packet |
| 3 | 1 | Version byte (=2). May affect rest of fields that follow. |
| 4 | 1 | Length of following material that is implicitly included in MD calculation. |
| 5 | 1 | Signature classification field (see below). Implicitly append this to message for MD calculation. |
| 6 | 4 | 32-bit timestamp of when signature was made. Implicitly append this to message for MD calculation. |
| 10 | 2 | Validity period, in number of DAYS (0 means forever) Implicitly append this to message for MD calculation. |
| 12 | 8 | 64-bit Key ID |
| 20 | 1 | Algorithm byte for public key scheme (RSA=0x01). -Algorithm byte affects field definitions that follow. |
| 21 | 1 | Algorithm byte for message digest (MD5=0x01). |
| 22 | 2 | First 2 bytes of the Message Digest inside the RSA-encrypted integer, to help us figure out if we used the right RSA key to check the signature. |
| 24 | ? | RSA-encrypted integer, encrypted message digest (MPI with bitcount prefix). |
If the plaintext that was signed is included in the same file as the signature packet, it begins right after the RSA secret-key-signed packet that contains the message digest. The plaintext has a "literal" CTB prefix.
The validity period field is generally only used for certifying keys. It should be set to 0 otherwise, for regular message signatures. It may be useful for PEM-like capabilities in future versions of PGP. PGP 2.3 will always just set it to 0, and will ignore it.
There is a length field that specifies how many bytes of material is implicitly included in the MD calculation. If this length field is 5, it means the following 1-byte classification field and the 4-byte timestamp are included in the signature packet. If the length byte is 7, it means the 2-byte validity period is also included. In PGP 2.3, we are using a length field of 5 for the material to be included in the MD calculation, so the validity period is unused and unincluded, and is assumed to be zeroed. This makes the whole signature certificate shorter.
The signature classification field describes what kind of signature certificate this is. There are various hex values:
| 00 | Signature of a message or document, binary image. |
| 01 | Signature of a message or document, canonical text. |
| 10 | Key certification, generic. Only version of key certification supported by PGP 2.0. Material signed is public key pkt and User ID pkt. |
| 11 | Key certification, persona. No attempt made at all to identify the user with a real name. Material signed is public key pkt and User ID pkt. |
| 12 | Key certification, casual identification. Some casual attempt made to identify user with his name. Material signed is public key pkt and User ID pkt. |
| 13 | Key certification, positive ID. Heavy-duty identification efforts, photo ID, direct contact with personal friend, etc. Material signed is public key pkt and User ID pkt. |
| 20 | Key compromise. User signs his own compromise certificate. Independent of user ID associations. Material signed is public key pkt ONLY. |
| 30 | Key/userid revocation. User can sign his own revocation to dissolve an association between a key and a user ID, or certifier may revoke his previous certification of this key/userid pair. Material signed is public key pkt and User ID pkt. |
| 40 | Timestamping a signature certificate made by someone else. Can be used to apply trusted timestamp, and log it in notary's log. Signature of a signature. |
When a signature is made to certify a key/UserID pair, it is computed across two packets-- the public key packet, and the separate User ID packet. See below.
The packet headers (CTB and length fields) for the public key packet and the user ID packet are both omitted from the signature calculation for a key certification.
A key compromise certificate may be issued by someone to revoke his own key when his secret key is known to be compromised. If that happens, a user would sign his own key compromise certificate with the very key that is being revoked. A key revoked by its own signature means that this key should never be used or trusted again, in any form, associated with any user ID. A key compromise certificate issued by the keyholder shall take precedence over any other key certifications made by anyone else for that key. A key compromise signed by someone other than the key holder is invalid.
Note that a key compromise certificate just includes the key packet in its signature calculation, because it kills the whole key without regard to any userid associations. It isn't tied to any particular userid association. It should be inserted after the key packet, before the first userid packet.
When a key compromise certificate is submitted to PGP, PGP will place it on the public keyring. A key compromise certificate is always accompanied in its travels by the public key and userIDs it affects. If the affected key is NOT already on the keyring, the compromise certificate (and its key and user ID) is merely added to the keyring anywhere. If the affected key IS already on the keyring, the compromise certificate is inserted after the affected key packet. This assumes that the actual key packet is identical to the one already on the key ring, so no duplicate key packet is needed. If a key has been revoked, PGP will not allow its use to encipher any messages, and if an incoming signature uses it, PGP will display a stern warning that this key has been revoked.
NOTE: Key/userid revocation certificates WILL NOT BE SUPPORTED in this version of PGP. But if we ever get around to supporting them, here are some ideas on how they should work...
A key/userid revocation certificate may be issued by someone to dissolve the association between his own key and a user ID. He would sign it with the very key that is being revoked. A key/userid revocation certificate issued by the keyholder shall take precedence over any other key certifications made by anyone else for that key/userid pair. Also, a third party certifier may revoke his own previous certification of this key/userid pair by issuing a key/userid revocation certificate. Such a revocation should not affect the certifications by other third parties for this same key/userid pair.
When a key/userid revocation certificate is submitted to PGP, PGP will place it on the public keyring. A key/userid revocation certificate is always accompanied in its travels by the public key it affects (the key packet and user ID packet precedes the revocation certificate). If the affected key is NOT already on the keyring, the revocation certificate (and its key and user ID) is merely added to the keyring anywhere. If the affected key IS already on the keyring, the revocation certificate is integrated in with the key's other certificates as though it were just another key certification. This assumes that the actual key packet is identical to the one already on the key ring, so no duplicate key packet is needed.
The Message digest has no CTB packet framing. It is stored packetless and naked, with padding, encrypted inside the MPI in the Signature packet.
PGP versions 2.3 and later use a new format for encoding the message digest into the MPI in the signature packet, a format which is compatible with RFC1425 (formerly RFC1115). This format is accepted but not written by version 2.2. The older format used by versions 2.2 and earlier is also accepted by version 2.3.
PGP versions 2.2 and earlier encode the MD into the MPI as follows:
MSB . . . LSB
0 1 MD(16 bytes) 0 FF(n bytes) 1
Enough bytes of FF padding are added to make the length of this whole string equal to the number of bytes in the modulus.
PGP versions 2.3 and later encode the MD into the MPI as follows:
MSB . . . LSB
0 1 FF(n bytes) 0 ASN(18 bytes) MD(16 bytes)
See RFC1423 for an explanation of the meaning of the ASN string. It is the following 18 byte long hex value:
3020300c06082a864886f70d020505000410
Enough bytes of FF padding are added to make the length of this whole string equal to the number of bytes in the modulus.
All this mainly affects the rsa_private_encrypt() and rsa_public_decrypt() functions in rsaglue.c.
There is no checksum included. We do include a copy of 2 bytes of the MD in the outer packet to help determine if we used the correct RSA key.
The DEK has no CTB packet framing. The DEK is stored packetless and naked, with padding, encrypted inside the MPI in the RSA public-key-encrypted packet.
PGP versions 2.3 and later use a new format for encoding the message digest into the MPI in the signature packet. (This format is not presently based on any RFCs due to the use of the IDEA encryption system.) This format is accepted but not written by version 2.2. The older format used by versions 2.2 and earlier is also accepted by version 2.3.
PGP versions 2.2 and earlier encode the MD into the MPI as follows:
MSB . . . LSB
0 1 DEK(16 bytes) CSUM(2 bytes) 0 RND(n bytes) 2
CSUM refers to a 16-bit checksum appended to the high end of the DEK. RND is a string of NONZERO pseudorandom bytes, enough to make the length of this whole string equal to the number of bytes in the modulus.
PGP versions 2.3 and later encode the MD into the MPI as follows:
MSB . . . LSB
0 2 RND(n bytes) 0 1 DEK(16 bytes) CSUM(2 bytes)
CSUM refers to a 16-bit checksum appended to the high end of the DEK. RND is a string of NONZERO pseudorandom bytes, enough to make the length of this whole string equal to the number of bytes in the modulus.
For both versions, the 16-bit checksum is computed on the rest of the bytes in the DEK key material, and does not include any other material in the calculation. In the above MSB-first representation, the checksum is also stored MSB-first. The checksum is there to help us determine if we used the right RSA secret key for decryption.
All this mainly affects the rsa_public_encrypt() and rsa_private_decrypt() functions in rsaglue.c.
| Offset | Length | Meaning |
| 0 | 1 | CTB for Conventional-Key-Encrypted data packet |
| 1 | 4 | 32-bit (or maybe 16-bit) length of packet |
| 5 | ? | conventionally-encrypted data. plaintext has 64 bits of random data prepended, plus 16 bits prepended for "key check" purposes |
The decrypted ciphertext may contain a compressed data packet or a literal plaintext packet.
After decrypting the conventionally-encrypted data, a special 8-byte random prefix and 2 "key check" bytes are revealed. The random prefix and key check prefix are inserted before encryption and discarded after decryption. This prefix group prefix is only visible only after decrypting the ciphertext in the packet.
The random prefix serves to start off the cipher feedback chaining process with 64 bits of random material. It may be discarded after decryption. The first 8 bytes is the random prefix material, followed by the 2-byte "key-check" prefix.
The key-check prefix is composed of two identical copies of the last 2 random bytes in the random prefix, in the same order. During decryption, the 9th and 10th bytes of decrypted plaintext are checked to see if they match the 7th and 8th bytes, respectively. If these key-check bytes meet this criterion, then the conventional key is assumed to be correct.
| Offset | Length | Meaning |
| 0 | 1 | CTB for Compressed data packet |
| 1 | 4 | 32-bit (or maybe 16-bit) length of packet |
| 5 | 1 | Compression algorithm selector byte (1=ZIP) |
| 6 | ? | compressed data |
The compressed data begins right after the algorithm selector byte. The compressed data may decompress into a raw literal plaintext data packet with its own CTB.
| Offset | Length | Meaning |
| 0 | 1 | CTB for raw literal data packet |
| 1 | 4 | 32-bit (or maybe 16-bit) length of packet |
| 5 | 1 | mode byte, 'b'= binary or 't'= canonical text |
| 6 | ? | filename, with leading string length byte |
| ? | 4 | Timestamp of last-modified date, or 0, or right now |
| ? | ? | raw literal plaintext data |
The timestamp may be have to be derived in a system dependent manner. ANSI C functions should be used to get it if available, otherwise store the current time in it. Or maybe store 0 if it's somehow not applicable.
Whne calculating a signature on a literal packet, the signature calculation only includes the raw literal plaintext data that begins AFTER the header fields in the literal packet-- after the CTB, the length, the mode byte, the filename, and the timestamp. The reason for this is to guarantee that detached signatures are exactly the same as attached signatures prefixed to the message. Detached signatures are calculated on a separate file that has no packet encapsulation.
A comment packet is generally just skipped over by PGP, although it may be displayed to the user when processed. It can be put in a keyring, or anywhere else.
| Offset | Length | Meaning |
| 0 | 1 | CTB for Comment packet |
| 1 | 1 | 8-bit length of packet |
| 2 | ? | ASCII comment, size is as in preceding length byte |
| Offset | Length | Meaning |
| 0 | 1 | CTB for secret key certificate |
| 1 | 2 | 16-bit (or maybe 8-bit) length of packet |
| 3 | 1 | Version byte (=2). May affect rest of fields that follow. |
| 4 | 4 | Timestamp |
| 8 | 2 | Validity period, in number of DAYS (0 means forever) |
| 10 | 1 | Algorithm byte for RSA (=1 for RSA). -Algorithm byte affects field definitions that follow. |
| ? | ? | MPI of RSA public modulus n |
| ? | ? | MPI of RSA public encryption exponent e |
| ? | 1 | Algorithm byte for cipher that protects following secret components (0=unencrypted, 1=IDEA cipher) |
| ? | 8 | Cipher Feedback IV for cipher that protects secret components (not present if unencrypted) |
| ? | ? | MPI of RSA secret decryption exponent d |
| ? | ? | MPI of RSA secret factor p |
| ? | ? | MPI of RSA secret factor q |
| ? | ? | MPI of RSA secret multiplicative inverse u |
| ? | 2 | 16-bit checksum of all preceding secret component bytes |
All secret fields in the secret key certificate may be password- encrypted, including the checksum. The checksum is calculated from all of the bytes of the unenciphered secret components. The public fields are not encrypted. The encrypted fields are done in CFB mode, and the checksum is used to tell if the password was good. The CFB IV field is just encrypted random data, assuming the "true" IV was zero.
NOTE: The secret key packet does not contain a User ID field. The User ID is enclosed in a separate packet that always follows the secret key packet on a keyring or in any other context.
| Offset | Length | Meaning |
| 0 | 1 | CTB for public key certificate |
| 1 | 2 | 16-bit (or maybe 8-bit) length of packet |
| 3 | 1 | Version byte (=2). May affect rest of fields that follow. |
| 4 | 4 | Timestamp of key creation |
| 8 | 2 | Validity period, in number of DAYS (0 means forever) |
| 10 | 1 | Algorithm byte for RSA (=1 for RSA). Algorithm byte affects field definitions that follow. |
| ? | ? | MPI of RSA public modulus n |
| ? | ? | MPI of RSA public encryption exponent e |
(All MPI's have bitcount prefixes)
NOTE: The public key packet does not contain a User ID field. The User ID is enclosed in a separate packet that always follows somewhere after the public key packet on a keyring or in any other context.
| Offset | Length | Meaning |
| 0 | 1 | CTB for User ID packet |
| 1 | 1 | 8-bit length of packet |
| 2 | ? | User ID string, size is as in preceding length byte |
The User ID packet follows a public key on a public key ring. It also follows a secret key on a secret key ring.
When a key is certified by a signature, the signature covers both the public key packet and the User ID packet. The signature certificate thereby logically "binds" together the user ID with the key. The user ID packet is always associated with the most recently occurring public key on the key ring, regardless of whether there are other packet types appearing between the public key packet and the associated user ID packet.
There may be more than one User ID packet after a public key packet. They all would be associated with the preceding public key packet.
The three different forms of this packet each come after: a public key packet, a user ID packet, or a signature packet on the public key ring. They exist only on a public key ring, and are never extracted with a key. Don't copy this separate trust byte packet from keyring, and do add it in back in when adding to keyring.
The meaning of the keyring trust packet is context sensitive. The trust byte has three different definitions depending on whether it follows a key packet on the ring, or follows a user ID packet on the ring, or follows a signature on the ring.
| Offset | Length | Meaning |
| 0 | 1 | CTB for Keyring trust packet |
| 1 | 1 | 8-bit length of packet (always 1 for now) |
| 2 | 1 | Trust flag byte, with context-sensitive bit definitions given below. |
For trust bytes that apply to the preceding key packet, the following bit
definitions apply:
For trust bytes that apply to the preceding user ID packet, the following bit definitions apply:
For a trust byte that applies to the preceding signature, the
following bit definitions apply:
Note that the other kinds of trust bytes are mainly derived from the OWNERTRUST bits. They are also derived from the BUCKSTOP bit (which will be set after creating a key, or after setting the owner trust to ultimate), and from the SIGTRUST bits, which is itself derived from a combination of OWNERTRUST bits and possibly the user's ratification.
When testing a key's integrity, we follow a trusted contiguous certification path back up to the owner of the key ring by following keyring trust bytes (for signatures) that have the CONTIG bits and SIGTRUST bits set, until we hit a keyring trust byte (for a key) that has BUCKSTOP bit set. Then we know we've reached the top of the trust pyramid, the keyring owner. Prior to this operation, we set all the CONTIG bits by navigating the pyramid from the top down, by testing the SIGTRUST bits that are "trustwise contiguous" with the top of the pyramid, in a special keyring maintenance pass.
The key legitimacy is ultimately determined by a probablistic fault-tolerant method, as follows. We also set KEYLEGIT if BUCKSTOP is set, which means that this is our own key. The OWNERTRUST bits can only become defined (nonzero) if KEYLEGIT is fully set already. At the moment KEYLEGIT becomes fully set (and not before), we ask the user to define the OWNERTRUST bits.
This probablistic fault-tolerant method of determining public key legitimacy is one of the principle strengths of PGP's key management architecture, as compared with PEM, for decentralized social environments.
The trust of a key owner (OWNERTRUST) does not just reflect our estimation of their personal integrity, it also reflects how competent we think they are at understanding key management and using good judgement in signing keys. The OWNERTRUST bits are not computed from anything -- it requires asking the user for his opinion.
To define the OWNERTRUST bits for a key owner, ask:
Would you always trust "Oliver North" to certify other public keys? (1=Yes, 2=No, 3=Usually, 4=I don't know) ? _
If a key is added to the key ring the trust bytes are initialized
to zero (undefined).
[--manual setting of SIGTRUST/OWNERTRUST not implemented] Normally, we derive the value of the SIGTRUST field by copying it directly from the signer key's OWNERTRUST field. Under special circumstances, if the user explicitly requests it with a special PGP command, we may let the user override the copied value for SIGTRUST by displaying an advisory to him and asking him for ratification, like so:
This key is signed by "Oliver North", whom you usually trust to sign keys. Do you trust "Oliver North" to certify the key for "Daniel Ellsberg"? (1=Yes, 2=No, 3=I don't know) ? _ <default is yes>
Or:
This key is signed by "Oliver North", whom you usually do not trust to sign keys. Do you trust "Oliver North" to certify the key for "Daniel Ellsberg"? (1=Yes, 2=No, 3=I don't know) ? _ <default is no>
An "I don't know" response to this question would have the same effect as a response of "no".
If we had no information about the trustworthyness of the signer (the OWNERTRUST field was uninitialized), we would leave the advisory note off.
Certifying a public key is a serious matter, essentially promising to the world that you vouch for this key's ownership. But sometimes I just want to make a "working assumption" of trust for someone's public key, for my own purposes on my own keyring, without taking the serious step of actually certifying it for the rest of the world. In that case, we can use a special PGP keyring management command to manually set the KEYLEGIT field, without relying on it being computed during a maintenance pass. Later, if a maintenance pass discovers a KEYLEGIT bit set that would not have been otherwise computed as set by the maintenance pass logic, it alerts me and asks me to confirm that I really want it set. [--end of not implemented section]
During routine use of the public keyring, we don't actually check the associated signatures certifying a public key. Rather, we always rely on trust bytes to tell us whether to trust the key in question. We depend on a separate maintenance pass to actually check the key signature certificates against the associated keys, and to set the trust bytes accordingly.
The maintenance pass operates in a top-of-pyramid-down manner as follows.
If at any time during any of these steps the KEYLEGIT field goes from not fully set to fully set, and the OWNERTRUST bits are still undefined, the user is asked a question to define the OWNERTRUST bits. First, for all keys with BUCKSTOP set, check if they are really present in the secret keyring, if not, the BUCKSTOP bit is cleared. SIGTRUST and KEYLEGIT is initialized to zero for non-buckstop keys.
The real maintenance pass is done in a recursive scan: Start with BUCKSTOP keys, find all userid/key pairs signed by a key and update the trust value of these signatures by copying the OWNERTRUST of the signer to the SIGTRUST of the signature. If this makes a key fully validated, start looking for signatures made by this key, and update the trust value for them.
If a signature fails to verify, obnoxiously alert the user, drop it from the key ring, and then do the maintenance pass to calculate all the ring-wide cascaded effects from this, if any. A failed signature should be exceedingly rare, and it may not even result in a KEYLEGIT field being downgraded. Having several signatures certifying each key should prevent damage from spreading too far from a failed certificate. But if dominoes do keep falling from this, it may indicate the discovery of an important elaborate attack.
A public key ring is comprised of a series of public key packets, keyring trust packets, user ID packets, and signature certificates.
Here is an example of an ordered collection of packets on a ring: