#### EDC

#### Synopsis

    HELP:   OpenSSL conform decryption of files
    TYPE:   OBJECT
    SYNTAX: EDC(BUFSIZ=num,METHOD=CLEAR/PASSWORD/LABEL,ALGO=AES/TDES/CAST5/CAST128/IDEA/BLOWFISH/BF/CAMELLIA,KEYLEN=num/KL064/KL128/KL192/KL256/KL08/KL16/KL24/KL32,MODE=ECB/CBC/OFB/CFB/CTR/GCM,PADDING=NONE/NOPAD/PKCS,KDF/PW2KEY=OLDSSL/PBKDF2,MD/HASH=MD5/SHA1/SHA224/SHA256/SHA384/SHA512/SHA3-224/SHA3-256/SHA3-384/SHA3-512,ITER=num,IV='bin-hex'/ZERO,SALT='bin-hex'/NONE/RANDOM,KEY='bin-hex',PASSWORD='str',LABEL='str',FKM5(),TAG='bin-hex',OFFSET=num,PARM='str'/STDOUT/STDERR,SKPFMT[=NOHEADER/SUPPORTED/REDUNDANT/SUPPREDU])

#### Description

The encryption/decryption converter can be used to encrypt or decrypt data streams. These streams are compatible with the `openssl enc` command, allowing to encrypt data with FLAM and decrypt with OpenSSL or vice versa. Crypto operations are performed in software, with hardware-acceleration (if available) or via FKM5 using a key label.

Encryption always results in binary blocks of data. The decryption result can be text or binary data. Record boundaries are lost. Please use delimiters, length fields or fixed length data if records must be exchanged.

The OpenSSL-compatible encryption output is raw encrypted data. An automatic detection of this format is not possible, except for the salt of the salted passphrase procedure, in which case a corresponding header is written.

The encrypted data can be ASCII-armored for transport/storage via/on text-based media. Compression should be done before encryption because the encrypted output looks like random data which is incompressible.

       cnv.gzp(...) cnv.edc(...) cnv.bas(... armor(...))

The component supports different methods, algorithms, key lengths, modes and paddings. Invalid or unsupported combinations will result in an error. Additional the password to key derivation function can be specified including the hashing procedure and amount of iterations.

Please have a look at the security considerations section before using this component.

#### Parameter side file

During encryption, a parameter side file can be generated. It contains information from the encryption process that is relevant for decryption (e.g. the GCM tag value if encrypting with AES-GCM). If decryption is done with `cnv.edc()` as well, the existing side file can be specified, avoiding the need to specify the relevant parameters manually. The example below uses an SSH server for the side file.

    WRITE:
       cnv.edc(IV=RANDOM KEY=RANDOM parm='ssh://user@server/sidefile.txt/&utf-8')
    READ:
       cnv.edc='ssh://user@server/sidefile.txt/&utf-8'
    OR:
       cnv.edc(parm='ssh://user@server/sidefile.txt/&utf-8')

The parameter side file is a text file which contains the parameter values used during encryption. The same parameter values must be passed to CNVEDC for decryption.

To facilitate this, you can generate this side file during encryption by assigning a file path to the `parm` parameter. This same file can then be used during decryption via the same parameter, allowing you to omit most parameters (except the key) during decryption. Hence, the parameter side file is mainly useful if the data will be decrypted with FLAM. To generate random IVs or random clear keys, the keyword `RANDOM` can be used for the respective parameters. The generated values are then written to the side file and required for decryption.

Please be aware that if `key=RANDOM` was specified, a new encryption key is generated per file and the generated encryption key is written to the side file (which is required in this case) because it would be impossible to decrypt the data without the randomly generated key. Anyone who can read this side file can decrypt your data! We do not recommend to store encrypted data and side file together or transmit both over the same channel. If you do not fully understand the security implications, we recommend not to use `key=RANDOM`.

If you specify your own clear key via `key` parameter, it is not written to the parameter side file. However, you can instruct to explicitly write the key to the side file with the `PRNKEY` flag. The warning above also applies in this case.

The parameter side file may contain:

- Method (as comment)
- Cipher algorithm
- Cipher mode
- Key length
- For password-based encryption:
  - Key derivation function (KDF)
  - Hash function used for key derivation
  - Iteration count for key derivation
- Offset (i.e. the position of the first encrypted byte)
- Clear key value (only if random or `PRNKEY` flag is set, see warning above)
- Initialization vector (only if random or `PRNKEY` flag is set)
- Salt (if password-based encryption is used with a random salt and no header is written or `PRNKEY` flag is set)
- Authentication Tag (only if no tag file defined, required by AEAD cipher modes like GCM to verify authenticity of encrypted data during decryption)

The side file is required if at least one of the following is true:

- GCM is used as cipher mode and no separate `TAG` file is provided
- `key=RANDOM` or `IV=RANDOM` was specified
- Password-based encryption with a random salt is used and no header is written

This corresponds to the following usage patterns:

       cnv.edc(... mode=GCM ...)
       cnv.edc(... iv=RANDOM key=RANDOM ...)
       cnv.edc(... format=NOHDR pass=a'hugo' ...)

If you use a clear key, passphrase or KFM5 key label for encryption and also specify a parameter side file, it is sufficient to specify the side file and your key/passphrase/label during decryption because all other parameters are contained in the side file:

       cnv.edc(parm='//ssh:user@server/sidefile.txt/&utf-8' pass=a'hugo')
       cnv.edc(parm='//ssh:user@server/sidefile.txt/&utf-8' key=00112233...)

#### Initialization vector & Salt

All cipher modes except ECB require an initialization vector. An initialization vector (IV) makes sure that the ciphertext is completely different for every encryption operation even if the same data is encrypted with the same key multiple times. Each IV value should only be used once with a given key for encryption. The same IV must be used for decryption, otherwise the output will be random data. Using the same IV more than once may allow an attacker to gather information about common prefixes in your plaintext (i.e. files that start identically). To avoid that, use a different IV for each file.

The default IV is all zeroes. You can and should either specify a (hexadecimal) IV that you generated on your own or you can generate a random IV with `iv=RANDOM` (recommended) which is written to the parameter side file. Alternatively, a clear key or key label can be combined with a password as additional factor. The IV is then derived from the password and a salt (with properties described below).

For password-based encryption, the key and IV are derived from the password and a salt by default. Multiple key derivation methods are available (e.g. OpenSSL, PBKDF2). Unless specified otherwise, the salt is randomly generated. The 8 bytes magic string `Salted__` followed by the generated 8 bytes salt value is prepended to the encryption output, making it compatible to OpenSSL's `enc` command. The prepended salt is automatically detected during decryption. Writing of the salt header can be disabled with `FORMAT=NOHDR`. The salt is then written to the parameter side file which, in this case, is required because otherwise decryption is impossible. You can specify a custom 8-byte salt value in hexadecimal form via the parameter `salt` or set it to `none` to not use a salt, which is not recommended because the same key and IV will be derived for each encryption operation. An IV can also explicitly be specified in which case it overrides the derived IV.

#### Defaults

When writing with `cnv.edc()` using key-based encryption, the defaults below apply:

- METHOD=CLEAR
- ALGO=AES
- KEYLEN=KL128
- MODE=CBC
- IV=ZERO
- KEY=RANDOM

For password-based encryption (i.e. a password is supplied), these defaults apply:

- METHOD=OLDSSL
- ALGO=AES
- KEYLEN=KL128
- MODE=CBC
- SALT=RANDOM
- IV=DERIVED FROM PASSWORD+SALT
- KEY=DERIVED FROM PASSWORD+SALT

These defaults may change in the future. Depending on your use case, you might want to explicitly specify all relevant parameters to avoid behavioral changes when upgrading to future FLAM versions.

#### AEAD modes like GCM (Authentication tag)

AEAD modes combine encryption and authentication in a single algorithm, making a separate HMAC unnecessary to ensure authenticity. In addition to protecting confidential data, AEAD ensures the authenticity and integrity of other data that is not encrypted, called associated data. The purpose is, for example, the transmission of a message header, which must be available as plaintext for processing. With AEAD, the associated data and the encrypted data become inseparable.

Currently, only GCM is supported as AEAD scheme. It was developed with the protection of network communication protocols in mind that often require header data that must be processed by involved entities without decrypting the content. Each transmitted message is typically relatively small and protocols usually re-negotiate encryption keys after a certain amount of data was transmitted. As a result, GCM has limitations that make it less suitable for file encryption. The maximum amount of data that can be encrypted with the same Key+IV combo is slightly below 64 GiB (2^36-32 = 68,719,476,704 bytes). The same key should not be used for more than 2^32 encryption operations because the IV is only 96 bits and the collision probability of randomly generated IVs rises significantly beyond that limit.

The authentication tag, which is mandatory for decrypting the data with GCM, can optionally be written to and read from a dedicated file by specifying a path for the `tag` parameter. If the `tag` parameter is not specified, the tag is written to the parameter side file. In any case, one of the two is needed during decryption to ensure authenticity.

Example for the `tag` parameter:

    WRITE:
       cnv.edc(... tag='ssh://user@server/tagfile.txt/&utf-8' ...)
    READ:
       cnv.edc(... tag=f'ssh://user@server/tagfile.txt/&utf-8' ...)

#### Partial encryption of files

With the parameter `offset`, you can start the encryption after offset amount of bytes. The data between the beginning and the offset will remain plaintext. This can be useful to keep some meta data (e.g. headers) at the beginning of the file in unencrypted form. With GCM, this unencrypted part becomes associated data and is thus part of the authentication tag, protecting it against tampering.

#### Use of wildcards (behavior in directory walk)

If the component is used on multiple files at once, the parameter side file will contain a concatenation of all metadata for all files processed. The original filenames are written as comment to the side file to separate the sections. However, such a parameter file cannot be used for decryption. Therefore, we don't recommend to use a side file when encrypting multiple files with a single invocation.

#### Considerations and security best practices

Some recommendations to safely and securely use this component:

- Default values may change in the future. We recommend to set at least the parameters mentioned in the defaults section if you don't want behavioral changes after FLAM updates.
- Do not use the same IV value more than once for encryption
- If you specify a custom IV and write multiple files (e.g. by using file name patterns), the same IV will be used to encrypt all files which leaks information about common prefixes if the files start with the same sequence of bytes.
- Avoid predictable IV values (unsafe against chosen plaintext attacks like BEAST)
- Use `iv=RANDOM` if possible
- Change encryption keys periodically
- For GCM: Do not encrypt files larger than 64 GiB (minus 32 bytes)
- For GCM: Do not encrypt more than 2^32 files with the same key
- Avoid password-based encryption without salt (results in constant key and IV for every encrypted file)
- If you write a parameter side file that contains a key (`key=RANDOM` or flag `PRNKEY`) make sure to not store it at the same location or transmit it over the same channel as the encrypted file.

#### OpenSSL compatibility

The component was implemented with compatibility to the `enc` command of the OpenSSL command line utility in mind. Data that was encrypted with this component can be decrypted with OpenSSL and vice versa as long as their parameterization is equivalent.

There are a few exceptions to OpenSSL compatibility:

- GCM is not a supported mode of the OpenSSL command line tool because of the limitations described above and resulting security implications.
- If the `offset` parameter is used, the file cannot be decrypted with the OpenSSL command line tool
- Too short keys or IVs will lead to an error, unlike the OpenSSL command line tool

#### Examples

##### Interoperability

Here is an example for interoperability using password-based encryption with PBKDF2 as key derivation method and a custom IV. The first two lines represent the parameters for encryption and decrytion with CNVEDC. The other two lines are the equivalent OpenSSL command line parameters.

    CNVEDC encrypt:  "ALGO=AES mode=CFB keylen=KL256 method=PBKDF2 pass=a'1234'     iv=0123456789ABCDEF0123456789ABCFFF parm=STDOUT"
    CNVEDC decrypt:  "ALGO=AES mode=CFB keylen=KL256 method=PBKDF2 pass=x'31323334' iv=0123456789ABCDEF0123456789ABCFFF parm=STDOUT "
    OpenSSL encrypt: "-e -AES-256-CFB -pass pass:1234 -pbkdf2 -iv 0123456789ABCDEF0123456789ABCFFF -p",
    OpenSSL decrypt: "-d -AES-256-CFB -pass pass:1234 -pbkdf2 -iv 0123456789ABCDEF0123456789ABCFFF -p",

##### File encryption

A simple example for file encryption with AES-128-CBC using a custom clear key and IV. Note that you should use a randomly generated IV per file.

     flcl xcnv "input(sav.fil(fio.blk(name=file.bin)))
                output(sav.fil(fio.blk(name=file.edc)
                  cnv.edc(algo=aes keylen=kl128 mode=cbc
                    iv='01234567801234567801234567801234'
                    key='0123456789ABCDEFFEDCBA9876543210'
                  )
                ))

#### Automatism

In order to be able to use the component in the detection when reading, its removal can be activated under certain circumstances via the parameter `SKPFMT`. On the one hand, with the selection `NOHEADER` (activated in the CONV command with DECRYPT.PWD()), which leads to removal if no salted header is found. The next stage is called `SUPPORTED`, where no decryption takes place if the data at hand corresponds to a format that is processed by other components. With `REDUNDANT` it can be prevented that the decryption takes place on data that show redundancies, which cannot be the case with a cryptogram (white noise). An exception to this is ECB mode, which can lead to recurring cryptograms if the data are identical. Therefore, `REDUNTANT` is not supported in ECB mode. With `SUPPREDU`, the latter two mechanisms can be requested together, which comes into play in the read string of the CONV command in DECRYPT.EDC().

#### Arguments

- `NUMBER: BUFSIZ=num - Initial buffer size for preallocation [65536]`
- `NUMBER: METHOD=CLEAR/PASSWORD/LABEL - Method for key determination [AUTO]`
  - `CLEAR - Clear key crypto (en/decrypt with raw unprotected key value)`
  - `PASSWORD - Password-based crypto (using key derivation function (KDF))`
  - `LABEL - Protected key crypto using label (FKM5 required)`
- `NUMBER: ALGO=AES/TDES/CAST5/CAST128/IDEA/BLOWFISH/BF/CAMELLIA - Algorithm used for decryption [AES]`
  - `AES - Advanced encryption standard (Rijndael)`
  - `TDES - Triple DES (Data encryption standard)`
  - `CAST5 - CAST5/CAST-128 (Carlisle Adams, Stafford Tavares)`
  - `CAST128 - CAST-128/CAST5 (Carlisle Adams, Stafford Tavares)`
  - `IDEA - International Data Encryption Algorithm`
  - `BLOWFISH - Blowfish from Bruce Schneier`
  - `BF - Blowfish from Bruce Schneier`
  - `CAMELLIA - Camellia from Japan (Mitsubishi Electric and NTT)`
- `NUMBER: KEYLEN=num/KL064/KL128/KL192/KL256/KL08/KL16/KL24/KL32 - Key length if passphrase used`
  - `KL064 - Key length 64 bits (8 bytes)`
  - `KL128 - Key length 128 bits (16 bytes)`
  - `KL192 - Key length 192 bits (24 bytes)`
  - `KL256 - Key length 256 bits (32 bytes)`
  - `KL08 - Key length 8 bytes (64 bits)`
  - `KL16 - Key length 16 bytes (128 bits)`
  - `KL24 - Key length 24 bytes (192 bits)`
  - `KL32 - Key length 32 bytes (256 bits)`
- `NUMBER: MODE=ECB/CBC/OFB/CFB/CTR/GCM - Mode of operation [if TAG then GCM else CBC]`
  - `ECB - Electronic codebook mode`
  - `CBC - Cipher block chaining mode`
  - `OFB - Output feedback mode`
  - `CFB - Cipher feedback mode`
  - `CTR - Counter mode`
  - `GCM - Galois/Counter Mode`
- `NUMBER: PADDING=NONE/NOPAD/PKCS - Padding for CBC or ECB mode [PKCS]`
  - `NONE - No padding (remaining data must have a multiple of the block length)`
  - `NOPAD - No padding (remaining data must have a multiple of the block length)`
  - `PKCS - OpenSSL conform PKCS padding`
- `NUMBER: KDF/PW2KEY=OLDSSL/PBKDF2 - Password to key derivation function [OLDSSL]`
  - `OLDSSL - Legacy OpenSSL key derivation (avoid use, default: SHA-256 with 1 round, hint: OpenSSL < 1.1.0 used MD5)`
  - `PBKDF2 - PBKDF2 key derivation function (default: SHA-256 with 10000 rounds, available from OpenSSL 1.1.1)`
- `NUMBER: MD/HASH=MD5/SHA1/SHA224/SHA256/SHA384/SHA512/SHA3-224/SHA3-256/SHA3-384/SHA3-512 - Hash algorithm for key derivation [SHA256]`
  - `MD5 - Message Digest 5 with 128 bits`
  - `SHA1 - Secure Hash Algorithm 1 with 160 bits`
  - `SHA224 - Secure Hash Algorithm 2 with 224 bits`
  - `SHA256 - Secure Hash Algorithm 2 with 256 bits`
  - `SHA384 - Secure Hash Algorithm 2 with 384 bits`
  - `SHA512 - Secure Hash Algorithm 2 with 512 bits`
  - `SHA3-224 - Secure Hash algorithm 3 with 224 bits`
  - `SHA3-256 - Secure Hash Algorithm 3 with 256 bits`
  - `SHA3-384 - Secure Hash Algorithm 3 with 384 bits`
  - `SHA3-512 - Secure Hash Algorithm 3 with 512 bits`
- `NUMBER: ITER=num - Iteration count for PBKDF2, implies METHOD=PBKDF2 if specified [10000]`
- `STRING: IV='bin-hex'/ZERO - Initialization vector for all modes except ECB [ZERO]`
  - `ZERO - Initialization vector of binary zeros`
- `STRING: SALT='bin-hex'/NONE/RANDOM - Salt for salted passphrase-based decryption [AUTO]`
  - `NONE - No salt (not recommended)`
  - `RANDOM - Random generated salt`
- `STRING: KEY='bin-hex' - Clear key value (direct entry of raw value (dangerous))`
- `STRING: PASSWORD='str' - Passphrase with or without salting`
- `STRING: LABEL='str' - Label to reference a protected key (FKM5 required)`
- `STRING: TAG='bin-hex' - The tag value for verification (only for GCM mode)`
- `NUMBER: OFFSET=num - Offset before decryption begins [0]`
- `STRING: PARM='str'/STDOUT/STDERR - Side file with parameter used for decryption (merged if not empty else written)`
  - `STDOUT - Write output to stdout`
  - `STDERR - Write output to stderr`
- `NUMBER: SKPFMT=NOHEADER/SUPPORTED/REDUNDANT/SUPPREDU - Skip decryption if format not valid [NO]`
  - `NOHEADER - Skip if no salted header found`
  - `SUPPORTED - Skip if format supported (PGP, GZIP, BZIP, XL, ZSTD, XML, ...)`
  - `REDUNDANT - Skip if data redundant (not a white noise)`
  - `SUPPREDU - Skip if format known or data redundant`