### DIFF

#### Synopsis

    HELP:   Compares two data sources
    TYPE:   OBJECT
    SYNTAX: > flcl DIFF(READ.{},COMPARE.{},LOGGING.{},MESSAGE(),FULLY,OFFSET,REPORT='str'/STDOUT/STDERR)

#### Description

The DIFF command uses all reading capabilities of the CONV command to compare the clear net content of two data sources independent of the data format and representation. The comparison takes place on a logical level, based on the neutral FLAM5 elements.

The DIFF function does not recognize insertions or deletions to/from either source. If is one byte or element more or less in the data of one of the sources, then the comparison will fail for the whole rest.

This command does not support wildcards.

As an example, the DIFF command can be used to compare the logical content of two XML files, where one file is in UTF-16LE and GZIP compressed and the other one is a host data set compressed, encrypted and in EBCDIC as FLAMFILE member. The DIFF command builds a neutral element list (UTF-8) and compares the parsed XML elements.

The following features are supported:

- Different kinds of remote access protocols (SSH, IP-Native, MQ-Series)
- Different kinds of data sources (Files, Streams, Tables)
- Different kinds of archive formats (ZIP, FLAMFILE, TAR)
- Different kinds of I/O methods (Binary, Character, Text, Record, FLAM)
- Different data encodings (HEX (Base16), Base32, Base64)
- Different encryption methods (FLAM, PGP)
- Different compression methods (GZIP, BZIP2, XZ)
- Automatic decoding of data encodings during read operations
- Auto-detection and decoding of length fields in data blocks
- Auto-detection of character sets and language based conversion
- Powerful character conversion based on CCSIDs with subset support and reporting
- Different logical data formatting (BIN, TEXT, XML, ...)

Depending on the data formats, a dedicated method is used, to compare the logical data records or streams. If a compare of the data formats makes no sense (XML with records or blocks) then an error is issued. For example: If you use read.file='filename' and compare.file='filename' and the first file is binary and the second file is the compressed version of this binary data, then a list of binary blocks with different length is built. In this case, it makes no sense to compare the elements (they are several blocks of different sizes). Hence, we compare the logical data stream and not the element list. If the first file contains records from a host data set and the second file contains text records with trailing whitespace and line delimiter (=rest element), then only the net record elements are compared. If the first file is block-oriented, then we include the remaining rest element from the text parser in a data stream compare.

With the switch 'FULLY' you can activate additional checks depending on the data formats, for example:

- TEXT/TEXT: Compare the remaining rest (trailing whitespace and delimiter)
- ASA-RECORD/ASA-RECORD: Compare the control character
- REL-RECORD/REL-RECORD: Compare the slot numbers
- XML/XML: Compare comments, formatting (whitespace) and DTD

If you want to verify that two files are identical independent of the platform and representation, then don't use this switch.

If you specified a report file and activate the 'OFFSET' switch, then byte offsets and byte values for each difference are written to the report file.

This can cause a very large report file!

To get syntax information, please use:

       flcl SYNTAX DIFF

To get help for a parameter, please use:

       flcl HELP DIFF.parameter[.parameter[...]]

To read the manual page for a parameter, please use:

       flcl MANPAGE DIFF.parameter[.parameter[...]]
          or
       flcl HELP DIFF.parameter[.parameter[...]] MAN

To generate the user manual for the command, please use:

       flcl GENDOCU DIFF=filename

Parameters can be defined via command line (directly or by parameter file) or via properties taken from the corresponding property file

#### Example

       flcl DIFF read.file='./swift.txt.gz' compare.file='~.swift.fba' report='out.txt'

This test reads text records and remaining rest elements from a text file which was compressed and for example encoded in ASCII-CP1252 and contains 0x0D0A as record delimiter. On the other side, an FBA host data set in EBCDIC is read and the ASA control characters are detached as attributes and trailing whitespeace is removed. Both files are converted to UTF-8 and only the net record elements are compared.

       flcl DIFF read.xml(file='test.xml.bz')
                   compare.xml(file='test.xml')
                   report='out.txt' fully offset

This example requires and compares two XML files including comments, DTD and whitespace element by element. The report file contains the byte offsets for each difference. If you don't use the 'FULLY' switch, then a data stream compare is done, i.e. only tags, attributes and data elements are compared.

       flcl DIFF read.text(file='./test.txt.xz' suptws)
                   compare.record(file='~.test.fba' suppad)
                   report='~.test.out' offset

The example reads a UNIX text from an LZMA compressed file and compares the text records with records from an FBA data set. In order to make this compare work, trailing whitespace must be removed from the UNIX file and padding characters from the fixed-block host file. The ASA control characters are detached by default. Both texts are converted to UTF-8 and only the net record data is compared.

To simplify usage of this compare feature, there is a sample utility program SCFCUDIF. This program takes options and a list of filenames from command line for a logical compare of the resulting net data after decoding, decryption, decompression, character conversion and formatting as neutral FLAM elements. It uses the FLUCDIFF subprogram and writes the report to STDOUT. The first file is compared to all remaining files in the list and the program stops at the first error.

    Parameters:
       :> SCFCUDIF "[-f] [-o] [-m] [-r] [reportfile] filename1 filename2 [filename3 ... filenameN]"
    Options:
       -f Make a full compare of data or elements
       -o Print byte offsets and values to report file
       -m Minimize protocol output on STDERR
       -r The first given filename is used for the report file

You can use this utility program to compare every type of supported file format. It supports comma separated lists, so that it can be used on mainframe system. The example below is taken from the JOBLIB(SCFCUDIF) member of your FLAM installation.

    //SCFCUDIF JOB 'SCFCUDIF',CLASS=A,MSGCLASS=X,NOTIFY=&SYSUID,REGION=0M
    //*
    //* SAMPLE JOB TO CALL THE SCFCUDIF SAMPLE PROGRAM
    //*
    //* THE SCFCUDIF SAMPLE PROGRAM USES THE FLUC SUB PROGRAM DIFF
    //* FOR A LOGICAL DATA COMPARE. YOU CAN PROVIDE SOME OPTIONS AND
    //* YOU MUST SUPPLY TWO OR MORE FILENAMES. TO SEE THE SYNTAX AND ALL
    //* OPTIONS RUN THE PROGRAM WITHOUT PARAMETERS. THE COMPLETE HELP IS
    //* WRITTEN TO SYSOUT.
    //*
    //SCFCUDIF EXEC PGM=SCFCUDIF,REGION=0M,
    //            PARM=('-o,DD:INPUT,~.odat.psfba.gz')
    //STEPLIB  DD DSN=&SYSUID..FLAM.LOAD,DISP=SHR
    //SYSOUT   DD SYSOUT=*
    //SYSPRINT DD SYSOUT=*
    //INPUT    DD DSN=TST.IDAT.PSFBA,DISP=SHR
    //*

The sample program SCFCUDIF is available on all platforms and the source is part of each FLAM installation (sample or SRCLIBC). You can simply adjust this program and use the provided build instructions (makefile or SBUILD job) to make your own utility.

#### Arguments

- `SWITCH: FULLY - Make a fully compare of data and elements [FALSE]`
- `SWITCH: OFFSET - Print byte offsets and values to report file [FALSE]`
- `STRING: REPORT='str'/STDOUT/STDERR - Generate a report file [NONE]`
  - `STDOUT - Write output to stdout`
  - `STDERR - Write output to stderr`