#### TVD

#### Synopsis

    HELP:   Write a data field with a previous tag and a subsequent delimiter
    TYPE:   OBJECT
    SYNTAX: TVD(ROOT='str',PATH='str',VALUE='str',CCSID='str'/DEFAULT/ASCII/EBCDIC/SYSTEM/LOCAL,SEPCHR[num/COMMA/COLON/SEMICOLON/TABULATOR/BLANK...],INDSIZE=num,INDCHAR/INDCHR=SPACE/TABULATOR,NLNCHAR/NLNCHR=HOST/BIN/TXT/NL/USS/LF/UNIX/CR/OLDMAC/CRLF/WINDOWS/DLM/SYSTEM,TLRCLS,USENID)

#### Description

This object is used to write a column with data that is enclosed by a tag and a trailing delimiter. The format was mainly developed to write SWIFT MT transaction data but can also be used for other complex hierarchical text formats such as XML or JSON.

The format can have an arbitrary hierarchical depth. This is expressed via a path specification with an opening tag and a closing delimiter, with each level separated by a slash. If TVD data from an underlying tag and delimiter are included, they can occur in any order. If this is not the case, the order must correspond to the information in the row specification. In such hierarchical formats, the beginning of the table is determined by the beginning of the repeating data. This beginning is indicated by the longest root path. The repeating data can itself be structured arbitrarily deep, which is indicated by a further path specification. The repeating data can be preceded by so-called headers or followed by trailer data. This header data is distinguished by a shorter root specification which must be placed before the repeating data as a template in the row specifications. The same applies to trailer data, except that it must follow after the repeating data. A short root in between will result in an error. A change in the header or trailer data, leads to an interruption of the table to output the changed header data.

The main differences between TVD and the explicit XML and JSON as part of the table support are:

- The format definitions for XML or JSON in the TVD root and path syntax are much more complex.
- The TVD processor knows nothing about JSON or XML. That is why no plausibility checks are possible.
- The explicit XML and JSON processors do not support mixed formats. The whole data stream must be in JSON or XML format. The TVD format can be mixed with any other format except XML and JSON.
- Some validations (e.g. exist a required branch with only optional fields) are not implemented with TVD. The format validation for XML or JSON is higher.

The column specification can be split into three parts. The middle part consists of the columns with the longest root, defining the repeating data portion. In front of it, it is possible to define header values with a shorter root and trailer values after the repeating data, also with a shorter root. Trailer values are not known at the beginning. Therefore they are optional by default. With the switch TLRCLS, you can enforce to close one additional level after the last trailer value was written. This is required for JSON but useless for SWIFT-MT, for example.

For the definition of root and path strings please refer to the documentation for reading TVD data. The differences for writing the TVD format are explained below:

Only the first tag or delimiter is written if alternative/aliases are defined in the strings. The default value is used to determine if an optional value must be written (identical) or not (value differs from the default). Additionally, the `USENID` switch can be activated to use the null indication. In this case, a tag containing the default value is written, if it was present in the original file (the null indicator was not defined).

When writing, indentation, the character code points used for the indentation and the code points for a line break can be defined. The indentation is done if a new line printed at the end of a tag or delimiter. With `\N` you can enforce new lines at the end of an tag or delimiter. In the case a tag ends with a new line, then the value is intended and also for the corresponding delimiter an indentation is done, if this delimiter starts with a new line.

The CCSID specified in format.tvd() is only valid for the tag and the delimiter. As default the CCSID of the value is used, which is determined by the type converter or the interconnected post processors. If using a definition with fixed length (`%%...%%` or `%n`), this specifies the number of characters which will be validated. In this case the CCSID defined in format.tvd() is used. That means that the fixed data will be interpreted in this character set and checked that they correspond to this length. In this case the charset for the tags and delimiters should be the same as the charset of the value, otherwise odd effects may be the result.

The escape sequences below are available when writing to enable the use of the same root/path for reading and writing the same file structure with the same string. They are usually not required to build a root/path string that is specified for writing only. These escape sequences are replaced by the following fixed values unless they are part of a push back:

- `\z` replaced by "a"
- `\m` replaced by "a"
- `\a` replaced by "a"
- `\l` replaced by "a"
- `\u` replaced by "A"
- `\d` replaced by "0"
- `\x` replaced by "0"
- `\p` replaced by "."
- `\s` replaced by newline ("n")

Normally these character classes are used in read mode to match different options of data items and should be part of a push back amount and not part of the written tag or delimiter.

The special escape sequences below can be used to write formats containing separators (like JSON) and for formatting the output with newline and space characters.

- `\B` is replaced by " " (whitespace)
- `\N` is replaced like "n" (newline)
- `\C` is replaced by a comma and a line break (i.e. ",n"), but only if there is another element that follows.
- `\c` is replaced by a comma and space characters (i.e. ", "), but only if there is another element that follows.
- `\k` is replaced by a comma (i.e. ','), but only if there is another element that follows.

The upper case character classes for termination of tags or delimiters described here are not relevant when writing. The tabulator ("t") is replaced by the code point 0x0009. The code points for newline ("n") can be set via the corresponding selection. The separator character's code point can also be set and defaults to comma. If a list of separators is defined, the first of them will be used.

Below is a JSON example (Row definition with possible data as comment):

    name='JSON2' default(type=string)
    COL(name='menu.id'                     root='\W{\N*}\C/"menu":\B{\N*}\C' path='"id":\B"*"\C')
    COL(name='menu.value'                  root='\W{\N*}\C/"menu":\B{\N*}\C' path='"value":\B"*"\C')
    COL(name='menu.popup.menuitem.value'   root='\W{\N*}\C/"menu":\B{\N*}\C/"popup":\B{\N*}\C/"menuitem":\B[\N*]\C' path='{\W*}\C/"value":\B"*"\c')
    COL(name='menu.popup.menuitem.onclick' root='\W{\N*}\C/"menu":\B{\N*}\C/"popup":\B{\N*}\C/"menuitem":\B[\N*]\C' path='{\W*}\C/"onclick":\B"*"\c')
    #
    {"menu": {
      "id": "file",
      "value": "File",
      "popup": {
        "menuitem": [
          {"value": "New", "onclick": "CreateNewDoc()"},
          {"value": "Open", "onclick": "OpenDoc()"},
          {"value": "Close", "onclick": "CloseDoc()"}
        ]
      }
    }}
    #

Now the same sample as XML (Row definition with data as comment):

    name='XML2' default(type=string)
    COL(name='menu.id'                     root='\W\<menu<*\<\/menu\>\N'                           path='\<menu*\>\N/\Bid="*"\W')
    COL(name='menu.value'                  root='\W\<menu<*\<\/menu\>\N'                           path='\<menu*\>\N/\Bvalue="*"\W')
    COL(name='menu.popup.menuitem.value'   root='\W\<menu<*\<\/menu\>\N/\<popup\>\N*\<\/popup\>\N' path='\<menuitem*\/\>\N/\Bvalue="*"\W')
    COL(name='menu.popup.menuitem.onclick' root='\W\<menu<*\<\/menu\>\N/\<popup\>\N*\<\/popup\>\N' path='\<menuitem*\/\>\N/\Bonclick="*"\W')
    #
    <menu id="file" value="File">
      <popup>
        <menuitem value="New" onclick="CreateNewDoc()" />
        <menuitem value="Open" onclick="OpenDoc()" />
        <menuitem value="Close" onclick="CloseDoc()" />
      </popup>
    </menu>
    #

The attribute list in the XML tags are parsed as additional level, where the start tag (`<menu`) and start end tag (`>`) are used as tag and delimiter. The start tag (`<menu`) has also a corresponding end tag (`</menu>`) in the level below. For this to work, this start tag must be provided with a push back (`\<menu\W<*` (in full length (no digit behind `<`))), so that it can be read again in the next level for the attribute list.

And last but not least the CSV portion for the same data (the data below will be the default data format, if the tables above written as text):

    name='CSV2' default(type=string)
    COL(name='menu.id'                      root='' path='"*"\k')
    COL(name='menu.value'                   root='' path='"*"\k')
    COL(name='menu.popup.menuitem.value'    root='' path='"*"\k')
    COL(name='menu.popup.menuitem.onclick'  root='' path='"*"\n')
    #
    "file","File","New","CreateNewDoc()"
    "file","File","Open","OpenDoc()"
    "file","File","Close","CloseDoc()"
    #

For this it would be better to use the CSV format for this (control about the headline), but it is also possible to parse CSV with TVD support.

By default, if a column value changes between two table rows, and this column is specified with a "short root", only the changed value is written out, even if multiple column specifications with the same root exist. In order to write out all columns with the same root if the data in one of the columns changes between rows, put an additional `!` after the mandatory/optional modifier at the beginning of the root string.

Below is an example for SWIFT MT103 message which demonstrates the additional `!` modifier. The complete definition is provided together with the example file MT101 in your FLAM installation.

    TABLE(ROW(name=MT103 DEFAULTS(CCSID=1141 TYPE=STRING FRASEP=comma ALWTRC)
        COL(name='HSPF'           root='!HSPF*TSPF\n'  path='*\n{<1'                      ) # value direct behind till begin of next tag #
        COL(name='BHB'            root='!!HSPF*TSPF\n' path='!{1:*}'                      ) # Basic header block is required #
        COL(name='AHB'            root='!!HSPF*TSPF\n' path='?{2:*}'                      ) # application header block is optional #
        COL(name='UHB-BPC'        root='!!HSPF*TSPF\n' path='?{3:*}/?{113:%%%%}           ) # user header block: optional bank priority code (fix 4 character)
        COL(name='UHB-MUR'        root='!!HSPF*TSPF\n' path='?{3:*}/?{108:*}' maxlen=16   ) # user header block: optional Message User Reference value (up to 16 character)
        COL(name='CB-SENDREV'     root='!HSPF*TSPF\n/{4:\n*-}\n' path=':20:*\n' maxlen=16 )
        COL(name='CB-BOPCODE'     root='!HSPF*TSPF\n/{4:\n*-}\n' path=':23B:*\n'          )
        COL(name='CB-VDCUISA-VD'  root='!HSPF*TSPF\n/{4:\n*-}\n' path=':32A:%%%%%%'       ) # split (fix 6 letter for the date) #
        COL(name='CB-VDCUISA-CU'  root='!HSPF*TSPF\n/{4:\n*-}\n' path='%%%'               ) # into (fix 3 letters for the currency (no TAG and no DELIM)) #
        COL(name='CB-VDCUISA-ISA' root='!HSPF*TSPF\n/{4:\n*-}\n' path='*\n'               ) # 3 fields (variable length amount) #
        COL(name='CB-CUINSAM'     root='!HSPF*TSPF\n/{4:\n*-}\n' path='?:33B:*\n'         )
        COL(name='CB-PORPUSE'     root='!HSPF*TSPF\n/{4:\n*-}\n' path='?:50K:*\n:<1'      ) #more than 1 line#
        ...
        COL(name='CB-DETAILS'     root='!HSPF*TSPF\n/{4:\n*-}\n' path='!:71A:*\n'         )
        COL(name='CB-DETAILS'     root='!HSPF*TSPF\n/{4:\n*-}\n' path='?:71F:*\n'         )
        COL(name='CB-DETAILS'     root='!HSPF*TSPF\n/{4:\n*-}\n' path='?:71G:*\n'         )
        COL(name='CB-DETAILS'     root='!HSPF*TSPF\n/{4:\n*-}\n' path='?:72:*\n-}<2|\n:<1')
        COL(name='CB-DETAILS'     root='!HSPF*TSPF\n/{4:\n*-}\n' path='?:77B:*\n-}<2'     )
        COL(name='OTB'            root='!HSPF*TSPF\n'            path='?{5:*}\n'          ) # optional trailer block #
    )

The HSPF header and TSPF trailer in the above example are printed if a change in the basic, application or user header block occurs. For more information about the root and path strings, please read the explanations about parsing the TVD format.

To write records, the delimiter of the TVD format must be used. The BLK2REC switch should be activated for this in the CONV command. If the XCNV command is used, then a block to record conversion (`CNV.REC()`) must be added, if each line should be a record in the data set or FLAMFILE.

#### Arguments

- `STRING: ROOT='str' - Root to XML element [taken from table/row/column]`
- `STRING: PATH='str' - Path to XML element [taken from column]`
- `STRING: VALUE='str' - Optional default value, to determine if a element optional ['0' for float and integer else '']`
- `STRING: CCSID='str'/DEFAULT/ASCII/EBCDIC/SYSTEM/LOCAL - Conversion from UTF-8 to this CCSID [system]`
  - `DEFAULT - Use default CCSID (environment)`
  - `ASCII - Use default ASCII CCSID (environment)`
  - `EBCDIC - Use default EBCDIC CCSID (environment)`
  - `SYSTEM - Use system character set (same as DEFAULT)`
  - `LOCAL - Use local character set (system/physical)`
- `NUMBER: SEPCHR[num/COMMA/COLON/SEMICOLON/TABULATOR/BLANK...] - Separator character as Unicode codepoint [COMMA]`
  - `COMMA - Comma separated (default)`
  - `COLON - Separated by colon (':')`
  - `SEMICOLON - Separated by semicolon (';')`
  - `TABULATOR - Separated by tabulator (x'0009')`
  - `BLANK - Separated by blank (x'0020')`
- `NUMBER: INDSIZE=num - Define indentation size for writing [2 if space, 1 if tab else 0]`
- `NUMBER: INDCHAR/INDCHR=SPACE/TABULATOR - Indentation character used to indent TVD tags [NONE]`
  - `SPACE - Use the space character as indentation character`
  - `TABULATOR - Use the tabulator character as indentation character`
- `NUMBER: NLNCHAR/NLNCHR=HOST/BIN/TXT/NL/USS/LF/UNIX/CR/OLDMAC/CRLF/WINDOWS/DLM/SYSTEM - Codepoints used for new lines (n) in tags and delimiters [SYSTEM]`
  - `HOST - Adds no delimiter (HOST)`
  - `BIN - Adds no delimiter (HOST)`
  - `TXT - Adds the system specific delimiter (DEFAULT)`
  - `NL - Adds delimiter new line (USS)`
  - `USS - Adds delimiter for USS (new line)`
  - `LF - Adds delimiter line feed (UNIX)`
  - `UNIX - Adds delimiter for UNIX (line feed)`
  - `CR - Adds delimiter carriage return (MAC)`
  - `OLDMAC - Adds delimiter for old MACs (carriage return)`
  - `CRLF - Adds delimiter carriage return line feed (WIN)`
  - `WINDOWS - Adds delimiter for WINDOWS (carriage return line feed)`
  - `DLM - Adds the system specific delimiter (DEFAULT)`
  - `SYSTEM - Adds the system specific delimiter (DEFAULT)`
- `SWITCH: TLRCLS - Close one additional level if trailer was written [FALSE]`
- `SWITCH: USENID - Use null indication for write [FALSE]`