MVS EBCDIC code pages and mappings files

IBM Z Open Editor and RSE API Plug-in for Zowe CLI (RSE CLI plug-in) can handle EBCDIC code page conversion mappings for MVS files. You can set an encoding default in the RSE profile, use an encoding tag with Upload and Download MVS commands, and use mappings files either provided by the RSE API host component or used at a project level. See this page with the list of all the supported code pages.

The RSE API Server provides the capability to manage mappings files that allow defining how MVS files should be converted when uploaded and downloaded. These mappings do not cover z/OS UNIX System Services; we recommend and use in IBM User Build a .gitattributes file for specifying the z/OS UNIX System Services mappings.

RSE CLI plug-in provides the ability to get a default mappings file, which is maintained by the RSE API server administrator, from the server that can be downloaded using the RSE CLI plug-in command zowe rse check conversion-mappings. You can make personal modifications to this file in the ~/.zowe/profiles/rse folder, and then have our tools use this mappings file automatically to interact with the server. The mapping is consistent with a similar behavior offered in IBM Developer for z/OS (IDz) via its user interface.

Capabilities

In addition to mappings files that are stored in the user's .zowe directory and used by the RSE CLI plug-in, ZAPP provides the same ability of defining mappings using the same schema, but offers YAML as additional syntax. This allows teams to share these mappings for their applications and make them part of the source code that could be stored in a Git repository or other SCM. ZAPP files can either be in YAML or JSON syntax; however, it is recommended to have your ZAPP file in one syntax only, not both, within the same project or directory.

IBM RSE API Plug-in for Zowe CLI examples

Many commands of the IBM RSE API Plug-in for Zowe CLI have additional parameters for specifying a code page directly or specifying a mappings file to be used for computing the code page encoding.

For example, assuming the member SAM1 was written and stored on MVS using the IBM-273 EBCDIC encoding, you can work with it via the following commands.

• To download the member SAM1 as a UTF-8 file (converting from IBM-273), issue the following command:

  zowe rse download ds "IBMUSER.SAMPLE.COBOL(SAM1)" --ec IBM-273
  

• To upload the member SAM1 using the IBM-273 EBCDIC encoding, issue the following command:

   zowe rse upload ftds ./ibmuser/sample/cobol/SAM1.txt "IBMUSER.SAMPLE.COBOL(SAM1)" --ec IBM-273
  

To download the default mappings file provided by your RSE API server administrator, issue the following command:

zowe rse check conversion-mappings

The downloaded mappings file will be placed in ~/.zowe/profiles/rse/conversion-mappings.json. You can add your personal conversion preferences to this file and the updates will be automatically used by all your RSE CLI plug-in commands without the need to specify a command line parameter.

However, if you manage more than one such file, for example for different projects, or you even prefer defining such mappings in a ZAPP file, then you can specify the file to use with a command line parameter as well:

zowe rse upload ftds ./ibmuser/sample/cobol/SAM1.txt "IBMUSER.SAMPLE.COBOL(SAM1)" --maps ./zapp.yaml

MVS encoding precedence rules

Tools such as the RSE CLI plug-in or Z Open Editor find mappings following these precedence rules with some rules applied to CLI only:

1. (CLI only) An encoding | ec parameter provided by the command should always have highest precedence, for example --encoding IBM-273.
2. (CLI only) An encoding found in a file that was provided via the parameter mappings-file | maps, for example --mappings-file ~/myproject/zapp.yaml.
3. An encoding found in the current workspace in a file named zapp.yaml or zapp.json.
4. An encoding found in the default mappings file in the user's ~/.zowe/profiles/rse folder that was downloaded via command line.
5. An encoding specified in the Zowe CLI profile itself.
6. No encoding is specified, resulting in the use of the server default mappings file residing on the server.

Mappings file schema

The JSON schema for defining RSE CLI plug-in profiles simply consists of an array of mappings and a default encoding to be used if none of the mappings provided apply. The mappings can be provided as a simple array called mappings of mappings objects in a JSON file or as part of a ZAPP profile with settings as shown in the diagram below. See examples further below.

Fig.1: ZAPP Schema Model for RSE API

When reading these mappings the following validation and interpretation rules apply:

• There can only be one ZAPP rse-mappings profile of type rseapi in a ZAPP file.
• Member mappings consist of the same properties as the mappings except for the member mappings itself, that is, only one level of nesting.
• When member mappings are matched, they will override values of the parent mappings. For example, the parent data set mapping might define a default encoding that is true for members that do not match, but if a member mapping is valid, any encoding value specified there will override the parent mapping.
• The resource property is a mandatory property for data set and member mappings. Note: Mapping against members in all data sets is not possible for performance reasons.
• The resource of the data set parent represents a pattern on the data set name.
• The resource of the member mapping represents a pattern on the member name.
• If no transfer is specified for a resource mapping, the default is text.
• If no encoding is defined for text, it will use the default specified in the local mappings file. If that is missing, it will use the default of the server.
• If a transfer is specified as binary, the encoding will be ignored.
• The extension property is only used for downloading files to determine which extension to append to the member name, which is to ensure editors can open the file correctly. For uploading files, only the mappings to resources are relevant.
• An extension entry can be present either in a member mapping or the data set mapping. If it is in both mappings, the member mapping takes precedence. If it is missing completely in mappings, the first resource match will be used.

Resources can be specified following the rules defined in Step 5 of the z/OS Explorer docs for Mapping data sets and partitioned data set members.

There are two wildcards allowed: ** and *

• ** matches in data set names across multiple name hierarchies. It cannot be used for member names. For example: **ERROR** matches:
  • USER.ERROR.TEST
  • USER.A.B.MYERROR.ABC
  • USER.A.B.MYERROR2.ABC
• * matches within a data set name segment or the member name. For example, USER**COB* matches:
  • USER.A.COB
  • USER.A.B.COBOL
  • USER.A.B.ACOBCOPY

Example JSON mappings file

The following shows a sample mappings file using JSON that you could place in your ~/.zowe/profiles/rse as conversion-mappings.json:

{
  "mappings": [
    {
      "resource": "**.SAMPLE.COB**",
      "extension": "cbl",
      "transfer": "text",
      "encoding": "IBM-273",
      "memberMappings": [
        {
          "extension": "dat",
          "transfer": "binary",
          "resource": "EIC*"
        }
      ]
    },
    {
      "resource": "**PLI*",
      "extension": "pl1",
      "transfer": "text",
      "encoding": "IBM-500"
    }
  ],
  "default.encoding": "IBM-1047"
}

This file defines that files located in data sets containing the string .SAMPLE.COB in their fully qualified name such as USER1.SAMPLE.COBOL would be transferred as text assuming an IBM-273 EBCDIC code page (Austria/Germany) on MVS. Hence, downloading a data set member with an RSE CLI plug-in command or Zowe Explorer using an RSE profile would therefore convert the content by RSE API from IBM-273 EBCDIC to a UTF-8 file, as that is the format Z Open Editor requires files to be in. Uploading such a file would convert the UTF-8 back to IBM-273 EBCDIC. The extension property, which is currently only supported by the CLI, specifies that the downloaded data set member would be stored as a file with a .cbl file extension.

In addition to the default rule for **.SAMPLE.COB** data sets, you see a nested memberMappings subrule that provides a refinement: If the member name contains EIC as a substring then the file will not be converted from IBM-273, but treated as a binary instead. This allows you to use data sets with members of different types.

Furthermore, the example shows how members in data sets that contain PLI in their names would be converted using IBM-500 (Belgium/Canada/Switzerland) as well as that in all other cases the default conversion code page would be IBM-1047.

Example ZAPP file

In a zapp.yaml file, you can specify the same mapping described in the previous section by defining a ZAPP profile of type rseapi. The zapp.yaml for this particular specification would contain text as follows:

name: sam
version: 1.1.0

profiles:
  - name: RSE-Mappings
    type: rseapi
    settings:
      mappings:
        - resource: "**.SAMPLE.COB**"
          extension: cbl
          transfer: text
          encoding: IBM-273
          memberMappings:
            - extension: dat
              transfer: binary
              resource: "*DAT"
        - resource: "**PLI*"
          extension: pl1
          transfer: text
          encoding: IBM-500
      default.encoding: IBM-1047

Using mappings files with Z Open Editor

Mappings files are not only used by RSE CLI plug-in commands, but also by Z Open Editor remote include file resolution operations. When you define a property group either in your ZAPP file or Settings, the retrieval of include files or copybooks will use mappings files based on the precedence rules defined above. For example, if you define mappings in a zapp.yaml file in your workspace, these mappings will be used. If you do not have a ZAPP file, but have a personal mappings file in ~/.zowe/profiles/rse/conversion-mappings.json, the personal file will be automatically used.

Using mappings files with Zowe Explorer

Zowe Explorer allows you to navigate your data sets and open data set members in the editor with a simple click. When you use Zowe Explorer with an RSE profile, conversion mappings can be used as well. To enable this, you need specify the absolute location of the mappings file to use in your user settings. The setting is called zopeneditor.zowe.defaultRseConversionMappingsFile. You can find it in the IBM Z Open Editor settings together with the other Zowe-related settings such as the defaultCliProfile. Conversion mappings defined in either a JSON-formatted RSE API user mappings file or a ZAPP file will override an encoding specified in a Zowe profile.

RSE CLI plug-in logging

When using encoding and mappings files with the RSE CLI plug-in commands, logging will be written and found at ~/.zowe/zowe/logs/zowe.log. This log file will contain details about the encoding precedence that was used during upload and download of MVS files, the location of the mappings file if available, and the outcome of the encoding or transfer type that was chosen according to the previously-mentioned values.

In addition to the logging, you can also refer to the Upload and Download output to troubleshoot. If binary transfer was used, the output includes a binary value of true. If the file was transferred as text and an encoding value was used, the output includes an encoding value for each file.