Preprocessor support

Z Open Editor provides the advanced capability to integrate parsing of COBOL and PL/I programs with a custom preprocessor to allow editing programs that contain such preprocessor statements without causing the editor to show syntax errors. The preprocessor would be executed on either a developer machine by Z Open Editor in the background (local) or a z/OS system (remote), taking COBOL or PL/I with custom code and transforming it to a valid program. Users can then continue editing such programs by using the preprocessor input to still receive full language support such as code completion, syntax error highlighting, and outline view for the regular COBOL or PL/I statements. Then at each save or editor focus change the preprocessor can run automatically again to update the processed representation used by the editor. The editor would not learn the preprocessor language, but rather ensure that the resulting preprocessed output is still valid COBOL or PL/I.

Local preprocessor

Example workflow

The following workflow is written from the perspective of developer, Deb.

Deb installs a command-line centric preprocessor program on her local development machine.
Deb creates a zapp.yaml file with a preprocessor profile type. The profile specifies the command-line for running the preprocessor on the currently edited program file as well as the output path for the preprocessed program files.
Deb edits a program file with preprocessor statements in Z Open Editor.
Deb either manually starts preprocessing the program via a right-click menu command or creates a user setting to automatically run the preprocessor on Save.
Z Open Editor starts the preprocessor command in the background and sends the output file to its COBOL (PL/I) parser.
Z Open Editor's COBOL (PL/I) parser will compare the program in the editor with the output of the preprocessor command.
Deb hovers over processor statements in Z Open Editor, which displays the resulting COBOL (PL/I).
Deb continues editing the program in Z Open Editor. Z Open Editor continues to provide full language support for all the parts of the program that are not preprocessor statements.

To learn and explore this workflow, check out the example preprocessor and code samples in our Github repository at https://github.com/ibm/zopeneditor-sample/tree/wazi-main/preprocessoropen in new window. Review the Readmeopen in new window file in this folder with detailed instructions for building and running the sample.

Requirements for the preprocessor

To be able to integrate your preprocessor with Z Open Editor, it must fulfill these requirements:

It needs to run locally on your development machine where Z Open Editor is running.
You need to be able to run the processor via command-line.
The command-line operation needs to create an output file that contains valid COBOL.
The output path can be computed by specifying a new file extension to be used with the program file name.

ZAPP profile

The integration of the preprocessor with the editor is handled via a ZAPP file profile. Such a profile might look like this:

- name: extended-cobol
  type: preprocessor
  language: cobol
  location: local
  settings:
    command: ${JAVA_HOME}/bin/java -jar ${WORKSPACE}/preprocessor/my-preprocessor/target/my-preprocessor-1.0-SNAPSHOT.jar ${input_file} ${output_file}
    outputPath: ./preprocessor/output
    fileExtension: cee
    environmentVariables:
      VARIABLE1: value1

The language property is mandatory and allows the values cobol or pl1 only.

In the settings object, the following properties can be used:

Property	Description
command	The command line to be executed. You can mix the command line with ZAPP Variables as well as use the predefined variables `${input_file}` and `${output_file}`. You need to define `${JAVA_HOME}` and `${WORKSPACE}` in the example above as ZAPP variables. Environment variables can only be used by specifying them with the `environmentVariables` property.
outputPath	The base path in which the output file should be created. This path must be inside the current VS Code workspace so that the editor can see and process the files.
fileExtension	The file extension to be used for the output files.
environmentVariables	An optional sub-object to specify the environment variables that should be created before executing the command.

Running the preprocessor

After you specify the ZAPP profile, you can run the preprocessor in the following ways:

Right-click and choose Execute local preprocessor command to manually run the preprocessor in the program file editor.
Automatically run the preprocessor via the VS Code user setting zopeneditor.autoPreprocessor. The setting can have one of the values:
- off: disables the automatic preprocessor execution.
- open: when a program file is opened in the editor.
- save: when a program file is saved in the editor.
- both: when opening and saving a program file in the editor.

When the preprocessor command runs, a progress dialog with a Cancel button is displayed. When the preprocessor finishes, it refreshes the editor and any syntax errors that were caused by preprocessor statements should go away or, if they contained problems, will show those instead.

If the processor command execution failed and produced errors, a dialog will be shown with a button to open the Z Open Editor log file viewer which lists the detailed error message produced by the preprocessor command.

Using the preprocessor output in the editor

After the preprocessor is executed, you can hover the mouse of preprocessor statements in their program file to see the code that was generated for the statement. You can edit these statements and all other code in the program.

Syntax errors for preprocessor statements will not be available until the preprocessor runs again. Syntax errors for other regular COBOL or PL/I code will be shown while typing as usual in Z Open Editor.

To get a side-by-side view of the program before and after preprocessing, you can use the right-click menu option Compare preprocessor input and output files.

Remote preprocessor

Example workflow

The following workflow is written from the perspective of developer, Deb.

Deb sets up a z/OS centric preprocessor data member on her remote z/OS development machine. This data member must be either a REXX or CLIST program.
Deb creates a zapp.yaml file with a preprocessor profile type. The profile specifies the TSO command for running the preprocessor on the currently edited program data member as well as the output dataset for the preprocessed data members.
Deb edits a dataset member with preprocessor statements in Z Open Editor through Zowe.
Deb either manually starts preprocessing the program via a right-click menu command or creates a user setting to automatically run the preprocessor on Save.
Z Open Editor uses Zowe to start the preprocessor command in the background and sends the output file to its parser.
Z Open Editor's COBOL or PL/I parser will compare the program in the editor with the output of the preprocessor command.
Deb hovers over processor statements in Z Open Editor, which displays the resulting COBOL or PL/I.
Deb continues editing the program in Z Open Editor. Z Open Editor continues to provide full language support for all the parts of the program that are not preprocessor statements.

Requirements for the preprocessor

To be able to integrate your preprocessor with Z Open Editor, it must fulfill these requirements:

It needs to run remotely on your z/OS machine.
The remote preprocessor must be either a REXX or CLIST program.
You need a RSE API or z/OSMF Zowe profile available in your workspace and configured as a default for Z Open Editor.
You need to be able to run the processor via a Zowe TSO command.
- You can read more about TSO commands and Zowe TSO profiles in the documentationopen in new window, and explore some examples in this blogopen in new window.
The TSO operation needs to create an output data member that contains valid COBOL or PL/I.
The program file to be processed needs to be in a PDS that is recognized by Z Open Editor to contain COBOL (PL/I) and the COBOL (PL/I) language support must be active. See Remote file associations for more details.

ZAPP profile

The integration of the preprocessor with the editor is handled via a ZAPP file profile. Such a profile could look like this:

- name: extended-cobol
  type: preprocessor
  language: cobol
  location: mvs
  settings:
    commandDataSet: ${HLQ}.PREPROC.REXX.(PREPROC)
    tempDataHLQ: ${HLQ}
    outputDataSet: ${HLQ}.PREPROC.OUTPUT()
    commandParameters: "var1=val1"

The language property is mandatory and allows the values cobol or pl1 only.

In the settings object the following properties can be used:

Property	Description
commandDataSet	The data member to be executed. You can mix with ZAPP Variables. You need to define `${HLQ}` in the example above as ZAPP variables. Environment variables can only be used by specifying them with the below `commandParameters` property.
tempDataHLQ	The High Level Qualifier (HLQ) to use to save an xml file that stores information for the `commandDataSet` to read from.
outputDataSet	The output dataset to save the data member output file of the preprocessor.
commandParameters	An optional parameter to specify any command parameters that should be used when executing the command.

Running the preprocessor

After you specify the ZAPP profile, you can run the preprocessor in the following ways:

Right-click and choose Execute remote preprocessor command to manually run the preprocessor in the program file editor.
Automatically run the preprocessor via the VS Code user setting zopeneditor.autoPreprocessor. The setting can have one of the values:
- off: disables the automatic preprocessor execution.
- open: when a program file is opened in the editor.
- save: when a program file is saved in the editor.
- both: when opening and saving a program file in the editor.

When the preprocessor command runs, a progress dialog with a Cancel button is displayed. A sequential data member is created using the tempDataHLQ to store information for the preprocessor to read. When the preprocessor finishes, it deletes the sequential data member and then refreshes the editor and any syntax errors that were caused by preprocessor statements should go away or, if they contained problems, will show those instead.

The following is an example of the sequential data member that gets created for the remote preprocessor to read based on the above ZAPP profile:

<PROGRAM>HLQ.COBOL(PRINTAPP)</PROGRAM><SYSLIB></SYSLIB><COMPILEOPTIONS></COMPILEOPTIONS><OUTPUT>HLQ.PREPROC.OUTPUT(PRINTAPP)</OUTPUT>

The ${HLQ} variable gets replaced with HLQ from user settings
The input data member's name was PRINTAPP
There were no mvs cobol libraries or compiler options specified in the ZAPP property groups

Using the preprocessor output in the editor

To get a side-by-side view of the program before and after preprocessing, you can use the right-click menu option Compare preprocessor input and output files.

Using the preprocessor for expanding PL/I macros

Our sample Git repository contains a sample REXX program that can be called by Z Open Editor's preprocessor to expand PL/I macros defined in a program by invoking the PL/I compiler on z/OS. The resulting output produced by this REXX program is expanded source code that can be used by Z Open Editor's language server in the same way as any other preprocessor output presenting hovers as well as side-by side views with the expanded macros in the source code.

To try and explore this sample, review the detailed instructions in the Preprocessor Readme of our Github repositoryopen in new window.

Limitations

Restriction: Several limitations are associated with using a preprocessor within Z Open Editor:

COBOL code formatting will be disabled for files that contain preprocessor statements.
Not all preprocessor statements within a COBOL EXEC block might be identified.
All preprocessor statements might not be identified if the code is reorganized.
Preprocessor statements that modify text in another location of a program might not be identified.
Real-time syntax checking of programs that contain preprocessor statements that contain only the beginning or ending of a language comment might not be identified.
Remote preprocessor command dataset members must be either REXX or CLIST files.
The side-by-side comparison of program and preprocessed output is not yet available for the remote preprocessor support.

# Preprocessor support

# Local preprocessor

# Example workflow

# Requirements for the preprocessor

# ZAPP profile

# Running the preprocessor

# Using the preprocessor output in the editor

# Remote preprocessor

# Example workflow

# Requirements for the preprocessor

# ZAPP profile

# Running the preprocessor

# Using the preprocessor output in the editor

# Using the preprocessor for expanding PL/I macros

# Limitations

Preprocessor support

Local preprocessor

Example workflow

Requirements for the preprocessor

ZAPP profile

Running the preprocessor

Using the preprocessor output in the editor

Remote preprocessor

Example workflow

Requirements for the preprocessor

ZAPP profile

Running the preprocessor

Using the preprocessor output in the editor

Using the preprocessor for expanding PL/I macros

Limitations