Managing secrets

Developing z/OS® applications requires you to interact with z/OS through Zowe CLI and Zowe Explorer using personal credentials, such as username and password or multi-factor authentication tokens, to access data sets, UNIX files, or job outputs. You also work with Git and Git server technologies such as GitLab or GitHub, which require personal SSH keys for operations such as cloning, pulling and pushing. You may have to access asset stores, such as Artifactory, which require personal access tokens. On a development laptop, you typically manage keys and configuration settings for these systems in your home directory or encrypted key stores like Windows Credentials Manager or MacOS Keychain, to have them available for all the different projects you work on. For instance, you use the same SSH keys for multiple Git repositories for many different projects on your GitHub server.

To achieve the same level of reuse and securely manage your secrets, OpenShift® Dev Spaces can manage these secrets and settings as Kubernetes secrets in your personal Kubernetes namespace, where your workspaces run. When a Dev Spaces workspace starts, these secrets and settings are mounted as either environment variables or files, provided as read-only symbolic links in your home directory. For example, a pair of private and public SSH keys is stored as one Kubernetes secret in Openshift and is then mounted as files ~/.ssh/rsa-id and ~/.ssh/rsa-id.pub in all your different Dev Spaces workspaces in your home directory whenever they start up.

Note that, by default, secrets in OpenShift might not meet your organization's security requirements and may require additional integrations with external secret vaults or drivers. As examples, see this article about the Secret Store CSI Driver Operator in OpenShift or this article about integrating OpenShift with IBM's HashiCorp Vaults (also see this Red Hat OpenShift enhances Vault integrations announcement).

The following instructions show examples of how to create environment variables with credentials for Zowe CLI and Explorer, as well as SSH files to interact with Git servers or execute Ansible playbooks from your Dev Spaces workspace. You can create these secrets using the OpenShift Web Console or by using the OpenShift oc command line tool. Check with your cluster administrator on how to log in to your OpenShift console to access your personal project, which is your Kubernetes namespace or where to download the correct version of the command-line tool.

Locating and managing secrets

All your secrets must be stored in your personal Dev Spaces OpenShift project so that they are not accessible to other users. You can find your project name in the OpenShift Console's Developer view following these steps:

1. If you are logged in to your OpenShift Dev Spaces dashboard, you can navigate to the OpenShift console by clicking the Applications icon (icon with nine squares) in the toolbar and select OpenShift console.

2. Alternatively, if you are inside your Dev Spaces workspace editor, click the icon all the way on the left of the editor's status bar and select Dev Spaces: Open OpenShift Console from the drop-down menu that appears at the top of the page.

3. After you are on the OpenShift Console page, make sure you switch to the Developer perspective, in case you are also an administrator of the cluster.

4. Select Project in the menu pane on the left.

5. Enter your OpenShift username in the search field.

6. The project typically is named <username>-<six-digit-id>.

7. Make note of your project name.

If you are already running a Dev Spaces workspace, an easier way is to

1. Open the VS Code terminal.

2. Run the command

   echo $WORKSPACE_NAMESPACE
   

After you have identified your project name:

1. Select Secrets in the OpenShift Developer Console's menu pane on the left.

2. In the Project drop-down at the top of the page, select your project name.

3. If you see a lot of projects in the drop-down, it is a good idea to mark your project as a favorite for easy access.

You can now see the list of default secrets already created by Dev Spaces. Use the Create menu button on the top-right of the page to create new secrets as described in the instructions in the next sections, or if you create secrets through command line, you can immediately see and verify the new secrets here.

You can use this page to modify secrets as well, for example, when your password changes.

1. Within the secrets list, click on a secret name.

2. Click the Reveal value button to display the contents of your secret. This is useful if you want to copy your public SSH key and add it to another system.

3. In the Actions drop-down, select Edit Secrets to open a form-based editor where you can make changes.

Mount secrets as Environment Variables

This section describes how to create and use secrets for your personal username and password to be used by Zowe CLI and Zowe Explorer in Dev Spaces.

1. The username and password that you want to store in the secret must be base64-encoded so that their content can be used with an OpenShift secret. The following commands, which you can run either in your workspace terminal or any other Unix-based terminal you have available, performs such an encoding for you.

   echo -n '<username>' | base64 | tr -d '\n' > encoded-user.txt
   echo -n '<password>' | base64 | tr -d '\n' > encoded-password.txt
   

Create the secret in the OpenShift console

1. If you want to use the OpenShift console as described in the previous section, ensure that you have selected your project on the OpenShift Secrets page.

2. Use the Create button with the From YAML option to open a text editor where you can paste and modify the following template:

   apiVersion: v1
   kind: Secret
   metadata:
     name: auth-secrets-zowe
     labels:
       controller.devfile.io/mount-to-devworkspace: "true"
       controller.devfile.io/watch-secret: "true"
     annotations:
       controller.devfile.io/mount-as: env
   type: Opaque
   data:
     ZOWE_OPT_USER: <paste-content-of-encoded-user.txt>
     ZOWE_OPT_PASSWORD: <paste-content-of-encoded-password.txt>
   

   As indicated, replace the two lines at the end with the content of your two text files created in the previous step. Note that currently only these two environment variable names are supported by Zowe Explorer.

3. Click the Create button and check in the Console that it was created correctly, as described in the previous section.

Create the secret with the command-line tool

1. If you are using the command-line tool, follow the steps in the previous section to find your personal OpenShift project name and set the context for oc commands with

   oc <my-project-name>
   

2. Save the contents of the modified template from above in a file called "secret.yaml" and run this command:

   oc apply -f secret.yaml
   

3. Another way to create secrets is by using command-line parameters like this, instead of the yaml file.

   oc create secret generic auth-secrets-zowe --from-literal=ZOWE_OPT_USER="<username>" --from-literal=ZOWE_OPT_PASSWORD="<password>" -n <my-project-name>
   

Testing the secret

After the secret is created, any currently running Dev Spaces workspace editor prompts you to reload the page.

1. After it finishes reloading, you can check the values of your environment variables in the VS Code terminal using the following command:

   echo $ZOWE_OPT_USER $ZOWE_OPT_PASSWORD
   

2. Now use the two environment variables $ZOWE_OPT_USER and $ZOWE_OPT_PASSWORD in your Zowe team configuration files as the values for the user and password properties. See the details in Using Environment Variables for Authentication in Zowe Explorer and Openshift Dev Workspaces

Create secrets for password-less access to your z/OS system

To ssh into a z/OS system, you need to create ssh private and public keys as well as a know_hosts file entry as K8s secrets.

In a nutshell, for experts and based on the instructions above, you need to perform the following steps to produce these secrets:

1. Create RSA key pairs and base64-encode them
2. Create a known_hosts entry and base64-encode it
3. Create a secret OpenShift resource and submit it against your workspace's namespace
4. Upload the public key to z/OS

Note, unless you are using vault technology as described at the beginning of this section, OpenShift administrators can access your secrets and decode your private key. It is strongly suggested that you create a fresh key pair with a passphrase to use for Dev Spaces. If you are a Windows user, you can perform some of the steps involving Unix commands in a running Dev Spaces workspace's VS Code terminal. If you do so, create a folder such as /project/ssh-keys or /projects/zopeneditor-sample/ssh-keys, and store the intermediate files produced by these instructions there. After you finish, delete the entire directory. In the following steps, assume that you have created and changed directory into that new empty directory called ssh-keys in the Dev Spaces terminal view.

1. Generate a new RSA key pair with the following command:

   ssh-keygen -t rsa -f devspaces-id -P ""
   

   The last -P parameter defines a passphrase, which you can provide to ensure that an OpenShift administrator cannot use your keys. If you leave it empty, it does not prompt you for the passphrase when using the keys.

2. Execute this command to generate the "known_hosts" entry required for this secret. Replace the word <hostname> with the IP address or the fully qualified hostname that you use to interact with the z/OS system. Replace <username> with your username on the z/OS system. Also, remember to provide the -p parameter for the SSH port if it is not the standard port. This command first prompts you to check the ECDSA key fingerprint and then enter your z/OS password.

   ssh <username>@<hostname> -o UserKnownHostsFile=./known-hosts
   exit
   

   If you do this for multiple machines, execute this command against each one and then manually concatenate all the outputs, separated by a newline character into one file.

3. You have three files in your folder that must be base64 encoded so their contents can be used with an OpenShift secret.

   cat devspaces-id | base64 | tr -d '\n' > devspaces-id-private.txt
   cat devspaces-id.pub | base64 | tr -d '\n' > devspaces-id-public.txt
   cat known-hosts | base64 | tr -d '\n' > known-hosts.txt
   

4. Create a new file called "secret.yaml". See the section above for alternative ways to perform these steps, either in the OpenShift Console or through command-line operations using the oc command.

   apiVersion: v1
   kind: Secret
   metadata:
     name: ssh-devspaces-secret
     labels:
       controller.devfile.io/mount-to-devworkspace: "true"
       controller.devfile.io/watch-secret: "true"
     annotations:
       controller.devfile.io/mount-path: "/home/user/.ssh"
   data:
     id_rsa: <paste-content-of-devspaces-id-private.txt>
     id_rsa.pub: <paste-content-of-devspaces-id-public.txt>
     known_hosts: <paste-contents-of-known_hosts.txt>
   

   As indicated, replace the three lines at the end with the content of your three text files created in the previous steps.

5. Create the secrets in the Console as described above or through the "oc" command with

   oc apply -f secret.yaml
   

6. After the secret is submitted, the Dev Spaces workspace editor prompts you to reload. After it finishes reloading, you can check if your secret is available using the VS Code terminal again.

   ll ~/.ssh
   total 0
   lrwxrwxrwx. 1 root 1000720000 13 May 30 22:20 id_rsa -> ..data/id_rsa
   lrwxrwxrwx. 1 root 1000720000 17 May 30 22:20 id_rsa.pub -> ..data/id_rsa.pub
   lrwxrwxrwx. 1 root 1000720000 18 May 30 22:20 known_hosts -> ..data/known_hosts
   

7. Now you can add your public key to the z/OS system through this command, which again prompts you (for the last time) for your z/OS password. Again add a -p parameter for a custom ssh port.

   ssh-copy-id -f -i ~/.ssh/id_rsa.pub <username>@<hostname>
   

8. Perform the final test by logging on to the z/OS system from the VS Code terminal in Dev Spaces this time without being prompted for your password.

   ssh <username>@<hostname>
   

Other security integrations

In addition to creating and managing personal secrets, OpenShift Dev Spaces administrators also set up OAuth-based authentication providers, such as GitHub Enterprise servers in your organization. This enables you to access the server without creating personal secrets, using your regular authentication method. Ask your administrator to review this documentation of the Red Hat administration guide with the details.

The Red Hat user documentation provides details on how to create and manage personal secrets. As a convenience, here are the instructions to create all the secrets required to access your remote z/OS® system without a password. These steps are a prerequisite to run Ansible® playbooks against your z/OS® system.