Cloud Pak for Data - DataStage Quickstart¶

Introduction¶

The instructions in this section will walk you through the steps necessary for setting up a demo environment that can be quickly shown to customers to showcase the art of the possible.

This demo in no way removes the need for following the tutorial style instructions that are documented in other sections of this guide. The instructions below will set up a pre-canned application demo environment to show to the customers.

Pre-requisites¶

Create a custom Git Organization¶

Create a new Git Organization to host the different GitOps repositories.

Instructions:

1. Log in to http://github.com and select the option to create a New organization.

   

2. Click on Create a free organization plan.

3. Complete the wizard by filling in the Organization account name and Contact email fields. Select the My personal account bullet and complete the verification step and click Next.

4. Skip the step to add members to the Organization.

5. Complete the Welcome to GitHub questionnaire and click Submit.

6. Congratulations, you have successfully created a new Github Organization.

   Note

   Please ensure your GitHub organization is Public and not Private.

Create a Git Personal Access Token (PAT)¶

Create a new Git Personal Access Token with the appropriate scopes. This will be required to run the application pipelines or to set up webhooks.

Instructions:

1. Log in to http://github.com and click on Settings.

   

2. Select Developer settings and click on Personal access tokens.

3. Provide a name for the token, set the Expiration to 90 days ,set the following scopes and click Generate token.

   GitHub Personal Access Token scopes

   
   

4. Copy and save the Personal Access Token. You will not be able to retrieve this value again later.

Red Hat OpenShift on IBM Cloud cluster with GitOps Configuration¶

Create the cluster¶

1. Provision a Red Hat OpenShift on IBM Cloud cluster with GitOps Configuration from IBM Technology Zone. Select the OpenShift + GitOps Configuration tile.

2. Click the Reserve now radio button.

3. Provide a name for the cluster, select Practice / Self-Education for the purpose and choose the region to provision the cluster.

4. Once a Preferred Geography has been selected, provide the appropriate Worker Node Count and Worker Node Flavor values based on the requirements for this quickstart listed in the note below. Finally, click Submit.

   Note

   For this Cloud Pak for Data - DataStage Quickstart we recommend you to request the Red Hat OpenShift on IBM Cloud cluster with GitOps Configuration with the following size:

   • Worker Node Count = 5
   • Worker Node Flavor = 16 vCPU x 64 GB - 300 GB Secondary Storage
   • NFS Size = 1 TB

Use the cluster¶

You will receive an email once the cluster provisioning is complete. The email will contain details on the cluster including the ArgoCD Console URL and admin credentials. This same information can also be found on the My reservations from IBM Technology Zone.

Once your cluster is ready, proceed to the next step to select resources to deploy.

Install required CLIs¶

• Install the Github CLI (version 1.14.0+)

• Install the OpenShift CLI oc (version 4.7 or 4.8)

  • Log into your OCP cluster, substituting the --token and --server parameters with your values:

    oc login --token=<token> --server=<server>
    

    If you are unsure of these values, click your user ID in the OpenShift web console and select Copy login command.

• Install the kubeseal CLI

  The kubeseal utility uses asymmetric crypto to encrypt secrets that only the controller can decrypt. These encrypted secrets are encoded in a SealedSecret resource which can safely be checked into your gitops repo. The controller will decrypt the secretes and install into your cluster.

Select resources to deploy¶

By now, you should already have a Red Hat OpenShift on IBM Cloud cluster with GitOps Configuration bootstrapped for you.

If you open ArgoCD, which is the GitOps tool being installed by the Red Hat OpenShift GitOps Operator, using the Argo CD URL provided in the email shown in the previous section, you will see that your ArgoCD GitOps application has been bootstrapped to monitor the multi-tenancy-gitops repository that has been forked into the GitHub organization you provided when requesting the quickstart environment.

As a result, anything you want to apply/do to your quickstart environment will be done through code changes on the aforementioned forked GitHub repository.

You can see in the image above of your ArgoCD web console that the profile within the multi-tenancy-gitops repository ArgoCD has been bootstrapped with is the single-cluster. As a result, anything you want to apply/do to your quickstart environment will be done within that GitOps profile.

You can also see that the ArgoCD applications for the infrastructure, services and applications layers are already created so that these will pick up any changes done, through code, at their respective layers.

Instructions¶

When you made the Red Hat OpenShift on IBM Cloud cluster request in the earlier section, automation has forked multi-tenancy-gitops GitHub repository into your GitHub organization, the name of which you provided in the cluster request form. You can execute the following steps either locally by cloning the multi-tenancy-gitops repository from your GitHub organization to your local workstation (and then make changes from you local workstation and commit and deliver those) or by using the new VSCode extension of GitHub Codespaces capability straight from your web browser (just open the multi-tenancy-gitops repository in your GitHub organization in your browser and press the . key):

1. Review the Infrastructure layer kustomization.yaml and un-comment the resources to deploy to match the Cloud Pak for Data - DataStage recipe.

2. Review the Services layer kustomization.yaml and un-comment the resources to deploy to match the Cloud Pak for Data - DataStage recipe.

3. Commit and push changes to your git repository (the following code refers to the commands when you cloned the multi-tenancy-gitops repository locally on your workstation)

git add .
git commit -m "initial bootstrap setup"
git push origin

Verifying the Installation¶

1. After approximately 30 minutes, get the status of the control plane (lite-cr)

   oc get ZenService lite-cr -n tools -o jsonpath="{.status.zenStatus}{'\n'}"
   

   Cloud Pak for Data control plane is ready when the command returns Completed. If the command returns another status, wait for some more time and rerun the command.

2. After approximately 2 hours, Get the status of DataStage (datastage-cr)

   oc get DataStage datastage-cr -n tools -o jsonpath="{.status.dsStatus} {'\n'}"
   

   DataStage is ready when the command returns Completed.

Cloud Pak for Data UI¶

1. Get the URL of the Cloud Pak for Data web client and open it in a browser.

   echo https://`oc get ZenService lite-cr -n tools -o jsonpath="{.status.url}{'\n'}"`
   

2. The credentials for logging into the Cloud Pak for Data web client are admin/<password> where password is stored in a secret.

   oc extract secret/admin-user-details -n tools --keys=initial_admin_password --to=-
   

3. Log into the IBM Cloud Pak for Data UI using the password from previous step.

   

4. Click on the navigation menu icon on the top left corner. Click on Services menu option to expand it, then select Services catalog.

   

5. Under Status, select "Enabled" to display only the services that are installed and enabled in Cloud Pak for Data.

   

6. Notice the Enabled tag next to DataStage. There might be other enabled services as well.