Skip to content

Deploying Watson Knowledge Catalog on Cloud Pak for Data with ArgoCD

Overview

In this section we're going to complete the installation of Watson Knowledge Catalog on Cloud Pak for Data using GitOps and ArgoCD.

cluster-tech

ArgoCD applications will be created for each of these resources. Specifically, ArgoCD applications will keep the cluster synchronized with the application YAML definitions stored in Git.

By the end of this topic we'll have a Watson Knowledge Catalog enabled on Cloud Pak for Data.

Note

If you have already completed the tutorial section to deploy Watson Studio using GitOps, Watson Knowledge Catalog will be enabled on the same Cloud Pak for Data instance.


Pre-requisites

Tip

In macOS, many of these utilities can be installed using Homebrew.

Please see these instructions for more information about how to install these utilities.

Before attempting this section, you must have completed the following tasks:

Please see the previous sections of this guide for information on how to do these tasks.

Note

If you have already completed the tutorial section to deploy Watson Studio using GitOps, when you follow the instructions in this tutorial, you will find that some lines in the kustomization.yaml file are already uncommented. In such cases, you can ignore that and those components are already deployed for you as part of that tutorial section. Please watch for specific call outs in the steps below.


Deploy services to the cluster

We've deployed IBM Cloud Pak for Data platform using GitOps. We can install additional services such as Watson Knowledge Catalog on this platform.

These components are part of the services layer in our architecture, and that requires us to access /0-bootstrap/single-cluster/2-services within our GitOps repository.

Installing Watson Knowledge Catalog using GitOps requires us to provide image registry access to specific service accounts such as zen-admin-sa, zen-editor-sa, zen-norbac-sa, wkc-iis-sa etc. Also, it requires specific Security Context Constraints to be defined for proper deployment of the service. This is provided through ArgoCD infra application.

  1. How to provide service account access

    As we saw earlier, the infra ArgoCD application uses the contents of the /0-bootstrap/single-cluster/1-infra folder to manage infrastructure resources.

  2. Review ArgoCD infra folder

    Let’s reexamine the contents of 0-bootstrap/single-cluster/1-infra/kustomization.yaml to see how ArgoCD manages the resources deployed to the cluster.

    Issue the following command:

    cat 0-bootstrap/single-cluster/1-infra/kustomization.yaml
    

    We can see the contents of the kustomization.yaml:

    resources:
    #- argocd/consolelink.yaml
    - argocd/consolenotification.yaml
    - argocd/namespace-ibm-common-services.yaml
    #- argocd/namespace-ci.yaml
    #- argocd/namespace-dev.yaml
    #- argocd/namespace-staging.yaml
    #- argocd/namespace-prod.yaml
    #- argocd/namespace-cloudpak.yaml
    #- argocd/namespace-istio-system.yaml
    #- argocd/namespace-openldap.yaml
    - argocd/namespace-sealed-secrets.yaml
    - argocd/namespace-tools.yaml
    #- argocd/namespace-instana-agent.yaml
    #- argocd/namespace-robot-shop.yaml
    #- argocd/namespace-openshift-storage.yaml
    #- argocd/namespace-spp.yaml
    #- argocd/namespace-spp-velero.yaml
    #- argocd/namespace-baas.yaml
    #- argocd/namespace-db2.yaml
    #- argocd/namespace-mq.yaml
    #- argocd/serviceaccounts-ibm-common-services.yaml
    #- argocd/serviceaccounts-tools.yaml
    #- argocd/serviceaccounts-db2.yaml
    #- argocd/serviceaccounts-mq.yaml
    #- argocd/scc-wkc-iis.yaml
    #- argocd/storage.yaml
    #- argocd/infraconfig.yaml
    #- argocd/machinesets.yaml
    patches:
    - target:
        group: argoproj.io
        kind: Application
        labelSelector: "gitops.tier.layer=infra"
      patch: |-
        - op: add
          path: /spec/source/repoURL
          value: https://github.com/tutorial-org-123/multi-tenancy-gitops-infra.git
        - op: add
          path: /spec/source/targetRevision
          value: master
    

    Note

    If you have already deployed Watson Studio using GitOps, you will see that the line for - argocd/serviceaccounts-tools.yaml is already uncommented for you. You can ignore that for now.

  3. Add service account permissions and enable WKC Security Context Constraints

    Open 0-bootstrap/single-cluster/1-infra/kustomization.yaml and make sure following resources are uncommented:

    - argocd/consolenotification.yaml
    - argocd/namespace-ibm-common-services.yaml
    - argocd/namespace-sealed-secrets.yaml
    - argocd/namespace-tools.yaml
    - argocd/serviceaccounts-tools.yaml
    - argocd/scc-wkc-iis.yaml
    

    If you take a diff, and just look for the added services:

    git diff  | grep "^\+" | grep -v "^\+++"
    

    You will have one of the following depending on whether you already installed Watson Studio or not. Remember, namespaces and consolenotification were already enabled.

    +- argocd/scc-wkc-iis.yaml
    

    or

    +- argocd/serviceaccounts-tools.yaml
    +- argocd/scc-wkc-iis.yaml
    

    This shows the resources to be deployed for infrastructure.

    Commit and push changes to your git repository:

    git add .
    git commit -s -m "Deploy Watson Knowledge Catalog SCC"
    git push origin $GIT_BRANCH
    

    The changes have now been pushed to your GitOps repository:

    [master c549ae2] Deploy Watson Knowledge Catalog SCC
     1 file changed, 2 insertions(+), 2 deletions(-)
    Enumerating objects: 11, done.
    Counting objects: 100% (11/11), done.
    Delta compression using up to 8 threads
    Compressing objects: 100% (6/6), done.
    Writing objects: 100% (6/6), 532 bytes | 177.00 KiB/s, done.
    Total 6 (delta 5), reused 0 (delta 0), pack-reused 0
    remote: Resolving deltas: 100% (5/5), completed with 5 local objects.
    To https://github.com/tutorial-org-123/multi-tenancy-gitops.git
       b49dff5..c549ae2  master -> master
    
  4. The infra application detects the change and resyncs

    Once these changes to our GitOps repository are seen by ArgoCD, it will resync the cluster to the desired new state.

    Switch to the ArgoCD UI Applications view, filter by infra project.

    argocd75

    Notice how the infra application has detected the changes and is automatically synching the cluster to create new sa-tools and scc-wkc-iis ArgoCD applications.

    Tip

    You can manually sync and refresh the infra ArgoCD application in the UI if you don't want to wait for ArgoCD to detect the change.

  5. How to deploy services

    As we saw earlier, the services application uses the contents of the /0-bootstrap/single-cluster/2-services folder to determine which Kubernetes resources should be deployed in the cluster.

  6. Review ArgoCD services folder

    Let’s reexamine the contents of 0-bootstrap/single-cluster/2-services/kustomization.yaml to see how ArgoCD manages the resources deployed to the cluster.

    Issue the following command:

    cat 0-bootstrap/single-cluster/2-services/kustomization.yaml
    

    We can see the contents of the kustomization.yaml:

    resources:
    # IBM Software
    
    ## Cloud Pak for Integration
    #- argocd/operators/ibm-ace-operator.yaml
    #- argocd/operators/ibm-apic-operator.yaml
    #- argocd/instances/ibm-apic-instance.yaml
    #- argocd/instances/ibm-apic-management-portal-instance.yaml
    #- argocd/instances/ibm-apic-gateway-analytics-instance.yaml
    #- argocd/operators/ibm-aspera-operator.yaml
    #- argocd/operators/ibm-assetrepository-operator.yaml
    #- argocd/operators/ibm-cp4i-operators.yaml
    #- argocd/operators/ibm-datapower-operator.yaml
    #- argocd/operators/ibm-eventstreams-operator.yaml
    #- argocd/operators/ibm-mq-operator.yaml
    #- argocd/operators/ibm-opsdashboard-operator.yaml
    #- argocd/operators/ibm-platform-navigator.yaml
    #- argocd/instances/ibm-platform-navigator-instance.yaml
    
    ## Cloud Pak for Business Automation
    #- argocd/operators/ibm-cp4a-operator.yaml
    #- argocd/operators/ibm-db2u-operator.yaml
    #- argocd/operators/ibm-process-mining-operator.yaml
    #- argocd/instances/ibm-process-mining-instance.yaml
    
    ## Cloud Pak for Data
    - argocd/operators/ibm-cpd-scheduling-operator.yaml
    #- argocd/instances/ibm-cpd-scheduling-instance.yaml
    - argocd/operators/ibm-cpd-platform-operator.yaml
    - argocd/instances/ibm-cpd-instance.yaml
    #- argocd/operators/ibm-cpd-ws-operator.yaml
    #- argocd/instances/ibm-cpd-ws-instance.yaml
    #- argocd/operators/ibm-cpd-wkc-operator.yaml
    #- argocd/instances/ibm-cpd-wkc-instance.yaml
    
    ## Cloud Pak for Security
    #- argocd/operators/ibm-cp4s-operator.yaml
    #- argocd/instances/ibm-cp4sthreatmanagements-instance.yaml
    
    ## IBM Foundational Services / Common Services
    #- argocd/operators/ibm-foundations.yaml
    #- argocd/instances/ibm-foundational-services-instance.yaml
    #- argocd/operators/ibm-automation-foundation-core-operator.yaml
    #- argocd/operators/ibm-automation-foundation-operator.yaml
    #- argocd/operators/ibm-license-service-operator.yaml
    #- argocd/instances/ibm-license-service-instance.yaml
    
    ## IBM Catalogs
    - argocd/operators/ibm-catalogs.yaml
    
    # B2BI
    #- argocd/instances/ibm-sfg-db2.yaml
    #- argocd/instances/ibm-sfg-mq.yaml
    #- argocd/instances/ibm-sfg-b2bi-setup.yaml
    #- argocd/instances/ibm-sfg-b2bi.yaml
    
    # Required for IBM MQ
    #- argocd/instances/openldap.yaml
    # Required for IBM ACE, IBM MQ
    #- argocd/operators/cert-manager.yaml
    #- argocd/instances/cert-manager-instance.yaml
    
    # Sealed Secrets
    - argocd/instances/sealed-secrets.yaml
    
    # CICD
    #- argocd/operators/grafana-operator.yaml
    #- argocd/instances/grafana-instance.yaml
    #- argocd/instances/grafana-instana.yaml
    #- argocd/instances/artifactory.yaml
    #- argocd/instances/chartmuseum.yaml
    #- argocd/instances/developer-dashboard.yaml
    #- argocd/instances/swaggereditor.yaml
    #- argocd/instances/sonarqube.yaml
    #- argocd/instances/pact-broker.yaml
    # In OCP 4.7+ we need to install openshift-pipelines and possibly privileged scc to the pipeline serviceaccount
    #- argocd/operators/openshift-pipelines.yaml
    
    # Service Mesh
    #- argocd/operators/elasticsearch.yaml
    #- argocd/operators/jaeger.yaml
    #- argocd/operators/kiali.yaml
    #- argocd/operators/openshift-service-mesh.yaml
    #- argocd/instances/openshift-service-mesh-instance.yaml
    
    # Monitoring
    #- argocd/instances/instana-agent.yaml
    #- argocd/instances/instana-robot-shop.yaml
    
    # Spectrum Protect Plus
    #- argocd/operators/spp-catalog.yaml
    #- argocd/operators/spp-operator.yaml
    #- argocd/instances/spp-instance.yaml
    #- argocd/operators/oadp-operator.yaml
    #- argocd/instances/oadp-instance.yaml
    #- argocd/instances/baas-instance.yaml
    
    patches:
    - target:
        group: argoproj.io
        kind: Application
        labelSelector: "gitops.tier.layer=services,gitops.tier.source=git"
      patch: |-
        - op: add
          path: /spec/source/repoURL
          value: https://github.com/tutorial-org-123/multi-tenancy-gitops-services.git
        - op: add
          path: /spec/source/targetRevision
          value: master
    - target:
        group: argoproj.io
        kind: Application
        labelSelector: "gitops.tier.layer=applications,gitops.tier.source=git"
      patch: |-
        - op: add
          path: /spec/source/repoURL
          value: https://github.com/tutorial-org-123/multi-tenancy-gitops-apps.git
        - op: add
          path: /spec/source/targetRevision
          value: master
    - target:
        group: argoproj.io
        kind: Application
        labelSelector: "gitops.tier.layer=services,gitops.tier.source=helm"
      patch: |-
        - op: add
          path: /spec/source/repoURL
          value: https://charts.cloudnativetoolkit.dev
    - target:
        name: ibm-automation-foundation-operator
      patch: |-
        - op: add
          path: /spec/source/helm/parameters/-
          value:
            name: spec.channel
            value: v1.1
    

    Note

    If you have already deployed Watson Studio using GitOps, you will see that following lines are already uncommented for you. You can ignore that for now.

    - argocd/operators/ibm-cpd-ws-operator.yaml
    - argocd/instances/ibm-cpd-ws-instance.yaml
    
  7. Add the services layer to the cluster

    Note

    The Watson Knowledge Catalog instance requires a RWX storageclass and it is set to managed-nfs-storage by default in the ArgoCD Application 0-bootstrap/single-cluster/2-services/argocd/instances/ibm-cp4d-watson-knowledge-catalog-instance.yaml. This storageclass is available for Red Hat OpenShift on IBM Cloud cluster provisioned from IBM Technology Zone with NFS storage selected.

    Open 0-bootstrap/single-cluster/2-services/kustomization.yaml and make sure following resources are uncommented:

    Note

    If you have already deployed Watson Studio using GitOps, leave any Watson Studio specific lines untouched.

    # Cloud Pak for Data
    - argocd/operators/ibm-cpd-scheduling-operator.yaml
    - argocd/operators/ibm-cpd-platform-operator.yaml
    - argocd/instances/ibm-cpd-instance.yaml
    - argocd/operators/ibm-cpd-wkc-operator.yaml
    - argocd/instances/ibm-cpd-wkc-instance.yaml
    
    # IBM Catalogs
    - argocd/operators/ibm-catalogs.yaml
    
    # Sealed Secrets
    - argocd/instances/sealed-secrets.yaml
    

    If you take a diff, and just look for the added services:

    git diff  | grep "^\+" | grep -v "^\+++"
    

    You will have the following.

    +- argocd/operators/ibm-cpd-wkc-operator.yaml
    +- argocd/instances/ibm-cpd-wkc-instance.yaml
    

    This shows the resources to be deployed for services.

    Commit and push changes to your git repository:

    git add .
    git commit -s -m "Deploy Watson Knowledge Catalog"
    git push origin $GIT_BRANCH
    

    The changes have now been pushed to your GitOps repository:

    [master 1dbac6f] Deploy Watson Knowledge Catalog
     1 file changed, 2 insertions(+), 2 deletions(-)
    Enumerating objects: 11, done.
    Counting objects: 100% (11/11), done.
    Delta compression using up to 8 threads
    Compressing objects: 100% (6/6), done.
    Writing objects: 100% (6/6), 528 bytes | 528.00 KiB/s, done.
    Total 6 (delta 5), reused 0 (delta 0), pack-reused 0
    remote: Resolving deltas: 100% (5/5), completed with 5 local objects.
    To https://github.com/tutorial-org-123/multi-tenancy-gitops.git
       b49dff5..1dbac6f  master -> master
    

    The intention of this operation is to indicate that we'd like the resources declared in 0-bootstrap/single-cluster/2-services/kustomization.yaml to be deployed in the cluster. Like the infra ArgoCD application, the resources created by the services ArgoCD application will manage the Kubernetes relevant services resources applied to the cluster.

  8. The services application detects the change and resyncs

    Once these changes to our GitOps repository are seen by ArgoCD, it will resync the cluster to the desired new state.

    Switch to the ArgoCD UI Applications view, click on the services application to see the start of this resync process:

    argocd78

    Notice how the services application has detected the changes and is automatically synching the cluster.

    argocd79

    Tip

    You can manually sync and refresh the services ArgoCD application in the UI if you don't want to wait for ArgoCD to detect the change.

  9. The new ArgoCD applications

    After a short while, you'll see lots of new ArgoCD applications have been created to manage the services we have deployed by modifying kustomization.yaml under 0-bootstrap/single-cluster/2-services folder:

    argo80

    See how most ArgoCD applications are Synced almost immediately, but some spend time in Progressing.

    argocd81

    Notice how many more Synced ArgoCD applications are now in the cluster; these are as a result of the newly added services layer in our architecture.

  10. Verify that Watson Knowledge Catalog has been deployed successfully.

    Switch to the ArgoCD UI Applications view, wait for all ArgoCD applications to become Synced and Healthy:

    argocd82

    Note

    The installation of Watson Knowledge Catalog will take approximately 3 hours.

    Alternatively, issue the following command on terminal and wait for the expected output.

    watch "oc get WKC wkc-cr -n tools -o jsonpath=\"{.status.wkcStatus} {'\n'}\""
    

    Expected output = Completed

    Tip

    Press Ctrl+C on your keyboard to exit out of watch window.

    We can now access the Cloud Pak for Data web client and see that Watson Knowledge Catalog is installed.

  11. Access the IBM Cloud Pak for Data web client

    Retrieve the URL for the IBM Cloud Pak for Data web client and and open it in a browser.

    echo https://`oc get ZenService lite-cr -n tools -o jsonpath="{.status.url}{'\n'}"`
    

    Retrieve the admin password

    oc extract secret/admin-user-details -n tools --keys=initial_admin_password --to=-
    

    Log in to the IBM Cloud Pak for Data web client with the admin credentials.

    CPD

    Click on the navigation menu icon on the top left corner. Click on Services menu option to expand it, then select Services catalog.

    CPD Nav

    Under Status, select "Enabled" to display only the services that are installed and enabled in Cloud Pak for Data.

    CPD Enabled Filter

    Notice the Enabled tag next to Watson Knowledge Catalog. There might be other enabled services as well.

    CPD WKC Services


Congratulations!

You've installed Watson Knowledge Catalog on Cloud Pak for Data. We installed it into the tools namespaces we created previously.

This chapter is now complete.