Preparing for application components¶
Audience: Architects, Application developers, Administrators
Timing: 20 minutes
Overview¶
In this topic, we will:
- Explore the sample
multi-tenancy-gitops-appsGitOps application components repository - Customize this repository for our cluster
By the end of this topic we'll have a cluster ready to deploy MQ applications and queue managers.
Introduction¶
In this topic we're going to explore then customize the GitOps repository that will store the MQ application and queue manager YAMLs. These YAMLs are built and tested by their corresponding pipelines ready for deployment. This GitOps repository will be accessed by ArgoCD applications that will apply these YAMLs to the cluster, resulting in the creation of running MQ applications and queue managers.
We'll examine these highlighted components in more detail throughout this topic; here's an initial overview of their function:
Git (config)repository stores the YAMLs built by the Tekton pipelines. These YAMLs will be applied to the cluster by ArgoCD applications to create MQ applications and queue managers.ArgoCD applicationsdeploy their corresponding MQ applications and queue managers to the cluster. ArgoCD will also ensure that these MQ resources remain configured according to the YAML definitions -- any configuration drift (deliberate or accidental) will be reverted to align with the YAML in theGit (config)repository. The YAML specification for each ArgoCD application also resides in theGit (config)repository.App1andQM1are instances of an MQ application and queue manager that have been created by their corresponding ArgoCD application. There is one ArgoCD application corresponding to each MQ application and each queue manager.
Pre-requisites¶
Before attempting this topic, you must have completed the following tasks:
- You have completed the previous sections of this tutorial.
Explore the -apps repository¶
The multi-tenancy-gitops-apps repository contains the YAMLs of the application components running in the cluster. For example, if there are 10 applications, 2 databases, 4 queue managers and 2 workflow engines running in the cluster, then there will be YAMLs in this repository corresponding to each of these instances.
Let's explore the structure of the multi-tenancy-gitops-apps repository to see how it works.
-
Ensure environment variables are set
Tip
Ensure you're in the
multi-tenancy-gitops-appsterminal window.Tip
If you're returning to the tutorial after restarting your computer, ensure that the $GIT_ORG, $GIT_BRANCH and $GIT_ROOT environment variables are set.
(Replace
<your organization name>appropriately:)export GIT_BRANCH=master export GIT_ORG=<your organization name> export GIT_ROOT=$HOME/git/$GIT_ORG-rootYou can verify your environment variables as follows:
echo $GIT_BRANCH echo $GIT_ORG echo $GIT_ROOT -
Locate the multi-tenancy-gitops-apps repository
Issue the following command to change to your GitOps repository:
cd $GIT_ROOT cd multi-tenancy-gitops-apps -
A folder for each application component type
Let's explore the top level folder structure of the the repository.
Issue the following command:
tree . -L 1to show the top level folders:
. ├── ace ├── apic ├── cntk ├── cp4a ├── mq ├── pm ├── scripts ├── shared ├── slack-notifications └── soapserverNotice that there's one high level folder for each of the different application components deployed to the cluster. For example, the YAMLs defining the API gateways, ACE integration servers and MQ queue managers are stored in the
apic,aceandmqfolders, respectively. -
An application component folder
Each of these application component folders has a similar structure. Let's look at the MQ folder as a typical example.
Issue the following command:
tree ./mq -L 2to show the structure of the MQ folder:
./mq ├── config │ └── argocd └── environments ├── base ├── ci ├── dev ├── prod ├── staging └── toolsNotice how:
- the
config/argocdfolder contains the ArgoCD applications that manage the MQ application components deployed to the cluster. - the
environmentsfolder contains one subfolder for each environment on the cluster, including MQ applications and queue managers.
- the
-
ArgoCD applications
Let's explore the folder structure to see how ArgoCD manages the cluster for these application components.
Issue the following command:
tree ./mq/config/argocd -L 2to show the ArgoCD applications:
./mq/config/argocd ├── ci │ ├── ci-app-rest.yaml │ └── ci-scc.yaml ├── dev │ ├── dev-mq-metric-samples-instance.yaml │ ├── dev-mq-qm01-instance.yaml │ └── dev-mq-spring-app-instance.yaml ├── prod │ ├── prod-mq-metric-samples-instance.yaml │ ├── prod-mq-qm01-instance.yaml │ └── prod-mq-spring-app-instance.yaml ├── staging │ ├── staging-mq-metric-samples-instance.yaml │ ├── staging-mq-qm01-instance.yaml │ └── staging-mq-spring-app-instance.yaml └── tools └── tools-cert-manager-artifacts.yamlNotice how:
- The
config/argocdfolder contains ArgoCD applications organized by folder namespace. - The name of the ArgoCD applications corresponds to the resource it manages; for example
dev/dev-mq-qm01-instance.yamlwill manage a queue manager in thedevnamespace.
The number of ArgoCD YAMLs in these folders will grow as application components are added to the cluster. For this tutorial, we've pre-populated the repository with the ArgoCD YAMLs for the initial application components we're going to deploy to the cluster.
- The
ArgoCD example¶
All ArgoCD applications work in a similar way. Let's examine one in a little more detail.
-
An example ArgoCD application
Let's look at the YAML of a typical ArgoCD application that manages a queue manager.
Issue the following command:
cat ./mq/config/argocd/dev/dev-mq-qm01-instance.yamlto see the YAML to manage an MQ queue manager running in the cluster:
apiVersion: argoproj.io/v1alpha1 kind: Application metadata: name: dev-mq-qm01-instance annotations: argocd.argoproj.io/sync-wave: "300" finalizers: - resources-finalizer.argocd.argoproj.io spec: destination: namespace: dev server: https://kubernetes.default.svc project: applications source: path: mq/environments/dev/mq-qm01/ repoURL: https://github.com/cloud-native-toolkit-demos/multi-tenancy-gitops-apps.git targetRevision: master syncPolicy: automated: prune: true selfHeal: true syncOptions: - Replace=trueNotice how this ArgoCD application will use:
- the
path: mq/environments/mq-infra/overlays/develement to identify the folder location containing the queue manager YAMLs that will be applied to the cluster. - the
repoURL:identifies the GitHUb repository; we'll need to customize this value for our deployment.
You can learn more about the different YAML elements in an ArgoCD application in the ArgoCD API reference.
There are many ArgoCD applications like this in the
-appsrepository; each one is responsible for managing the deployment of any application component to the cluster, including its related resources such as configmaps and certificates. - the
-
The
environmentsfolderAs we've seen in our example, the ArgoCD applications refer to YAMLs in the
environmentsfolder; let's explore theenvironmentsfolder in more detail.Issue the following command:
tree ./mq/environments -L 2to see the wide range of YAMLs in this folder:
./mq/environments ├── base │ ├── kustomization.yaml │ ├── mq-metric-samples │ ├── mq-qm01 │ └── mq-spring-app ├── ci │ ├── certificates │ ├── configmaps │ ├── eventlisteners │ ├── kustomization.yaml │ ├── pipelines │ ├── roles │ ├── routes │ ├── scc │ ├── secrets │ ├── tasks │ ├── triggerbindings │ └── triggertemplates ├── dev │ ├── kustomization.yaml │ ├── mq-metric-samples │ ├── mq-qm01 │ └── mq-spring-app ├── prod │ ├── kustomization.yaml │ ├── mq-metric-samples │ ├── mq-qm01 │ └── mq-spring-app ├── staging │ ├── kustomization.yaml │ ├── mq-metric-samples │ ├── mq-qm01 │ └── mq-spring-app └── tools ├── certificates ├── cluster-issuers └── kustomization.yaml 31 directories, 6 filesAs the cluster grows, more YAMLs will be added to the
environmentsfolder. These YAMLs are referenced by corresponding ArgoCD YAMLs in the.../config/argocd/folder. -
The resource YAMLs
When a pipeline successfully builds and tests an application component from a source repository, it stores its output YAML in the
-appsrepository.Issue the following command:
tree ./mq/environments/ -L 3to show the contents of the folder that will contain the YAMLs for the
mq-qm01,mq-spring-appandmq-metric-samples:./mq/environments/ ├── base │ ├── kustomization.yaml │ ├── mq-metric-samples │ │ ├── deployment.yaml │ │ ├── kustomization.yaml │ │ ├── role.yaml │ │ ├── rolebinding.yaml │ │ ├── route.yaml │ │ ├── secret.yaml │ │ ├── service.yaml │ │ ├── serviceaccount.yaml │ │ └── servicemonitor.yaml │ ├── mq-qm01 │ │ ├── kustomization.yaml │ │ └── queuemanager.yaml │ └── mq-spring-app │ ├── deployment.yaml │ ├── jaegerconfigmap.yaml │ ├── kustomization.yaml │ ├── route.yaml │ ├── service.yaml │ └── serviceaccount.yaml ├── ci │ ├── certificates │ │ ├── ci-mq-client-certificate.sh │ │ ├── ci-mq-client-certificate.yaml_template │ │ ├── ci-mq-server-certificate.sh │ │ └── ci-mq-server-certificate.yaml_template │ ├── configmaps │ │ ├── gitops-repo-configmap.sh │ │ └── gitops-repo-configmap.yaml_template │ ├── eventlisteners │ │ ├── cntk-event-listener.sh │ │ ├── cntk-event-listener.yaml_template │ │ ├── mq-qm-dev-eventlistener.yaml │ │ ├── mq-qm-stage-eventlistener.yaml │ │ ├── mq-spring-app-dev-eventlistener.yaml │ │ └── mq-spring-app-stage-eventlistener.yaml │ ├── kustomization.yaml │ ├── pipelines │ │ ├── ibm-test-pipeline-for-dev.yaml │ │ ├── ibm-test-pipeline-for-stage.yaml │ │ ├── java-maven-dev-pipeline.yaml │ │ ├── mq-metric-samples-dev-pipeline.yaml │ │ ├── mq-pipeline-dev.yaml │ │ └── mq-spring-app-dev-pipeline.yaml │ ├── roles │ │ ├── custom-ci-pipeline-sa-rolebinding.yaml │ │ ├── custom-dev-pipeline-sa-rolebinding.yaml │ │ ├── custom-pipeline-sa-clusterrole.yaml │ │ ├── custom-pipeline-sa-role.yaml │ │ ├── custom-prod-pipeline-sa-rolebinding.yaml │ │ └── custom-staging-pipeline-sa-rolebinding.yaml │ ├── routes │ │ ├── cntk-route.yaml │ │ ├── mq-qm-dev-route.yaml │ │ ├── mq-qm-stage-route.yaml │ │ ├── mq-spring-app-dev-route.yaml │ │ └── mq-spring-app-stage-route.yaml │ ├── scc │ │ ├── Chart.yaml │ │ └── values.yaml │ ├── secrets │ │ ├── artifactory-access-secret.sh │ │ ├── artifactory-access-secret.yaml │ │ ├── git-credentials-secret.sh │ │ ├── git-credentials-secret.yaml │ │ ├── ibm-entitled-registry-credentials-secret.sh │ │ ├── ibm-entitled-registry-credentials-secret.yaml │ │ ├── mq-client-jks-password-secret.sh │ │ └── mq-client-jks-password-secret.yaml │ ├── tasks │ │ ├── 10-gitops-for-mq.yaml │ │ ├── 10-gitops-promotion.yaml │ │ ├── 10-gitops.yaml │ │ ├── 11-app-name.yaml │ │ ├── 12-functional-tests.yaml │ │ ├── 13-cphtestp-performance-test.yaml │ │ ├── 13-jmeter-performance-test.yaml │ │ ├── 4-smoke-tests-mq.yaml │ │ ├── 4-smoke-tests.yaml │ │ ├── ibm-build-tag-push-v2-6-13.yaml │ │ ├── ibm-helm-release-v2-6-13.yaml │ │ ├── ibm-img-release-v2-6-13.yaml │ │ ├── ibm-img-scan-v2-6-13.yaml │ │ ├── ibm-java-maven-test-v2-6-13.yaml │ │ ├── ibm-setup-v2-6-13.yaml │ │ ├── ibm-tag-release-v2-6-13.yaml │ │ └── mq-metrics-build-tag-push.yaml │ ├── triggerbindings │ │ └── cntk-binding.yaml │ └── triggertemplates │ ├── mq-qm-dev-triggertemplate.sh │ ├── mq-qm-dev-triggertemplate.yaml_template │ ├── mq-qm-dev.yaml │ ├── mq-qm-stage-triggertemplate.sh │ ├── mq-qm-stage-triggertemplate.yaml_template │ ├── mq-spring-app-dev-triggertemplate.sh │ ├── mq-spring-app-dev-triggertemplate.yaml_template │ ├── mq-spring-app-dev.yaml │ ├── mq-spring-app-stage-triggertemplate.sh │ └── mq-spring-app-stage-triggertemplate.yaml_template ├── dev │ ├── kustomization.yaml │ ├── mq-metric-samples │ │ ├── components │ │ ├── configmap │ │ ├── kustomization.yaml │ │ └── mq-metric-samples │ ├── mq-qm01 │ │ ├── certificates │ │ ├── components │ │ ├── configmap │ │ ├── kustomization.yaml │ │ ├── queuemanager │ │ └── secrets │ └── mq-spring-app │ ├── certificates │ ├── components │ ├── configmap │ ├── deployment │ ├── kustomization.yaml │ └── secrets ├── prod │ ├── kustomization.yaml │ ├── mq-metric-samples │ │ ├── components │ │ ├── configmap │ │ ├── kustomization.yaml │ │ └── mq-metric-samples │ ├── mq-qm01 │ │ ├── certificates │ │ ├── components │ │ ├── configmap │ │ ├── kustomization.yaml │ │ ├── queuemanager │ │ └── secrets │ └── mq-spring-app │ ├── certificates │ ├── components │ ├── configmap │ ├── deployment │ ├── kustomization.yaml │ └── secrets ├── staging │ ├── kustomization.yaml │ ├── mq-metric-samples │ │ ├── components │ │ ├── configmap │ │ ├── kustomization.yaml │ │ └── mq-metric-samples │ ├── mq-qm01 │ │ ├── certificates │ │ ├── components │ │ ├── configmap │ │ ├── kustomization.yaml │ │ ├── queuemanager │ │ ├── roles │ │ └── secrets │ └── mq-spring-app │ ├── certificates │ ├── components │ ├── configmap │ ├── deployment │ ├── kustomization.yaml │ └── secrets └── tools ├── certificates │ └── mq-selfsigned-certificate ├── cluster-issuers │ ├── mq-selfsigned-clusterissuer │ └── selfsigned-clusterissuer └── kustomization.yaml 74 directories, 99 filesNotice how:
- There is a folder corresponding to the
dev,stageandprodinstances of the queue manager, sample spring applications and metrics sample. - Each of these folders will use Kustomize to override the
baseYAMLs.
We'll explore this folder structure in more detail as we progress through the tutorial.
- There is a folder corresponding to the
Tailor the -apps repository¶
As we've just seen, before the cloned multi-tenancy-gitops-apps can be used to store application component YAML files, it must be customized.
Once we've customized the local repository, we'll push our updates back to our repository branch on GitHub where it can be accessed by ArgoCD running in our cluster.
-
Locate the multi-tenancy-gitops-apps repository
Tip
Ensure you're in the
multi-tenancy-gitops-appsterminal window before you start to customize the-appsrepository.Issue the following command to change to your GitOps repository:
cd $GIT_ROOT cd multi-tenancy-gitops-apps -
Run the customization script
Let's customize the cloned
multi-tenancy-gitops-appsrepository with the relevant$GIT_BRANCHand$GIT_ORGvalues for our cluster.The sample GitOps repository provides the
set-git-source.shscript to make this task easy.Run script to replace the git url and branch to your git organization where you created the git repositories:
./scripts/set-git-source.shThe script will list customizations it will use and all the files that it customizes:
Setting kustomization patches to https://github.com/prod-ref-guide/multi-tenancy-gitops.git on branch master Setting kustomization patches to https://github.com/prod-ref-guide/multi-tenancy-gitops-infra.git on branch master Setting kustomization patches to https://github.com/prod-ref-guide/multi-tenancy-gitops-services.git on branch master Setting kustomization patches to https://github.com/prod-ref-guide/multi-tenancy-gitops-apps.git on branch master done replacing variables in kustomization.yaml files git commit and push changes nowYou can examine your local clone of the GitOps repository to verify these customizations. You should see lots of amended ArgoCD YAMLs that point to your GitOps repository on GitHub.
-
Add the changes to a git index, ready to push to GitHub
We've now customized our local clone of the
multi-tenancy-gitops-appsrepository. Let's commit these changes and make the customized repository available to the cluster via GitHub.Add all changes in the current folder to a git index:
git add . -
Commit the changes to git
Use the following command to create a commit record:
git commit -s -m "GitOps customizations for organization and cluster"See the Git commit message for the customized files:
[master a900c39] GitOps customizations for organization and cluster 36 files changed, 49 insertions(+), 49 deletions(-) -
Push changes to GitHub
Push this commit back to the branch on GitHub:
git push origin $GIT_BRANCHThe changes have now been pushed to your GitOps repository:
Enumerating objects: 139, done. Counting objects: 100% (139/139), done. Delta compression using up to 8 threads Compressing objects: 100% (76/76), done. Writing objects: 100% (87/87), 4.70 KiB | 1.57 MiB/s, done. Total 87 (delta 40), reused 0 (delta 0) remote: Resolving deltas: 100% (40/40), completed with 14 local objects. To https://github.com/tutorial-org-123/multi-tenancy-gitops-apps.git d95eca5..a900c39 master -> masterWe've now successfully pushed the customized the
-appsrepository to our GitHub organization, ready for use by Tekton and ArgoCD.
Congratulations!
You've explored and customized the multi-tenancy-gitops-apps repository that will store the application component YAMLs for your cluster.
This section of the tutorial is now complete. In the following section, we're going to use the Git repositories, pipelines and ArgoCD applications we've configured to deploy MQ applications and queue managers to the cluster.