Db2u on Native Kubernetes!!

Db2u is here as a technical preview! I’ll cover the basics of how to set it up for a local sandbox, and leave the details and whys to future articles. IBM has stated that db2u will be available in GA later this year on Amazon EKS and Azure EKS, but meanwhile, I’ve got the technical preview up and running on my MacBook Pro!

The tutorial for the technical preview was published by Baheer Kamal, an IBMer, on Medium in January. I’d recommend reading it in detail for all of the details. He understands all of this on a much deeper level than I do. He’s also sometimes on the Db2 discord if you have feedback. There are a few typos in the Medium article – notably a missing namespace in a couple of places and a missing dash that might trip you up if you’re not familiar with Kubernetes and associated technologies.

Setting up db2u locally

I’m sharing the steps that I used to run db2u on my computer. Note that this was done on a Mac, and also that this is an old enough Mac that it does NOT have an M1 chip. I am 99% sure that this will fail on an M1. I and others are pushing IBM to get working on the M1 architecture, but it has not made their list of priorities yet. Please be vocal with IBM if this is something that you also want.

Kubernetes

First, note that, whatever you call the Kubernetes that comes with Docker, is not enough. You’ll need MiniKube to follow the directions provided by the tutorial. I also know someone who got it working on k3d.

MiniKube

MiniKube is easy to work with. If you go to their website, it will give you the exact command you need to download and install it based on the computer you’re working from. Here are the commands I used, along with the output I saw:

$ curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-darwin-amd64
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 66.6M  100 66.6M    0     0  13.8M      0  0:00:04  0:00:04 --:--:-- 14.8M
$ sudo install minikube-darwin-amd64 /usr/local/bin/minikube
Password:
$ minikube start
😄  minikube v1.25.1 on Darwin 10.15.6
✨  Automatically selected the docker driver. Other choices: hyperkit, ssh
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
💾  Downloading Kubernetes v1.23.1 preload ...
    > preloaded-images-k8s-v16-v1...: 504.42 MiB / 504.42 MiB  100.00% 8.72 MiB
    > gcr.io/k8s-minikube/kicbase: 378.98 MiB / 378.98 MiB  100.00% 6.49 MiB p/
🔥  Creating docker container (CPUs=2, Memory=2200MB) ...
🐳  Preparing Kubernetes v1.23.1 on Docker 20.10.12 ...
    ▪ kubelet.housekeeping-interval=5m
    ▪ Generating certificates and keys ...
    ▪ Booting up control plane ...
    ▪ Configuring RBAC rules ...
🔎  Verifying Kubernetes components...
    ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟  Enabled addons: storage-provisioner, default-storageclass

❗  /usr/local/bin/kubectl is version 1.21.2, which may have incompatibilites with Kubernetes 1.23.1.
    ▪ Want kubectl v1.23.1? Try 'minikube kubectl -- get pods -A'
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default

After that, I added this alias to my .bash_profile:

alias kubectl="minikube kubectl --"

This works for me because I don’t use kubectl locally, but use it through the Rancher CLI.

OLM

Next, you need to download and install the Operator Lifecycle Manager (OLM). My understanding of all the moving pieces here is still developing, but as I understand it, this provides the framework needed for the db2 operator. That looks something like this:

$ curl -sL https://github.com/operator-framework/operator-lifecycle-manager/releases/download/v0.20.0/install.sh | bash -s v0.20.0
customresourcedefinition.apiextensions.k8s.io/catalogsources.operators.coreos.com created
customresourcedefinition.apiextensions.k8s.io/clusterserviceversions.operators.coreos.com created
customresourcedefinition.apiextensions.k8s.io/installplans.operators.coreos.com created
customresourcedefinition.apiextensions.k8s.io/olmconfigs.operators.coreos.com created
customresourcedefinition.apiextensions.k8s.io/operatorconditions.operators.coreos.com created
customresourcedefinition.apiextensions.k8s.io/operatorgroups.operators.coreos.com created
customresourcedefinition.apiextensions.k8s.io/operators.operators.coreos.com created
customresourcedefinition.apiextensions.k8s.io/subscriptions.operators.coreos.com created
customresourcedefinition.apiextensions.k8s.io/catalogsources.operators.coreos.com condition met
customresourcedefinition.apiextensions.k8s.io/clusterserviceversions.operators.coreos.com condition met
customresourcedefinition.apiextensions.k8s.io/installplans.operators.coreos.com condition met
customresourcedefinition.apiextensions.k8s.io/olmconfigs.operators.coreos.com condition met
customresourcedefinition.apiextensions.k8s.io/operatorconditions.operators.coreos.com condition met
customresourcedefinition.apiextensions.k8s.io/operatorgroups.operators.coreos.com condition met
customresourcedefinition.apiextensions.k8s.io/operators.operators.coreos.com condition met
customresourcedefinition.apiextensions.k8s.io/subscriptions.operators.coreos.com condition met
namespace/olm created
namespace/operators created
serviceaccount/olm-operator-serviceaccount created
clusterrole.rbac.authorization.k8s.io/system:controller:operator-lifecycle-manager created
clusterrolebinding.rbac.authorization.k8s.io/olm-operator-binding-olm created
olmconfig.operators.coreos.com/cluster created
deployment.apps/olm-operator created
deployment.apps/catalog-operator created
clusterrole.rbac.authorization.k8s.io/aggregate-olm-edit created
clusterrole.rbac.authorization.k8s.io/aggregate-olm-view created
operatorgroup.operators.coreos.com/global-operators created
operatorgroup.operators.coreos.com/olm-operators created
clusterserviceversion.operators.coreos.com/packageserver created
catalogsource.operators.coreos.com/operatorhubio-catalog created
Waiting for deployment "olm-operator" rollout to finish: 0 of 1 updated replicas are available...
deployment "olm-operator" successfully rolled out
deployment "catalog-operator" successfully rolled out
Package server phase: Installing
Package server phase: Succeeded
deployment "packageserver" successfully rolled out

The output there tells me it is also creating a namespace and other needed objects. I need to invest some time here to understand all of the components.

Next, we use YAML to pull and create the db2u operator. Here, I’m pasting in the YAML, so multiple lines of text. There aren’t really changes or tweaks to make here as I understand it.

$ kubectl create -f - << EOF
> ---
> apiVersion: v1
> kind: Namespace
> metadata:
>   name: db2u
> ---
> apiVersion: operators.coreos.com/v1alpha1
> kind: CatalogSource
> metadata:
>   name: db2u-catalog
>   namespace: olm
> spec:
>   sourceType: grpc
>   image: icr.io/db2u/db2-catalog@sha256:e6ac7746ae2d6d3bb4fbf75144d94e400db887c1702b19e08b9533752d896178
>   displayName: Db2u Operators
>   publisher: IBM Db2
> ---
> apiVersion: operators.coreos.com/v1alpha2
> kind: OperatorGroup
> metadata:
>   name: db2u-oprator-og
>   namespace: db2u
> spec:
>   targetNamespaces:
>   - db2u
> ---
> apiVersion: operators.coreos.com/v1alpha1
> kind: Subscription
> metadata:
>   name: db2u-operator-v2-0-0-sub
>   namespace: db2u
> spec:
>   name: db2u-operator
>   source: db2u-catalog
>   sourceNamespace: olm
>   installPlanApproval: Automatic
> EOF
namespace/db2u created
catalogsource.operators.coreos.com/db2u-catalog created
operatorgroup.operators.coreos.com/db2u-oprator-og created
subscription.operators.coreos.com/db2u-operator-v2-0-0-sub created

I can see here that it is creating more resources for me. This will take a bit to get fully created and ready. Wait until the below command shows both a “STATUS” of Running and “READY” of 1/1:

$ while true; do sleep 30; kubectl get -n db2u pod; done
No resources found in db2u namespace.
NAME                                    READY   STATUS              RESTARTS   AGE
db2u-operator-manager-6f5b9b496-56hd7   0/1     ContainerCreating   0          10s
NAME                                    READY   STATUS    RESTARTS   AGE
db2u-operator-manager-6f5b9b496-56hd7   0/1     Running   0          40s
NAME                                    READY   STATUS    RESTARTS   AGE
db2u-operator-manager-6f5b9b496-56hd7   1/1     Running   0          71s

You can see that this took approximately 71 seconds or a bit less for me.

Running the Db2 Image (Creating Pod, Container, Instance and Database)

Now we’re getting to the parts that are usually the DBA’s responsibility when building traditional systems. The next thing to do is to create a CRD in YAML that defines what we want created and how we want it created. This is also where I expect to have to make changes to configure my system a bit differently. I haven’t done those experiments yet, but I think this is where I’ll be doing it. I’ll first provide the command as I used it, and then provide the results of running it separately for clarity.

kubectl create -f - << EOF
apiVersion: db2u.databases.ibm.com/v1
kind: Db2uCluster
metadata:
  name: demo
  namespace: db2u
spec:
  license:
    accept: true
  version: "11.5.7.0"
  size: 1
  environment:
    dbType: db2oltp
    database:
      name: bludb
  storage:
    - name: meta
      type: "create"
      spec:
        storageClassName: "standard"
        accessModes:
          - ReadWriteMany
        resources:
          requests:
            storage: 10Gi
    - name: data
      type: "create"
      spec:
        storageClassName: "standard"
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 20Gi
EOF

If you've worked with Db2, I'm sure you can see things here that might be interesting to change. I know there are different types for oltp and wh (BLU - maybe DPF?) databases. I could change the database name here, and if I weren't just starting to experiment, I would - BLUDB doesn't make sense for an OLTP database. Also note that there are a couple of different storage resources here - one that is allowing access by multiple containers and one that is not. I'd speculate that shared area is something that is more used by a warehousing implementation, or by HADR if you set that up. I must resist the urge to dig into the details on HADR and how to deal with topics like anti-affinity that we'd need to run that properly on Kubernetes.

Here's what it looks like when I ran that:

$ kubectl create -f - << EOF
> apiVersion: db2u.databases.ibm.com/v1
> kind: Db2uCluster
> metadata:
>   name: demo
>   namespace: db2u
> spec:
>   license:
>     accept: true
>   version: "11.5.7.0"
>   size: 1
>   environment:
>     dbType: db2oltp
>     database:
>       name: bludb
>   storage:
>     - name: meta
>       type: "create"
>       spec:
>         storageClassName: "standard"
>         accessModes:
>           - ReadWriteMany
>         resources:
>           requests:
>             storage: 10Gi
>     - name: data
>       type: "create"
>       spec:
>         storageClassName: "standard"
>         accessModes:
>           - ReadWriteOnce
>         resources:
>           requests:
>             storage: 20Gi
> EOF
db2ucluster.db2u.databases.ibm.com/demo created

This part takes even longer to spin up. I'd assume that is because this is installing db2 and creating an instance and all of that. To watch the progress of this use a command like the one below. You're looking for the status of Completed on the pod with "restore-morph" in its name.

$ while true; do sleep 30; kubectl get po -n db2u; done
NAME                                    READY   STATUS    RESTARTS   AGE
db2u-operator-manager-6f5b9b496-xwwr8   1/1     Running   0          6m57s
NAME                                    READY   STATUS              RESTARTS   AGE
c-demo-instdb-fqsl8                     0/1     ContainerCreating   0          7s
c-demo-ldap-6b9dcd85b5-6qs6x            0/1     ContainerCreating   0          7s
db2u-operator-manager-6f5b9b496-xwwr8   1/1     Running             0          7m27s
NAME                                    READY   STATUS              RESTARTS   AGE
c-demo-instdb-fqsl8                     0/1     ContainerCreating   0          37s
c-demo-ldap-6b9dcd85b5-6qs6x            0/1     ContainerCreating   0          37s
db2u-operator-manager-6f5b9b496-xwwr8   1/1     Running             0          7m57s
NAME                                    READY   STATUS              RESTARTS   AGE
c-demo-etcd-0                           0/1     ContainerCreating   0          8s
c-demo-instdb-fqsl8                     0/1     Completed           0          68s
c-demo-ldap-6b9dcd85b5-6qs6x            0/1     Running             0          68s
db2u-operator-manager-6f5b9b496-xwwr8   1/1     Running             0          8m28s
NAME                                    READY   STATUS      RESTARTS   AGE
c-demo-db2u-0                           0/1     Init:0/2    0          8s
c-demo-etcd-0                           0/1     Running     0          38s
c-demo-instdb-fqsl8                     0/1     Completed   0          98s
c-demo-ldap-6b9dcd85b5-6qs6x            1/1     Running     0          98s
db2u-operator-manager-6f5b9b496-xwwr8   1/1     Running     0          8m58s
NAME                                    READY   STATUS            RESTARTS   AGE
c-demo-db2u-0                           0/1     PodInitializing   0          38s
c-demo-etcd-0                           1/1     Running           0          68s
c-demo-instdb-fqsl8                     0/1     Completed         0          2m8s
c-demo-ldap-6b9dcd85b5-6qs6x            1/1     Running           0          2m8s
db2u-operator-manager-6f5b9b496-xwwr8   1/1     Running           0          9m28s
NAME                                    READY   STATUS            RESTARTS   AGE
c-demo-db2u-0                           0/1     PodInitializing   0          68s
c-demo-etcd-0                           1/1     Running           0          98s
c-demo-instdb-fqsl8                     0/1     Completed         0          2m38s
c-demo-ldap-6b9dcd85b5-6qs6x            1/1     Running           0          2m38s
db2u-operator-manager-6f5b9b496-xwwr8   1/1     Running           0          9m58s
NAME                                    READY   STATUS            RESTARTS   AGE
c-demo-db2u-0                           0/1     PodInitializing   0          99s
c-demo-etcd-0                           1/1     Running           0          2m9s
c-demo-instdb-fqsl8                     0/1     Completed         0          3m9s
c-demo-ldap-6b9dcd85b5-6qs6x            1/1     Running           0          3m9s
db2u-operator-manager-6f5b9b496-xwwr8   1/1     Running           0          10m
NAME                                    READY   STATUS            RESTARTS   AGE
c-demo-db2u-0                           0/1     PodInitializing   0          2m9s
c-demo-etcd-0                           1/1     Running           0          2m39s
c-demo-instdb-fqsl8                     0/1     Completed         0          3m39s
c-demo-ldap-6b9dcd85b5-6qs6x            1/1     Running           0          3m39s
db2u-operator-manager-6f5b9b496-xwwr8   1/1     Running           0          10m
NAME                                    READY   STATUS            RESTARTS   AGE
c-demo-db2u-0                           0/1     PodInitializing   0          2m39s
c-demo-etcd-0                           1/1     Running           0          3m9s
c-demo-instdb-fqsl8                     0/1     Completed         0          4m9s
c-demo-ldap-6b9dcd85b5-6qs6x            1/1     Running           0          4m9s
db2u-operator-manager-6f5b9b496-xwwr8   1/1     Running           0          11m
NAME                                    READY   STATUS      RESTARTS   AGE
c-demo-db2u-0                           0/1     Running     0          3m10s
c-demo-etcd-0                           1/1     Running     0          3m40s
c-demo-instdb-fqsl8                     0/1     Completed   0          4m40s
c-demo-ldap-6b9dcd85b5-6qs6x            1/1     Running     0          4m40s
db2u-operator-manager-6f5b9b496-xwwr8   1/1     Running     0          12m
NAME                                    READY   STATUS      RESTARTS   AGE
c-demo-db2u-0                           1/1     Running     0          3m40s
c-demo-etcd-0                           1/1     Running     0          4m10s
c-demo-instdb-fqsl8                     0/1     Completed   0          5m10s
c-demo-ldap-6b9dcd85b5-6qs6x            1/1     Running     0          5m10s
c-demo-restore-morph-r788c              1/1     Running     0          17s
db2u-operator-manager-6f5b9b496-xwwr8   1/1     Running     0          12m
NAME                                    READY   STATUS      RESTARTS   AGE
c-demo-db2u-0                           1/1     Running     0          4m10s
c-demo-etcd-0                           1/1     Running     0          4m40s
c-demo-instdb-fqsl8                     0/1     Completed   0          5m40s
c-demo-ldap-6b9dcd85b5-6qs6x            1/1     Running     0          5m40s
c-demo-restore-morph-r788c              1/1     Running     0          47s
db2u-operator-manager-6f5b9b496-xwwr8   1/1     Running     0          13m
NAME                                    READY   STATUS      RESTARTS   AGE
c-demo-db2u-0                           1/1     Running     0          4m40s
c-demo-etcd-0                           1/1     Running     0          5m10s
c-demo-instdb-fqsl8                     0/1     Completed   0          6m10s
c-demo-ldap-6b9dcd85b5-6qs6x            1/1     Running     0          6m10s
c-demo-restore-morph-r788c              1/1     Running     0          77s
db2u-operator-manager-6f5b9b496-xwwr8   1/1     Running     0          13m
NAME                                    READY   STATUS      RESTARTS   AGE
c-demo-db2u-0                           1/1     Running     0          5m10s
c-demo-etcd-0                           1/1     Running     0          5m40s
c-demo-instdb-fqsl8                     0/1     Completed   0          6m40s
c-demo-ldap-6b9dcd85b5-6qs6x            1/1     Running     0          6m40s
c-demo-restore-morph-r788c              0/1     Completed   0          107s
db2u-operator-manager-6f5b9b496-xwwr8   1/1     Running     0          14m

In my case, this took about 7-8 minutes to get through everything.

You now have a usable database up and running on db2u!

Using db2u

There are some details you'll need to get in order to actually connect to the database.

Port

First, depending on how you're using Db2, you may need to know the nodePort assigned. Kubernetes assigns this for you, and I don't know if it's possible to change or control it at all. The command to find it is:

$ kubectl get svc c-demo-db2u-engn-svc -n db2u -o jsonpath='{.spec.ports[?(@.name=="legacy-server")].nodePort}{"\n"}'
30161

This was one of the places in the Medium article where the namespace was missing from the command. If you don't add the namespace, as I have above, you will get an error that looks like this:

$ kubectl get svc c-demo-db2u-engn-svc -o jsonpath='{.spec.ports[?(@.name=="legacy-server")].nodePort}{"\n"}'
Error from server (NotFound): services "c-demo-db2u-engn-svc" not found

Setting up Network Connectivity

This command, you'll have to abandon the session you start it from. It runs in the foreground and not the background. This may be one of the disadvantages of using MiniKube. I know someone who got this working with k3d specifically so he didn't have to leave something running in the foreground to be able to connect. The warning message does indicate this may have something to do with running it on Mac. Also note that the medium article had a minor syntax error in this command, and I've corrected it below.

$ minikube service --url c-demo-db2u-engn-svc -n db2u
🏃  Starting tunnel for service c-demo-db2u-engn-svc.
|-----------|----------------------|-------------|------------------------|
| NAMESPACE |         NAME         | TARGET PORT |          URL           |
|-----------|----------------------|-------------|------------------------|
| db2u      | c-demo-db2u-engn-svc |             | http://127.0.0.1:59145 |
|           |                      |             | http://127.0.0.1:59146 |
|-----------|----------------------|-------------|------------------------|
http://127.0.0.1:59145
http://127.0.0.1:59146
❗  Because you are using a Docker driver on darwin, the terminal needs to be open to run it.

Password

Finally, a password is created for db2inst1 in the above process. You can change the CRD to set a password, and in a real implementation, I'd want that to be passed in as a secret from somewhere. In this case we can extract that password. This is another command that was missing the namespace in the Medium article and I've added it.

$ echo $(kubectl get secret c-demo-instancepassword -n db2u -o jsonpath={.data.password} | base64 -d)
YSFOgoNlb9Gauij

I do find it interesting that the password can be extracted in this manner.

Connecting

Now you have all the information you need to connect to the database. Here's a simple python script I used to test:

import ibm_db
import ibm_db_dbi
conn_str='database=bludb;hostname=localhost;port=59145;protocol=tcpip;uid=db2inst1;pwd=YSFOgoNlb9Gauij'
ibm_db_conn = ibm_db.connect(conn_str,'','')
#Drop the table if exist
drop_table="drop table if exists mytable"
ibm_db.exec_immediate(ibm_db_conn, drop_table)
 
#Create table
create="create table mytable(id int, name varchar(50))"
ibm_db.exec_immediate(ibm_db_conn, create)
 
#Insert data into table mytable using prepared statement
insert = "insert into mytable values(?,?)"
params=((1,'Sanders'),(2,'Pernal'),(3,'OBrien'))
stmt_insert = ibm_db.prepare(ibm_db_conn, insert)
ibm_db.execute_many(stmt_insert,params)
# Fetch data
select="select id, name from mytable"
stmt_select = ibm_db.exec_immediate(ibm_db_conn, select)
for x in range(3):
 cols = ibm_db.fetch_tuple( stmt_select )
 print("%s, %s" % (cols[0], cols[1]))
ibm_db.close(ibm_db_conn)

One thing to note here - the port number I'm using is not the NodePort, but is the value supplied from the MiniKube service command. The NodePort may be more critical to know in other scenarios.

Documentation

Since db2u outside of OpenShift is still in technical preview, the complete documentation does not yet exist. I'm assured it will when they go GA with it later this year. For now, check out the db2u documentation for OpenShift to find some details you might need.

Summary

I'm excited for working more with db2u. This is just the beginning, but I wanted to share my progress so far. Let me know your successes and failures!