Accessing postgres running on your local machine from Minikube cluster

I was looking to containerize my application running in Wildfly  and Tomcat connecting to Postgres database. For experimenting with this I installed Minikube on my CentOS 7 Laptop.

I did not want to move my postgres database to Kubernetes cluster because of ephemeral nature of containers. I wanted to move my Wildfly and Tomcat instances to Kubernetes and connect to the database running on my local machine from these containers. In my production environment I plan to use AWS Postgres service.

Connecting to postgres server on my local machine was not straight forward, but once you know how to do it, it is an easy process. Here I am describing what I learned in this process.

Running postgres service on IP address interfaced with Minikube

Minikube runs inside a docker container. Docker creates many virtual networks on your machine. These interfaces are like multiple LAN cables connected to your computer and other end of these LAN cables it connected to different containers. Your computer have multiple IP addresses assigned to it. One for each interface. When you try to connect back from container you need to connect on the IP address of your computer assigned by the network on which your container is hosted. 

You can run following command to list out all interfaces attached to your computer.

ip addr

On my computer it returned the following output.

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: enp1s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast state DOWN group default qlen 1000
    link/ether 70:5a:0f:2a:78:76 brd ff:ff:ff:ff:ff:ff
3: wlp2s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 44:85:00:58:b2:a9 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.7/24 brd 192.168.1.255 scope global noprefixroute dynamic wlp2s0
       valid_lft 61564sec preferred_lft 61564sec
    inet6 fe80::c8fe:901b:5a99:57ca/64 scope link noprefixroute 
       valid_lft forever preferred_lft forever
4: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
    link/ether 52:54:00:d1:46:6b brd ff:ff:ff:ff:ff:ff
    inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
       valid_lft forever preferred_lft forever
5: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master virbr0 state DOWN group default qlen 1000
    link/ether 52:54:00:d1:46:6b brd ff:ff:ff:ff:ff:ff
7: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    link/ether 02:42:28:3e:b3:9e brd ff:ff:ff:ff:ff:ff
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
    inet6 fe80::42:28ff:fe3e:b39e/64 scope link 
       valid_lft forever preferred_lft forever
8: br-fdf8159cfe40: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default 
    link/ether 02:42:22:25:29:3e brd ff:ff:ff:ff:ff:ff
    inet 192.168.49.1/24 brd 192.168.49.255 scope global br-fdf8159cfe40
       valid_lft forever preferred_lft forever
    inet6 fe80::42:22ff:fe25:293e/64 scope link 
       valid_lft forever preferred_lft forever
14: veth683e46d@if13: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-fdf8159cfe40 state UP group default 
    link/ether 1e:b5:dd:3f:2c:87 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet6 fe80::1cb5:ddff:fe3f:2c87/64 scope link 
       valid_lft forever preferred_lft forever

Now run following command to find IP address of Minikube container.

minikube ip

On my local machine it returned following IP address:

192.168.49.2

Now you can analyze output of  "ip addr" and find out that interface 8 is connected to Minikube and IP address for virtual inteface connected to that network is 192.168.49.1

For your postgres instance to be connectable from Minikube containers it need to listen at 192.168.49.1

You need to modify /var/lib/pgsql/13/data/postgresql.conf file to include 192.168.49.1 in listen addresses. My file reads as follows:

listen_addresses = 'localhost,192.168.49.1'             # what IP address(es) to listen on;

Now you need to modify /var/lib/pgsql/13/data/pg_hba.conf file to allow access form Minikube network. My file reads like this:

# TYPE  DATABASE        USER            CIDR-ADDRESS            METHOD

# "local" is for Unix domain socket connections only
local   all             all                                     trust
#local   all         postgres                             password
# IPv4 local connections:
host    all             all             127.0.0.1/32           password
# IPv6 local connections:
host    all             all             ::1/128                password 
#localhost and kubernetes connection
host    all             all             192.168.49.0/24         password
local   all             dashboard                              password

Now restart postgres service using following command:

sudo systemctl restart postgresql-13

Now you are ready to connect to your local postgres server from Minikube cluster containers. You need to use 192.168.49.1 as host IP address for connecting to your local postgres server.

Testing connectivity to local postgres instance from Minikube cluster

For testing connectivity to your local postgres server from Minikube cluster you need to run a postgres image on Minikube cluster. This image contains "psql" command which can be used to connect to any remote postgres server. We will use this command to connect to our local postgres server from a pod running postgres image to confirm that connectivity is established.

Now run following command to run postgres image on one pod in Minikube cluster.

kubectl run postgres --image=postgres --env="POSTGRES_PASSWORD=mysecretpassword"

Now get a list of running pods using following command:

kubectl get pods

On my computer it returned following output:

NAME                                     READY   STATUS    RESTARTS   AGE
balanced-5744b548b4-fnvff                1/1     Running   3          13d
hello-minikube-6ddfcc9757-vfpcj          1/1     Running   3          13d
hello-node-7567d9fdc9-bpcfd              1/1     Running   3          13d
kubernetes-bootcamp-769746fd4-q65cm      1/1     Running   3          13d
kubernetes-bootcamp-769746fd4-tv2bd      1/1     Running   3          13d
postgres                                 1/1     Running   0          2m44s
tomcat-jas-deployment-85f8f68b99-jk6zr   2/2     Running   0          3h46m

I can see that my pod is running. Now run following command to connect to postgres pod and open a terminal window for this pod.

kubectl exec postgres -t -i -- /bin/bash

Transcript for my session is as follows:

localhost.localdomain: ~/docker-files/tomcat-jas >kubectl exec postgres -t -i -- /bin/bash
root@postgres:/# psql -h 192.168.49.1 -U postgres
Password for user postgres: 
psql (13.3 (Debian 13.3-1.pgdg100+1))
Type "help" for help.

Password used here is password for my local postgres server instance. I can access to all my tables from this postgre prompt.

Initially I faced connectivity problems and they were resolved after running following command:

sudo iptables --flush

Use the above command only if you need it. 

Please let me know in comments below if you face any problem in following this post.