Getting started with Django
This repository is an example of how to run a Django app on Kubernetes. It uses the Writing your first Django app Polls application (parts 1 and 2) as the example app to deploy. From here on out, we refer to this app as the 'polls' application.
What we will do ?
In order to deploy this app on Kubernetes we will need to complete the following tasks:
- Create a docker registry access token in order to permit Kubernetes to fetch the docker image we are going to build
- Declare the docker registry access token in variables that will be used in the Continous Integration/Deployment (CI/CD)
- Create a Dockerfile in order to build your Docker image
- Setup Gitlab CI in order to build automaticaly the Docker image at each git push (CI)
- Define Kubernetes manifest in order to deploy
- A PostgreSQL server with some persistent storage
- Your Django App()
- Define a URL in order to access to your application
- Setup Gitlab CI in order to define the Continuous Delivery (CD)
Step by step guide
Create Docker Registry Access Token
One of the many Gitlab feature is the ability to host docker images. In order to permit Kubernetes to fetch those images, you need to provide an access token. Please follow the following steps in order to create the access token :
Declare the docker registry access token in variables
We will reuse the previously created access token in order to reuse them in the Continuous Integration pipeline. To do that we will store them in CI variables that we will use during our CI pipeline.
This will prevent from storing credentials in the Git repository. Note that only owners of the project can access to those variables.
Create the Dockerfile
In the root directory of your project create Dockerfile file and fill it like the following :
# https://docs.docker.com/engine/reference/builder/
# FROM refer the image where we start from
# Here we will use a basic python 3 debian image
FROM python:3
# Use the RUN command to pass command to be run
# Here we update the image and add python3 virtualenv
RUN apt-get update && apt-get upgrade -y && apt-get install \
-y --no-install-recommends python3 virtualenv
# Create a virtualenv for the application dependencies.
# # If you want to use Python 2, use the -p python2.7 flag.
RUN virtualenv -p python3 /env
ENV PATH /env/bin:$PATH
# Install pip dependencies
ADD requirements.txt /app/requirements.txt
RUN /env/bin/pip install --upgrade pip && /env/bin/pip install -r /app/requirements.txt
# We add the current content of the git repo in the /app directory
ADD . /app
# We use the CMD command to start the gunicorn daemon
# when we start the container.
# Note the $PORT variable, we will need to define it when we start the container
CMD gunicorn -b :$PORT mysite.wsgiSetup Gitlab CI
At the root of your git repository create a .gitlab-ci.yaml file.
We will use a special docker image which contain the docker binary. We will attach to the running docker daemon in order to build the image.
stages:
- build
- deploy
services:
- docker:dind
variables:
DOCKER_HOST: tcp://localhost:2375
build:
image: docker:latest
stage: build
script:
- docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY
- docker build -t ${CI_REGISTRY}/${CI_PROJECT_NAME}/polls:${CI_COMMIT_SHORT_SHA} .
- docker tag ${CI_REGISTRY}/${CI_PROJECT_NAME}/polls:${CI_COMMIT_SHORT_SHA} ${CI_REGISTRY}/${CI_PROJECT_NAME}/polls:latest
- docker push -- ${CI_REGISTRY}/${CI_PROJECT_NAME}/polls
tags:
- k8sSave, commit and push; you should be abble to see your first running pipeline
Once succesfully completed, you can see the docker image in the Registrysection on the left pane.
Create manifest yaml file
Create a file `manifest.yaml` at the root directory of your git repository and fill it with the following definition.
> Keep in mind that yaml formating require that you seperate each declaration with `---` line.
* PostgreSQL Server
In order to deploy a Postgresql server we need :
- [ ] Storage
- [ ] Configuration
- [ ] Deployment
- [ ] Service
* Persistent Volume Claim
As a Docker image is immutable, you may need to define some persistent storage. In the case of a PostgreSQL container we need to persist the data of the database.
We do this using a `Persistent Volume Claim`.
> You can see that we define an `accessModes`to `ReadWriteOnce`, this mean that the Persistent Volume will only be accessed by one container.
```yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: postgres-claim
labels:
app: postgresql
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
```
* PostgreSQL secret
We are here defining the PostgreSQL basic parameters : username, password and database. This `Secret` will be reused later in `Deployments`.
> Note: the data have to be base64 encoded. This can be done online or by command line on MacOS or Linux
```yaml
apiVersion: v1
kind: Secret
metadata:
name: postgresql-credentials
type: Opaque
data:
username: cG9sbHNfdXNlcgo=
password: c2xsb3BfYzNiaQo=
database: cG9sbHMK
```
* PostgreSQL Deployment
```yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: postgresql
labels:
app: postgresql
spec:
strategy:
type: Recreate
template:
metadata:
labels:
app: postgresql
tier: postgreSQL
spec:
containers:
- image: postgres:9.6.2-alpine
name: postgresql
env:
- name: POSTGRES_USER
valueFrom:
secretKeyRef:
name: postgresql-credentials
key: username
- name: POSTGRES_DB
valueFrom:
secretKeyRef:
name: postgresql-credentials
key: database
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: postgresql-credentials
key: password
ports:
- containerPort: 5432
name: postgresql
volumeMounts:
- name: postgresql
mountPath: /var/lib/postgresql/data
subPath: data
volumes:
- name: postgresql
persistentVolumeClaim:
claimName: postgres-claim
- name: postgresql-credentials
secret:
secretName: postgresql
```
* PostgreSQL Service
```yaml
apiVersion: v1
kind: Service
metadata:
name: postgresql
labels:
app: postgresql
spec:
ports:
- port: 5432
selector:
app: postgresql
tier: postgreSQL
````
* Django Application
* Deployment
```yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: polls
labels:
app: polls
spec:
replicas: 3
template:
metadata:
labels:
app: polls
spec:
containers:
name: polls-app
image: ${CI_REGISTRY}/${CI_PROJECT_NAME}/polls:${CI_COMMIT_SHORT_SHA}
# This setting makes nodes pull the docker image every time before
# starting the pod. This is useful when debugging, but should be turned
# off in production.
imagePullPolicy: Always
env:
- name: DATABASE_NAME
valueFrom:
secretKeyRef:
name: postgresql-credentials
key: database
- name: DATABASE_USER
valueFrom:
secretKeyRef:
name: postgresql-credentials
key: username
- name: DATABASE_PASSWORD
valueFrom:
secretKeyRef:
name: postgresql-credentials
key: password
ports:
- containerPort: 8080
volumes:
- name: postgresql-credentials
secret:
secretName: postgresql
```
* Service
```yaml
apiVersion: v1
kind: Service
metadata:
name: polls
labels:
app: polls
spec:
type: ClusterIP
ports:
- port: 80
targetPort: 8080
selector:
app: polls
```
* Ingress Resource
```yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: traefik
labels:
app: polls
name: polls
spec:
rules:
- host: ${CI_PROJECT_NAME}.pasteur.cloud
http:
paths:
- backend:
serviceName: polls
servicePort: 80
path: /
```
* Kubernetes Job
We will use a `Job` in order to manage django migrations.
> Note: Kubernetes jobs are run only once opposed to `Deployments` that run continiously.
```yaml
apiVersion: batch/v1
kind: Job
metadata:
name: polls-migrations
spec:
template:
spec:
containers:
- name: django
image: ${CI_REGISTRY}/${CI_PROJECT_NAME}/polls:${CI_COMMIT_SHORT_SHA}
command: ['python', 'manage.py', 'migrate']
env:
- name: DATABASE_NAME
valueFrom:
secretKeyRef:
name: postgresql-credentials
key: database
- name: DATABASE_USER
valueFrom:
secretKeyRef:
name: postgresql-credentials
key: username
- name: DATABASE_PASSWORD
valueFrom:
secretKeyRef:
name: postgresql-credentials
key: password
restartPolicy: Never
backoffLimit: 5
```Setup Continuous Delivery in Gitlab CI
deploy:
stage: deploy
image: registry-gitlab.pasteur.fr/dsi-tools/docker-images:docker_kubernetes_image
variables:
NAMESPACE: "mynamespace"
environment:
name: mynamespace
url: https://mynamespace.pasteur.cloud
script:
- kubectl delete secret registry-gitlab -n ${NAMESPACE} --ignore-not-found=true
- kubectl create secret docker-registry -n ${NAMESPACE} registry-gitlab --docker-server=registry-gitlab.pasteur.fr --docker-username=${DEPLOY_USER} --docker-password=${DEPLOY_TOKEN} --docker-email=kubernetes@pasteur.fr
- envsubst < polls.yaml | kubectl apply -f -
- kubectl patch deployment polls -p "{\"spec\":{\"template\":{\"metadata\":{\"labels\":{\"date\":\"`date +'%s'`\"}}}}}"
tags:
- k8s