Skip to content
Snippets Groups Projects
Select Git revision
  • 38d91d9e79e4d3a1e24501fdc46ebb9506db79dc
  • master default protected
2 results

python-django-template

  • Clone with SSH
  • Clone with HTTPS
  • Getting started with Django

    This repository is an example of how to run a Django app on Kubernetes. It uses the Writing your first Django app Polls application (parts 1 and 2) as the example app to deploy. From here on out, we refer to this app as the 'polls' application.

    What we will do ?

    In order to deploy this app on Kubernetes we will need to complete the following tasks:

    • Create a docker registry access token in order to permit Kubernetes to fetch the docker image we are going to build
    • Declare the docker registry access token in variables that will be used in the Continous Integration/Deployment (CI/CD)
    • Create a Dockerfile in order to build your Docker image
    • Setup Gitlab CI in order to build automaticaly the Docker image at each git push (CI)
    • Define Kubernetes manifest in order to deploy
      • A PostgreSQL server with some persistent storage
      • Your Django App()
      • Define a URL in order to access to your application
    • Setup Gitlab CI in order to define the Continuous Delivery (CD)

    Step by step guide

    Create Docker Registry Access Token

    One of the many Gitlab feature is the ability to host docker images. In order to permit Kubernetes to fetch those images, you need to provide an access token. Please follow the following steps in order to create the access token :

    alt text

    alt text

    alt text

    Declare the docker registry access token in variables

    We will reuse the previously created access token in order to reuse them in the Continuous Integration pipeline. To do that we will store them in CI variables that we will use during our CI pipeline.

    This will prevent from storing credentials in the Git repository. Note that only owners of the project can access to those variables.

    alt text

    alt text

    alt text

    Create the Dockerfile

    In the root directory of your project create Dockerfile file and fill it like the following :

    # https://docs.docker.com/engine/reference/builder/
    # FROM refer the image where we start from
    # Here we will use a basic python 3 debian image
    
    FROM python:3
    
    # Use the RUN command to pass command to be run
    # Here we update the image and add python3 virtualenv
    
    RUN apt-get update && apt-get upgrade -y && apt-get install \
      -y --no-install-recommends python3 virtualenv
    
    # Create a virtualenv for the application dependencies.
    # # If you want to use Python 2, use the -p python2.7 flag.
    RUN virtualenv -p python3 /env
    ENV PATH /env/bin:$PATH
    
    # Install pip dependencies
    ADD requirements.txt /app/requirements.txt
    RUN /env/bin/pip install --upgrade pip && /env/bin/pip install -r /app/requirements.txt
    
    # We add the current content of the git repo in the /app directory
    ADD . /app
    
    # We use the CMD command to start the gunicorn daemon
    # when we start the container.
    # Note the $PORT variable, we will need to define it when we start the container
    CMD gunicorn -b :$PORT mysite.wsgi

    Setup Gitlab CI

    At the root of your git repository create a .gitlab-ci.yaml file.

    We will use a special docker image which contain the docker binary. We will attach to the running docker daemon in order to build the image.

    stages:
      - build
      - deploy
    
    services:
      - docker:dind
    
    variables:
      DOCKER_HOST: tcp://localhost:2375
    
    build:
      image: docker:latest
      stage: build
      script:
        - docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY
        - docker build -t ${CI_REGISTRY}/${CI_PROJECT_NAME}/polls:${CI_COMMIT_SHORT_SHA} .
        - docker tag ${CI_REGISTRY}/${CI_PROJECT_NAME}/polls:${CI_COMMIT_SHORT_SHA} ${CI_REGISTRY}/${CI_PROJECT_NAME}/polls:latest
        - docker push -- ${CI_REGISTRY}/${CI_PROJECT_NAME}/polls
      tags:
        - k8s

    Save, commit and push; you should be abble to see your first running pipeline

    alt text

    Once succesfully completed, you can see the docker image in the Registrysection on the left pane.

    Create manifest yaml file

    Create a file manifest.yaml at the root directory of your git repository and fill it with the following definition.

    Keep in mind that yaml formating require that you seperate each declaration with --- line.

    PostgreSQL Server

    In order to deploy a Postgresql server we need :

    • Storage
    • Configuration
    • Deployment
    • Service
    Persistent Volume Claim

    As a Docker image is immutable, you may need to define some persistent storage. In the case of a PostgreSQL container we need to persist the data of the database.

    We do this using a Persistent Volume Claim.

    You can see that we define an accessModesto ReadWriteOnce, this mean that the Persistent Volume will only be accessed by one container.

    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
    name: postgres-claim
    labels:
        app: postgresql
    spec:
    accessModes:
        - ReadWriteOnce
    resources:
        requests:
        storage: 1Gi
    PostgreSQL secret

    We are here defining the PostgreSQL basic parameters : username, password and database. This Secret will be reused later in Deployments.

    Note: the data have to be base64 encoded. This can be done online or by command line on MacOS or Linux

    alt text

    apiVersion: v1
    kind: Secret
    metadata:
    name: postgresql-credentials
    type: Opaque
    data:
    username: cG9sbHNfdXNlcgo=
    password: c2xsb3BfYzNiaQo=
    database: cG9sbHMK
    PostgreSQL Deployment
    apiVersion: extensions/v1beta1
    kind: Deployment
    metadata:
    name: postgresql
    labels:
        app: postgresql
    spec:
    strategy:
        type: Recreate
    template:
        metadata:
        labels:
            app: postgresql
            tier: postgreSQL
        spec:
        containers:
            - image: postgres:9.6.2-alpine
            name: postgresql
            env:
                - name: POSTGRES_USER
                valueFrom:
                    secretKeyRef:
                    name: postgresql-credentials
                    key: username
                - name: POSTGRES_DB
                valueFrom:
                    secretKeyRef:
                    name: postgresql-credentials
                    key: database
                - name: POSTGRES_PASSWORD
                valueFrom:
                    secretKeyRef:
                    name: postgresql-credentials
                    key: password
            ports:
                - containerPort: 5432
                name: postgresql
            volumeMounts:
                - name: postgresql
                mountPath: /var/lib/postgresql/data
                subPath: data
        volumes:
            - name: postgresql
            persistentVolumeClaim:
                claimName: postgres-claim
            - name: postgresql-credentials
            secret:
                secretName: postgresql
    PostgreSQL Service
    apiVersion: v1
    kind: Service
    metadata:
    name: postgresql
    labels:
        app: postgresql
    spec:
    ports:
        - port: 5432
    selector:
        app: postgresql
        tier: postgreSQL

    Django Application

    Deployment
    apiVersion: extensions/v1beta1
    kind: Deployment
    metadata:
    name: polls
    labels:
        app: polls
    spec:
    replicas: 3
    template:
        metadata:
        labels:
            app: polls
        spec:
        containers:
        name: polls-app
            image: ${CI_REGISTRY}/${CI_PROJECT_NAME}/polls:${CI_COMMIT_SHORT_SHA}
            # This setting makes nodes pull the docker image every time before
            # starting the pod. This is useful when debugging, but should be turned
            # off in production.
            imagePullPolicy: Always
            env:
                - name: DATABASE_NAME
                valueFrom:
                    secretKeyRef:
                    name: postgresql-credentials
                    key: database
                - name: DATABASE_USER
                valueFrom:
                    secretKeyRef:
                    name: postgresql-credentials
                    key: username
                - name: DATABASE_PASSWORD
                valueFrom:
                    secretKeyRef:
                    name: postgresql-credentials
                    key: password
            ports:
            - containerPort: 8080
        volumes:
            - name: postgresql-credentials
            secret:
                secretName: postgresql
    Service
    apiVersion: v1
    kind: Service
    metadata:
    name: polls
    labels:
        app: polls
    spec:
    type: ClusterIP
    ports:
    - port: 80
        targetPort: 8080
    selector:
        app: polls
    Ingress Resource
    apiVersion: extensions/v1beta1
    kind: Ingress
    metadata:
    annotations:
        kubernetes.io/ingress.class: traefik
    labels:
        app: polls
    name: polls
    spec:
    rules:
    - host: https://${CI_PROJECT_NAME}.k8s-dev.pasteur.fr
        http:
        paths:
        - backend:
            serviceName: polls
            servicePort: 80
            path: /
    Kubernetes Job

    We will use a Job in order to manage django migrations.

    Note: Kubernetes jobs are run only once opposed to Deployments that run continiously.

    apiVersion: batch/v1
    kind: Job
    metadata:
    name: polls-migrations
    spec:
    template:
        spec:
        containers:
            - name: django
            image: ${CI_REGISTRY}/${CI_PROJECT_NAME}/polls:${CI_COMMIT_SHORT_SHA}
            command: ['python', 'manage.py', 'migrate']
            env:
                - name: DATABASE_NAME
                valueFrom:
                    secretKeyRef:
                    name: postgresql-credentials
                    key: database
                - name: DATABASE_USER
                valueFrom:
                    secretKeyRef:
                    name: postgresql-credentials
                    key: username
                - name: DATABASE_PASSWORD
                valueFrom:
                    secretKeyRef:
                    name: postgresql-credentials
                    key: password
        restartPolicy: Never
    backoffLimit: 5

    Setup Continuous Delivery in Gitlab CI

    deploy:
      stage: deploy
      image: registry-gitlab.pasteur.fr/dsi-tools/docker-images:docker_kubernetes_image
      variables:
        NAMESPACE: "mynamespace"
      environment:
        name: mynamespace
        url: https://${CI_PROJECT_NAME}.k8s-dev.pasteur.fr
      script:
        - kubectl delete secret registry-gitlab -n ${NAMESPACE} --ignore-not-found=true
        - kubectl create secret docker-registry -n ${NAMESPACE} registry-gitlab --docker-server=registry-gitlab.pasteur.fr --docker-username=${DEPLOY_USER} --docker-password=${DEPLOY_TOKEN} --docker-email=kubernetes@pasteur.fr
        - envsubst < polls.yaml | kubectl apply -f -
        - kubectl patch deployment polls -p "{\"spec\":{\"template\":{\"metadata\":{\"labels\":{\"date\":\"`date +'%s'`\"}}}}}"
      tags:
        - k8s

    What else ?

    Kubernetes dashboard

    View your running workload

    Increase the number of replicas

    What happen if I kill a Pod ?

    Grafana : Metrics and Logs

    Best Practices

    Always use Gitlab