Skip to content
Snippets Groups Projects

Compare revisions

Changes are shown as if the source revision was being merged into the target revision. Learn more about comparing revisions.

Source

Select target project
No results found
Select Git revision
  • master
1 result

Target

Select target project
  • kubernetes-workshop/python-django-template
  • aghozlan/python-django-template
  • tru/python-django-template
3 results
Select Git revision
  • master
1 result
Show changes
Commits on Source (11)
......@@ -19,6 +19,24 @@ In order to deploy this app on Kubernetes we will need to complete the following
* Define a URL in order to access to your application
* Setup Gitlab CI in order to define the Continuous Delivery (CD)
## Pre-requisits
### On your computer
* Git ust be installed
* Nice to have : a code editor (Atom, Visual Studio code ....)
* Connected to Pasteur VPN (or you won't be able to use all the tools)
### Gitlab
If you have never used Gitlab before, you will need to add an `ssh key` in your profile or define `HTTP` password in order to fetch the git repository.
Use basic git command :
```
git add .
git commit -m 'My comment'
git push
```
## Step by step guide
### Create Docker Registry Access Token
......@@ -68,7 +86,7 @@ RUN pip install --default-timeout=100 --upgrade pip && pip install -r /app/requi
# We add the current content of the git repo in the /app directory
ADD . /app
WORKDIR /app
RUN rm -rf .gitlab-ci.yml Dockerfile README.md img
RUN rm -rf .gitlab-ci.yml Dockerfile README.md img solution
# We use the CMD command to start the gunicorn daemon
# when we start the container.
# Note the $PORT variable, we will need to define it when we start the container
......@@ -77,16 +95,26 @@ CMD python manage.py runserver 0.0.0.0:$PORT
### Setup Gitlab CI
At the root of your git repository create a `.gitlab-ci.yaml` file.
At the root of your git repository create a `.gitlab-ci.yml` file.
> Name it correctly `.gitlab-ci.yml` and not `.gitlab-ci.yaml` !
We will use a special docker image which contain the docker binary, we build a docker in a docker :-)
We will use a special docker image which contain the docker binary.
We will attach to the running docker daemon in order to build the image.
What do we do here ?
* We login on the docker registry enbended in Gitlab (a docker registry is the place where we store docker images to redistribute them)
* We build the docker image and we add a `tag`to it (actually the id of the git commit)
* We tag the previously builded docker image with a special tag `latest` (this can be sometime usefull)
* We push the image to the docker registry
```yaml
stages:
- build
- deploy
variables:
DOCKER_HOST: tcp://localhost:2375/
services:
- docker:dind
......@@ -98,26 +126,50 @@ build:
- docker build -t "$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_NAME/polls:${CI_COMMIT_SHORT_SHA}" .
- docker tag "$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_NAME/polls:${CI_COMMIT_SHORT_SHA}" "$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_NAME/polls:latest"
- docker push "$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_NAME/polls:${CI_COMMIT_SHORT_SHA}"
tags:
- k8s
```
Save, commit and push; you should be abble to see your first running pipeline
> For those not familiar with `git` you can just do the following in a terminal
```
git add .
git commit -m 'My comment'
git push
```
![alt text](img/ci_pipeline.png)
Once succesfully completed, you can see the docker image in the `Registry`section on the left pane.
### Create manifest yaml file
### Create manifests yaml files
In order to deploy your application on Kubernetes we need to define how to deploy :
* What is the source of the image ?
* Do I need parameters in order to configure my application ?
* Do I need some storage ?
* What will be it's name ?
* Do I have some "secure" information ? (login, password...)
* What would be it's dns name ?
....
So we are going to create manifest files at the root directory of your git repository and fill it with the following definition.
Create a file `manifest.yaml` at the root directory of your git repository and fill it with the following definition.
> Keep in mind that yaml formating require that you seperate each declaration with `---` line.
#### PostgreSQL Server
> Create the `postgresql.yaml` file at the root directory of your project
In order to deploy a Postgresql server we need :
- [ ] Storage
- [ ] Configuration
- [ ] Secrets (passwords....)
- [ ] Deployment
- [ ] Service
- [ ] Service (Load balancing even if it's for one Pod)
##### Persistent Volume Claim
......@@ -126,6 +178,7 @@ As a Docker image is immutable, you may need to define some persistent storage.
We do this using a `Persistent Volume Claim`.
> You can see that we define an `accessModes`to `ReadWriteOnce`, this mean that the Persistent Volume will only be accessed by one container.
> Insert this in the `postgresql.yaml` file :
```yaml
apiVersion: v1
kind: PersistentVolumeClaim
......@@ -149,6 +202,7 @@ We are here defining the PostgreSQL basic parameters : username, password and da
![alt text](img/base64.png)
> Insert this in the `postgresql.yaml` file with `---` line before
```yaml
apiVersion: v1
kind: Secret
......@@ -163,6 +217,7 @@ data:
##### PostgreSQL Deployment
> Insert this in the `postgresql.yaml` file with `---` line before
```yaml
apiVersion: apps/v1
kind: Deployment
......@@ -227,6 +282,7 @@ spec:
##### PostgreSQL Service
> Insert this in the `postgresql.yaml` file with `---` line before
```yaml
apiVersion: v1
kind: Service
......@@ -240,11 +296,14 @@ spec:
selector:
app: postgresql
tier: postgreSQL
````
```
#### Django Application
##### Deployment
> Create `polls.yaml` file
##### Deployment
> Insert this in the `polls.yaml` file
```yaml
apiVersion: apps/v1
kind: Deployment
......@@ -305,6 +364,7 @@ spec:
##### Service
> Insert this in the `polls.yaml` file with `---` line before
```yaml
apiVersion: v1
kind: Service
......@@ -322,6 +382,7 @@ spec:
##### Ingress Resource
> Insert this in the `polls.yaml` file with `---` line before
```yaml
apiVersion: extensions/v1beta1
kind: Ingress
......@@ -333,7 +394,7 @@ metadata:
name: polls
spec:
rules:
- host: ${GITLAB_USER_LOGIN}.k8s-dev.pasteur.fr
- host: ${GITLAB_USER_LOGIN}-${CI_PROJECT_NAME}.k8s-dev.pasteur.fr
http:
paths:
- backend:
......@@ -349,6 +410,7 @@ Create a file `job.yaml` at the root directory of your git repository and fill i
We will use a `Job` in order to manage django migrations.
> Note: Kubernetes jobs are run only once opposed to `Deployments` that run continiously. We put it in a seperate file because a Job is immutable and cannot be updated.
> Insert this in the `job.yaml` file
```yaml
---
apiVersion: batch/v1
......@@ -361,7 +423,8 @@ spec:
containers:
- name: django
image: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_NAME/polls:${CI_COMMIT_SHORT_SHA}
command: ['python', '/app/manage.py', 'migrate']
command: ["/bin/sh","-c"]
args: ["python manage.py makemigrations && python manage.py migrate"]
env:
- name: DATABASE_HOST
value: postgresql
......@@ -392,34 +455,82 @@ spec:
### Setup Continuous Delivery in Gitlab CI
> Insert this in the `.gitlab-ci.yml` file after the previous part (docker build)
```yaml
deploy:
stage: deploy
image: registry-gitlab.pasteur.fr/dsi-tools/docker-images:docker_kubernetes_image
variables:
NAMESPACE: "<your gitlab user>-django"
NAMESPACE: ${GITLAB_USER_LOGIN}-${CI_PROJECT_NAME}
environment:
name: <your gitlab user>-django
url: https://<your gitlab user>-django.k8s-dev.pasteur.fr
name: ${GITLAB_USER_LOGIN}-${CI_PROJECT_NAME}
url: https://${GITLAB_USER_LOGIN}-${CI_PROJECT_NAME}.k8s-dev.pasteur.fr
script:
- yum install -y gettext
- kubectl delete secret registry-gitlab -n ${NAMESPACE} --ignore-not-found=true
- kubectl create secret docker-registry -n ${NAMESPACE} registry-gitlab --docker-server=registry-gitlab.pasteur.fr --docker-username=${DEPLOY_USER} --docker-password=${DEPLOY_TOKEN} --docker-email=kubernetes@pasteur.fr
- envsubst < manifest.yaml | kubectl apply -n ${NAMESPACE} -f -
- envsubst < postgresql.yaml | kubectl apply -n ${NAMESPACE} -f -
- kubectl wait --for=condition=available --timeout=600s deployment/postgresql
- kubectl delete job polls-migrations -n ${NAMESPACE} --ignore-not-found=true
- envsubst < job.yaml | kubectl apply -n ${NAMESPACE} -f -
- kubectl wait --for=condition=complete --timeout=600s job/polls-migrations
- envsubst < polls.yaml | kubectl apply -n ${NAMESPACE} -f -
- kubectl patch deployment polls -p "{\"spec\":{\"template\":{\"metadata\":{\"labels\":{\"date\":\"`date +'%s'`\"}}}}}"
tags:
- k8s
```
### Take a look at your Gitlab CI pipeline
If everything goes well, you should see that you have a pipeline with two steps:
- build
- deploy
You can see the log output by clicking on each one
### Informations available in Gitlab
Gitlab offer a basic set of features in order to manage your Kubernetes web application.
In the `Operations` section on the left panel you are able to view :
* Metrics : It will display basic memory and cpu graphs
* Environements : It will list all the environements you have created (you can have several deploy stages in your `.gitlab-ci.yml` : dev, stagging, production....) and give direcrt access to :
* Website link
* Monitoring
* Open a remote shell on the container
* Error Tracking : If your application implement error tracking on [sentry]: https://sentry.io/ errors are displayed here.
* Serverless : Feature not available here at Pasteur
## What else ?
### Is my deployed application is working ?
Now we want to know if what we have done is working; to do that just go in `Operations/Environements` and click on the first icon on the right hand side, it should open your web app.
You may have a `404 page not found` error, that's can be unfortunately normal for the first deployement, all you have to do is to restart the deploy job in the CI section of your gitlab project.
### Kubernetes dashboard
#### View your running workload
> We provide a web interface to visualise your workloads you have access to, you can connect to https://console.k8s-dev.pasteur.fr and follow the login process. On the left panel, you will find a drop down list and select your namespace. You will be able to view your Deployements, Pods, Services and so on.
#### Increase the number of replicas
If you want to increase the number of replicas of your web application, you can go in the Deployement section and select the 3 dots on the `polls` line; select `Scale`. You can increase or decrease the number of replicas. Kubernetes will automaticaly add or remove Pods.
> You can also update the file `polls.yaml` to change the replicas value (by default 1)
> Kubernetes will automatically load balance the traffic to the n PODS.
#### What happen if I kill a Pod ?
You can try to `Delete` the polls `Pod` using the 3 dots icon and see that Kubernetes will automaticaly restart a new one.
### Grafana : Metrics and Logs
You can log on https://grafana.k8s-dev.pasteur.fr/ and browse the various dashboard available.
### Best Practices
Always use Gitlab !!!
\ No newline at end of file
You git repository must be the source of truth, never change something directly on Kubernetes that needs to be persistent.
Integrate Kubernetes at the beginning of your project, you'll waste less time.
......@@ -2,6 +2,9 @@ stages:
- build
- deploy
variables:
DOCKER_HOST: tcp://localhost:2375/
services:
- docker:dind
......@@ -13,20 +16,26 @@ build:
- docker build -t "$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_NAME/polls:${CI_COMMIT_SHORT_SHA}" .
- docker tag "$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_NAME/polls:${CI_COMMIT_SHORT_SHA}" "$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_NAME/polls:latest"
- docker push "$CI_REGISTRY_IMAGE/$CI_COMMIT_REF_NAME/polls:${CI_COMMIT_SHORT_SHA}"
tags:
- k8s
deploy:
stage: deploy
image: registry-gitlab.pasteur.fr/dsi-tools/docker-images:docker_kubernetes_image
variables:
NAMESPACE: "tmenard-django"
NAMESPACE: ${GITLAB_USER_LOGIN}-${CI_PROJECT_NAME}
environment:
name: tmenard-django
url: https://tmenard-django.k8s-dev.pasteur.fr
name: ${GITLAB_USER_LOGIN}-${CI_PROJECT_NAME}
url: https://${GITLAB_USER_LOGIN}-${CI_PROJECT_NAME}.k8s-dev.pasteur.fr
script:
- kubectl delete secret registry-gitlab -n ${NAMESPACE} --ignore-not-found=true
- kubectl create secret docker-registry -n ${NAMESPACE} registry-gitlab --docker-server=registry-gitlab.pasteur.fr --docker-username=${DEPLOY_USER} --docker-password=${DEPLOY_TOKEN} --docker-email=kubernetes@pasteur.fr
- envsubst < manifest.yaml | kubectl apply -n ${NAMESPACE} -f -
- envsubst < postgresql.yaml | kubectl apply -n ${NAMESPACE} -f -
- kubectl wait --for=condition=available --timeout=600s deployment/postgresql
- kubectl delete job polls-migrations -n ${NAMESPACE} --ignore-not-found=true
- envsubst < job.yaml | kubectl apply -n ${NAMESPACE} -f -
- kubectl wait --for=condition=complete --timeout=600s job/polls-migrations
- envsubst < polls.yaml | kubectl apply -n ${NAMESPACE} -f -
- kubectl patch deployment polls -p "{\"spec\":{\"template\":{\"metadata\":{\"labels\":{\"date\":\"`date +'%s'`\"}}}}}"
tags:
- k8s
......@@ -14,6 +14,9 @@ RUN apt-get update && apt-get upgrade -y && apt-get install \
ADD requirements.txt /app/requirements.txt
RUN pip install --default-timeout=100 --upgrade pip && pip install -r /app/requirements.txt
RUN addgroup --gid 1001 django && \
useradd --uid 1001 --gid 1001 django
# We add the current content of the git repo in the /app directory
ADD . /app
WORKDIR /app
......@@ -21,4 +24,7 @@ RUN rm -rf .gitlab-ci.yml Dockerfile README.md img
# We use the CMD command to start the gunicorn daemon
# when we start the container.
# Note the $PORT variable, we will need to define it when we start the container
USER django
CMD python manage.py runserver 0.0.0.0:$PORT
......@@ -36,4 +36,4 @@ spec:
- name: postgresql-credentials
secret:
secretName: postgresql-credentials
backoffLimit: 5
backoffLimit: 10
apiVersion: apps/v1
kind: Deployment
metadata:
name: polls
labels:
app: polls
spec:
replicas: 1
selector:
matchLabels:
app: polls
template:
metadata:
labels:
app: polls
spec:
containers:
- name: polls-app
image: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_NAME/polls:${CI_COMMIT_SHORT_SHA}
imagePullPolicy: Always
env:
- name: DATABASE_HOST
value: postgresql
- name: DATABASE_NAME
valueFrom:
secretKeyRef:
name: postgresql-credentials
key: database
- name: DATABASE_USER
valueFrom:
secretKeyRef:
name: postgresql-credentials
key: username
- name: DATABASE_PASSWORD
valueFrom:
secretKeyRef:
name: postgresql-credentials
key: password
- name: PORT
value: "8080"
ports:
- containerPort: 8080
resources:
requests:
memory: "64Mi"
cpu: "50m"
limits:
memory: "128Mi"
cpu: "100m"
imagePullSecrets:
- name: registry-gitlab
volumes:
- name: postgresql-credentials
secret:
secretName: postgresql-credentials
---
apiVersion: v1
kind: Service
metadata:
name: polls
labels:
app: polls
spec:
type: ClusterIP
ports:
- port: 8080
selector:
app: polls
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: traefik
labels:
app: polls
name: polls
spec:
rules:
- host: ${GITLAB_USER_LOGIN}.k8s-dev.pasteur.fr
http:
paths:
- backend:
serviceName: polls
servicePort: 8080
path: /
......@@ -93,91 +93,4 @@ spec:
- port: 5432
selector:
app: postgresql
tier: postgreSQL
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: polls
labels:
app: polls
spec:
replicas: 1
selector:
matchLabels:
app: polls
template:
metadata:
labels:
app: polls
spec:
containers:
- name: polls-app
image: $CI_REGISTRY_IMAGE/$CI_COMMIT_REF_NAME/polls:${CI_COMMIT_SHORT_SHA}
imagePullPolicy: Always
env:
- name: DATABASE_HOST
value: postgresql
- name: DATABASE_NAME
valueFrom:
secretKeyRef:
name: postgresql-credentials
key: database
- name: DATABASE_USER
valueFrom:
secretKeyRef:
name: postgresql-credentials
key: username
- name: DATABASE_PASSWORD
valueFrom:
secretKeyRef:
name: postgresql-credentials
key: password
- name: PORT
value: "8080"
ports:
- containerPort: 8080
resources:
requests:
memory: "64Mi"
cpu: "50m"
limits:
memory: "128Mi"
cpu: "100m"
imagePullSecrets:
- name: registry-gitlab
volumes:
- name: postgresql-credentials
secret:
secretName: postgresql-credentials
---
apiVersion: v1
kind: Service
metadata:
name: polls
labels:
app: polls
spec:
type: ClusterIP
ports:
- port: 8080
selector:
app: polls
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: traefik
labels:
app: polls
name: polls
spec:
rules:
- host: ${GITLAB_USER_LOGIN}.k8s-dev.pasteur.fr
http:
paths:
- backend:
serviceName: polls
servicePort: 8080
path: /
tier: postgreSQL
\ No newline at end of file