Skip to content
Snippets Groups Projects
Unverified Commit f76e8981 authored by Henry Chen's avatar Henry Chen Committed by GitHub
Browse files

Fix the cuda 12 base image (#46)

parent b62838ee
No related branches found
No related tags found
No related merge requests found
# syntax=docker/dockerfile:1 # syntax=docker/dockerfile:1
FROM nvidia/cuda:12.2.0-base-ubuntu20.04 FROM nvidia/cuda:12.2.0-base-ubuntu22.04
RUN apt-get update && apt-get install -y -q python3 python3-pip RUN apt-get update && apt-get install -y -q python3 python3-pip
WORKDIR /app WORKDIR /app
COPY requirements.txt requirements.txt COPY requirements.txt requirements.txt
......
...@@ -85,6 +85,12 @@ helm install llama2-7b-chat-cuda11 ialacol/ialacol -f examples/values/llama2-7b- ...@@ -85,6 +85,12 @@ helm install llama2-7b-chat-cuda11 ialacol/ialacol -f examples/values/llama2-7b-
Deploys llama2 7b model with 40 layers offloadind to GPU. The inference is accelerated by CUDA 11. Deploys llama2 7b model with 40 layers offloadind to GPU. The inference is accelerated by CUDA 11.
### CUDA Driver Issues
If you see `CUDA driver version is insufficient for CUDA runtime version` when making the request, you are likely using a Nvidia Driver that is not [compatible with the CUDA version](https://docs.nvidia.com/cuda/cuda-toolkit-release-notes/index.html).
Upgrade the driver manually on the node (See [here](https://github.com/awslabs/amazon-eks-ami/issues/1060) if you are using CUDA11 + AMI). Or try different version of CUDA.
## Tips ## Tips
### Creative v.s. Conservative ### Creative v.s. Conservative
......
apiVersion: v2 apiVersion: v2
appVersion: 0.7.0 appVersion: 0.7.1
description: A Helm chart for ialacol description: A Helm chart for ialacol
name: ialacol name: ialacol
type: application type: application
version: 0.7.1 version: 0.7.2
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment