To install Docker Engine on Ubuntu 24.04, the official and recommended method is to set up Docker's official apt repository. This ensures you get the latest stable version and automatic future updates.

Before starting, remove any default or unofficial Docker packages to prevent conflicts:

sudo apt remove docker.io docker-doc docker-compose docker-compose-v2 podman-docker containerd runc

Update your package index and install the prerequisite security certificates:

sudo apt update
sudo apt install -y ca-certificates curl gnupg

Next, add Docker's official GPG key to verify package authenticity:

sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://docker.com -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

Add the stable repository to your system's apt sources:

echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://docker.com \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

Update your repository index to include Docker's packages, then install Docker Engine, the CLI, and Docker Compose V2:

sudo apt update
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

Check if the service is up and running:

sudo systemctl status docker

Run the default test container to verify it pulls and executes properly:

sudo docker run hello-world

By default, Docker requires root privileges. To run commands as a regular user, add yourself to the docker group:

• Create the group (usually exists already):

sudo groupadd docker

• Add your user to the group:

sudo usermod -aG docker $USER

• Apply the changes immediately without logging out:

newgrp docker

• Verify by running without sudo:

docker run hello-world

For detailed troubleshooting or specific alternative guides, you can refer to the Official Docker Engine Installation Docs.

To enable NVIDIA AI and GPU acceleration on Docker, you must install and configure the NVIDIA Container Toolkit. This acts as the bridge that allows Docker containers to access your host machine's physical GPU.

Here is how to set it up on a Linux host (such as Ubuntu).

Ensure you have the official NVIDIA drivers installed on your host system, along with Docker Engine. Validate your driver installation by running:

nvidia-smi

(You should see a table displaying your GPU information.)

Add the official package repositories and install the toolkit:

# Add the NVIDIA package repository
curl -fsSL https://github.io | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg

curl -s -L https://github.io | \
sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

# Update package list and install
sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit

Configure the Docker daemon to automatically recognize the NVIDIA container runtime, then restart the Docker service:

# Configure Docker to use the NVIDIA runtime
sudo nvidia-ctk runtime configure --runtime=docker

# Restart the Docker service to apply changes
sudo systemctl restart docker

Test the configuration by running a lightweight, official CUDA container. Passing the –gpus all flag tells Docker to expose your graphics cards to the container:

sudo docker run --rm --gpus all ubuntu nvidia-smi

If successful, the container will print out your host GPU details exactly like the native nvidia-smi command did.

If you are developing locally via Docker Desktop, you do not need to manually install the toolkit. Instead, you can leverage native AI features:

• WSL 2 (Windows): Ensure WSL integration is turned on under Settings > Resources > WSL Integration in Docker Desktop.
• Docker Model Runner (DMR): Go to Settings > AI, check Enable Docker Model Runner, and select Enable GPU-backed inference to pull and manage local AI models natively using commands like docker model run.

If you plan to deploy enterprise-grade AI workloads, you can also authenticate your Docker client to the NVIDIA NGC Container Registry using an API key. This gives you direct access to fine-tuned, GPU-optimized AI models and microservices.