Differences

This shows you the differences between two versions of the page.

--- tech:linux [2026/06/14 22:27] – created glong
+++ tech:linux [2026/06/15 18:53] (current) – glong
@@ Line 2: / Line 2: @@
 Ubunut 24.04
+[[:tech:Linux:AI Setup]] \\
+====== Install all drivers needed for using V100 on Ubuntu 24.04 LTS ======
+Setting up an enterprise data center GPU like the NVIDIA V100 (Volta architecture) on **Ubuntu 24.04 LTS** requires the proprietary driver and the CUDA Toolkit. Since the V100 is typically used in a headless server environment for compute, machine learning, and AI workloads, a **compute-only/headless driver installation** is ideal. This avoids dragging in unnecessary graphical desktop packages.
+Here is the clean, streamlined method using the official NVIDIA network repositories for Ubuntu 24.04.
+==== Step 1: Clean Up Existing Drivers ====
+To prevent conflicts with generic open-source or older drivers, remove any current installations:
+<code bash\>
+sudo apt-get purge 'nvidia\*' 'cuda\*' -y
+sudo apt-get autoremove -y
+</code>
+==== Step 2: Install Linux Kernel Headers ====
+Ensure you have the correct kernel headers installed so the NVIDIA kernel modules can compile properly:
+<code bash\>
+sudo apt-get update
+sudo apt-get install linux-headers-$(uname -r) build-essential -y
+</code>
+==== Step 3: Setup the NVIDIA & CUDA Network Repository ====
+NVIDIA provides a repository pinned to Ubuntu 24.04 (noble). Fetch the repository configurations and GPG keys so apt can safely pull the latest stable packages:
+<code bash\>
+# Download the repository pin file to set priority
+wget <https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86>\_64/cuda-ubuntu2404.pin
+sudo mv cuda-ubuntu2404.pin /etc/apt/preferences.d/cuda-repository-pin-600
+# Download and install the repository package
+wget <https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2404/x86>\_64/cuda-keyring\_1.1-1\_all.deb
+sudo dpkg -i cuda-keyring\_1.1-1\_all.deb
+# Update apt package listings
+sudo apt-get update
+</code>
+==== Step 4: Install the Drivers and CUDA Toolkit ====
+For enterprise cards like the V100, the stable branch (such as the nvidia-driver-550-server or nvidia-driver-565-server depending on repository syncs) is highly recommended.
+To fetch the **headless, data-center optimal drivers** alongside the CUDA toolkit, run:
+<code bash\>
+sudo apt-get -y install cuda-toolkit-12-8 nvidia-headless-server-550 nvidia-utils-550
+</code>
+*Note: If you plan on deploying Docker containers that utilize the V100, you should install the container runtime package as well:* `sudo apt-get install -y nvidia-container-toolkit`
+==== Step 5: Update Environment Variables ====
+To ensure your system paths find the CUDA binaries, append them to your shell configuration (\~/.bashrc):
+<code bash\>
+echo 'export PATH=/usr/local/cuda/bin${PATH:+:${PATH}}' \>\> \~/.bashrc
+echo 'export LD\_LIBRARY\_PATH=/usr/local/cuda/lib64${LD\_LIBRARY\_PATH:+:${LD\_LIBRARY\_PATH}}' \>\> \~/.bashrc
+source \~/.bashrc
+</code>
+==== Step 6: Reboot and Verify ====
+Restart the machine to initialize the newly compiled kernel modules:
+<code bash\>
+sudo reboot
+</code>
+Once the system is back up, verify that the OS recognizes the V100 and that the drivers are operational:
+<code bash\>
+nvidia-smi
+</code>
+You should see an output matrix detailing your V100, its current temperature, power consumption, and the driver/CUDA version currently running.
+====== GeForce 210 Driver Override Guide ======
+We cannot fix this the ''nouveau'' driver cannot live with the V100 NVidia driver
 ====== Installing Docker on Ubuntu 24.04 ======
@@ Line 88: / Line 178: @@
 For detailed troubleshooting or specific alternative guides, you can refer to the [[https://docker.com|Official Docker Engine Installation Docs]].
+====== Enabling NVIDIA AI and GPU Acceleration on Docker ======
+To enable NVIDIA AI and GPU acceleration on Docker, you must install and configure the **NVIDIA Container Toolkit**. This acts as the bridge that allows Docker containers to access your host machine's physical GPU.
+Here is how to set it up on a Linux host (such as Ubuntu).
+===== 1. Install Prerequisites =====
+Ensure you have the official NVIDIA drivers installed on your host system, along with Docker Engine. Validate your driver installation by running:
+<code>
+nvidia-smi
+</code>
+//(You should see a table displaying your GPU information.)//
+===== 2. Install the NVIDIA Container Toolkit =====
+https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/latest/install-guide.html
+Add the official package repositories and install the toolkit:
+<code>
+# Add the NVIDIA package repository
+curl -fsSL https://github.io | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
+curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg \
+  && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | \
+    sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | \
+    sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
+sudo sed -i -e '/experimental/ s/^#//g' /etc/apt/sources.list.d/nvidia-container-toolkit.list
+# Update package list and install
+sudo apt-get update
+export NVIDIA_CONTAINER_TOOLKIT_VERSION=1.19.1-1
+  sudo apt-get install -y \
+      nvidia-container-toolkit=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \
+      nvidia-container-toolkit-base=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \
+      libnvidia-container-tools=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \
+      libnvidia-container1=${NVIDIA_CONTAINER_TOOLKIT_VERSION}
+</code>
+===== 3. Configure the Docker Runtime =====
+Configure the Docker daemon to automatically recognize the NVIDIA container runtime, then restart the Docker service:
+<code>
+# Configure Docker to use the NVIDIA runtime
+sudo nvidia-ctk runtime configure --runtime=docker
+# Restart the Docker service to apply changes
+sudo systemctl restart docker
+</code>
+===== 4. Verify GPU Access in Docker =====
+Test the configuration by running a lightweight, official CUDA container. Passing the ''--gpus all'' flag tells Docker to expose your graphics cards to the container:
+<code>
+docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi
+</code>
+If successful, the container will print out your host GPU details exactly like the native ''nvidia-smi'' command did.
+----
+===== Alternative: Docker Desktop (Windows / Mac) =====
+If you are developing locally via Docker Desktop, you do not need to manually install the toolkit. Instead, you can leverage native AI features:
+  * **WSL 2 (Windows)**: Ensure WSL integration is turned on under **Settings > Resources > WSL Integration** in Docker Desktop.
+  * **Docker Model Runner (DMR)**: Go to **Settings > AI**, check **Enable Docker Model Runner**, and select **Enable GPU-backed inference** to pull and manage local AI models natively using commands like ''docker model run''.
+If you plan to deploy enterprise-grade AI workloads, you can also authenticate your Docker client to the NVIDIA NGC Container Registry using an API key. This gives you direct access to fine-tuned, GPU-optimized AI models and microservices.
+====== Installing LM Studio on Ubuntu 24.04 ======
+To install LM Studio on Ubuntu 24.04 LTS, you have two primary options: the **headless lms daemon** (ideal for servers, command-line usage, or API orchestration) or the **AppImage** (for the full graphical interface).
+---
+===== Option 1: Headless Daemon (lms) =====
+If you intend to use LM Studio for backend services, API hosting, or CI/CD, the ''lms'' daemon is the official, recommended approach.
+==== 1. Install the Daemon ====
+Run the following command in your terminal to download and execute the official installer:
+<code>
+curl -fsSL https://lmstudio.ai/install.sh | bash
+</code>
+==== 2. Usage ====
+Once the installation script completes, you can interact with the service directly using the ''lms'' CLI.
+To start the background service:
+<code>
+lms daemon up
+</code>
+**Note:** If you encounter an "Illegal instruction" error upon execution, ensure your system's ''libc'' and environment variables are fully updated. This can sometimes occur on bleeding-edge hardware configurations.
+---
+===== Option 2: Graphical AppImage (Full GUI) =====
+If you require the standard LM Studio desktop interface, you must download the AppImage. Note that Ubuntu 24.04 enforces strict sandboxing rules that require manual adjustment.
+==== 1. Download and Make Executable ====
+Download the latest Linux AppImage from the official LM Studio website, then give it execution permissions:
+<code>
+chmod +x LM_Studio-*.AppImage
+</code>
+==== 2. Extract and Configure Sandbox ====
+The application will likely fail to launch unless you fix the ''chrome-sandbox'' permissions. Extract the AppImage and update the owner permissions:
+<code>
+# Extract the AppImage
+./LM_Studio-*.AppImage --appimage-extract
+# Navigate to the extracted folder
+cd squashfs-root
+# Fix sandbox permissions
+sudo chown root:root chrome-sandbox
+sudo chmod 4755 chrome-sandbox
+</code>
+==== 3. Launch the Application ====
+You can now run the application from within the extracted directory:
+<code>
+./lm-studio
+</code>
+If you still encounter sandboxing issues and prefer to bypass it, you can launch with the ''--no-sandbox'' flag (use with caution):
+<code>
+./lm-studio --no-sandbox
+</code>
+---
+===== Troubleshooting & Performance =====
+^ Issue ^ Resolution ^
+| **Missing Dependencies** | Install base libraries: \\ <code>sudo apt install libatk1.0-0 libatk-bridge2.0-0 libcups2 libgdk-pixbuf2.0-0 libgtk-3-0 libpango-1.0-0 libcairo2 libxcomposite1 libxdamage1 libasound2t64 libatspi2.0-0</code> |
+| **Hardware Acceleration** | For integrated or discrete GPUs, ensure your drivers (such as ''intel-media-va-driver-non-free'' for Intel setups) are properly configured via ''mesa'' to ensure local model inference acceleration. |