The NVIDIA CUDA Toolkit is a powerful development suite for accelerating computationally intensive applications using the GPU’s parallel processing capabilities. CUDA, short for Compute Unified Device Architecture, is a parallel computing platform and programming model developed by NVIDIA. It allows developers to leverage the processing power of NVIDIA GPUs (Graphics Processing Units) to perform general-purpose computing tasks that go beyond graphics rendering.

CUDA is particularly important for high-performance computing, artificial intelligence (AI), machine learning (ML), and deep learning applications. It provides developers with a software environment for writing applications that can run in parallel on thousands of GPU cores, offering significant speed improvements compared to traditional CPU-based computations.

In this guide, we’ll show you how to install and setup the NVIDIA CUDA Toolkit on your Atlantic.Net GPU server.

Prerequisites

  • An Ubuntu 22.04 GPU Server
  • A root or sudo privileges

Step 1: Update System Packages

Start by updating the system package list to ensure your environment is equipped with the latest updates and patches.

apt update -y

Step 2: Verify Your GPU Compatibility

To confirm that your GPU supports CUDA, run the following command:

lspci | grep -i nvidia

This command lists all NVIDIA GPUs detected by your system. Example output:

06:00.0 VGA compatible controller: NVIDIA Corporation GA102GL [A40] (rev a1)

If the output lists an NVIDIA GPU, your system is CUDA-compatible. If no NVIDIA GPU is listed, update the PCI hardware database and rerun the lspci command:

update-pciids

Step 3: Check GCC Compiler Version

CUDA requires a supported version of the GNU Compiler Collection (GCC). Verify the installed version with:

gcc --version

Example output:

gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

If your GCC version is outdated, install or update it using:

apt install gcc -y

Step 4: Download and Install NVIDIA CUDA Toolkit

Visit the CUDA Toolkit Archive files page and download the NVIDIA CUDA keyring using the below command.

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb

Install the downloaded package with dpkg command:

dpkg -i cuda-keyring_1.1-1_all.deb

Refresh the package index to recognize NVIDIA’s repository:

apt-get update

Install the CUDA Toolkit version 12.6 using:

apt-get -y install cuda-toolkit-12-6

This command downloads and installs the necessary files, including the CUDA compiler, libraries, and tools.

Step 5: Configure Environment Variables

To ensure the CUDA Toolkit functions seamlessly, you will need to update the system environment variables.

Append the CUDA binary directory to the PATH variable:

echo "export PATH=/usr/local/cuda-12.6/bin${PATH:+:${PATH}}" >> ~/.bashrc

Include the CUDA library directory in the LD_LIBRARY_PATH:

echo "export LD_LIBRARY_PATH=/usr/local/cuda-12.6/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}" >> ~/.bashrc

Activate the updated environment variables:

source ~/.bashrc

By configuring these variables, the system knows where to find CUDA executables and libraries.

Step 6: Verify the Installation

Verify the GPU driver installation using NVIDIA System Management Interface (SMI):

nvidia-smi

Output:

Ensure the NVIDIA CUDA Compiler (NVCC) is installed:

nvcc --version

Output:

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2024 NVIDIA Corporation
Built on Tue_Oct_29_23:50:19_PDT_2024
Cuda compilation tools, release 12.6, V12.6.85
Build cuda_12.6.r12.6/compiler.35059454_0

Step 7: Test CUDA Installation with Sample Programs

Download sample CUDA programs for testing the installation.

git clone https://github.com/NVIDIA/cuda-samples.git

Switch to the directory containing the deviceQuery sample:

cd cuda-samples/Samples/1_Utilities/deviceQuery

Build the deviceQuery program:

make

Execute the deviceQuery program to test CUDA functionality:

./deviceQuery

If the program runs successfully, your CUDA installation is complete.

Conclusion

Installing the NVIDIA CUDA Toolkit on an Atlantic.Net GPU server enables you to harness the power of GPU computing for complex applications. With this guide, you’ve verified your GPU compatibility, installed the CUDA Toolkit, configured environment variables, and tested the setup with sample programs. You’re now ready to develop and run CUDA-accelerated applications. Dive into Atlantic.Net’s GPU server hosting options and explore the possibilities CUDA offers!