torch.cuda.is_available() Is False: Fix PyTorch CUDA Detection

You installed PyTorch on a machine with an NVIDIA GPU, but this check prints False:

import torch
print(torch.cuda.is_available())

Start with three checks before reinstalling anything:

1. Run nvidia-smi on the host. If that fails, fix the driver or host GPU visibility first.
2. Run python -c "import torch; print(torch.version.cuda, torch.cuda.is_available())" in the exact environment used by the application. If torch.version.cuda is None, you likely installed a CPU-only build.
3. If host PyTorch works but Docker fails, test GPU passthrough inside the container.

A False result means this Python process cannot currently use a CUDA device; it does not identify which layer failed. Isolate driver, wheel, environment, and container visibility one at a time.

A five-minute fault-isolation sequence

Use the row matching your first failed check, then verify the fix by repeating the same command.

Step 1: Establish which Python and torch you are testing

Execute these commands in the same shell, virtual environment or notebook kernel that runs the failing application:

python -m pip -V
python -m pip show torch
python -c "import sys, torch; print('python:', sys.executable); print('torch:', torch.__version__); print('torch file:', torch.__file__); print('build CUDA:', torch.version.cuda); print('available:', torch.cuda.is_available()); print('count:', torch.cuda.device_count())"

The python -m pip form reduces ambiguity between the package installer and interpreter. In Jupyter, sys.executable is especially important because the notebook kernel may use another environment than your terminal.

Interpret torch.version.cuda carefully: it describes the CUDA version used for the installed PyTorch build, not a guarantee that a CUDA device is accessible. A None value normally means you do not have a CUDA-enabled build. An installed CUDA toolkit alone does not convert a CPU wheel into a GPU wheel.

Step 2: Check the driver separately

On the host operating system, run:

nvidia-smi

If the tool cannot communicate with the NVIDIA driver, fix the host GPU/driver path before reinstalling Python libraries. If it works, inspect the GPU name, the active processes, and available device memory.

The CUDA version displayed by nvidia-smi is a driver capability indicator. It is not automatically the installed nvcc compiler version, and it does not need to numerically match the PyTorch wheel’s build label. Driver support and the chosen wheel still need to be compatible.

Most users running a prebuilt PyTorch wheel do not need to compile PyTorch or install an entire standalone CUDA toolkit merely to use CUDA. Building custom extensions is a different requirement.

Step 3: Repair a CPU-only or incorrect PyTorch installation

If the host driver works but torch.version.cuda is None, examine the environment and installation command before making changes. Use the selector on the official PyTorch Start Locally page for your operating system, package manager, Python version and supported CUDA build. Run its current command inside the intended environment.

Avoid copying an old cuXXX index URL from an unrelated tutorial. The published wheel choices and platform support change. After the install, repeat the same interpreter check from Step 1.

If torch.version.cuda is non-null but availability remains false, repeatedly reinstalling the same wheel is unlikely to reveal the blocker. Continue to runtime visibility and compatibility.

Step 4: Check whether the process can see a GPU

First inspect process-level visibility without exposing private values in a public bug report:

python -c "import os; print('CUDA_VISIBLE_DEVICES:', os.environ.get('CUDA_VISIBLE_DEVICES')); print('NVIDIA_VISIBLE_DEVICES:', os.environ.get('NVIDIA_VISIBLE_DEVICES'))"

CUDA_VISIBLE_DEVICES can restrict which GPU indices a process sees. A deliberately empty or disabled device selection can hide an otherwise working card. Schedulers, notebooks, containers and launch scripts may set this variable without a visible reminder in your application code.

Check the running application’s environment, not just a new terminal. Multiple Python processes may have different variables. On AMD ROCm builds, the public PyTorch torch.cuda API also exposes GPU functionality; on Apple Silicon, check the separate MPS backend instead of assuming that CUDA should work.

Step 5: If this happens inside Docker, test the GPU boundary

A container seeing the host filesystem does not mean it can see the host GPU. NVIDIA’s Container Toolkit documentation describes selecting GPU devices for Docker workloads.

Compare the host and container, starting with a known-good GPU-enabled image and an appropriate runtime configuration. A typical diagnostic command is:

docker run --rm --gpus all ubuntu nvidia-smi

This command is illustrative and depends on the host’s NVIDIA Container Toolkit configuration and the chosen image containing the needed utility. If the container cannot access the GPU while the host can, fix GPU allocation and runtime configuration before debugging the application code.

If container nvidia-smi succeeds, run the PyTorch interpreter checks inside the actual application image. A CPU-only wheel there can fail even though GPU passthrough works. Conversely, a correct CUDA wheel cannot overcome a container that was launched without access to devices.

Step 6: Distinguish detection from a usable allocation

Once availability becomes true, test a tiny allocation rather than inferring that an entire training run is healthy:

python -c "import torch; x=torch.ones(1, device='cuda'); print(x, torch.cuda.get_device_name(0))"

If this passes but a larger workload fails with CUDA out-of-memory, that is a capacity problem, not a detection problem. Inspect device memory, other GPU processes, batch size and model placement. A missing driver, a CPU-only wheel and CUDA OOM require different fixes.

What to include in a useful issue report

Capture the exact Python path, torch.__version__, torch.version.cuda, torch.cuda.is_available(), device count, host driver status, whether the code is containerized, and the first relevant exception. Redact usernames, tokens, internal paths, IP addresses and unrelated process arguments.

The effective decision order is host driver -> exact Python/wheel -> process GPU visibility -> container GPU boundary -> first real tensor allocation. That sequence is usually more informative than blindly updating CUDA.

Primary sources

• PyTorch: official installation selector and verification
• PyTorch CUDA semantics and programming notes
• NVIDIA Container Toolkit: Docker GPU selection
• NVIDIA System Management Interface

Related: Why Is My Local LLM Slow? and How to Reduce LLM VRAM Usage.