Ollama error 500 “unable to load model”: what it means and how to fix it

What the error looks like

The text changes depending on the version, the cause and where you call Ollama from, but it always starts with 500. In the terminal you usually see “Error: 500 Internal Server Error:” and one detail: “unable to load model:” with the path to the model file, or “llama-server process has terminated: exit status 1”, as in this Windows report on GitHub. If you call the API from a program, you may only get “HTTP Error 500: Internal Server Error”, with no more clues: that is what our test script received.

All of these variants say the same thing: the model never loaded. The reason isn’t in that message.

Find the cause in the log (server.log)

With the Ollama app, the log is at %LOCALAPPDATA%\Ollama\server.log, and older ones are kept as server-1.log, server-2.log… If you need more detail, close the app and start it with OLLAMA_DEBUG=1 (Ollama troubleshooting). Look for the last “Load failed” line and read what is just above it: that shows whether memory ran out, whether the format wasn’t recognized, or whether CUDA failed (the library the NVIDIA graphics card uses for its calculations).

What to do, in this order

Change one thing at a time and test again; if you change three, you won’t know which one fixed it.

• Free up memory: ollama ps shows what is loaded, ollama stop <model> unloads it, and you should close games or video editors. nvidia-smi tells you how much VRAM is left.
• Lower the context: inside ollama run, /set parameter num_ctx 2048; through the API, "options": {"num_ctx": 2048}. If it loads with less context, the problem was memory.
• Put fewer layers on the graphics card: /set parameter num_gpu 20. It didn’t help in the case below, but it solves cases where the card is just too small.
• Turn off flash attention (a calculation optimization) to rule it out: $env:OLLAMA_FLASH_ATTENTION = "0"; ollama serve.
• Update Ollama (ollama -v shows your version). A recent model may not load in an old version.
• Use a path without special characters: if your Windows user name has accents, tildes or another alphabet, move the models with [Environment]::SetEnvironmentVariable("OLLAMA_MODELS", "C:\ollama-modelos", "User"), restart Ollama and download the models again. This was the fix in the GitHub report linked above.
• Download it again: ollama rm <model> and then ollama pull <model>, in case the file is damaged.
• Use a smaller or more compressed (quantized) model. Before you buy hardware for a model, test it: the 35B one couldn’t be used at all with 16 GB.

How to check that it’s fixed

Loading once isn’t enough: in the case below, the first load worked and the next ones didn’t. Load the model, unload it with ollama stop and repeat two or three times, with the context you will really use. Then confirm with ollama ps that it is loaded and where (PROCESSOR column), and that server.log has no new “Load failed” since your test. If PROCESSOR shows a split between CPU and GPU, it will work, but slowly: we explain it in Ollama not using the GPU on Windows.

A real case: a 35B model on a 16 GB graphics card

On a test PC with Windows 11, a 16 GB graphics card and about 31 GB of RAM (9.3 GB free at that moment), we downloaded qwen3.6:35b, which takes up 22 GB on disk. The log did the math: “projected to use 20912 MiB of device memory vs. 15172 MiB of free device memory”, meaning it needed more memory than the card had free. Ollama split it on its own: 12.4 GB on the graphics card and 8.3 GB in RAM.

The first load, with a 4,096 context, worked. The next ones, with 8,192 and then 4,096 again, ended in a 500 error. The log showed a CUDA failure while it prepared the model, “CUDA error: shared object initialization failed”, inside a flash attention function, and the process exited with code 0xc0000409.

We tried the two fixes most often recommended: turning off flash attention (OLLAMA_FLASH_ATTENTION=0; the log confirmed “flash_attn = disabled”) and forcing 20 layers onto the graphics card (num_gpu 20; the log said “offloaded 20/42 layers to GPU”). Both failed with the same “CUDA error” at ggml-cuda.cu:110. We found no single cause: on that PC and with that version, the 35B model could not be used, while the 4B–12B models worked without problems. The data is in our speed and VRAM test of local models.

What 0xc0000409 means (and why it is not an attack)

Windows pairs that code with a scary message: “The system detected an overrun of a stack-based buffer in this application”. It doesn’t mean someone attacked you, nor necessarily that memory overflowed. As Raymond Chen of Microsoft explains, that code is now used for any immediate “fail fast” shutdown of a program that detects a serious error, including one caused by abort() (STATUS_STACK_BUFFER_OVERRUN doesn’t mean that there was a stack buffer overrun). In the case above, llama-server shut itself down after the CUDA error.

Rare cases

• GGUF models downloaded from Hugging Face: they may carry data or an architecture that your Ollama version doesn’t recognize, and the log will say so when it loads them. Update Ollama or use the version from the Ollama library.
• Models that can see images: they load a second file (the image projector). With the 35B model, the log calculated 1,134 MiB for it alone. If that file is the one that fails, the log names it.
• A 500 error that has nothing to do with loading: if OpenCode or another agent fails while the model does load, the context may be the problem. See OpenCode with Ollama not working.
• What counts is the free memory, not the installed memory: the Windows desktop alone already used about 3 GB of VRAM at rest.