Why ggml reaching 1.0 matters

Local inference projects often depend on a small native library below the model runner. In the llama.cpp ecosystem, that library is ggml: a C/C++ tensor library designed for portable, memory-conscious machine-learning workloads.

The repository dates back to 2022. The news is the release: ggml v1.0.0 was published on November 13, 2026. It reached the Hacker News front page and sparked a large r/LocalLLaMA discussion. For developers who embed ggml or package it for other applications, the important change is the move to semantic-versioned releases with an explicit signal for breaking public API changes.

ggml is not llama.cpp

It is worth separating the two projects:

ggml is the lower-level tensor library. It defines tensors, computation graphs, memory management, and backend execution.
llama.cpp is a higher-level inference engine that uses ggml to run language models.
Projects such as Ollama and LM Studio build on the llama.cpp ecosystem, but their exact dependency arrangements differ by project and release.

The ggml README describes a small C/C++ implementation with no mandatory third-party dependencies, support for multiple CPU architectures, quantized tensor types, and backends for CPU, GPU, NPU, and browsers. It is closer to a low-level runtime than to PyTorch or TensorFlow.

That distinction matters. ggml gives systems developers control over memory, tensor layouts, and backend execution. It does not provide the same high-level training workflow or automatic differentiation experience as PyTorch.

What v1.0.0 changes

The official release notes group the work into API, core, and backend changes.

The API changes are the ones to review first:

Deprecated public API was removed.
The legacy DIAG_MASK_INF and DIAG_MASK_ZERO operations were removed.
The RPC protocol was bumped to v9, so incompatible peers are rejected during the handshake.
New functions allow callers to control YaRN correction-bound truncation.
FP8 support and additional precision tests were added.

The release also includes backend work across CUDA, Metal, Vulkan, SYCL, OpenCL, WebGPU, Hexagon, CPU, CANN, ZDNN, and OpenVINO. Examples include few-row matrix multiplication improvements, fused mixture-of-experts operations, new WebGPU kernels, and scheduler changes that avoid uploading inputs a graph does not read.

Do not assume every operation is available on every device. Backend coverage is still specific to the operation and hardware. The maintainers' Introduction to ggml guide also notes that some operations may work on CPU but not on CUDA, and that low-level development requires understanding the library's memory and graph model.

What the versioning policy promises

The maintainers documented release and versioning rules in discussion #1579. For ggml, release builds use semantic version tags:

A major version signals a breaking public API change.
A minor version is for backward-compatible features and API additions.
A patch version is for fixes that do not change the API.

This gives downstream developers a more useful signal than tracking arbitrary commits. Still, pin the version you have tested and upgrade deliberately.

There is another detail for llama.cpp consumers: the llama.cpp repository keeps an internal copy of ggml for development builds, and that copy can differ from the separately released ggml tag. The maintainers explain how official release tags are aligned when the internal and released copies match. If your application links against a system-installed ggml, verify the compatibility requirements of the llama.cpp release you use.

Build ggml from source

The current README documents a simple CMake build. On Ubuntu or another Debian-based Linux distribution, install the compiler and build tools first:

sudo apt install build-essential cmake git

Clone the repository and build the release configuration:

git clone https://github.com/ggml-org/ggml cd ggml mkdir build && cd build cmake .. cmake --build . --config Release -j 8

The repository includes a commented matrix-multiplication example that demonstrates tensor creation and backend handling. Build the minimal context example from the project root:

cmake -B build cmake --build build --config Release --target simple-ctx ./build/bin/simple-ctx

The example multiplies two small matrices and prints the transposed result. It is a good first check that the compiler, CMake configuration, and CPU backend are working. For GPU execution, use the backend example and configure the relevant backend in CMake; the simple context example is intentionally CPU-focused.

The key operations in the example are:

Allocate a ggml context for tensor metadata and data.
Create tensors with the required shapes and data types.
Build a computation graph with operations such as matrix multiplication.
Expand the graph from the result tensor.
Execute the graph through a backend.
Retrieve the output and release the context or backend resources.

The full source is in examples/simple. Prefer it over copying an old blog snippet because low-level APIs can change between releases.

A migration checklist for v1.0

Before upgrading an application that embeds ggml:

Search for deprecated API calls and the removed diagonal-mask operations.
Rebuild every native target rather than assuming the CPU build covers all backends.
If you use the RPC backend, upgrade compatible peers together because the protocol is now v9.
Check whether your dependency is the standalone ggml release or an internal copy bundled by llama.cpp.
Run numerical tests for tensor operations that matter to your application, especially when changing backend or precision.
Pin the version in your package manager or build manifest until your integration tests pass.

When should you use ggml?

Choose ggml when you need a low-level tensor runtime, quantized inference-oriented operations, and control over CPU or accelerator backends. It fits native applications, embedded inference components, and developers building an engine rather than just calling a hosted model API.

Choose PyTorch when your main job is model research, training, or high-level tensor experimentation. Consider ONNX Runtime or ExecuTorch when your deployment flow is built around their model formats and execution APIs. These tools overlap in some deployment scenarios, but they are not drop-in replacements for ggml's C API.

The trade-off is engineering effort. ggml is compact and portable, but it exposes low-level concepts. You must understand tensor dimensions, graph lifetimes, memory allocation, backend availability, and the exact version of the API you are linking.

The r/LocalLLaMA v1.0.0 discussion covers the broader llama.cpp release, which includes ggml v1.0.0, and shows what local-inference developers are asking about the new release.

Sources and further reading:

If you maintain a native application that embeds ggml, what would you test first: API compatibility, RPC interoperability, or backend numerical parity?

Written by Shaswat. Find more work on GitHub, LinkedIn, and StackShade on YouTube.