TornadoVM 7.3 Drops JNI for Java's FFM API and Ships Three Releases in Nine Days

TornadoVM, the open-source framework that compiles Java code to run on GPUs, moved its backends from the Java Native Interface (JNI) to the Foreign Function & Memory (FFM) API in version 7, InfoQ reported on Oct. 21. Author Ben Evans called 7.3 "a major milestone for the project".

The project's releases page shows how fast that landed: 7.2.0 on Oct. 7, 7.3.0 on Oct. 14 and 7.3.1 on Oct. 15. The README, which calls the code Apache License 2.0, lists 7.3.1 as current, supporting JDK 21 through 27, with OpenCL as the default backend alongside CUDA and Apple Metal.

Release	Date	Headline items from the release notes
7.2.0	Oct. 7	Native arrays beyond 2,147,483,647 elements; device indexing fix for identical GPUs
7.3.0	Oct. 14	Async MLX dispatch on Metal; new tornado-nccl module; per-GPU CUDA contexts
7.3.1	Oct. 15	tornado-mlx and tornado-nccl published to Maven Central

What changed under the hood

InfoQ lists three architectural changes. The backends use FFM in place of JNI, the experimental JVM Compiler Interface (JVMCI) dependency is gone, and compiler metadata now comes through a single reflection, ASM and Unsafe layer on every supported JDK. It also lists support for the cuTile programming model on CUDA, calls into the cuBLAS and CUTLASS libraries from task graphs, and multi-GPU execution.

The article says TornadoVM offers four ways to program Nvidia GPUs from Java, then names three: a parallel annotation on loops, a kernel context API added because the annotation alone was not enough for production performance, and the new TileContext. It also spells Apple's MLX library "MXL"; the project's release notes spell it MLX.

For a team weighing it, the useful fine print is the workload. InfoQ says the best candidates are data-parallel loops with no dependencies between iterations, and that not every Java computation benefits from offloading.

The numbers are the project's own

InfoQ's piece contains no benchmark numbers and does not say who ran any. The figures that exist are on the project's pages and are vendor-supplied in the sense that the maintainers produced them.

The 7.3.0 notes report an "about 6x" LLM decode speedup on Metal from sharing command buffers, and thread-block sizing gains of up to 99x on CUDA and 2.7x on Metal for loops with no usable divisor. The README reports its jitLLM inference engine at 117 tokens a second on an RTX 5090. None of these names a baseline, a model or a run count, which the Tov benchmark dispute shows is where comparisons go wrong.

jitLLM and the commercial layer

InfoQ says jitLLM v2.0 shipped alongside 7.3 as a Java inference engine in a single JAR, positioned by the team against llama.cpp and Ollama. The README calls it the official GPU engine for LangChain4j and Quarkus. InfoQ adds that TornadoVM and jitLLM are commercialized through Beestream.ai, a spin-off founded by the TornadoVM team, a detail the README did not mention.

The project's integrations with Quarkus, Flink and Spring AI are reachable through SDKMan. In the Flink example InfoQ describes, a developer writes a standard Table API query and sets a flag, and the task manager checks for a local GPU before a library provider decides whether the query suits GPU execution.

A reader choosing between tools can compare this with the Wild linker's 0.11.0 release, which also reported a performance cost alongside a feature. The next release to watch is 7.3.2 or 7.4, neither of which had been tagged when the releases page was read.