Pinned
Powered by TornadoVM🚀🚀🚀 www.infoq.com/news/2025/06... GPULlama3.java Brings GPU-Accelerated LLM Inference to Pure Java - InfoQ
Christos Kotselidis
@kotselidis.net
Associate Professor at The University of Manchester Co-founder at Beestream.ai Project Lead #TornadoVM #Java #AI #GPU
126 followers163 following47 posts
Pinned
Powered by TornadoVM🚀🚀🚀 www.infoq.com/news/2025/06... GPULlama3.java Brings GPU-Accelerated LLM Inference to Pure Java - InfoQ
Christos Kotselidis reposted
TornadoVM 8.0.0 is out! After 10 years: thank you, Graal. Java kernels are now compiled by TC, our own pure-Java compiler to CUDA, OpenCL and Metal. • AOT ready • No JVMCI • Now Apache 2.0 • Your code doesn't change 👉 www.tornadovm.org/blogs/tornad... #OpenSource #Java #AI #GPU
Great to see @tornadovm.org and @jitllm.io hit #1 on Hacker News!” Check out the latest article on @infoq.com by @ogkittylyst.bsky.social : www.infoq.com/news/2026/10...
Christos Kotselidis reposted
www.infoq.com/news/2026/10... - TornadoVM version 7 brings GPU workloads directly from Java
Christos Kotselidis reposted
Had so much fun at #JDD 2026 in Kraków yesterday. I presented “Java Goes CUDA-Native”: new @tornadovm.org features that let @jitllm.io run LLM inference at GPU-native speed, from Java. tornadovm.org jitllm.io #opensource #Java #TornadoVM #GPU #AI
Christos Kotselidis reposted
Yesterday my colleagues @maryxek.bsky.social and @mikepapadim.bsky.social showed live demos about using NVIDIA cuFFT and cuBLAS with @tornadovm.org for signal filtering and @jitllm.io! NVIDIA performance is a jar away from Java. #Java #CUDA #TornadoVM
Optimized GPU libraries (e.g. cuBLAS, cuDNN, etc.) by NVIDIA, now one jar away for your Java - without caring about JNI. We also show demos with Apache Flink and @jitllm.io. Watch our talk in @devoxx.com 👇 youtu.be/6SPnBmQrYEk?... #opensource #AI #TornadoVM #Java #GPU
With the latest #TornadoVM version you can accelerate end-to-end Apache #Flink pipelines on the GPUs with native #LLM inference. No Python, no external APIs, all in one deployable jar!
Optimized GPU libraries (e.g. cuBLAS, cuDNN, etc.) by NVIDIA, now one jar away for your Java - without caring about JNI. We also show demos with Apache Flink and @jitllm.io. Watch our talk in @devoxx.com 👇 youtu.be/6SPnBmQrYEk?... #opensource #AI #TornadoVM #Java #GPU
Christos Kotselidis reposted
We are at @devoxx.com, getting ready for our first hands-on #GPU lab for #Java! We will be in BOF2 room at 16:50, with live demos! @tornadovm.org @jitllm.io
Christos Kotselidis reposted
jitLLM is part of the @langchain4j.dev 1.21.0 release! 🚀 Read more here: docs.langchain4j.dev/integrations... #opensource #LLM #GPU #Java #LangChain4j #Inference
#LangChain4j 1.21.0 is out! 🚀 Meet Decision Models: instant yes/no, choice and scale answers with probabilities. Decision Services turn any Java interface into a classifier. Plus model routing, re-ranking, tool selection, guardrails and agentic routing! github.com/langchain4j/...
Christos Kotselidis reposted
Next week we will be in @devoxx.com and speak about @tornadovm.org, @jitllm.io and how we accelerate Java on GPUs, using #NVIDIA native libraries! Come and talk to us, to learn more! #opensource #Java #AI #TornadoVM #jitLLM #GPU
Our latest addition to the TornadoVM family. Blazing fast AI inference at native speed!
Can Java compete with llama.cpp/ollama? YES! With @jitllm.io! jitLLM uses NVIDIA CUDA features and libraries (cuBLAS, cuDNN) and reaches 0.90× to 1.12× of llama.cpp's throughput! jitLLM is empowered by @tornadovm.org and will be soon in @langchain4j.dev and @quarkus.io. github.com/beehive-lab/...
Christos Kotselidis reposted
📣 New Press Release: AXHEL, an open-source C++ library from @DII_Pisa accelerating the arithmetic behind Homomorphic Encryption on AArch64. 👉 aero-project.eu/wp-content/u... Repo: github.com/AERO-Project-EU/AXHEL @hipeac.bsky.social @eucloudedgeiot.bsky.social #opensource #he #encryption
Christos Kotselidis reposted
TornadoVM 7.0.0 is out! New TileContext API: Write #GPU kernels from #Java in tiles, not threads, via #NVIDIA #cuTile model. Now, cuTile methods can be mixed with cuBLAS functions in one TaskGraph, optimizing data transfers. 👉 github.com/beehive-lab/TornadoVM/releases/tag/v7.0.0 #opensource #AI
Christos Kotselidis reposted
Christos Kotselidis reposted
Last week, at ApplePies Conference 2026, the Department of Information Engineering at the University of Pisa presented new results from our project on #FPGA acceleration for #homomorphicencryption. Read the preprint: zenodo.org/records/2277...
Christos Kotselidis reposted
It’s been a busy week in the #Java ecosystem ➤ The GA release of TornadoVM 6.0; point releases of JReleaser, LangChain4j, Java Operator SDK, JHipster, Kotlin Toolchain and Yupiik Fusion; and maintenance releases of Micronaut and GraalVM Development Kit. 🔗 #InfoQ News Roundup: buff.ly/W01plzL
Christos Kotselidis reposted
New #TornadoVM release, and it's JVMCI-free. 📌No more separate jdk25 branch, no back-syncing every PR from develop - one build for JDK 21–27. 📌Native code also migrated from JNI to Panama's FFM API - 7.5k lines of C++ gone. Kudos to all contributors! 👏 #opensource #Java #GPU #AI
TornadoVM 6.0.0 is here - JVMCI-free and JNI-free. ✅ Compatible with any JDK 21-27 distribution. ✅ Cold start down 37%. ✅ 4.1× faster GEMM with Tensor-Core MMA. ✅ ~7,400 lines of C/C++ glue deleted. 📖 New blog: www.tornadovm.org/blogs/tornad... #Java #GPU #CUDA #TornadoVM #OpenSource #HPC #AI
Christos Kotselidis reposted
TornadoVM is now modernized and forwards compatible with any JDK > 21 while receiving significant performance boosts! Can't wait to see those benefits translating to our AI inference engine in our next release!
TornadoVM 6.0.0 is here - JVMCI-free and JNI-free. ✅ Compatible with any JDK 21-27 distribution. ✅ Cold start down 37%. ✅ 4.1× faster GEMM with Tensor-Core MMA. ✅ ~7,400 lines of C/C++ glue deleted. 📖 New blog: www.tornadovm.org/blogs/tornad... #Java #GPU #CUDA #TornadoVM #OpenSource #HPC #AI
Great news! Three talks accepted at @devoxx.com 2026! Hope to see you all and discuss the future of #Java and #AI!
3 sessions accepted at @devoxx.com Belgium! 🛠️ Tools in Action: NVIDIA ecosystem from Java via #TornadoVM's Hybrid API 🎤 Talk: Building LLM inference engines in Java with TornadoVM 🧑💻 Hands-on Lab: From install to insight — a GPU lab for Java devs Stay tuned #Java #CUDA #LLM #AI #GPU #opensource
Christos Kotselidis reposted
Great effort and what an achievement! #LLM inference implemented in #Java, automatically accelerated on #GPUs with #TornadoVM & CUDA native support! #opensource #AI #Java #GPU
GPULlama3 v1.0.0 is out - now #CUDA-native ⚡ ✅ TornadoVM CUDA backend w/ tensor-core (MMA) batch prefill ✅ FP16 & Q8_0 · Llama, Qwen, Devstral & more models ✅ An OpenAI-compatible server (llama-tornado --server) Pure #Java. No JNI. github.com/beehive-lab/GPULlama3.java #opensource #AI #LLM #GPU
Our #Java #AI inference engine just got another major update with a huge performance leap and native #CUDA integration!
GPULlama3 v1.0.0 is out - now #CUDA-native ⚡ ✅ TornadoVM CUDA backend w/ tensor-core (MMA) batch prefill ✅ FP16 & Q8_0 · Llama, Qwen, Devstral & more models ✅ An OpenAI-compatible server (llama-tornado --server) Pure #Java. No JNI. github.com/beehive-lab/GPULlama3.java #opensource #AI #LLM #GPU
Christos Kotselidis reposted
The final release of the AERO software stack is here! Our new v2.0 leaflet showcases how we've optimized cloud deployments on top of the EU processor designs & accelerators. 📄 Get the brochure: aero-project.eu/posts/aero-v... #OpenSource #cloud @hipeac.bsky.social @euprojectpulse.bsky.social
Christos Kotselidis reposted
It’s been a busy week in the #Java ecosystem ➤ a reintroduction of Value Objects (Preview); the GA release of WildFly 41; the July 2026 edition of Open Liberty 26.0.0.7; point releases of TornadoVM, Apache TomEE, Java Operator SDK and LangChain4j; and more. 🔗 #InfoQ News Roundup: bit.ly/4pvegky
Christos Kotselidis reposted
New results presented at #IOLTS2026 by the Dept. of Information Engineering, Università di Pisa! Our work shows how #ARM #SVE accelerates Microsoft SEAL, boosting CKKS #homomorphicencryption performance. 👉 aero-project.eu/posts/aero-i... #Cybersecurity #Research
Christos Kotselidis reposted
JVM Weekly by @askowronski.bsky.social features @tornadovm.org 5.0.0 release that brings native CUDA support (cuBLAS, cuDNN, etc.) via the new Hybrid API. Same hardware-agnostic Java, can be extended for AMD & Apple Metal libraries. www.linkedin.com/pulse/java-g... #Java #GPU #TornadoVM #opensource
Christos Kotselidis reposted
It’s been a busy week in the #Java ecosystem ➤ The GA release of TornadoVM 5.0; point releases of JHipster, Keycloak and Google ADK; maintenance releases of GraalVM Native Build Tools and Micronaut; the OmniFish Build of Payara and introducing Vidocq, and more. 🔗 #InfoQ News Roundup: bit.ly/4pq19Bl
Christos Kotselidis reposted
TornadoVM 5.0.0 beacons a very important milestone: Native CUDA support! JIT-compiled Java kernels and native cuBLAS/cuDNN/cuFFT calls now live in the same task graph - same GPU buffers, same CUDA stream, no round-trips between them. #opensource #Java #TornadoVM #GPU #AI 👇
TornadoVM 5.0.0 is CUDA-native. One line of #Java. 74× faster. TornadoVM's Hybrid API drops a native cuBLAS call straight into a TaskGraph - same buffers, same #CUDA stream, zero host round-trips. ⚡ Run with the Hybrid API: bit.ly/4vqL7sq 📖 Full engineering deep dive: bit.ly/44VecBf #AI #GPU
TornadoVM has officially transitioned into a new phase. One line of code, > 150x performance. Becoming #CUDA native unlocks new potentials for high performance #Java #AI inference with zero dependencies. Checkout our latest release, blogs, and new website 👇
TornadoVM 5.0.0 is CUDA-native. One line of #Java. 74× faster. TornadoVM's Hybrid API drops a native cuBLAS call straight into a TaskGraph - same buffers, same #CUDA stream, zero host round-trips. ⚡ Run with the Hybrid API: bit.ly/4vqL7sq 📖 Full engineering deep dive: bit.ly/44VecBf #AI #GPU
Christos Kotselidis reposted
TornadoVM 5.0.0 is CUDA-native. One line of #Java. 74× faster. TornadoVM's Hybrid API drops a native cuBLAS call straight into a TaskGraph - same buffers, same #CUDA stream, zero host round-trips. ⚡ Run with the Hybrid API: bit.ly/4vqL7sq 📖 Full engineering deep dive: bit.ly/44VecBf #AI #GPU
Christos Kotselidis reposted
Final month of #AERO - and we're not slowing down! Run GPU-accelerated LLMs in pure Java from a @quarkus.io #Langchain4j apps. No Python. No CUDA. Just #Java with @tornadovm.org. A collab between @manchester.ac.uk & Red Hat! 🔗 aero-project.eu/posts/runnin... #Java #LLM #AI #LLM #GPU #opensource
Christos Kotselidis reposted
Last week I was in Sofia for @jprimeconf.bsky.social! I presented our latest work on @tornadovm.org and #GPULlama3 for seamless #GPU acceleration of local #AI inference with @quarkus.io and @langchain4j.dev. Thanks to organizers, speakers, attendees & photographers for a great event. #opensource
Christos Kotselidis reposted
Excited to attend #jPrime2026 in Sofia this week! Looking forward to sharing our work on #GPULlama3, @tornadovm.org and their integration with @quarkus.io @langchain4j.dev. Join me on Thursday (4/6) at 14:05, in Hall A: jprime.io/agenda/275 #jPrime26 #Java #GPUs #AI #TornadoVM
Christos Kotselidis reposted
All the way from From JVM to GPUs: Building GPU-Accelerated AI Libraries in Java with TornadoVM! Learn about it from Thanos Stratikopoulos! Live at #jPrime26! Grab Your Pass: 🗓️ 3-4 June 2026! 📍 Sofia 🇧🇬! 🎫 jprime.io/tickets
Christos Kotselidis reposted
This week, Mary Xekalaki @maryxek.bsky.social presents the latest work at #ICPE2026! “Pulse: A Profiling & Visualization Infrastructure for Heterogeneous Managed Systems” 📅 Thu, May 7 🕒 14:40–15:00 🔗 github.com/beehive-lab/... 📄 zenodo.org/records/1895... #opensource #Java #GPU #TornadoVM
New paper accepted at #ICPE2026! We present TornadoVMPulse, a profiling & visualization tool for #TornadoVM to analyze and optimize #Java apps on #GPUs. Results for #GPULlama3: ⚡ 18.4% faster execution 📉 38% lower data-transfer cost 🇪🇺 @aero-project.eu 📍 Florence, May26 zenodo.org/records/1895...
Christos Kotselidis reposted
Great to see this work from the TornadoVM team! As part of the AERO project, we’re excited to support open-source innovation that pushes the boundaries of running local LLMs on GPUs directly from #Java. @euprojectpulse.bsky.social @hipeac.bsky.social @eucloudedgeiot.bsky.social #opensource
Last week we gave two presentations at #OCX26 and @devoxxgreece.bsky.social - and the recordings are now available, scan the QR code! From open-source evolution to running LLMs on GPUs directly from Java - there’s a lot to explore. Stay tuned for more! #opensource #Java #TornadoVM #AI #LLM #GPUs
Christos Kotselidis reposted
TornadoVM team at @devoxxgreece.bsky.social Great discussions, insightful sessions, and a fantastic community pushing the boundaries of high-performance #Java and #GPUs. 📸 Photo credit: @ddoutsiopoulos.bsky.social
Christos Kotselidis reposted
Last week we gave two presentations at #OCX26 and @devoxxgreece.bsky.social - and the recordings are now available, scan the QR code! From open-source evolution to running LLMs on GPUs directly from Java - there’s a lot to explore. Stay tuned for more! #opensource #Java #TornadoVM #AI #LLM #GPUs
Christos Kotselidis reposted
TornadoVM is featured in the latest Java Pro Magazine special issue (“Java in the Age of AI”, April 2026). 📌https://javapro.io/wp-content/uploads/2026/04/JAVAPRO_04-2026.pdf#page=42 @hipeac.bsky.social @euprojectpulse.bsky.social @eucloudedgeiot.bsky.social #horizoneurope #opensource #community
TornadoVM is featured in the April 2026 issue of Java Pro Magazine - “Java in the Age of AI”! 👉 javapro.io/wp-content/u... In this article, @mikepapadim.bsky.social shared our experiences on building #GPU accelerated #LLM inference libraries with #TornadoVM. #Java #community #AI
Christos Kotselidis reposted
TornadoVM is featured in the April 2026 issue of Java Pro Magazine - “Java in the Age of AI”! 👉 javapro.io/wp-content/u... In this article, @mikepapadim.bsky.social shared our experiences on building #GPU accelerated #LLM inference libraries with #TornadoVM. #Java #community #AI
Christos Kotselidis reposted
airhacks.tv episode 145 Q&A: BCE pattern from 1992 meets Java 25—Agent Smith (~130KB, zero dependencies) for multi-agent delegation, GPULlama: local LLMs via TornadoVM, ZUnit test runner in ~300 lines. No Maven, no Gradle, just Java 25 👉 adambien.blog/roller/145th... #airhacks #java #llm #airhacks
Christos Kotselidis reposted
At Devoxx Greece today? Check out "Building and Running LLMs on GPUs Directly from Java with TornadoVM and GPULlama3.java" with Michalis Papadimitriou at 10:55-11:35 in MC 3. github.com/quarkusio/qu... #quarkusworldtour
Christos Kotselidis reposted
We are at #OCX26 this week! Yesterday, Thanos Stratikopoulos from University of Manchester presented the #opensource practices used in @tornadovm.org project for resulting into a production ready software! Stay tuned for more! @euprojectpulse.bsky.social #opensource #cloud #GPU #Java #TornadoVM
Christos Kotselidis reposted
On my way to Brussels 🇧🇪 for #OCX2026 “TornadoVM: From Academia to Open Source Success” 🕓 Tomorrow, 16:00–16:45 | 📍 Studio 4 👉 www.ocxconf.org/event/2026/a... Let’s connect if you’re there! @tornadovm.org #OpenSource #Java #TornadoVM
Christos Kotselidis reposted
Our @cern.voxxeddays.ch 2026 talk is now live! We introduce TornadoVM, and show how it’s already used in the ESA #GAIA mission. Highlight: #GPULlama3, a #GPU accelerated #LLM inference in #Java, integrated with @langchain4j.dev + #Quarkus 👉 www.tornadovm.org/gpullama3 📽️ youtu.be/oRVtTEbtt_Y?...
Official #Apple Metal support just landed to #TornadoVM + #CUDA graphs 🚀
TornadoVM v4.0.0 is out! ✔ Apple Metal backend → native Apple Silicon ✔ SIMD intrinsics for PTX → more performance ✔ CUDA Graphs → reduced overhead ✅ Downloads at TornadoVM → www.tornadovm.org/downloads Big thanks to our contributors! 👏 #Java #JDK25 #JDK21 #TornadoVM #opensource #AI #GPUs
Christos Kotselidis reposted
If you're a #Java developer on #Apple computers, #TornadoVM now enables native execution on Apple Silicon via Metal (MSL). Big applause to all contributors! 👏 #opensource #Java #AI #GPUs
TornadoVM v4.0.0 is out! ✔ Apple Metal backend → native Apple Silicon ✔ SIMD intrinsics for PTX → more performance ✔ CUDA Graphs → reduced overhead ✅ Downloads at TornadoVM → www.tornadovm.org/downloads Big thanks to our contributors! 👏 #Java #JDK25 #JDK21 #TornadoVM #opensource #AI #GPUs
Christos Kotselidis reposted
We’re thrilled to share that the sessions from Voxxed Days CERN 2026 have officially started shipping on the @devoxx.com YouTube channel! 📺 Watch the full playlist here: www.youtube.com/playlist?lis...
Christos Kotselidis reposted
We’re excited to see TornadoVM featured at OCX 2026. Thanos Stratikopoulos will share how TornadoVM evolved from academic research into an open source success story. 🕓 16:00-16:45 📍 Studio 4 www.ocxconf.org/event/2026/a... #OCX2026 #TornadoVM #opensource #Java #HPC #research
Christos Kotselidis reposted
This is a very useful tool for TornadoVM developers working on optimizing Java applications for GPUs. GitHub 👉 github.com/beehive-lab/... Preprint 👉 zenodo.org/records/1895... #opensource #TornadoVM #Java #GPU #ICPE2026
New paper accepted at #ICPE2026! We present TornadoVMPulse, a profiling & visualization tool for #TornadoVM to analyze and optimize #Java apps on #GPUs. Results for #GPULlama3: ⚡ 18.4% faster execution 📉 38% lower data-transfer cost 🇪🇺 @aero-project.eu 📍 Florence, May26 zenodo.org/records/1895...