Vizuara Kernel Engineering Founding Cohort · 2026
This is the waitlist, not registration yet. Full workshop registration opens in about two weeks, and the founding cohort begins in September 2026. Join below to be first in line.
Vizuara
Vizuara AI Labs

Vizuara's Kernel Engineering
Workshop is here

The most modern GPU kernel-engineering program anywhere, built over two years, taught live, and explained from the ground up, from the silicon up to Flash Attention 4, Blackwell, and kernels written by AI. If you want to actually be ready for GPU performance-engineering roles at the frontier, this is it.

The third and final chapter of Vizuara's GPU trilogy, after 5D Parallelism and the Inference Engineering Workshop.

25 live sessions 6 parts lectures paired with live coding capstone sponsored by a top SF GPU startup starts late September 2026 · dates TBC
Apply to the waitlist → See the curriculum
Built on the modern inference stack
NVIDIA Hopper · H100 Blackwell · GB200 CUDA Triton CUTLASS PyTorch vLLM DeepSeek
The exact tools and hardware you'll write kernels for: Hopper & Blackwell tensor cores, CUDA, Triton, CUTLASS/CuTe, and the production inference stack.
Why this is different

The most modern kernel curriculum anywhere

Most GPU courses stop at 2022, and the few modern ones are written for people who already know everything. This one lives on the 2025-2026 frontier but explains each idea from first principles, with, for every topic, why it matters, where it's used, and which companies use it.

Explained simply

Every session opens with the idea in simple words before the jargon, you'll leave knowing what the terms actually mean.

The real frontier

Flash Attention 4, Blackwell, NVFP4, DeepSeek, AI-written kernels, the newest material, not a 2022 rerun.

Built by hand, live

Every lecture is paired with a live-coding session. You write the kernels yourself, step by step.

Job-ready output

Maps 1:1 to what frontier companies like Wafer, NVIDIA and the top labs actually hire for.

The curriculum

Six parts, from the CPU to AI-written kernels

A single, deliberate arc. Each concept lecture (Dr. Raj Dandekar) is paired with a live-coding session (Shubham Panchal) so theory and practice go hand in hand. Click any session, each one explains the idea simply, plus why it matters, where it's used, and who uses it.

Part IParallelism, from the CPU up1 session + 3 pairs

Get comfortable with "doing many things at once" on a CPU first, then meet the GPU and how you program it, so every kernel that follows sits on solid foundations.

CPUParallelism on the CPUThe three ways a computer does many things at once, before you watch the GPU do it at massive scale.Live session · Shubham›
Inside
  • Do the same math on many numbers at once (SIMD): one instruction, eight results, using AVX, Arm Neon, and Google Highway, and watching what the compiler speeds up for you automatically.
  • Run many tasks at the same time: lightweight "threads" done the modern way (Go goroutines, Java virtual threads, Rust's tokio) versus the classic way (pthreads, OpenMP).
  • How one CPU core secretly overlaps work (pipelining, out-of-order execution), the same trick the GPU takes to the extreme.
Why it matters It's the mental model behind all fast code, do more work at the same time.
Where it's used Every high-performance library, long before you touch a GPU.
Who uses it Game engines, high-frequency trading, NumPy & PyTorch's CPU paths.
Lecture 1 · Raj
How fast can this go?
↔
Coding session 1 · Shubham
Roofline lab: predict then measure
›
The lecture
What's the speed limit, and why?
  • Every kernel is held back by one of two things: how fast the chip can do math, or how fast it can move data.
  • The "roofline": a simple chart that tells you the fastest a task could run before you write a line of code.
  • One number ("math done per byte loaded") predicts almost everything.
The live build
Predict, then check on a real GPU.
  • Time real PyTorch operations and see which limit each one hits.
  • Guess "math-bound or memory-bound?", then confirm it on hardware.
Why it matters You learn the speed limit before optimizing, so you never waste effort chasing the wrong thing.
Where it's used The first thing any performance engineer does with a new kernel.
Who uses it NVIDIA, PyTorch, and every ML-infra team.
Lecture 2 · Raj
The CUDA programming model
↔
Coding session 2 · Shubham
Your first CUDA kernels + GPU Puzzles
›
The lecture
How a GPU actually runs your code.
  • Thousands of tiny threads, grouped into "warps" of 32 that move in lockstep.
  • What it costs when threads in a group disagree and have to wait ("warp divergence").
  • How your code turns into the real machine instructions, and why it pays to read them.
The live build
Write your first kernels.
  • Launch your very first GPU programs from scratch.
  • Work through GPU Puzzles, bite-sized kernel challenges.
  • Look at the actual assembly your code compiled to.
Why it matters The foundation, you can't speed up what you don't understand.
Where it's used Every CUDA program ever written.
Who uses it Anyone doing GPU work, NVIDIA and every AI lab.
Lecture 3 · Raj
The memory hierarchy in anger
↔
Coding session 3 · Shubham
The matrix-transpose ladder
›
The lecture
Why moving data is the real problem.
  • GPUs are fast at math but slow to fetch data, so how you arrange memory access is everything.
  • When neighbouring threads read neighbouring memory, it's ~10× faster ("coalescing"), the single biggest lever.
  • Two quiet traps that halve your speed: "bank conflicts" and low "occupancy".
The live build
Make a simple kernel fast, step by step.
  • Take a matrix "transpose" and speed it up one fix at a time.
  • Use NVIDIA's profiler (Nsight Compute) to see exactly where the time goes.
Why it matters Memory, not math, is usually the bottleneck, fixing it gives the biggest wins.
Where it's used Every GPU kernel that touches a lot of data.
Who uses it NVIDIA, DeepSeek, vLLM, and every inference team.
Part IIThe GEMM worklog: naive → cuBLAS3 pairs

Take the single most important operation in AI, matrix multiply, from a version that uses ~1% of the GPU to one that beats NVIDIA's own library, one improvement at a time.

Lecture 4 · Raj
GEMM worklog I
↔
Coding session 4 · Shubham
GEMM by hand I
›
The lecture
The first four versions of matrix multiply.
  • Matrix multiply ("GEMM") is ~90% of what a neural network does.
  • A naive version uses only 1.3% of the GPU, we'll see exactly why.
  • Two classic fixes (coalescing + "tiling") take it to 36.5%, measuring each gain.
The live build
Write them yourself.
  • Code the first four matrix-multiply kernels by hand.
  • Profile each and watch the number climb from 1.3% → 36.5%.
Why it matters Matrix multiply is the heart of every model, make it fast and you make everything fast.
Where it's used Training and running every AI model on Earth.
Who uses it NVIDIA (cuBLAS) and every AI lab.
Lecture 5 · Raj
GEMM worklog II
↔
Coding session 5 · Shubham
GEMM by hand II
›
The lecture
The advanced tricks that reach library speed.
  • Reuse data in the fastest memory (registers), load four numbers at once ("float4"), and let the code auto-pick its best settings.
  • "Warptiling", the technique that gets you to 93.7% of NVIDIA's hand-tuned library.
The live build
Reach near-library speed by hand.
  • Push your own matrix multiply toward 93.7% of cuBLAS.
  • Watch the compiled code shrink from 8 memory loads to 2, the "aha" moment.
Why it matters This is the gap between a hobby kernel and a production one, real, measurable speed.
Where it's used The core of every fast ML library.
Who uses it NVIDIA, OpenAI, Meta, anyone training big models.
Lecture 6 · Raj
Tensor cores, the second worklog
↔
Coding session 6 · Shubham
A WMMA tensor-core GEMM
›
The lecture
The GPU's special matrix-math units.
  • Modern GPUs have "tensor cores" that do matrix math ~10× faster than normal cores.
  • How to feed them the exact data layout they demand.
  • The precision menu (FP16 / BF16 / TF32): trading a little accuracy for a lot of speed.
The live build
Program the tensor cores directly.
  • Use the tensor cores (WMMA) and beat your own best regular kernel.
Why it matters Tensor cores are why modern AI is affordable, using them well is a must-have skill.
Where it's used All modern training and inference.
Who uses it NVIDIA and every frontier lab.
Part IIIAttention & profiling2 pairs

Learn to find why a kernel is slow like a pro, then build the operation at the heart of every LLM, FlashAttention, and kick off your capstone.

Lecture 7 · Raj
Profiling & debugging like a pro
↔
Coding session 7 · Shubham
Debug 3 sabotaged kernels
›
The lecture
Find the real problem, don't guess.
  • Read NVIDIA's profiler like a pro and pinpoint the true bottleneck.
  • The professional debugging toolkit, the same workflow the vLLM team uses.
The live build
Fix broken kernels, live.
  • Hunt down and fix the bugs in three sabotaged kernels, under time pressure.
Why it matters Finding why a kernel is slow or wrong is half the job.
Where it's used Daily work at every GPU / inference team.
Who uses it vLLM, NVIDIA, and all inference teams.
Lecture 8 · Raj
Attention: the kernel that ate the world
↔
Coding session 8 · Shubham
Build FlashAttention v1 live
›
The lecture
The operation behind every LLM.
  • "Attention" is what lets a model look back over your whole prompt, and it's the biggest memory hog.
  • The clever trick ("online softmax") that made it fit in fast memory and unlocked long context.
  • Why generating text one word at a time is a different, memory-bound problem.
The live build
Build the famous fast-attention kernel.
  • Write FlashAttention from scratch, and start your capstone.
Why it matters FlashAttention is why long-context, ChatGPT-style models are even possible.
Where it's used Every LLM, everywhere.
Who uses it OpenAI, Anthropic, Meta, Google, DeepSeek.
Part IVThe modern frontier6 deep-dives

The 2025-2026 kernels the best labs ship right now: Hopper, Blackwell, DeepSeek, and Flash Attention 4, each one explained from the ground up, not assumed.

Deep-dive 1FlashAttention from scratchRebuild the famous fast-attention kernel properly, then see what changed in FA2 and FA3.›
Inside
  • Rebuild the full fast-attention kernel, cleanly.
  • The trick that keeps the math correct while streaming through data, plus "masking" so the model only looks backwards.
  • What improved in the newer versions: FA2 (splits the work better) and FA3 (uses the H100's async features).
Why it matters The reference kernel every inference stack quietly depends on.
Where it's used LLM training and serving.
Who uses it OpenAI, Anthropic, Together AI, vLLM, SGLang.
Deep-dive 2Beating cuBLAS on an H100Use the H100's newest features to build a matrix multiply that beats NVIDIA's own library.›
Inside
  • Use the H100's newest features to move data (TMA) and do matrix math (WGMMA) in the background, so nothing waits.
  • "Warp specialization": some threads fetch data while others compute, like an assembly line.
  • Put it together into a matrix multiply that beats NVIDIA's own library.
Why it matters Squeezing the last 10-20% from the most expensive hardware saves millions of dollars.
Where it's used Frontier training clusters.
Who uses it NVIDIA, DeepSeek (DeepGEMM), OpenAI.
Deep-dive 3Triton → CUTLASS → CuTe-DSLThe modern, faster ways to write kernels, including writing them in Python.›
Inside
  • Triton: write a fast kernel in ~40 lines of Python instead of hundreds of lines of C++.
  • CUTLASS: NVIDIA's powerful (but hard) C++ toolkit, and its layout system.
  • The big 2025 shift: top kernels (FA4, DeepGEMM) are now written in Python that compiles down to the metal.
Why it matters The modern way to write kernels, far less code, the same speed.
Where it's used Most new kernel development today.
Who uses it OpenAI (Triton), NVIDIA (CUTLASS), DeepSeek.
Deep-dive 4Inference-serving kernelsThe kernels that make AI products fast and cheap enough to actually ship.›
Inside
  • Two very different jobs: reading your prompt ("prefill") vs writing the answer one word at a time ("decode").
  • PagedAttention: how vLLM packs many users' conversations into memory efficiently.
  • Speculative decoding: a small model drafts, a big model checks, 2-3× faster replies.
  • Running models in lower precision (FP8, 4-bit) to serve more users per GPU.
Why it matters This is what makes AI products fast and cheap enough to ship to millions.
Where it's used Every AI API and chatbot you've used.
Who uses it vLLM, SGLang, OpenAI, Anthropic, NVIDIA (TensorRT-LLM).
Deep-dive 5Blackwell & NVFP4NVIDIA's newest chip and its 4-bit number format that roughly doubles throughput.›
Inside
  • NVIDIA's newest chip (Blackwell) has brand-new tensor cores ("tcgen05") and a new fast on-chip memory ("TMEM").
  • NVFP4: a tiny 4-bit number format that roughly doubles inference throughput.
  • How two chips can even team up on a single operation.
Why it matters The very newest hardware, being early here is a rare, in-demand skill.
Where it's used 2025-26 frontier data centres.
Who uses it NVIDIA and the labs deploying on GB200.
Deep-dive 6Flash Attention 4The fastest attention kernel on Earth right now, explained from the ground up.New in 2026›
Inside: the FlashAttention story, FA1 → FA4
  • Attention has been rewritten four times as GPUs changed. FA4 is the newest, built for Blackwell.
  • The key idea, simply: the matrix units got so fast that the "softmax" step (a normalization the model does) became the slow part, so FA4 rearranges the whole kernel so the fast units never sit idle.
  • Its clever move ("adaptive softmax") is to only redo that normalization when the numbers actually change enough, skipping most of the work. It's the first attention kernel to break a "petaflop" (a thousand trillion operations a second), and it's written in Python.
Why it matters The fastest attention that exists today, you'll understand it before almost anyone.
Where it's used Cutting-edge Blackwell inference (brand-new, still in beta).
Who uses it Tri Dao / Together AI, and NVIDIA (folded into cuDNN).
Part VAI-written kernels2 deep-dives

The newest twist of all: tiny hand-crafted kernels that beat huge libraries, and AI models that now write GPU kernels themselves, and how a kernel engineer guides them.

Deep-dive 7DeepSeek: FlashMLA & DeepGEMMProof that a small, well-crafted kernel can beat the giants.›
Inside
  • DeepSeek open-sourced tiny, super-fast kernels that beat much bigger libraries.
  • FlashMLA: fast attention for their memory-saving design.
  • DeepGEMM: a ~300-line matrix multiply that outperforms huge codebases, the lesson being that smart scheduling beats brute-force code.
Why it matters Proof a small, sharp kernel can beat the giants, a great story for your portfolio.
Where it's used DeepSeek's models, and now widely copied.
Who uses it DeepSeek and the open-source inference community.
Deep-dive 8LLM-generated kernels: the agent + profiler loopAI models are now writing GPU kernels, and your new job is to guide them.New in 2025-26›
Inside
  • The frontier: AI models are starting to write GPU kernels themselves.
  • KernelBench: the standard test for "can an AI write a correct, fast kernel?"
  • AlphaEvolve (Google DeepMind): an AI that discovered a faster matrix multiply and sped up a real attention kernel by ~32%.
  • Your new role: guiding these AI agents with a profiler in the loop, plus an honest look at where the hype breaks down.
Why it matters The future of the job, kernel engineers increasingly supervise AI that writes kernels.
Where it's used Research today, production tomorrow.
Who uses it Google DeepMind, NVIDIA, Stanford, Meta.
Part VICapstone, sponsored by a GPU startupcapstone

A real production problem, sponsored by our GPU-startup partner and judged by their engineers at demo day, the closest thing to an on-site interview before the on-site interview.

CapstoneYour capstone: a real GPU-startup kernel problemSolve a genuine production problem and present it to the partner's team.›
Inside
  • Our partner (a top San Francisco GPU startup) gives you a real problem their team actually cares about.
  • You solve it end to end, using everything from Parts I-V.
  • At demo day, their engineers grade the capstones and pick the standouts.
Why it matters A real, industry-graded project for your portfolio and interviews.
Where it's used Exactly the kind of work these companies hire for.
Who uses it Our partner + the frontier labs hiring GPU engineers.
Who teaches it

Taught live by practitioners

Every session is live, concepts built from first principles, then written into working kernels in front of you.

Dr. Raj DandekarLectures
MIT PhD · Founder, Vizuara

Has designed and taught Vizuara's 5D Parallelism and Inference Engineering workshops, the first two chapters of this GPU trilogy, and is known for taking genuinely hard systems and ML material and making it click from first principles. He teaches the concept lectures.

5D Parallelism WorkshopInference Engineering WorkshopMIT PhD
Shubham PanchalLive coding
ML + on-device engineer · creator of SmolChat

An Android developer since 2017, Shubham specializes in deploying ML models on-device in Android apps, the creator of SmolChat (run LLMs locally, on-device). An active StackOverflow contributor and frequent Medium writer on ML, math and Android, he now works across Rust, C and C++ alongside Java/Kotlin backends. He leads the live-coding, where you write every kernel with him.

SmolChat, on-device LLMsRust / C / C++Kotlin / JavaStackOverflow · Medium
What you'll build

The kernels you write become your book

Every project is a chapter you author with your own hands. By the end, your work assembles into a beautifully illustrated engineering book, your personal record of everything you built, yours to keep, plus a capstone graded on a real production problem.

Part I-III→CPU parallelism, first kernels, the transpose ladder
GEMM→The worklog to ~94% of cuBLAS + a tensor-core GEMM
Attention→FlashAttention from scratch
Frontier→Hopper, Blackwell, Flash Attention 4, DeepSeek
AI kernels→The KernelBench / AlphaEvolve agent loop
Capstone→A real problem from our GPU-startup partner
You take it home
Your Kernel Engineering worklog

A hand-illustrated book assembled from the kernels you wrote and the numbers you measured, your proof of depth and your interview-prep reference for the frontier.

Something cool in store

A top GPU startup is sponsoring your capstone

Here's something we're genuinely excited about. We're partnering with a leading GPU startup based in San Francisco to sponsor the capstone, and it's built right into the workshop. Your final project won't be a toy: it'll be a real production problem from their team, judged by their engineers. We'll reveal exactly who they are soon.

PartnerRevealing soon
A top San Francisco GPU startup is sponsoring the capstone and bringing a real production problem into the cohort. The reveal is coming.

A real capstone problem

Not a toy. The partner brings a genuine production problem their team actually cares about.

Sponsored by the partner

The capstone is sponsored by a leading GPU startup, part of the workshop from day one.

Judged by their engineers

At demo day, the partner's engineers evaluate and grade the capstones and pick the standouts.

A door to the frontier

The strongest graduates get real visibility with a company hiring at the edge of AI infrastructure.

Your capstone is a real problem from a top San Francisco GPU startup, judged by their engineers at demo day, the closest thing to an on-site loop before the on-site loop. We'll announce the partner soon.
Be ready for the frontier

This gears you for real GPU-engineering jobs

GPU performance engineering is one of the most sought-after, hardest-to-fill skills in AI. This isn't a toy syllabus, it maps section-for-section onto what companies like Wafer, NVIDIA, and the top labs actually hire for.

Matmul → cuBLAS parity + tensor cores

The core screen for every GPU-perf role, tiling, vectorization, tensor cores, reasoning about speed byte-for-byte.

NVIDIAAnthropic JDWafer

FlashAttention + inference kernels

FlashAttention 1-4, PagedAttention, KV-cache, speculative decoding, the exact stack in frontier inference roles.

vLLM / SGLangTensorRT-LLM

Triton, CUTLASS & CuTe internals

The modern kernel-authoring layer, warp-specialized GEMM, and epilogue fusion.

Kernel Engineer (CUDA/Triton)

Profiling, PTX/SASS & the AI-kernel loop

Nsight/NCU, reading the compiled assembly, and the agent + profiler loop that frontier GPU startups productionize.

WaferKernelBench

These roles are among the highest-paid, most in-demand engineering jobs in the industry, the kind where a single week's pay dwarfs the cost of this workshop. You leave able to point at each line of a job description and say: built that, graded on it, wrote the chapter.

Investment

Worth every bit of it

$3,000
per person · founding cohort
Live & graded: every session taught live; your capstone judged by a top GPU startup.
A book you keep: your own illustrated worklog of every kernel you build.
Job-ready: maps 1:1 to frontier GPU-engineering roles that pay multiples of the fee.
Explained from first principles: from the CPU up to Flash Attention 4, nothing assumed.

Nobody else sells a single live, graded, book-producing path through all of this. Against what these skills unlock, $3,000 is a fraction of what the job it prepares you for pays in a single week.

When. Dates aren't finalised yet, but the founding cohort is expected to begin in the third or fourth week of September 2026. Join the waitlist and we'll confirm the schedule with you first.

This is an application, not a purchase. Fill it out only if you're a serious, dedicated learner who truly intends to join, ready to invest the time and the money to do it properly. We read every response.

Apply

Join the founding-cohort waitlist

Tell us why you want in. We are selecting a small group of committed people, so be honest and specific. Dates aren't finalised yet, but the cohort is expected to begin in the third or fourth week of September 2026.

By submitting, you confirm you are a serious applicant intending to enroll in the $3,000 founding cohort.

✓

You're on the waitlist

Thank you, your application has been received. We personally read every response. If you're a strong fit for the founding cohort, we'll be in touch by email with the next steps.