KDD 2026 Hands-on Tutorial

A Hands-on Tutorial for Diffusion Language Models with dLLM

Content

Focus

This tutorial is centered on the dLLM open-source repository, a unified framework for training, inference, and evaluation of diffusion language models.

Hands-on outcomes

Participants will run interactive inference, adapt models with reproducible recipes, reproduce benchmark results, and extend dLLM with custom samplers or objectives.

Audience

The tutorial is designed for researchers and practitioners who want practical, reproducible access to diffusion language model experiments and tooling.

Schedule

Tutorial Agenda

Module 1
50 min Foundations and quick demo Reproducibility challenges, dLLM repository structure, and a first open-model inference run.

Participants start from the practical motivation for diffusion language models: bidirectional denoising, non-autoregressive sampling, and why reproducible tooling is still hard to compare across papers.

Demo

Repository walkthrough plus a minimal inference command that loads an open dLLM checkpoint and generates text from a masked prompt.

Placeholder: add the exact command, checkpoint name, and expected sample output.

Module 2
25 min Inference experiments Compare decoding settings, generation order, and sampler behavior.

This module turns inference into a controlled experiment: attendees vary step count, temperature, remasking policy, and scheduler choices, then compare qualitative output and latency.

Demo

Side-by-side sampler comparison using the same prompt and seed, with token order visualization for the denoising trajectory.

Placeholder: add sampler presets, visualization screenshot, or notebook link.

Module 3
40 min Training and open recipes Unified trainer interface, BERT-Chat, Tiny-A2D, and adaptation workflows.

Participants inspect the recipe format, connect data configuration to the trainer, and see how small-scale adaptation can be reproduced without rewriting the pipeline.

Demo

Run a small recipe dry run, inspect logged configuration, and identify the files that control model, data, objective, and optimization settings.

Placeholder: add the training recipe path and the expected hardware/runtime note.

Module 4
20 min Unified evaluation Run evaluation and compare how configuration choices affect benchmark numbers.

This section focuses on making evaluation comparable: dataset selection, decoding configuration, metric reporting, and where small configuration differences can change reported results.

Demo

Launch one evaluation job, read the generated result file, and compare it against a reference result table.

Placeholder: add benchmark name, result table, and expected output file location.

Module 5
20 min Framework extension Implement and verify a minimal scheduler or sampler change.

Attendees see the extension points that make dLLM useful as research infrastructure: adding a sampler, swapping an objective, or modifying scheduling behavior while keeping the rest of the pipeline unchanged.

Demo

Modify a minimal sampler hook, run a smoke test, and compare the changed token trajectory with the baseline sampler.

Placeholder: add target source file, patch snippet, and smoke-test command.

Module 6
25 min Discussion and troubleshooting Tradeoffs, evaluation sensitivity, tooling gaps, and attendee questions.

The closing section ties the hands-on modules back to research judgment: when dLLMs are preferable to autoregressive baselines, where the tooling is mature, and where new contributions are still needed.

Discussion prompts

Collect attendee blockers from the exercises and map each one to debugging steps, configuration checks, or open-source contribution opportunities.

Placeholder: add FAQ entries from previous dry runs or audience questions.

Resources

Related Resources

Code dLLM GitHub Repository

Unified training, inference, and evaluation framework for diffusion language models.

Report dLLM: Simple Diffusion Language Modeling

Technical report introducing the dLLM framework and reproducible recipes.

Models dLLM Hub on Hugging Face

Released checkpoints and collections including Tiny-A2D and BERT-Chat.

People

Presenter and Contributors

In-person presenter

Lingjie Chen

Ph.D. student, University of Illinois Urbana-Champaign

Works on diffusion language models, post-training, open recipes, model adaptation, and unified evaluation workflows.

lingjie7@illinois.edu

Contributor

Zhanhui Zhou

Ph.D. student, University of California, Berkeley

Works on diffusion language models, agentic AI, and reproducible open-source tooling.

zhanhui@berkeley.edu

Contributor

Hanghang Tong

Professor, University of Illinois Urbana-Champaign

Works on large-scale data mining and machine learning for graph and multimedia data.

htong@illinois.edu

Contributor

Dawn Song

Professor, University of California, Berkeley

Works on AI safety and security, agentic AI, privacy, and trustworthy AI systems.

dawnsong@cs.berkeley.edu