Selected tutorial · ICVGIP 2026
Modern image and video generation increasingly relies on latent-space transformers, including Diffusion Transformers (DiTs) and Multimodal Diffusion Transformers (MMDiTs). Their effectiveness is fundamentally constrained by the visual tokenizer that defines the latent representation: what visual information is preserved, how it is compressed, how semantics are organized, and what sequence structure the generator must learn.
This tutorial presents a focused framework for co-designing visual tokenizers and MMDiTs as one latent generative system. It examines discrete, continuous, hybrid, semantic, adaptive, and representation-autoencoder-based tokenizers, connecting their choices to generator design decisions such as patchified latent tokens, positional representations, timestep conditioning, modality-specific projections, joint and efficient attention, and diffusion or rectified-flow objectives.
The central focus is the tokenizer–MMDiT interface: token rate, latent dimensionality, topology, normalization, semantic structure, positional geometry, modality identifiers, attention layout, and decoding constraints. Through design principles, failure modes, and compact system-design exercises, participants examine how these choices affect optimization stability, attention cost, controllability, semantic fidelity, and generation quality in image and video generation, editing, and multimodal conditional generation.
Participants will leave with a practical methodology for selecting tokenizers, configuring MMDiTs, diagnosing interface mismatches, designing informative ablations, and evaluating tokenizer and generator choices as one integrated pipeline.
The session progresses from representation design to end-to-end generative system evaluation, with a dedicated question-and-answer segment.
Frame visual tokenization as the step that defines the latent space in which generation occurs. Introduce the trade-off between compressing images, videos, and related modalities and preserving the information, geometry, and semantic structure needed by the generator.
Compare discrete tokenizers, continuous latent autoencoders, representation autoencoders, semantic tokenizers, adaptive tokenization, and hybrid discrete–continuous approaches. Discuss latent capacity, compression ratio, token count, channel dimensionality, and semantic alignment, and how these choices affect reconstruction fidelity, temporal consistency, optimization stability, attention cost, and generative learnability.
Introduce latent diffusion and rectified-flow objectives, then derive DiTs as transformer denoisers over patchified latent tokens. Cover positional representations, timestep conditioning, adaptive normalization, self-attention, velocity or noise prediction, and efficient attention. Extend to MMDiTs through modality-specific projections and joint attention, including how cross-attention, dual-stream, single-stream, and hybrid designs depend on tokenizer output structure.
Analyze latent dimensionality, token density, normalization, semantic organization, positional geometry, modality identifiers, and attention masks as interface design choices. Discuss why strong reconstruction or denoising alone is insufficient when the representation and generator are misaligned, and why systems benefit from joint tokenizer–MMDiT design.
Cover text-to-image generation, video generation, image editing, multimodal conditioning, controllable synthesis, and domain-specific generation. Discuss fine-tuning, adapters, feature injection, guidance, inversion, and control branches for adapting systems to these settings.
Reserve time for participant questions, clarification of design trade-offs, and open research directions, connecting the tutorial principles to attendees’ own research problems and system-design constraints.
The tutorial is intended for graduate students, academic researchers, and industry practitioners working in computer vision, generative modeling, multimodal learning, and representation learning. Familiarity with deep learning, transformers, and diffusion models is helpful but not required. The material starts with foundational concepts and progresses to current research directions, making it useful to newcomers and experienced researchers interested in tokenizer–generator co-design.
By the end of the tutorial, participants should be able to:
Dolby Laboratories
Guan-Ming Su is Director at Dolby Laboratories, leading image technology development. His expertise spans multimedia coding, computer vision, 3D imaging, HDR, and machine learning. He has held roles in Dolby Vision, Marvell, and ESS Technology, contributing to video codec architecture and advanced imaging systems.
Dolby Laboratories
Shiv Gehlot is a Staff Researcher at Dolby Laboratories, working on computer vision, multimodal learning, and generative AI.
Dolby Laboratories
Aupendu Kar is a Senior Researcher at Dolby Laboratories working on computer vision and deep learning, with a focus on low-level vision.
Dolby Laboratories
Sutanu Bera is a Senior Researcher at Dolby Laboratories, working on video restoration and low-level image processing using deep learning.
ICVGIP 2026 takes place December 21–24, 2026, in Kolkata. Tutorials and workshops are scheduled for December 21 at the Indian Statistical Institute. Visit the ICVGIP 2026 website.