Mosaic memory profiling tutorial #3744
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Introduces a beginner tutorial demonstrating how to use Mosaic for GPU memory analysis in PyTorch. The tutorial covers: - Analyzing memory savings from activation checkpointing - Debugging unexpected memory usage from abandoned code - Integrating Mosaic into training pipelines for CI/CD Includes graceful handling for environments without GPU access.
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/tutorials/3744
Note: Links to docs will display an error until the docs builds have been completed. This comment was automatically generated by Dr. CI and updates every 15 minutes. |
Add HAS_MOSAIC_CLI check to skip Mosaic CLI subprocess calls when the mosaic package is not installed. This prevents FileNotFoundError in CI environments that have CUDA but don't have Mosaic installed.
Remove check=True from subprocess.run calls to prevent exceptions when Mosaic CLI commands fail. Instead, check return codes and print informative messages. This allows the tutorial to run in environments where Mosaic is partially installed or configured differently.
Set __main__.__file__ to a valid file path if not present. Transformers library reads this file to inspect source code, so we provide the tutorial file path or fall back to the transformers module path if __file__ is not available.
basilwong
force-pushed
the
mosaic-memory-profiling-tutorial
branch
from
16248f0 to
a64f6c0
Compare
Wrap buggy model instantiation in try/except to handle ValueError from newer transformers versions that don't support experts implementation on GPT2Model. Falls back gracefully when the demo cannot run.
svekars
reviewed
Jan 27, 2026
svekars
reviewed
Jan 27, 2026
Co-authored-by: Svetlana Karslioglu <[email protected]>
- Add tutorial to "What's new in PyTorch tutorials" section - Add customcarditem in Profiling section of index.rst - Add customcarditem and toctree entry in ecosystem.rst
sekyondaMeta
added
the
skip-link-check
svekars
approved these changes
Jan 27, 2026
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
This PR adds a comprehensive tutorial on using Mosaic for GPU memory profiling in PyTorch.
Mosaic is a post-analysis tool for memory usage that was instrumental in debugging OOM issues during the 405B LLaMA training.
What users will learn
Tutorial structure
Requirements
pip install git+https://github.com/facebookresearch/mosaic.gitChecklist
_static/img/mosaic/Related Links