[docs] Use autodoc2 and remove automodule (NVIDIA#2542)

Author: Phlip79

.gitignore

@@ -18,4 +18,5 @@ runs/
 **/dist/
 
 # Sphinx documentation
-docs/_build
+docs/_build
+docs/apidocs

docs/api-guide/datasets.md

@@ -3,101 +3,3 @@
 ```{include} datasets_readme.md
 ```
 
-## Submodules
-
-### datasets.blended_megatron_dataset_config module
-
-```{automodule} core.datasets.blended_megatron_dataset_config
-:members:
-:undoc-members:
-:show-inheritance:
-```
-
-### datasets.blended_megatron_dataset_builder module
-
-```{automodule} core.datasets.blended_megatron_dataset_builder
-:members:
-:undoc-members:
-:show-inheritance:
-```
-
-### datasets.megatron_tokenizer module
-
-```{automodule} core.datasets.megatron_tokenizer
-:members:
-:undoc-members:
-:show-inheritance:
-```
-
-### datasets.indexed_dataset module
-
-```{automodule} core.datasets.indexed_dataset
-:members:
-:undoc-members:
-:show-inheritance:
-```
-
-### datasets.megatron_dataset module
-
-```{automodule} core.datasets.megatron_dataset
-:members:
-:undoc-members:
-:show-inheritance:
-```
-
-### datasets.gpt_dataset module
-
-```{automodule} core.datasets.gpt_dataset
-:members:
-:undoc-members:
-:show-inheritance:
-```
-
-### datasets.masked_dataset module
-
-```{automodule} core.datasets.masked_dataset
-:members:
-:undoc-members:
-:show-inheritance:
-```
-
-### datasets.bert_dataset module
-
-```{automodule} core.datasets.bert_dataset
-:members:
-:undoc-members:
-:show-inheritance:
-```
-
-### datasets.t5_dataset module
-
-```{automodule} core.datasets.t5_dataset
-:members:
-:undoc-members:
-:show-inheritance:
-```
-
-### datasets.blended_dataset module
-
-```{automodule} core.datasets.blended_dataset
-:members:
-:undoc-members:
-:show-inheritance:
-```
-
-### datasets.utils module
-
-```{automodule} core.datasets.utils
-:members:
-:undoc-members:
-:show-inheritance:
-```
-
-## Module contents
-
-```{automodule} core.datasets
-:members:
-:undoc-members:
-:show-inheritance:
-```
-

docs/api-guide/dist_checkpointing.md

@@ -80,61 +80,3 @@ You can combine formats to optimize both flexibility and performance:
 dist_checkpointing.strategies
 ```
 
-## Submodules
-
-### dist_checkpointing.serialization module
-
-```{automodule} core.dist_checkpointing.serialization
-:members:
-:undoc-members:
-:show-inheritance:
-```
-
-### dist_checkpointing.mapping module
-
-```{automodule} core.dist_checkpointing.mapping
-:members:
-:undoc-members:
-:show-inheritance:
-```
-
-### dist_checkpointing.optimizer module
-
-```{automodule} core.dist_checkpointing.optimizer
-:members:
-:undoc-members:
-:show-inheritance:
-```
-
-### dist_checkpointing.core module
-
-```{automodule} core.dist_checkpointing.core
-:members:
-:undoc-members:
-:show-inheritance:
-```
-
-### dist_checkpointing.dict_utils module
-
-```{automodule} core.dist_checkpointing.dict_utils
-:members:
-:undoc-members:
-:show-inheritance:
-```
-
-### dist_checkpointing.utils module
-
-```{automodule} core.dist_checkpointing.utils
-:members:
-:undoc-members:
-:show-inheritance:
-```
-
-## Module contents
-
-```{automodule} core.dist_checkpointing
-:members:
-:undoc-members:
-:show-inheritance:
-```
-

docs/api-guide/dist_checkpointing.strategies.md

@@ -5,37 +5,3 @@ Package defining different checkpoint formats (backends) and saving/loading algo
 Strategies can be used for implementing new checkpoint formats or implementing new (more optimal for a given use case) ways of saving/loading of existing formats.
 Strategies are passed to `dist_checkpointing.load` and `dist_checkpointing.save` functions and control the actual saving/loading procedure.
 
-## Submodules
-
-### dist_checkpointing.strategies.base module
-
-```{automodule} core.dist_checkpointing.strategies.base
-:members:
-:undoc-members:
-:show-inheritance:
-```
-
-### dist_checkpointing.strategies.tensorstore module
-
-```{automodule} core.dist_checkpointing.strategies.tensorstore
-:members:
-:undoc-members:
-:show-inheritance:
-```
-
-### dist_checkpointing.strategies.two_stage module
-
-```{automodule} core.dist_checkpointing.strategies.two_stage
-:members:
-:undoc-members:
-:show-inheritance:
-```
-
-## Module contents
-
-```{automodule} core.dist_checkpointing.strategies
-:members:
-:undoc-members:
-:show-inheritance:
-```
-

docs/api-guide/distributed.md

@@ -8,44 +8,3 @@ synchronize gradients across different parallelism modes (e.g., 'tied'
 layers on different pipeline stages, or gradients for experts in a MoE on
 different ranks due to expert parallelism).
 
-## Submodules
-
-### distributed.distributed_data_parallel
-
-Model wrapper for distributed data parallelism. Stores gradients in a
-contiguous buffer, and supports the option of overlapping communication
-(all-reduce or reduce-scatter) with backprop computation by breaking up
-full model's gradients into smaller buckets and running all-reduce /
-reduce-scatter on each bucket asynchronously.
-
-```{automodule} core.distributed.distributed_data_parallel
-:members:
-:undoc-members:
-:show-inheritance:
-```
-
-### distributed.finalize_model_grads
-
-Finalize model gradients for optimizer step across all used parallelism modes.
-Synchronizes the all-reduce / reduce-scatter of model gradients across DP replicas,
-all-reduces the layernorm gradients for sequence parallelism, embedding gradients
-across first and last pipeline stages (if not tied), and expert gradients for expert
-parallelism.
-
-```{automodule} core.distributed.finalize_model_grads
-:members:
-:undoc-members:
-:show-inheritance:
-```
-
-## Module contents
-
-Contains functionality to synchronize gradients across different ranks before
-optimizer step.
-
-```{automodule} core.distributed
-:members:
-:undoc-members:
-:show-inheritance:
-```
-

docs/api-guide/fusions.md

@@ -9,55 +9,3 @@ functionality for doing just-in-time compilation
 in recent versions), or call into custom kernels in external libraries
 such as Apex or TransformerEngine.
 
-## Submodules
-
-### fusions.fused_bias_dropout module
-
-This module uses PyTorch JIT to fuse the bias add and dropout operations. Since dropout is not used during inference, different functions are used when in train mode and when in inference mode.
-
-```{automodule} core.fusions.fused_bias_dropout
-:members:
-:undoc-members:
-:show-inheritance:
-```
-
-### fusions.fused_bias_gelu module
-
-This module uses PyTorch JIT to fuse the bias add and GeLU nonlinearity operations.
-
-```{automodule} core.fusions.fused_bias_gelu
-:members:
-:undoc-members:
-:show-inheritance:
-```
-
-### fusions.fused_layer_norm module
-
-This module provides a wrapper around various fused LayerNorm implementation in Apex.
-
-```{automodule} core.fusions.fused_layer_norm
-:members:
-:undoc-members:
-:show-inheritance:
-```
-
-### fusions.fused_softmax module
-
-This module provides wrappers around variations of Softmax in Apex.
-
-```{automodule} core.fusions.fused_softmax
-:members:
-:undoc-members:
-:show-inheritance:
-```
-
-### fusions.fused_cross_entropy_loss module
-
-This module uses PyTorch JIT to fuse the cross entropy loss calculation and batches communication calls.
-
-```{automodule} core.fusions.fused_cross_entropy
-:members:
-:undoc-members:
-:show-inheritance:
-```
-

docs/api-guide/index.md

@@ -1,7 +1,7 @@
 # API Guide
 
 ```{toctree}
-:maxdepth: 4
+:maxdepth: 2
 
 models
 tensor_parallel

docs/api-guide/models.bert.md

@@ -2,21 +2,3 @@
 
 Useful package for training bert and bert like encoder only models. It optionally comes with a binary head that can be used for classification tasks .
 
-## Submodules
-
-### models.bert.bert_model module
-
-```{automodule} core.models.bert.bert_model
-:members:
-:undoc-members:
-:show-inheritance:
-```
-
-## Module contents
-
-```{automodule} core.models.bert
-:members:
-:undoc-members:
-:show-inheritance:
-```
-

docs/api-guide/models.gpt.md

@@ -2,21 +2,3 @@
 
 This is the implementation of the popular GPT model. It supports several features like model parallelization (Tensor Parallel, Pipeline Parallel, Data Parallel) , mixture of experts, FP8 , Distributed optimizer etc. We are constantly adding new features. So be on the lookout or raise an issue if you want to have something added.
 
-## Submodules
-
-### models.gpt.gpt_model module
-
-```{automodule} core.models.gpt.gpt_model
-:members:
-:undoc-members:
-:show-inheritance:
-```
-
-## Module contents
-
-```{automodule} core.models.gpt
-:members:
-:undoc-members:
-:show-inheritance:
-```
-

docs/api-guide/models.md

@@ -12,11 +12,3 @@ models.t5
 models.bert
 ```
 
-## Module contents
-
-```{automodule} core.models
-:members:
-:undoc-members:
-:show-inheritance:
-```
-