Skip to content

on_boarding.md

Latest commit

 

History

History
191 lines (161 loc) · 25.1 KB

on_boarding.md

File metadata and controls

191 lines (161 loc) · 25.1 KB
graph LR
    Core_Utilities["Core Utilities"]
    Data_Management_Preprocessing["Data Management & Preprocessing"]
    Model_Training_Evaluation["Model Training & Evaluation"]
    Natural_Language_Processing_NLP_Models_Components["Natural Language Processing (NLP) Models & Components"]
    Computer_Vision_CV_Models_Components["Computer Vision (CV) Models & Components"]
    Data_Management_Preprocessing -- "calls" --> Core_Utilities
    Model_Training_Evaluation -- "calls" --> Core_Utilities
    Data_Management_Preprocessing -- "provides input data to" --> Model_Training_Evaluation
    Data_Management_Preprocessing -- "prepares data for" --> Natural_Language_Processing_NLP_Models_Components
    Model_Training_Evaluation -- "trains" --> Natural_Language_Processing_NLP_Models_Components
    Model_Training_Evaluation -- "trains" --> Computer_Vision_CV_Models_Components
    Natural_Language_Processing_NLP_Models_Components -- "consumes data from" --> Data_Management_Preprocessing
    Natural_Language_Processing_NLP_Models_Components -- "is trained by" --> Model_Training_Evaluation
    Computer_Vision_CV_Models_Components -- "consumes data from" --> Data_Management_Preprocessing
    Computer_Vision_CV_Models_Components -- "is trained by" --> Model_Training_Evaluation
    click Core_Utilities href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main//d2l-zh/Core_Utilities.md" "Details"
    click Model_Training_Evaluation href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main//d2l-zh/Model_Training_Evaluation.md" "Details"
    click Natural_Language_Processing_NLP_Models_Components href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main//d2l-zh/Natural_Language_Processing_NLP_Models_Components.md" "Details"
    click Computer_Vision_CV_Models_Components href "https://github.com/CodeBoarding/GeneratedOnBoardings/blob/main//d2l-zh/Computer_Vision_CV_Models_Components.md" "Details"
Loading

CodeBoardingDemoContact

Component Details

The d2l-zh project is structured to provide a comprehensive and framework-agnostic educational resource for deep learning. The architecture is designed to be modular, allowing for clear separation of concerns across different stages of a deep learning pipeline and across various deep learning frameworks (MXNet, PaddlePaddle, TensorFlow, PyTorch).

Core Utilities

This component provides fundamental, cross-cutting helper functions essential for the entire project. These include utilities for visualization (e.g., set_figsize, plot), timing operations (Timer), and general data handling (e.g., download_extract, download_all). These utilities are crucial for setting up experiments, displaying results, and managing datasets, regardless of the specific deep learning task or framework.

Related Classes/Methods:

Data Management & Preprocessing

This component is responsible for the entire data pipeline, from loading raw datasets to transforming them into a format suitable for deep learning models. It encompasses dataset loading for various domains (e.g., Fashion MNIST, Time Machine, IMDB, VOC), text tokenization, vocabulary creation, and handling image data for computer vision tasks. It acts as the primary data provider for all downstream model components.

Related Classes/Methods:

Model Training & Evaluation

This component encapsulates the core logic for training and evaluating deep learning models across different tasks. It handles the iterative process of training, including iterating over epochs, processing mini-batches, computing loss, performing backpropagation, and updating model parameters. It also includes functionalities for calculating and reporting performance metrics (e.g., accuracy) on validation or test datasets.

Related Classes/Methods:

Natural Language Processing (NLP) Models & Components

This comprehensive component covers all aspects of text processing and modeling within the project. It includes functionalities for building vocabularies and creating/loading token embeddings (e.g., GloVe, Word2Vec) to convert text into numerical representations. Furthermore, it implements various sequence models like Recurrent Neural Networks (RNNs), the advanced Transformer architecture, and the powerful BERT model, along with their foundational Attention Mechanisms. This component provides the specialized tools and models required for language-related tasks such as language modeling, machine translation, and text classification.

Related Classes/Methods:

Computer Vision (CV) Models & Components

This component focuses on image processing and computer vision tasks. It includes implementations of popular convolutional neural network (CNN) architectures like ResNet-18 for image classification. Additionally, it provides specialized utilities for object detection (e.g., displaying bounding boxes, IoU calculation, NMS) and image segmentation (e.g., VOC dataset handling). This component provides the necessary models and helper functions to work with visual data.

Related Classes/Methods:

FAQ