Implementing from scratch: Attention is all you need - PyTorch Tutorial

Contents

• Introduction Introduction

  • Upgrade your transformer and computer vision knowledge

    43s
• 1. Course Setup 1. Course Setup

  • (Locked)
    Getting comfortable with Codespaces

    6m 39s
• 2. What Is a Transformer? 2. What Is a Transformer?

  • So, what is a transformer?

    5m 57s
  • A brief history: Life before transformers

    12m 1s
  • (Locked)
    Why transformers are revolutionary for computer vision

    5m 57s
  • (Locked)
    Comparing convolutional neural networks (CNNs) to transformers

    10m 18s
• 3. Getting Started with Vision Transformers 3. Getting Started with Vision Transformers

  • (Locked)
    Overview of PyTorch and Hugging Face transformers

    12m 45s
  • (Locked)
    Building a dataset

    16m 32s
  • (Locked)
    Running a pretrained model for classification

    6m 5s
• 4. Implementing Vision Transformers from Scratch 4. Implementing Vision Transformers from Scratch

  • (Locked)
    Implementing from scratch: Attention is all you need

    23m 30s
  • (Locked)
    Tokenization of images: How vision transformers (ViTs) see

    11m 42s
  • (Locked)
    Building a simple ViT

    17m 12s
• 5. Fine-Tuning Transformers with Transfer Learning 5. Fine-Tuning Transformers with Transfer Learning

  • (Locked)
    Transformers in the wild: How to find the right one for you

    16m 24s
  • (Locked)
    Fine-tuning in practice: Adapting ViTs to new tasks

    5m 51s
  • (Locked)
    Training strategies for vision transformers

    11m 54s
  • (Locked)
    Fine-tuning Swin Transformers

    7m 37s
• 6. Real-World Use Cases 6. Real-World Use Cases

  • (Locked)
    Choosing the right ViT for the job

    14m 32s
  • (Locked)
    Evaluating ViTs

    15m 2s
  • (Locked)
    Optimizing inference at deployment

    7m 27s
• 7. Explainability and Attention Visualization 7. Explainability and Attention Visualization

  • (Locked)
    What is an attention heatmap?

    6m 4s
  • (Locked)
    Visualizing attention

    11m 5s
• Conclusion Conclusion

  • (Locked)
    Next steps

    22s