Recent Advancements in ViT Tokenization

Advancements in Tokenization for Vision Transformers (ViT)

1. Dynamic Mixed-Scale Tokenization (MSViT)

   • Reference: Source

   • Description: MSViT introduces a conditional gating mechanism for ViT, allowing dynamic tokenization instead of static, uniform tokenization. This approach enhances flexibility in capturing image features.

2. Mixed-Resolution Tokenization

   • Reference: Paper

   • Description: This technique reduces the number of patches through tokenization while preserving global attention across the entire image. It offers a valuable orthogonal improvement in ViT models.

3. Scale-space Tokenization (SRVT)

   • Reference: Link

   • Description: Implemented in SRVT, this method incorporates scale-space patch embedding to improve the robustness of transformers. It enhances the model's ability to handle variations in scale within images.

4. Intra-token Refinement

   • Reference: Research

   • Description: Addressing limitations of naïve patch-based approaches, intra-token refinement employs stride-p convolution to capture rich context within tokens. This refinement enhances the representation of image features.

These advancements highlight ongoing efforts to enhance Vision Transformers' performance by refining tokenization techniques. Their effectiveness may vary depending on specific application requirements.