Second Brain

Tag: transformers

33 items with this tag.

Feb 14, 2025
Simplifying DINO via Coding Rate Regularization
Dec 04, 2024
Beyond cls - Exploring the true potential of Masked Image Modeling representations
Oct 28, 2024
HoPE - A Novel Positional Encoding Without Long-Term Decay for Enhanced Context Awareness and Extrapolation
Oct 08, 2024
Round and Round We Go! What makes Rotary Positional Encodings useful?
Oct 01, 2024
nGPT - Normalized Transformer with Representation Learning on the Hypersphere
Jun 24, 2024
Building on Efficient Foundations - Effectively Training LLMs with Structured Feedforward Layers
Jun 17, 2024
Refusal in Language Models Is Mediated by a Single Direction
Jun 05, 2024
Exact Conversion of In-Context Learning to Model Weights in Linearized-Attention Transformers
- transformers
Jun 04, 2024
Block Transformer - Global-to-Local Language Modeling for Fast Inference
- efficient_dl
- transformers
May 23, 2024
Grokked Transformers are Implicit Reasoners - A Mechanistic Journey to the Edge of Generalization
- transformers
- mechinterp
Mar 26, 2024
The Unreasonable Ineffectiveness of the Deeper Layers
Feb 03, 2024
Parameter-Efficient Fine-Tuning for Pre-Trained Vision Models - A Survey
Jan 17, 2024
Vision Mamba - Efficient Visual Representation Learning with Bidirectional State Space Model
Jan 11, 2024
Surgical-DINO - Adapter Learning of Foundation Models for Depth Estimation in Endoscopic Surgery
Jan 01, 2024
Block Expansion
- efficient_dl
- transformers
Jan 01, 2024
Mean Attention Distance
- dl_theory
- transformers
Jan 01, 2024
Residual stream
- mechinterp
- transformers
Oct 30, 2023
Battle of the Backbones - A Large-Scale Comparison of Pretrained Models across Computer Vision Tasks
Oct 20, 2023
Learning with Unmasked Tokens Drives Stronger Vision Learners
Jul 18, 2023
Llama 2 - Open Foundation and Fine-Tuned Chat Models
- foundation_models
- transformers
May 01, 2023
What Do Self-Supervised Vision Transformers Learn?
Dec 15, 2022
FlexiViT - One Model for All Patch Sizes
Nov 22, 2022
DETRs with Collaborative Hybrid Assignments Training
Mar 30, 2022
Exploring Plain Vision Transformer Backbones for Object Detection
Jan 03, 2022
Vision Transformer with Deformable Attention
- transformers
- computer_vision
Dec 22, 2021
A Mathematical Framework for Transformer Circuits
- mechinterp
- transformers
Nov 25, 2021
BoxeR - Box-Attention for 2D and 3D Transformers
- transformers
- object_detection
Jun 28, 2021
Early Convolutions Help Transformers See Better
May 09, 2021
FNet - Mixing Tokens with Fourier Transforms
Oct 22, 2020
An image is worth 16x16 words - Transformers for image recognition at scale
- vit
- transformers
Oct 08, 2020
Deformable DETR - Deformable Transformers for End-to-End Object Detection
May 26, 2020
End-to-End Object Detection with Transformers
Nov 08, 2019
On the Relationship between Self-Attention and Convolutional Layers

Created with Quartz v4.5.2 © 2025