Second Brain

Tag: dl_theory

79 items with this tag.

Oct 08, 2025
Into the Rabbit Hull - From Task-Relevant Concepts in DINO to Minkowski Geometry
Apr 02, 2025
Learning from Streaming Video with Orthogonal Gradients
Mar 11, 2025
Principal Components Enable A New Language of Images
Mar 03, 2025
Deep Learning is Not So Mysterious or Different
Feb 26, 2025
Rate–Distortion–Perception Trade-Off in Information Theory, Generative Models, and Intelligent Communications
- information_theory
- dl_theory
Feb 14, 2025
Simplifying DINO via Coding Rate Regularization
Jan 01, 2025
Are less inductive biases better or worse?
- dl_theory
- question
Jan 01, 2025
Rate Distortion and Spectral Analysis on Representations
Jan 01, 2025
Should we separate semantics and pose?
Dec 04, 2024
Beyond cls - Exploring the true potential of Masked Image Modeling representations
Nov 12, 2024
Interaction Asymmetry - A General Principle for Learning Composable Abstractions
- dl_theory
Oct 28, 2024
HoPE - A Novel Positional Encoding Without Long-Term Decay for Enhanced Context Awareness and Extrapolation
Oct 28, 2024
LoRA vs Full Fine-tuning - An Illusion of Equivalence
- dl_theory
Oct 08, 2024
Round and Round We Go! What makes Rotary Positional Encodings useful?
Oct 03, 2024
Searching for Efficient Linear Layers over a Continuous Space of Structured Matrices
- dl_theory
Oct 01, 2024
nGPT - Normalized Transformer with Representation Learning on the Hypersphere
Jul 05, 2024
Revealing the Utilized Rank of Subspaces of Learning in Neural Networks
- dl_theory
Jul 04, 2024
Improving Self Consistency in LLMs through Probabilistic Tokenization
- llm
- dl_theory
Jul 03, 2024
How JEPA Avoids Noisy Features - The Implicit Bias of DeepLinear Self Distillation Networks
- dl_theory
Jun 14, 2024
LieRE - Generalizing Rotary Position Encodings
- dl_theory
- vit
Jun 13, 2024
An Image is Worth More Than 16x16 Patches - Exploring Transformers on Individual Pixels
Jun 10, 2024
Learning Continually by Spectral Regularization
Jun 03, 2024
The Geometry of Categorical and Hierarchical Concepts in Large Language Models
- dl_theory
- geometry
May 30, 2024
Symmetries in Overparametrized Neural Networks - A Mean-Field View
- dl_theory
- equivariance
May 30, 2024
The Empirical Impact of Neural Parameter Symmetries, or Lack Thereof
May 17, 2024
Using Degeneracy in the Loss Landscape for Mechanistic Interpretability
Apr 09, 2024
Simultaneous linear connectivity of neural networks modulo permutation
Mar 29, 2024
Rewrite the Stars
- dl_theory
- efficient_dl
Mar 28, 2024
DenseNets Reloaded - Paradigm Shift Beyond ResNets and ViTs
Mar 05, 2024
Emergent Equivariance in Deep Ensembles
- equivariance
- dl_theory
Jan 01, 2024
Maximal pruning and functional recovery
- dl_theory
- efficient_dl
Jan 01, 2024
Mean Attention Distance
- dl_theory
- transformers
Jan 01, 2024
Multiple global minima
- optimizability
- dl_theory
Jan 01, 2024
Priors over Neural Network weights
- dl_theory
- equivariance
Dec 13, 2023
Harmonics of Learning - Universal Fourier Features Emerge in Invariant Networks
Nov 22, 2023
White-Box Transformers via Sparse Rate Reduction - Compression Is All There Is
- compression
- dl_theory
Nov 09, 2023
Are - Hierarchical - Visual Representations Hierarchical
Oct 03, 2023
Discovering Symmetry Breaking in Physical Systems with Relaxed Group Convolution
Sep 28, 2023
Vision Transformers Need Registers
Jul 31, 2023
Stochastic positional embeddings improve masked image modeling
- ssl
- dl_theory
Jul 11, 2023
Memorization Through the Lens of Curvature of Loss Function Around Samples
- dl_theory
- llm
May 24, 2023
Fourier Transformer - Fast Long Range Modeling by Removing Sequence Redundancy with FFT Operator
- dl_theory
- spectral
May 22, 2023
Improving Convergence and Generalization Using Parameter Symmetries
May 01, 2023
What Do Self-Supervised Vision Transformers Learn?
Apr 24, 2023
A Cookbook of Self-Supervised Learning
Apr 19, 2023
To Compress or Not to Compress- Self-Supervised Learning and Information Theory - A Review
- information_theory
- dl_theory
Mar 23, 2023
Optimization Dynamics of Equivariant and Augmented Neural Networks
Mar 08, 2023
A general theory of correct, incorrect, and extrinsic equivariance
Feb 14, 2023
Self-supervised learning of Split Invariant Equivariant representations
Dec 27, 2022
The Forward-Forward Algorithm - Some Preliminary Investigations
- dl_theory
- optimization
Oct 24, 2022
Robust Self-Supervised Learning with Lie Groups
Oct 13, 2022
The Hidden Uniform Cluster Prior in Self-Supervised Learning
- dl_theory
- ssl
Oct 05, 2022
RankMe - Assessing the downstream performance of pretrained self-supervised representations by their rank
- ssl
- dl_theory
Sep 29, 2022
In Search of Projectively Equivariant Networks
- equivariance
- dl_theory
Sep 29, 2022
Variance Covariance Regularization Enforces Pairwise Independence in Self-Supervised Representations
Jul 07, 2022
Equivariant Representation Learning via Class-Pose Decomposition
Jun 27, 2022
Guillotine Regularization - Why removing layers is needed to improve generalization in Self-Supervised Learning
Jun 03, 2022
On the duality between contrastive and non-contrastive self-supervised learning
May 27, 2022
On the Symmetries of Deep Learning Models and their Internal Representations
- dl_theory
May 26, 2022
Matryoshka Representation Learning
- dl_theory
May 13, 2022
Toward a Geometrical Understanding of Self-supervised Contrastive Learning
Mar 30, 2022
How Does SimSiam Avoid Collapse Without Negative Samples A Unified Understanding with Self-supervised Contrastive Learning
Jan 28, 2022
Approximately equivariant networks for imperfectly symmetric dynamics
Jan 10, 2022
A ConvNet for the 2020s
May 09, 2021
FNet - Mixing Tokens with Fourier Transforms
Feb 25, 2021
How to represent part-whole hierarchies in a neural network
- dl_theory
Feb 20, 2021
Provably Strict Generalisation Benefit for Equivariant Models
- dl_theory
- equivariance
Feb 04, 2021
CKConv - Continuous Kernel Convolution For Sequential Data
- convolutions
- dl_theory
Dec 08, 2020
Neural Mechanics - Symmetry and Broken Conservation Laws in Deep Learning Dynamics
- dl_theory
Nov 20, 2020
Exploring Simple Siamese Representation Learning
May 20, 2020
Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere
Dec 03, 2019
Towards Understanding the Spectral Bias of Deep Learning
- dl_theory
- spectral
Jan 29, 2019
An Investigation into Neural Net Optimization via Hessian Eigenvalue Density
Jan 23, 2019
Rethinking Lossy Compression - The Rate-Distortion-Perception Tradeoff
- dl_theory
- information_theory
Dec 05, 2018
Towards a Definition of Disentangled Representations
- disentanglement
- dl_theory
Feb 11, 2018
G-SGD - Optimizing ReLU Neural Networks in its Positively Scale-Invariant Space
- dl_theory
Nov 03, 2015
Understanding symmetries in deep networks
- dl_theory
Jun 08, 2015
Learning both Weights and Connections for Efficient Neural Networks
Jan 01, 1989
Optimal Brain Damage

Created with Quartz v4.5.2 © 2025