A Unified Algebraic-Geometric Framework for Analyzing Neural Network Weight Tensors: Theory, Structural Properties, and the Algebraic Tensor Complexity Index (ATCI)

Deep neural networks rely on high-dimensional tensors that encode the parameters of convolutional filters, fully connected layers, and attention mechanisms. Understanding the structural properties of these tensors is crucial for interpreting model behavior, improving compressibility, and developing principled complexity measures. Existing approaches typically rely on numerical tools such as singular value decomposition, matrix norms, or heuristic pruning criteria. However, these methods do not capture the algebraic constraints that shape the space of admissible weight tensors. In this paper we introduce a unified algebraic-geometric framework for analyzing neural network weight tensors using concepts from multilinear algebra and algebraic geometry, including Segre varieties, secant varieties, tensor rank stratification, and polynomial flattening constraints. We propose the Algebraic Tensor Complexity Index (ATCI), a composite geometric measure that integrates tensor rank, the algebraic dimension and degree of associated varieties, and the Frobenius distance to low-rank secant varieties. We discuss computational approximations to ATCI and demonstrate the framework with analytical case studies on fully connected, convolutional, and attention-layer tensors. Applications to model compression, overfitting diagnostics, and stability analysis are explored. The work aims to bridge algebraic geometry and deep learning, providing a mathematically grounded basis for structural analysis of models.

کلیدواژه ها:

Tensor Rank ، Segre Variety ، Deep Learning ، Algebraic Geometry ، Model Compression