arXiv:2512.18586v2 Announce Type: replace Abstract: Neural networks often exhibit spectral bias, a phenomenon where high-frequency components may converge substantially more slowly than low-frequency ones. To address this imbalance, we propose a cross-attention-based network that combines a multiscale random Fourier feature bank with learnable amplitude weighting. The amplitude envelope modulates the Fourier features according to their frequencies, while the cross-attention residual structure provides an input-dependent mechanism to emphasize informative scales. As a result, the proposed design accelerates high-frequency convergence relative to comparison networks built on the same multiscale bank. Moreover, the feature bank can be incrementally enriched with dominant modes identified through discrete Fourier analysis of intermediate approximations, without modifying the hidden-layer architecture. We further extend the framework to PDE learning through a two-network representation in which one subnetwork approximates higher-frequency solution components and the other approximates lower-frequency components, with a learnable or analytically computed mixing coefficient balancing their contributions. Numerical experiments on function approximation, image reconstruction, and representative elliptic PDEs demonstrate the accuracy of the proposed method for problems involving high-frequency, multiscale, and discontinuous features.