This paper introduces provenance networks, a class of neural models that embed explainability by learning to link each prediction to its supporting training examples as part of normal inference, operating like a differentiable end-to-end k-nearest-neighbor system. Unlike post-hoc methods, interpretability is intrinsic to the architecture, directly addressing model opaqueness, hallucination, and data attribution.