In this monograph, firstly, we introduce integrable renormalization formalism as a method of studying quantum integrable systems underlying the Hopf algebra of Feynman diagrams. Besides we consider an alternative approach to these systems which is connected with the Lie group of diffeographisms of a physical theory. Secondly, we consider the process which allows us to develop the Connes-Marcolli approach to the study of nonperturbative QFT. For this purpose, at the first step, we work on a new Hall tree type framework of the universal Hopf algebra of renormalization which can be lifted onto the level of universal counterterms. At the second step, we associate a categorical construction to each equation DSE which can encode this class of equations with respect to a special family of connections. At the third step, we consider the universality of the category of the equisingular flat vector bundles at the level of DSEs. At the fourth step, applying the mentioned categorical investigation, we determine a new family of Picard-Fuchs type equations related to each DSE. Further, we relate a category of Feynman motivic sheaves as a subcategory of the Arapura category to each DSE.