Graphene, with its high strength and electrical conductivity, is rapidly gaining new applications in a variety of industrial and scientific fields. Graphene-modified polymer and cementitious materials demonstrate unique mechanical, thermal, electrical, and barrier characteristics due to covalent and non-covalent bonds between matrices and graphene. These nanocomposites have demonstrated potential for applications in several technological fields. This review focuses on the latest developments in the field of graphene-filled polymeric and cementitious materials. The main types of graphene fillers and methods of their functionalization are given. Methods of obtaining several polymer-graphene and cement-graphene nanocomposites, their morphology, and physical and mechanical properties are considered.