The mining activities create considerable problems all over the world that affect nature in several aspects. The disposal of waste materials causes contamination of water sources, environmental pollution, and the destruction of local ecology. Therefore any strategy put forward for the utilization of waste materials will help to sustainable development. This study, therefore, aims to research the possibility of utilizing quarry waste material (QWM) in cement composites. The QWM was initially dried and then ground to powder to obtain micron-sized quarry waste powder (QWP). The QWP was then incorporated into the cement mortar by using two different methods, namely, the cement substitution method and the aggregate substitution method. In both methods, the substitution ratio of QWP was set as 10, 20 and 30% by weight of cement and the fine aggregate, respectively. The fresh and hardened properties of the QWP-modified mortars were obtained by determining the flow value, fresh density, oven-dry density, water absorption, voids content, and compressive strength. Test results showed that the flow value was significantly reduced with the incorporation of QWP and that the aggregate substitution method caused higher drops in the flow value. The effect of QWP on the physical properties varied depending on the incorporation method. The substitution of QWP by cement resulted in higher water absorption, lower oven-dry density, and higher voids content compared to those of the reference. However, substituting QWP with fine aggregate did not significantly alter the water absorption and the oven-dry density but reduced the voids content. The compressive strength reduced at higher levels when QWP was substituted with cement compared to the reference. Therefore substituting QWP with fine aggregate suggests a better way of utilizing this material in cement composites where compressive strength is a significant parameter.