Human mesenchymal stem cells (MSCs) are currently recognized as a powerful cell source for regenerative medicine, notably for their capacity to differentiate into multiple cell types. The combination of MSCs with biomaterials functionalized with instructive cues can be used as a strategy to direct specific lineage commitment, and can thus improve the therapeutic efficacy of these cells. In terms of biomaterial design, one common approach is the functionalization of materials with ligands capable of directly binding to cell receptors and trigger specific differentiation signaling pathways. Other strategies focus on the use of moieties that have an indirect effect, acting, for example, as sequesters of bioactive ligands present in the extracellular milieu that, in turn, will interact with cells. Compared with complex biomolecules, the use of simple compounds, such as chemical moieties and peptides, and other small molecules can be advantageous by leading to less expensive and easily tunable biomaterial formulations. This review describes different strategies that have been used to promote substrate-mediated guidance of osteogenic differentiation of immature osteoblasts, osteoprogenitors and MSCs, through chemically conjugated small moieties, both in two- and three-dimensional set-ups. In each case, the selected moiety, the coupling strategy and the main findings of the study were highlighted. The latest advances and future perspectives in the field are also discussed.