BaM hexaferrites substituted with both Ca2+ and Mg2+ ions, namely, Ba1-2×CaxMgxFe12O19 (0.0 ≤ x ≤ 0.1), synthesized during a sol–gel auto-combustion route. The hexaferrite phase and morphology of all samples were investigated using X-ray powder diffraction, a field emission scanning electron microscope, a high-resolution transmission microscope, and Fourier transform infrared spectroscopy. In addition, an M-type hexagonal structure was confirmed using XRD for all samples. FE-SEM and TEM revealed the shape of the hexagonal plate. Measurements of the magnetization versus the field M(H) of Ba1-2×CaxMgxFe12O19 (0.0 ≤ x ≤ 0.1) nanohexaferrites were conducted at 300 and 10 K. A hard-ferrimagnetic behavior at both 300 and 10 K was noted for the different products produced. The squareness ratio indicates the uniaxial anisotropy for various products. The deduced values of saturation magnetization (Ms) in all substituted samples are higher than in the pristine sample (x = 0). The Ba0.96Ca0.02Mg0.02Fe12O19 nanosized hexaferrite showed the highest values of Ms, remanence Mr, magneton number (nB), and magnetocrystalline anisotropy constant (Keff). In contrast, the values of the coercive field (Hc) and intrinsic coercivity (Hci) diminish with the increase in the amount of the substituted Ca and Mg elements.