In this work, we investigate the magnetic structures of (Fe1-xMnx)2AlB2 solid-solution quaternaries in the x=0 to 1 range using x-ray and neutron diffraction, magnetization measurements, and mean-field theory calculations. While Fe2AlB2 and Mn2AlB2 are known to be ferromagnetic (FM) and antiferromagnetic (AFM), respectively, herein we focused on the magnetic structure of their solid solutions, which is not well understood. The FM ground state of Fe2AlB2 becomes a canted AFM at x≈0.2, with a monotonically diminishing FM component until x≈0.5. The FM transition temperature (TC) decreases linearly with increasing x. These changes in magnetic moments and structures are reflected in anomalous expansions of the lattice parameters, indicating a magnetoelastic coupling. Lastly, the magnetocaloric properties of the solid solutions were explored. For x=0.2 the isothermal entropy change is smaller by 30% than it is for Fe2AlB2, while the relative cooling power is larger by 6%, due to broadening of the temperature range of the transition.