Symmetric quasiconvexity plays a key role for energy minimization in geometrically linear elasticity theory. Due to the complexity of this notion, a common approach is to retreat to necessary and sufficient conditions that are easier to handle. This article focuses on symmetric polyconvexity, which is a sufficient condition. We prove a new characterization of symmetric polyconvex functions in the two- and three-dimensional setting, and use it to investigate relevant subclasses like symmetric polyaffine functions and symmetric polyconvex quadratic forms. In particular, we provide an example of a symmetric rank-one convex quadratic form in 3d that is not symmetric polyconvex. The construction takes the famous work by Serre from 1983 on the classical situation without symmetry as inspiration. Beyond their theoretical interest, these findings may turn out useful for computational relaxation and homogenization.