A variety of magnetic devices can be manipulated remotely using a single permanent “actuator” magnet positioned in space by a robotic manipulator. This paper presents the spherical-actuator-magnet manipulator (SAMM), which is designed to replace or augment the singularity-prone spherical wrist used by prior permanent-magnet manipulation systems. The SAMM uses three omniwheels to enable holonomic control of a spherical magnet's heading and enable the magnet's instantaneous axis-of-rotation to be set arbitrarily. The SAMM performs closed-loop control of its dipole using field measurements obtained from Hall-effect sensors. We describe the operation and construction of the SAMM, develop and characterize a controller for the SAMM's spherical magnet, and demonstrate remote actuation of an untethered magnetic device in a lumen.