Probabilistic swarm guidance enables autonomous agents to generate their individual trajectories independently so that the entire swarm converges to the desired distribution shape. In contrast with previous homogeneous or inhomogeneous Markov chain based approaches [1], this paper presents an optimal transport based approach which guarantees faster convergence, minimizes a given cost function, and reduces the number of transitions for achieving the desired formation. Each agent first estimates the current swarm distribution by communicating with neighboring agents and using a consensus algorithm and then solves the optimal transport problem, which is recast as a linear program, to determine its transition probabilities. We discuss methods for handling motion constraints and also demonstrate the superior performance of the proposed algorithm by numerically comparing it with existing Markov chain based strategies.