We explore the algebra of the sets of iterated Stratonovich or Ito integrals that appear in the stochastic Taylor series expansion of the solution to a stochastic differential equation. The algebra of iterated Stratonovich integrals with pointwise multiplication is a shuffle algebra, allowing us to apply results from work on shuffle algebras. The pointwise product of Ito integrals is a modified shuffle product. Lyndon words provide an algebraic basis for both sets of iterated integrals. This basis is similar to, but simpler than, that obtained by Sussmann using Hall words.