We study the possibility of heat conductivity by two-level systems in amorphous solids. Transport requires a macroscopic cluster of weakly interacting, almost degenerate two-level systems. We find that such a cluster exists for a sufficiently large spectral diffusion kernel. The resulting heat conductivity of TLS is, however, by several orders of magnitude smaller than that of thermal phonons. As a consequence, transport through TLS can be ruled out as an explanation for possible anomalies in the measured heat conductivity in the milliKelvin range.