The nonlinear dynamics of semiconductor optical amplifiers (SOAs) are investigated. We focus on the nonlinear gain, phase and chirp dynamics that can be used for high speed all-optical switching. Ultrafast component in the phase dynamics is observed. We explain this phenomenon by using a numerical model that includes the impact of sub-picosecond intra-band effects in the SOA. In addition, we also investigate the chirp properties, and analyze the influences of chirp dynamics upon the detuned-filtering-based optical switches that can realize more than 160 Gbit/s operating speed. The results are useful for SOA-based high speed all-optical switching in photonic networks.