Chaotic neural networks have been proved to be powerful tools to solve function and combinatorial optimization problems. Several chaotic neural units were studied and their activation functions are obviously different. The reversed bifurcation and Lyapunov exponent figures were respectively given. To improve the search-optimization capacity of chaotic neural network, a new transiently chaotic neural network was presented and its activation function is composed by Morlet and Sigmoid function. Then it was applied to function and combinatorial optimization problems. The simulation results show that the new transiently chaotic neural network is superior to the other transiently chaotic neural networks