A special kind of recurrent neural networks (RNN) with implicit dynamics has recently been proposed by Zhang et al, which could be generalized to solve online various time-varying problems. In comparison with conventional gradient neural networks (GNN), such RNN (or termed specifically as Zhang neural networks, ZNN) models are elegantly designed by defining matrix-valued indefinite error functions. In this paper, we generalize and investigate the ZNN and GNN models for online solution of time-varying matrix square roots. In addition, software modeling techniques are investigated to model and simulate both neural-network systems. Computer-modeling results verify that superior convergence and efficacy could be achieved by such ZNN models in this time-varying problem solving, as compared to the GNN models.