Heterogeneous convergence networks will integrate multiple access technologies to provide seamless mobility for users with high-speed connectivity. This will give rise to a heterogeneous networks environment that can provide open standard interfaces regardless of the diversity of heterogeneous networks, where handoff mechanism is used to keep the services continued. In this paper, a heterogeneous network platform is constructed according to a novel architecture based on network adaptation layer. And a handoff mechanism based on capability adaptation is proposed to implement seamless handoff process, where the capability adaptation algorithm is given to select the optimal network and terminal. In addition, an illustration is given in detail to present the adaptation and handoff process with stream media service, which is effective to satisfy the requirements of the applications for modern service industry.