This paper considers the problem of switched Wiener system identification from a Kernel based manifold embedding perspective. Our goal is to identify both the Kernel mapping and the dynamics governing the evolution of the data on the manifold from noisy output measurements and with minimal assumptions about the nonlinearity and the affine portion of the systems. While in principle this is a very challenging problem, the main result of the paper shows that a computationally efficient solution can be obtained using a polynomial optimization approach that allows for exploiting the underlying sparse structure of the problem and provides optimality certificates. As an alternative, we provide a low complexity algorithm for the case where the affine part of the system switches only between 2 sub models.