In this paper, we investigate the problem of the joint transceiver and offset design for point-to-point multiple-input–multiple-output (MIMO) and multiple user multiple-input–single-output visible light communication (VLC) systems. Both uplink and downlink multi-user scenarios are considered. The shot noise induced by the incoming signals is considered, leading to a more realistic MIMO VLC channel model. Under key lighting constraints, we formulate non-convex optimization problems aimed at minimizing the sum mean squared error. To optimize the transceiver and the offset jointly, a gradient projection-based procedure is resorted to. When only imperfect channel state information is available, a semidefinite programming-based scheme is proposed to obtain robust transceiver and offset. The proposed method is shown to non-trivially outperform the conventional scaled zero forcing and singular. value decomposition-based equalization methods. The robust scheme works particularly well when the signal is much stronger than the noise.