We investigate the stability of cooperation agreements, such as those agreed by cartels, among firms in a Cournot model of oligopolistic competition embedded in a multimarket contact setting. Our analysis considers a broad array of 64 potential market structural configurations under linear demand and quadratic production costs. We establish that for an appropriate range of parameter values there exists a unique core stable market configuration in which an identical two‐firm cartel is sustained in both markets. Our result highlights the significance of multimarket presence for cartel formation in light of the well‐known result from the single‐market setting where cartels are non‐profitable.