Factories are examples of system of systems with all related problems of integral ownership, lack of overview, emerging properties, and many stakeholders making decisions locally. In the past 7 years, engineers at the GKN Aerospace factory in Kongsberg have been modeling parts of the factory to tackle the complexity of such a facility. In successive papers, they have shown that modeling helps in understanding, reasoning, communication, and decision making. In this paper, we zoom out and describe from a systems of systems perspective the challenges of modeling the system to improve factory level performance indicators, such as inventory levels and production cycle times.