The purpose of this paper is to formally define and solve ethical problems of how an artificial vehicle (AV) determines its driving behavior when there are some passengers in the AV and some pedestrians on a street. We construct a mathematical model introducing mainly two Bentham- and Nash-types social welfare functions, and derive optimal solutions. We show the optimal solutions are completely different depending on the functions and their parameters. Our contribution is that policymakers or managers of AVs can discuss the problem and determine an algorithm for autonomous driving by formalizing the situation and offering the optimal solutions.