We propose and evaluate an agenda- and justification-based architecture for discovery systems that selects the next tasks to perform, as well as heuristics for use in discovery systems. This framework has many desirable properties: (1) it selects its own tasks to perform based upon how plausible they are judged to be; (2) it facilitates the encoding of general discovery strategies using a variety of background knowledge; and (3) it tailors its behavior toward a users interests. Many experiments with a prototype discovery program called HAMB demonstrate that both reasons and estimates of interestingness contribute to performance in the domains of protein crystallization and patient rehabilitation data. The programs heuristics provide good initial solutions to problems encountered when implementing fully autonomous discovery systems.