We propose a generic framework for the automatic usability evaluation of web sites by combining traditional automatic usability methods with affective computing techniques. To evaluate a framework a pilot study was carried out where users (n=4) reported their affective states using dimensional and categorical models. Binary task completion, time, mouse clicks, and error rates as an indicator of web usability were automatically captured for each page. Results suggested that frustration experienced when error rates and time for the task were higher. Delight on the other hand was at the other side of the spectrum. In the case that usability measurements had almost same values (e.g. confusing or engaging pages), affective states may be a way to show the difference.