The field of learning has been deadlocked for quite some time by apparently contradictory conclusions as to which is a better theory of learning. This article attempts to resolve this inconsistency by pointing out that learning models have different objectives that imply different model comparison criteria. The different criteria are expected to lead to the same conclusions if the models are well specified, but might lead to different conclusions when they are used to compare approximations. The magnitude of the bias that results from a mis-specification is evaluated experimentally.