A common finding in oddity search, a search in which the target is unknown but defined to be different from the distractors, is that human performance remains insensitive or even improves with number of distractors (set-size). A number of explanations based on perceptual and attentional mechanisms have been proposed to explain the anomalous set-size effect. Here, we consider whether the shallower set-size function for oddity search could be explained by stimulus information and task demands. We developed an ideal observer and a difference coding (standard deviation) model for single fixation oddity search and compared it to the ideal observer in the standard target known search as well as to human performance in both search tasks. Performance for the ideal and difference coding model in the oddity search resulted in a shallower set-size function than the target known ideal observer and were good predictors of human search accuracy. However, the ideal observer model was a better predictor than the standard deviation model for ten of the twelve data sets. The results highlight the importance of using ideal observer analysis to separate contributions to human performance arising from perceptual/attentional mechanisms inherent to the human brain from those contributions arising from differences in task demands.