Origin of Model Selection Criteria
It is the preconception that model is selected from the hypothesis space which can explain data.
･Consistency for Data: Accuracy, Minimize Error
･Coverage for Data: Increasing covering event/feature
Ockham’s razor → MDL､AIC
“Simplest model is selected with increasing Consistency for Data”
Matchable principal (maximizing Matching Opportunity)
“Simplest model is selected with increasing Coverage for Data”