The importance of being ignorant using entropy for interpretation and inference

Abstract

In many real life situations, we have to draw conclusions from data which are not complete and have been affected by measurement errors. Such problems have been addressed from the time of Bayes and Laplace (late 1700's) using concepts which parallel Boltzmann's use of entropy in thermal physics. The idea is to assign probabilities to different possible conclusions from a given set of data. A critical - and sometimes controversial - input is a 'prior probability', which represents our knowledge before any data are given or taken! This body of ideas is introduced in this article with simple examples.