Cross-Entropy

This measure, also known as directed divergence, measures how good a distribution approximates another distribution. It is used in reconstructability analysis (section 3.3.2) as a distance measure between reconstructed hypothesis compared to the original distribution.

$$H(f,f^h) = \sum_{x \in dom(X)} f(x) \cdot \log_2\left(\frac{f(x)}{f^h(x)}\right)$$ (12)

There are several more interesting measures for measuring structure and information which I will refer to in section 3.