From: carlas on
If what you say is true “that this result is off because of general statistics” then limit of negE when the number of samples go to infinity should go to zero. This is not the case, since it explodes, so there should be something wrong with the normalization.


I = randn(10000000,1);
p = hist(I,1000000);
% remove zero entries in p
p(p==0) = [];
% normalize p so that sum(p) is one.
p = p ./ numel(I);
Eg = -sum(p.*log(p));

I = randn(10000000,1);
p = hist(I,1000000);
% remove zero entries in p
p(p==0) = [];
% normalize p so that sum(p) is one.
p = p ./ numel(I);
Et = -sum(p.*log(p));


negE=Et-Eg