It's not answering quite the question you asked, but Shannon - who invented much of this stuff - has some really nice practical arguments in the start of his amazing paper that introduced "information theory" [1]. It's a really readable paper, much less intimidating than you might think, and worth a look.
Hartley was (as far as I know) the first person to recognise the usefulness of log probabilities in the context of measuring information [2]. It's a really amazing paper, for several reasons ... but one thing that really strikes me about it is how it's aged: the first half has an essentially timeless presentation of the essence of information, and the second has a now-quite-irrelevant presentation of how to make a better TV. I guess that was the really interesting problem at the time!
Hartley was (as far as I know) the first person to recognise the usefulness of log probabilities in the context of measuring information [2]. It's a really amazing paper, for several reasons ... but one thing that really strikes me about it is how it's aged: the first half has an essentially timeless presentation of the essence of information, and the second has a now-quite-irrelevant presentation of how to make a better TV. I guess that was the really interesting problem at the time!
[1] https://people.math.harvard.edu/~ctm/home/text/others/shanno...
[2] http://dotrose.com/etext/90_Miscellaneous/transmission_of_in...