Computer Science > Information Theory
[Submitted on 18 Mar 2024 (v1), revised 22 Apr 2024 (this version, v2), latest version 6 Feb 2025 (v6)]
Title:Towards a Theory of Pragmatic Information
View PDF HTML (experimental)Abstract:Standard information theory says nothing about how much meaning is conveyed by the information. We fill this gap with a rigorously justifiable, quantitative definition of ``pragmatic information'' as the amount of meaning in a message that is relevant to a particular decision (as opposed to the {\it value} that the decision maker places on that message). We posit that such a message updates a ``state of the world'' random variable, $\omega$, that informs the decision. The pragmatic information of a single message is then defined as the Kulbach-Leibler divergence between the {\it a priori} and updated probability distributions of $\omega$; the pragmatic information of a message ensemble is defined as the expected value of the pragmatic information values of the ensemble's component messages. We then show, first, that the pragmatic information of a single message is the expected difference between the shortest binary encoding of $\omega$ under the {\it a priori} and updated distributions, and, second, that the average of the pragmatic values of individual messages, when sampled a large number of times from the ensemble, approaches its expected value.
Pragmatic information thus defined has many hoped-for properties, such as non-negativity and additivity for independent decisions and ``pragmatically independent'' messages. We also sketch three applications: The first is the single play of a slot machine, a.k.a. ``one armed bandit'', with an unknown payout probability; the second being a characterization of the rate of biological evolution in a simple model of reproduction with mutation (the so-called ``quasi-species'' model); the third being a reformulation of the efficient market hypothesis of financial economics. We conclude by noting the importance of the computational capacity of the receiver in processing pragmatic information.
Submission history
From: Edward D. Weinberger [view email][v1] Mon, 18 Mar 2024 23:33:58 UTC (13 KB)
[v2] Mon, 22 Apr 2024 03:00:06 UTC (15 KB)
[v3] Sun, 4 Aug 2024 21:58:28 UTC (16 KB)
[v4] Sat, 5 Oct 2024 04:58:05 UTC (16 KB)
[v5] Wed, 20 Nov 2024 20:10:06 UTC (16 KB)
[v6] Thu, 6 Feb 2025 21:49:56 UTC (55 KB)
Current browse context:
cs.IT
References & Citations
Bibliographic and Citation Tools
Bibliographic Explorer (What is the Explorer?)
Connected Papers (What is Connected Papers?)
Litmaps (What is Litmaps?)
scite Smart Citations (What are Smart Citations?)
Code, Data and Media Associated with this Article
alphaXiv (What is alphaXiv?)
CatalyzeX Code Finder for Papers (What is CatalyzeX?)
DagsHub (What is DagsHub?)
Gotit.pub (What is GotitPub?)
Hugging Face (What is Huggingface?)
Papers with Code (What is Papers with Code?)
ScienceCast (What is ScienceCast?)
Demos
Recommenders and Search Tools
Influence Flower (What are Influence Flowers?)
CORE Recommender (What is CORE?)
arXivLabs: experimental projects with community collaborators
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.