Jaynes

Introduction

I recommend reading my summary of each chapter before reading the chapter, occasionally Jaynes explains things terribly, and you want to know about that ahead of time.

You should also check out the Lectures by Aubrey Clayton which follow Jaynes, they’re great, I recommend you watch them all (at 3x speed with occasional rewinding).

Another useful resource is this Unofficial Errata and Commentary

Plausible reasoning

Learning objectives

• Logic and Probability are about logical causation not physical causation
• Notation and basic properties of Boolean algebra
• Adequate sets of operations for Boolean algebra
  • All logical functions can be represented as a combination of basic operations
  • Reduction to disjunctive normal form
• Basic desiderata for plausibility

Boolean algebra

Most of the Boolean algebra identities are inherited from algebra in the case where true and false are 1 and 0 respectively. The only novel identities are about duality, converting and’s into or’s and vise versa

• \overline{AB} = \overline{A} + \overline{B}
• \overline{A+B} = \overline{A}\;\overline{B}

Keep in mind \overline{AB} \ne \overline{A}\;\overline{B}, logical negation is not like complex conjugation.

Jaynes mentions the following theorem (1.13) without explanation

• If \overline{B} = AD then A\overline{B} = \overline{B} and B\overline{A} = \overline{A}.

This won’t be used until a proof in Chapter 2 which I recommend you skip, hence you may also skip this theorem, unless you want to practice Boolean algebra.

If you want an intuition for (1.13) consider how the statement A \Rightarrow B is logically equivalent to saying A = AB meaning we can rewrite (1.13) as

• If \overline{B} = AD then \overline{B} \Rightarrow A and \overline{A} \Rightarrow B

Recall that P \Rightarrow Q is false if and only if P is true and Q is false. We can check this for (1.13) by considering the two cases for \overline{B}’s truth value.

Adequate sets of operations

The basic desiderata

Jaynes uses the strange notation A|B to denote the plausibility of A given B

The quantitative rules

Learning objectives

• Cox’s theorem (aka: the derivation of probability from our basic desiderata)

I strongly recommend you skim most of the proof as it’s very hard to follow and understanding all the details isn’t worth much. Aubrey Clayton agrees and explains why Jaynes goes to this much trouble.

I also have a lesswrong post with my take on why Cox’s theorem is philosophically interesting.

The commentary provided here is also good to read.

Godel’s theorem

Jaynes states that

These considerations seem to open up the possibility that, by going into a wider field by invoking principles external to probability theory, one might be able to prove the consistency of our rules. At the moment, this appears to us to be an open question.

See the commentary here

Actually, it is not an open question: the rules of probability theory can easily be proven consistent, and the proof can be found in any undergraduate mathematical text discussing set-theoretic probability theory.