 |
|
Title: An Introduction to Information Theory
ISBN: 0486240614
Author:
John R. Pierce
Publicate Date: 1980-11-01
Publish: 1980-11-01
List Price: $12.95
Average Customer Rating: 4.5
Format: Paperback
 |
|
|
|
|
Amazon Lowest New Price: $6.40
Amazon Lowest Used Price: $4.00
Amazon Merchant Price: $7.77
|
|
|
|
|
|
|
|
|
|
|
| Customer Review: |
|
1: Excellent explanation for the core concepts
You cannot learn the math of information theory from this book but this book is perfect for you to get the core concepts.
An Excellent place to start!
|
2: A wonderfully engaging introduction to Information Theory
This book is a delight to read. It is thoughtfully written so the text flows effortlessly. Everything is described in an intuitive yet concise manner. In fact, despite the technical terms in its title and body, this book is accessible to any casual reader of "popular science" material.
The basic concepts of the "amount" of information, coding, information rate, noise, power, signal space, and channel capacity are described so clearly that the book can be an indictment of some of the more technical texts! The author delivers on his promise of a "non-mathematical" book, and does so the "right" way - namely, like a good teacher, by making you understand the essential ideas.
In short, reading the book is like sitting in front of a knowledgeable scientist who talks in a measured and engaging (and almost reflective and informal) manner. He conveys the significance of the main subject by touching on a variety of related ideas and even raising philosophical questions here and there. The result is a good context for understanding information theory itself.
It relates the main subject to fundamental concepts in physics such as thermal noise, deep space radiation, and entropy. A short discussion of linguistics and grammar eventually leads to an introductory discussion of "communication and control" - filtering and prediction, servo's and negative feedback, automata and computing - and what today may be considered parts of the field of artificial intelligence!
Final note: perhaps except for a few specific paragraphs (e.g. accounts of developments in computer technology), virtually everything in the book is at the level of fundamental science and, therefore, remains relevant and useful.
|
3: Best intro book
This is by far the best introductory book on info theory. The author has a talent for making difficult concepts easy and interesting. A definite page turner! Note, this book is for the lay person wanting to know what information theory is about. If you're taking a course in information theory, look somewhere else!
|
4: Good intro but dated
The update of this book should have been updated. While it is understandable that at the time of the first print of this book in 1961 the author saw little or no practical use for Shannon's information theory (other than perhaps his channel capacity theorem) it was well known by the second printing in 1980 that it has profound implications in studying biology (and modern technology). For instance in an article published in Nature in 1967, A. L. MacKay showed how the genetic code is highly optimal using Huffman's algorithm. More recently Ardell and Sella (with summaries available on the net) have 'demonstrated that the code's present structure was also shaped by natural selection (though non-Darwinian, see below). In this process, the codons - the triplets of nucleotides that map a particular nucleic acid sequence into proteins - are arranged to minimize the negative effects of genetic error, and to optimize the process of 'readout' of genes during protein synthesis. By permuting all 20 amino acids across all possible codon sets, both groups found that the 'universal' genetic code - the one found in nearly every organism on earth...-falls in the best .0001% of all possible codes and perhaps even better, in terms of its capacity to be an error-correcting code...' By showing modifications are possible in one generation the evidence points away from Crick's thesis of the genetic code being a 'fozen accident' but instead possible Lamarckian beginnings with horizantal gene transfer leading to Carl Woese's early RNA World hypothesis before Darwinian vertical descent begins.
The author also tends to perpetuate the widespread misunderstanding (generally by physicists who tend to contort the meaning away from Shannon's into 'available' states or choices such as with Black Holes) that information is uncertainty; he confuses (readers potentially with) surprise versus information by not taking into account the other half of the necessary equation for information transmission, being noise. He says "The amount of information conveyed by the message increases as the amount of uncertainty as to what message actually will be produced becomes greater." [pg 23] While he clears this up in a later chapter on noise it becomes so technical that it appears most readers of Shannon's theory have been mislead. At this point the scientists (usually physicists who actually work with a different concept of 'available information') typically equate the uncertainty with Kolmogorov complexity and assume that maximum information and complexity is randomess.
For instance consider Philip Nelson's comment in his book Biological Physics that 'random messages carry the most information!' In one footnote of his nearly 600 page book he effectively dismisses all of Nobel Prize winner Shannon's information achievements.
Much of the trouble is with terminology. We think of noise as impure sound. Shannon tried to avoid this problem by introducing the term 'equivocation' but on the other hand this seems to have no intuitive meaning in this context. One really has to go to the math to sort it out. The critical equation to potentially eradicate the confusion does not appear in the book -
R = Hbefore - Hafter
H is an entropy-like formula without Boltzman's constant; however the concepts are very different. (Reportedly Von Neuman told Shannon in the 1940's to call his uncertainty 'entropy, as noone will know what you mean!' Apparently this is still working!) Entropy of the universe apparently increases under the 2nd law of thermodynamics (at least ignoring gravity and extensivity), information begins and ends with life (one needs a recognizer to measure it). A random message in fact carries no information as there is no resolution (reduction) of uncertainty. This is all explained at molecular biologist's Dr. Tom Schneider's website, I know of no other comprehensive source and certainly no book that gets it right. (As yet! 'Hope springs eternal!' A. Pope; 1688 - 1744)
|
5: One of the best books I recently read!
A very good introductory text to information theory. Written in a plain, comprehensive way without too many unnecessary equations. I recommend this book to anyone looking for a book in such topic!
|
|
|
|
|
