Shannon summarized his war work on secret communications in a 114-page opus, \u201cA Mathematical Theory of Cryptography,\u201d which he finished in 1945. The paper was immediately deemed classified and too sensitive for publication, but those who read it found a long treatise exploring the histories and methodologies of various secrecy systems. Moreover, he had offered a persuasive analysis of which methods might be unbreakable (what he called \u201cideal\u201d) and which cryptographic systems might be most practical if an unbreakable system were deemed too complex or unwieldy. His mathematical proofs presented the few people cleared to read it with a number of useful insights and an essential observation that language, especially the English language, was filled with redundancy and predictability. Indeed, he later calculated that English was about 75 to 80 percent redundant. This had ramifications for cryptography: The less redundancy you have in a message, the harder it is to crack its code. And this also, by extension, had implications for how you might send a message more efficiently. Shannon would often demonstrate that the sender of a message could strafe its vowels and still make it intelligible. To illustrate Shannon\u2019s point, David Kahn, a historian of cryptography who wrote extensively on Shannon, used the following example:<\/p>\n\n\n\n
F C T S S T R N G R T H N F C T N<\/p>\n\n\n\n
To transmit the message fact is stranger than fiction <\/em>one could send fewer letters. You could, in other words, compress <\/em>it without subtracting any of its content. Shannon suggested, moreover, that it wasn\u2019t only individual letters or symbols that were sometimes redundant. Sometimes you could take entire words out of a sentence without altering its meaning.<\/em><\/p>\n\n\n\n The completion of the cryptography paper coincided with the end of the war. But Shannon\u2019s personal project\u2014the one he had been laboring on at home in the evenings\u2014was largely worked out a year or two before that. Its subject was the general nature of communications. \u201cThere is this close connection,\u201d Shannon later said of the link between sending an encoded message and an uncoded one. \u201cThey are very similar things, in one case trying to conceal information, and in the other case trying to transmit it.\u201d In the secrecy paper, he referred briefly to something he called \u201cinformation theory.\u201d This was a bit of a coded message in itself, for he offered no indication of what this theory might say.<\/p>\n\n\n\n ALL WRITTEN AND SPOKEN EXCHANGES, to some degree, depend on code\u2014the symbolic letters on the page, or the sounds of consonants and vowels that are transmitted (encoded) by our voices and received (decoded) by our ears and minds. With each passing decade, modern technology has tended to push everyday written and spoken exchanges ever deeper into the realm of ciphers, symbols, and electronically enhanced puzzles of representation. Spoken language has yielded to written language, printed on a press; written language, in time, has yielded to transmitted language, sent over the air by radio waves or through a metal cable strung on poles. First came telegraph messages\u2014which contained dots and dashes (or what might have just as well been the 1s and 0s of Boolean algebra) that were translated back into English upon reception. Then came phone calls, which were transformed during transmission\u2014changing voices into electrical waves that represented sound pressure and then interleaving those waves in a cable or microwave transmission. At the receiving end, the interleaved messages were pulled apart\u2014decoded, in a sense\u2014by quartz filters and then relayed to the proper recipients.<\/p>\n\n\n\n In the mid-1940s Bell Labs began thinking about how to implement a new and more efficient method for carrying phone calls. PCM\u2014short for pulse code modulation\u2014was a theory that was not invented at the Labs but was perfected there, in part with Shannon\u2019s help and that of his good friend Barney Oliver, an extraordinarily able Bell Labs engineer who would later go on to run the research labs at Hewlett-Packard. Oliver would eventually become one of the driving forces behind the invention of the personal calculator. Shannon and Oliver had become familiar with PCM during World War II, when Labs engineers helped create secret communication channels between the United States and Britain by using the technology. Phone signals moved via electrical waves. But PCM took these waves (or \u201cwaveforms,\u201d as Bell engineers called them) and \u201csampled\u201d them at various points as they moved up and down. The samples\u20148,000 per second\u2014could then be translated into on\/off pulses, or the equivalent of 1s and 0s. With PCM, instead of sending waves along phone channels, one could send information that described the numerical coordinates of the waves. In effect what was being sent was a code. Sophisticated machines at a receiving station could then translate these pulses describing the numerical coordinates back into electrical waves, which would in turn (at a telephone) become voices again without any significant loss of fidelity. The reasons for PCM, if not its methods, were straightforward. It was believed that transmission quality could be better preserved, especially over long distances that required sending signals through many repeater stations, by using a digital code rather than an analog wave. Indeed, PCM suggested that telephone engineers could create a potentially indestructible format that could be periodically (and perfectly) regenerated as it moved over vast distances.<\/p>\n\n\n\n Shannon wasn\u2019t interested in helping with the complex implementation of PCM\u2014that was a job for the development engineers at Bell Labs, and would end up taking them more than a decade. \u201cI am very seldom interested in applications,\u201d he later said. \u201cI am more interested in the elegance of a problem. Is it a good problem, an interesting problem?\u201d For him, PCM was a catalyst for a more general theory about how messages move\u2014or in the future could <\/em>move\u2014from one place to another. What he\u2019d been working on at home during the early 1940s had become a long, elegant manuscript by 1947, and one day soon after the press conference in lower Manhattan unveiling the invention of the transistor, in July 1948, the first part of Shannon\u2019s manuscript was published as a paper in the Bell System Technical Journal<\/em>; a second installment appeared in the Journal <\/em>that October. \u201cA Mathematical Theory of Communication\u201d\u2014 \u201cthe magna carta of the information age,\u201d as Scientific American <\/em>later called it\u2014wasn\u2019t about one particular thing, but rather about general rules and unifying ideas. \u201cHe was always searching for deep and fundamental relations,\u201d Shannon\u2019s colleague Brock McMillan explains. And here he had found them. One of his paper\u2019s underlying tenets, Shannon would later say, \u201cis that information can be treated very much like a physical quantity, such as mass or energy.\u201d To consider it on a more practical level, however, one might say that Shannon had laid out the essential answers to a question that had bedeviled Bell engineers from the beginning: How rapidly, and how accurately, can you send messages from one place to another?<\/p>\n\n\n\n \u201cThe fundamental problem of communication,\u201d Shannon\u2019s paper explained, \u201cis that of reproducing at one point either exactly or approximately a message selected at another point.\u201d Perhaps that seemed obvious, but Shannon went on to show why it was profound. If \u201cuniversal connectivity\u201d remained the goal at Bell Labs\u2014if indeed the telecommunications systems of the future, as Kelly saw it, would be \u201cmore like the biological systems of man\u2019s brain and nervous system\u201d\u2014then the realization of those dreams didn\u2019t only depend on the hardware of new technologies, such as the transistor. A mathematical guide for the system\u2019s engineers, a blueprint for how to move data around with optimal efficiency, which was what Shannon offered, would be crucial, too. Shannon maintained that all communications systems could be thought of in the same way, regardless of whether they involved a lunchroom conversation, a postmarked letter, a phone call, or a radio or telephone transmission. Messages all followed the same fairly simple pattern:<\/p>\n\n\n\n All messages, as they traveled from the information source to the destination, faced the problem of noise. This could be the background clatter of a cafeteria, or it could be static (on the radio) or snow (on television). Noise interfered with the accurate delivery of the message. And every channel that carried a message was, to some extent, a noisy channel.<\/em><\/p>\n\n\n\n To a non-engineer, Shannon\u2019s drawing seemed sensible but didn\u2019t necessarily explain anything. His larger point, however, as he proved in his mathematical proofs, was that there were ways to make sure messages got where they were supposed to, clearly and reliably so. The first place to start, Shannon suggested, was to think about the information <\/em>within a message. The semantic aspects of communication were irrelevant to the engineering problem, he wrote. Or to say it another way: One shouldn\u2019t necessarily think of information in terms of meaning. <\/em>Rather, one might think of it in terms of its ability to resolve uncertainty. Information provided a recipient with something that was not previously known, was not predictable, was not redundant. \u201cWe take the essence of information as the irreducible, fundamental underlying uncertainty that is removed by its receipt,\u201d a Bell Labs executive named Bob Lucky explained some years later. If you send a message, you are merely choosing from a range of possible messages. The less the recipient knows about what part of the message comes next, the more information you are sending. Some language choices, Shannon\u2019s research suggested, occur with certain probabilities, and some messages are more frequent than others; this fact could therefore lead to precise calculations of how much information these words or messages contained. (Shannon\u2019s favorite example was to explain that one might need to know that the word \u201cquality\u201d begins with q<\/em>, for instance, but not that a u <\/em>follows after. The u <\/em>gives a recipient no information if they already have the q<\/em>, since u <\/em>always follows q<\/em>; it can be filled in by the recipient.)<\/p>\n\n\n\n Shannon suggested it was most useful to calculate a message\u2019s information content and rate in a term that he suggested engineers call \u201cbits\u201d\u2014a word that had never before appeared in print with this meaning. Shannon had borrowed it from his Bell Labs math colleague John Tukey as an abbreviation of \u201cbinary digits.\u201d The bit, Shannon explained, \u201ccorresponds to the information produced when a choice is made from two equally likely possibilities. If I toss a coin and tell you that it came down heads, I am giving you one bit of information about this event.\u201d All of this could be summed up in a few points that might seem unsurprising to those living in the twenty-first century but were in fact startling\u2014\u201ca bolt from the blue,\u201d as one of Shannon\u2019s colleagues put it\u2014to those just getting over the Second World War: (1) All communications could be thought of in terms of information; (2) all information could be measured in bits; (3) all the measurable bits of information could be thought of, and indeed should be thought of, digitally<\/em>. This could mean dots or dashes, heads or tails, or the on\/off pulses that comprised PCM. Or it could simply be a string of, say, five or six 1s and 0s, each grouping of numerical bits representing a letter or punctuation mark in the alphabet. For instance, in the American Standard Code for Information Interchange (ASCII), which was worked out several years after Shannon\u2019s theory, the binary representation for FACT IS STRANGER THAN FICTION would be as follows:<\/p>\n\n\n\n 010001100100000101000011010101000010000001001001010 1001100100000010100110101010001010010010000010100111 001000111010001010101001000100000010101000100100001 000001010011100010000001000110010010010100001101010 100010010010100111101001110<\/p>\n\n\n\n Thus Shannon was suggesting that all information, at least from the view of someone trying to move it from place to place, was the same, whether it was a phone call or a microwave transmission or a television signal.<\/p>\n\n\n\n This was a philosophical argument, in many respects, and one that would only infiltrate the thinking of the country\u2019s technologists slowly over the next few decades. To the engineers at the Labs, the practical mathematical arguments Shannon was also laying out made a more immediate impression. His calculations showed that the information content of a message could not exceed the capacity of the channel through which you were sending it. Much in the same way a pipe could only carry so many gallons of water per second and no more, a transmission channel could only carry so many bits of information at a certain rate and no more. Anything beyond that would reduce the quality of your transmission. The upshot was that by measuring the information capacity of your channel and by measuring the information content of your message you could know how fast, and how well, you could send your message. Engineers could now try to align the two\u2014capacity and information content. For anyone who actually designed communications systems with wires or cables or microwave transmitters, Shannon had handed them not only an idea, but a new kind of yardstick.<\/p>\n\n\n\n