Claude Shannon, a legacy transcending digital technology

Claude Shannon, a major scientist from the second half of the 20^th century, marked his era with his communication theory. His work triggered a digital metamorphosis that today affects all levels of our societies. To celebrate what would have been Shannon’s 100^th birthday this year, the Institut Henri Poincaré will pay tribute to the scientist with a conference on October 26 to 28. At this event, Olivier Rioul, a researcher at Télécom ParisTech, will provide insight into the identity of this pioneer in the communication field, and will present part of the legacy he left behind.

 

 

Claude Elwood Shannon. The name is not well known by the general public. And yet, if the digital revolution the world is experiencing today had a father, it would doubtless be this man, born in 1916 in a small town in Michigan. His life, which ended in 2001, received little media coverage. Unlike Alan Turing, no Hollywood blockbusters have been dedicated to him. Nor has his identity been mythicized by artistic circles, as was the case for Einstein. “Shannon led an ordinary life, and perhaps that is why nobody talks about him,” observes Olivier Rioul, researcher in digital communications at Télécom ParisTech.

Though his life was not particularly extraordinary, Claude Shannon’s work, on the other hand, was thrilling in many ways. In 1948, he wrote an article entitled A mathematical theory of communication. “Its publication came as revolution in the scientific world,” explains Olivier Rioul. In this article, Claude Shannon introduced the concept of bits of information. He also outlined – for the first time – a schematic diagram of a communication channel, which included all the active parts involved in transmitting a signal, from its source to its destination.

 

Shannon and his magic formula

Yet in addition to his channel diagram, it was above all a formula published in a paper in 1948 that went on to mark the scientific community: C=W.log(1+SNR). With this mathematical expression, Shannon defined the maximum capacity of a transmission channel, in other words, the quantity of information that can be transmitted in a reliable manner. It shows that this capacity depends solely on the channel’s bandwidth, and the relationship between the strength of the transmitted signal and the noise in the channel. Based on these results, every channel has a throughput limit, below which the message transmitted from the transmitter to the receiver is not altered.

“Shannon’s strength lies in having succeeded in obtaining this result in a theoretical way,” insists Olivier Rioul. “Shannon did not provide the solution required to reach this limit, but showed that it exists, for all channels.” It would not be until 43 years later, with the work of Berrou, Glavieux and Thitimajshima in 1991, that Shannon’s limit would be nearly reached for the first time with the development of turbo codes.

Olivier Rioul believes the story behind this formula is out of the ordinary, and has been the subject of many historical approximations. “And the time was ripe. In 1948 – the year in which Claude Shannon made his results public – seven other scientists published similar formulas,” he explains, based on research carried out with José Carlos Magossi on the history of this formula.

However, the results obtained by Shannon’s peers were sometimes inaccurate and sometimes inspired by Shannon’s prior work, and therefore not very original. And all of them were part of the same environment, were in contact with each other or participated in the same conferences. All except Jacques Laplume, a French engineer who obtained a correct formula similar to Shannon’s at almost the same time. Yet what he lacked and what kept him from leaving his mark on history was the enormous contribution of the rest of Shannon’s theory.

Read the blog post What are turbo codes?

A pioneer in communications, but that’s not all…

While his work represents the beginnings of modern digital communications, Claude Shannon also left behind a much greater legacy. In 1954, behavioral psychologist Paul Fitts published his law, named after him, which is used to model human movements. In his scientific article, he explicitly cited Shannon’s theorem, referring to his channel capacity formula. “Today we use Fitt’s formula to study human-computer interactions,” explains Olivier Rioul, who worked with a PhD student on reconciling this law with Shannon’s theory.

The scope of Shannon’s work therefore far exceeds the realms of information and communication theory. As a lover of games, he developed one of the first machines for playing chess. He was also one of the pioneers of artificial intelligence and machine learning, with his demonstration in 1950 of a robotic mouse that could find its way through a labyrinth and remember the optimal route.

Although Shannon’s life was not necessarily extraordinary in the literal sense, he was undeniably an extraordinary man. As for his lack of fame – which the centenary celebration of his birth seeks to remedy – he himself had said, referring to his information theory, “In the beginning I didn’t think it would have an enormous impact. I enjoyed working on these problems, just like I enjoyed working on lots of other problems, without any ulterior motives for money or fame. And I think that a lot of other scientists have this same approach, they work because of their love of the game.”

 

Read more on our blog :

Simplicity theory: teaching relevance to artificial intelligences