20 KiB
authors, created_at, title
| authors | created_at | title | |
|---|---|---|---|
|
2020/04/23 | The Past, Present, and the Future |
Introduction
For eons, the bravest kings, noblest queens, mightiest commanders, and fearsome scouts have all relied on effective communication to win wars, stop rebellions, command armies, and safeguard against betrayal. Yet, they all understood if their communication was intercepted, and should this information fall into enemy hands, it might lead to demise, and ultimately death. To prevent interception of vital communication through betrayal or usage of force, they encouraged scholars to come up with what's now known as ciphers, which are techniques for disguising information.
The desire for secrecy means, that someone is out there to get that information. This has resulted in two different factions, the codemakers and the codebreakers. From the beginning of time, there has been this eternal war between both factions. The codemakers strive to make better and better codes, the codebreakers have come with stronger and powerful methods for attacking these codes. In fact, this war between the codemakers and the codebreakers has been responsible for many breakthrough in mathematics and the sciences including the most important one, computers.
This document will constantly evolve to document this everlasting war. We don't really talk about how the techniques work in this document. For those head over to different sections where we discuss those in detail. For those who are new to cryptography, however, it is essential, in my opinion, to understand the history of cryptography and how the events from past resulted in present, and how they will affect the future.
The Past
The War of Xerxes & Greeks
Some of the earliest accounts of hiding secrets date back to Herodotus, a legendary figure known as "the father of history". Herodotus in one of his accounts, chronicles the conflict of Persia and Xerxes, the despotic king of Persians. It was art of hiding secrets that saved Persia from falling into slavery.
Greek and Persia are both known for their well running feud. This feud reached a never before seen crisis after Xerxes started construction of the new capital of his great kingdom. Gifts and tributes arrived from kings, merchants, and the likes with the sole exception of Athens and Sparta. Xerxes was extremely proud and this insolence has reached his bottom line. Xerxes started mobilizing a force and declared that he will conquer the world and Persia's boundary would be the heavens itself. He actually assembled the greatest fighting force in the history of mankind, and that too in secrecy, and was ready to launch an attack.
This buildup, however, was witnessed by Demaratus. Demaratus was a greek who was exiled from his homeland, and lived in a Persian city. Yet, he still felt some loyalty towards his home, so he decided to warn them about the incoming disaster. The question was, how would I do that without getting caught by the Persian Guards. Unlike fantasy novels, no pigeons were willing to come to take his letter to the Greek commanders. He decided to scrape wax from the wooden folding tablets, write the message, and then cover it with wax again. This way, it appears blank to the guards, and when the message reaches the destination, the wax can be scraped off, and the message could be received.
Thanks to this message the defenseless Greeks began to arm themselves. They were able to create two hundred warships that could be used to fight off the invasion. Xerxes has lost the element of surprise, and the Greeks were ready, waiting for the ambush. As the Greeks were prepared, they were able to bait the Persian warships to fight in the confines of the bay. Soon panic took hold of the Persian forces. Princess Artemisia was surrounded by three sides, and scared for her life decided to head back to the sea, only to ram her ship to another ship. Soon, chaos took over, and Greeks launched a bloody assault on the Persian forces. The mightiest army assembled had been outmaneuvered.
This simple strategy by Demaratus revolved around hiding the message. In fact, this is not the first time a simple concealment of a message has worked. Herodotus' account also tells of another extremely funny tale. Histiaeus wanted to encourage Aristagoras to revolt against the persian king. To convey this message without being intercepted, he shaved the head of one of his messengers, and wrote the message on his scalp, and then waited until his hair had grown back. Since the messenger seemingly carried nothing with him, the king's guards didn't give him a hard time. When the messenger finally reached the recipient of the message, he shaved off his head and pointed it at the intended recipient.
What really is cryptography and steganography?
The technique of hiding the existence of messages is called steganography. The word was derived from two, you guessed it, greek words steganos, meaning covered, and graphein meaning to write. Other examples of this technique being used include the ancient Chinese silk ball, where a message was written, which was then covered in wax and swallowed by the messenger. The usage of invisible ink, which disappears until heated is also considered steganography. The "milk" of tithymalus plant was originally first used as ink. As the ink is transparent after drying, gentle heating will char the ink to visible brown. In fact, many organic fluids work similarly, and who knows for how many modern spies have improvised by using their own urine.
While steganography is certainly useful, and still used quite extensively, it suffers from a fundamental weakness that if the message is discovered, then the content of the message is revealed right away. There will be events where someday the message is discovered. Scraping wax tablets, heating paper, shaving heads, while unlikely might happen some day. Thus in parallel, we started the evolution of cryptography. Cryptography originates from two, you guessed it (it's always either latin or greek), greek words crypto which means "hidden" and graphein which means "to write". The aim is to make the message unintelligible. The protocol is discussed beforehand, and thus only those who are privy to the details are able to reverse it. Without the protocol, it is quite difficult to scramble the message, although when the message is of utmost importance, history has told us that codebreakers have always been successful. Poor Mary learnt it the hard way.
Both can be combined for maximum security. During World War II, microdot was a form of steganography that Germans extensively used. They'd shrink a page full of text down to less than 1 millimeter, then hide it on top of a letter. A tip from an anonymous source to the FBI, who told them to look out for a shiny gleam was the first time Americans had caught this. Therefore, Americans were able to read the contents of the page, only that the Germans were extra careful and had scrambled the text and then reduced it. This combines cryptography and steganography for maximum security. Cryptography is considered to be the more powerful of the two branches, because cryptography is the only technique that prevents messages from being processed by the enemy.
Ancient cryptography can be thought off with two branches. Transposition and Substitution Ciphers.
In the former, we rearrange the letters. While this is insecure for smaller messages, for bigger numbers it's extremely difficult to bruteforce. For example, in a short email of 40 characters, there are 40 to the power of 26 variations. A thousand lifetimes is not enough to decode this one by one. In fact, the sun would have probably swallowed us whole long ago before we finish. While this might be super secure, if we randomly jumble these letters without reason, then the intended recipient would also have to spend a thousand lifetimes before getting the message. It's a little too late by then. Which means, it has to be fairly straightforward to be understood by the recipient, that has been previously agreed.
The later, called substitution revolves around substituting something for another. One of the earliest descriptions of this appears in Kama Sutra, a text written in the fourth century about 64-arts women should study to be a good wife. One of those arts is the art of cryptography. One of the techniques recommended is to pair one set of letters with another.
Caesar and his ciphers
One of the first documented uses of a substitution cipher is documented in Julius Caesar's Gallic Wars. Caesar describes how he sent a message to Cicero, who was on the verge of surrendering that his army is on his way to reinforce them. Caesar instructed his messenger, who was ahead of the army rushing towards Cicero, that if he couldn't approach the tower to give the news to Cicero, fasten the letter to the spear, and hurl it towards the tower. The letter substituted Roman Letters with Greek Letters effectively making it incomprehensible to the enemy soldiers. The messenger, fearing for his life, hurled his spear towards the tower, and by sheer luck it struck one of the tower's walls. After two days, the letter was sighted by one of Cicero's soldiers, and was brought to him. Cicero was extremely happy, and recited the letter to his parade of soldiers, thus increasing their morale to maximum.
In fact Caesar was one of the firmest believer of cryptography. In fact, he used cryptography so much that Valerius Probus wrote an entire treaty on his ciphers, which unfortunately didn't survive time. The only Cipher of his, that we have a detailed description of, is called the Caesar Shift Cipher, which works by shifting the position of every character by some number.
Arabic Golden Age
In 750AD, the start of the Abbasid dynasty healded the golden age of Islam. The arts and the sciences flourished. The legacy of Islamic scientists from that era is still felt to this day evident from arabic words such as algebra, that are found all around the sciences. This richness was largely due to rich and peaceful society. There was not much interest in conquest, and wanted to establish an affluent society. Lower taxes encouraged greater economy, and all of this centred around effective administration, which relied on secure communication achieved through encryption. Officials protected sensitive information and tax records through cryptography. They usually used a monoalphabetic substitution cipher, where letters could be replaced by different letters. Though these ciphers aren't much of a big deal, arabs alongside being able to use ciphers, were also able to destroy them. They invented cryptanalysis, which is the science to unscramble encryption without knowing it's key. While cryptographers develop new methods of encrypting data, cryptanalysts are the ones responsible for breaking these. Arabic scientists were able to find a way to break the monoalphabetic substitution cipher, which had been invulnerable for centuries.
Cryptanalysis could not be invented yet because it requires study of mathematics, statistics and linguistics. Since Islam pursued knowledge above all, the scientists were equipped with the tools to dabble in it. The economic success also meant that the scholars could focus on research instead of worrying about survival. The art was able to be distributed as they had acquired the knowledge to make paper from the Chinese.
The arabic scholars analyzed individual letters, and not just words. They later realized that certain words, especially a and i, are very common in arabic. This small discovery is actually a giant breakthrough that later evolved to form frequency analysis. Instead of scrambling billions of characters, frequency of characters could easily reveal the contents of the message. While highly effective, this is not a silver bullet. Short texts deviate greatly from the averages, for example. Similarly for longer content, it's not necessary for it to follow that certain frequency.
The Dark Ages
While the Arab scholars were having a good time, europe was struck in Dark Ages. Arabs had already pioneered cryptanalysis, while Europe was still struggling with the basics. The study of cryptography was mostly done by monks studying bible, in search of hidden meaning, much like a bible study group. These monks were fascinated by the fact that the old testament included many example of cryptography. For example, atbash cipher appears quite often in the bible. Thanks to this fascination, the usage of cryptography got more and more popular in Europe.
Centuries later, cryptography became a norm in Europe, with alchemists and scientists using it to encrypt their discoveries. Soon, politics came into play, and interest in cryptography reached unprecedented heights for secret communications.
The Cipher of Mary, Queen of Scots
Mary Stuart was born to King James V, and was his only legitimate surviving child. The distraught Scottish King had suffered a complete mental and physical breakdown, after a crushing battle with Henry VIII. Just a week later after Mary was born, as if he was waiting for the news of an heir, died. The untimely death of her father meant that Mary inherited the throne as an infant. At the age of nine, she was officially crowned as the queen of Scots. Her young age prevented Henry VIII from attacking Scotland, as he would receive backlash for invading a country whose king recently died. Henry VIII decided to strike a deal by offering his son and Mary to be tied in a sacred bond of marriage. His idea, that this way his line could peacefully gain the crown of Scotland. The Scots, however, instead struck a deal with the French, that Mary and Francis, the dauphin of France, would marry when they were of age, and would unite France and Scotland. As France was a Catholic nation, Scotland themselves would prefer to ally with a catholic country. In the meantime, France would deter England from attacking Scotland.
Soon Mary at the age of six, set sail to France. Mary's first few years in French Court were the definition of luxury. She was protected from harm, lived in a grand castle, and fell in love with her future husband. At the age of sixteen they married, and Mary became the Queen of France. Soon, however, Francis succumbed to an ear infection, and Mary was widowed. From this point onwards, her life became full of tragedy.
When she returned back to Scotland, the nation was completely different. Scotland had moved mostly towards the Protestant church. At first, Mary was able to keep her subjects in check. However, her second marriage to her cousin Darnley, led to decline in her popularity. Darnley was a vicious man, whose greed for power resulted in Mary losing faith of Scottish nobles. Following year, Darnley viciously murdered Mary's secretary in front of her. It was clear that Darnley was not a sane man, and he had a lot of power that an insane man should not. Soon, his house blew up, and when he attempted to escape was strangled. Her son was the only good thing to come out of the marriage.
Her next marriage was hardly more successful. James Hepburn, was publically considered the murderer of Mary's husband. Further events occurred, and the protestant nobles rebelled against Mary. Mary was imprisoned. However, next year she escaped from her prison, and fled for her life to England through a fishing boat. She had hoped that Elizabeth, her cousin, would provide her refuge. This was a big mistake, as English Catholics considered Mary, a catholic, to be the true queen of England. Thus Elizabeth imprisioned Mary, on paper the reason was her connection with hthe murder of Darnley fearing her rise.
Mary was imprisoned for 18 years. Her sole crime was her existence. This had pushed her to limit. Some of her catholics supporters who were increasingly being harassed and disposed by the protestants, hatched a plan to rescue her. Mary, then masterminded a plan to assassinate Elizabeth. As she is next in line for the throne, she would effectively be able to escape imprisonment. Mary designed a cipher to communicate with her correspondents, consisting of cipher alphabets and code words. Her plans were going well, and she would have succeeded, but her courier was actually double agent to Walsingham, principal secretary to the Queen. While Mary used a fairly complicated cipher, her correspondents only sent an extremely simple cipher when replying to her. Walsingham was a firm believer in cryptography. He has learnt his lessons and understood the power of cryptography after he was able to deter an invasion of England thanks to it. So he was very ready to fight off this threat. He laid down waiting for the right moment to strike.
Walsingham had to convince Queen Elizabeth of Mary's guilt. While Queen Elizabeth despised Mary, she was a Scottish Queen and there would be a giant controversy if a head of a state could execute a foreign head of state. Elizabeth wasn't willing to try this gamble as this would set an interesting precedent, which means that rebels wouldn't have much reservations about killing another Queen. Her name? Elizabeth. Then they were related by blood, and it's not exactly a great feeling to execute your own family, especially someone who's life was such a tragedy. The essence of this assasination was the exact opposite of what caused her abdication from the throne. In this case, some catholic noblemen wanted to replace Elizabeth,a protestant with a catholic queen, and Mary perfectly fit the bill. It was clear that Mary was involved in the incident, but they were not sure if Mary was a figurehead, or if she was the mastermind herself. Thus, Walsingham's challenge was to find definitive proof that could link Mary as the mastermind behind plotting the assasination of the Queen of England.
Mary was a careful woman. She had ensured that all her communication with other conspirators had been written in the cipher she designed. This cipher turned this wonderful content about murdering a head of state into meaningless symbols that made absolutely no sense. Mary was sure that even if Walsingham intercepted one of her letters, the contents were a complete mystery. Just like how in the modern world, encrypted data cannot be used as evidence by law enforcement agencies, similarly it could not be used in the English Court as well.
Unfortunately for her, Walsingham had another designation. He was England's spymaster. He exactly knew who might be capable of deciphering these meaningless symbols. Thomas Phellippes, while little is known about him, he's a legendary fellow, who was known for deciphering similar cipher messages by those who plotted against Queen Elizabeth, and thus responsible for providing evidence to punish them. Once again, it was the battle of wits against a code maker, and the codebreaker. If Mary's cipher is strong enough, there is a good chance she will survive. Once again, Mary's life hangs on a paper thin margin. However, Phellippes was a genius. He would devour any message he received, and have solutions in matter of time. He was the master of frequency analysis, and with how many times each character repeated, he was able to easily decode majority of the content. As for codewords? That could be easily understood from the context of rest of the decoded message. This is a good example of how sometimes bad cipher is worse than no cipher. If they were meeting in public, they would have acted far more discreetly, but in this case believing their communication to be secure, they threw caution to the wind. Lessons to be learnt here. The correspondents were caught and suffered the most humiliating and horrid execution. In the words of the Elizabethan historian William Camden, “they were all cut down, their privities were cut off, bowelled alive and seeing, and quartered.”
Mary was soon caught, and was sent to trial. Although, they used a cipher, there belonged to an era where ciphers were constantly being weakened by advances in cryptanalysis. It stood no chance against Phellippes. There was sufficient proof, and death penalty was recommended. Elizabeth signed her death warrant. With her family motto in mind, "In my end is my beginning", she approached the block. The executioners requested her forgiveness, and she was finally released from her misery.
This document is a work in progress, and will be constantly updated.