Michael C. Libby
1/9/2005 1:18:00 PM
On Sun, 2005-01-09 at 15:31 +0900, Dave Burt wrote:
> You looked at matching words. I'm wondering if you could get as good a go
> with reasonable-sized ciphertexts using a letter frequencies method? (i.e.
> the most common letters in an English text are E, then T, etc.) Perhaps you
> could apply letter frequencies first before moving on to match common words,
> even.
Using letter frequencies really was not something we could do for the
Quiz because our inputs were just a list of dictionary words and a
cryptogram. Going beyond the Quiz parametes, we could generate a list of
frequencies (using texts from gutenberg.org or a web crawler) or we
could find some cryptological authority's list and use that.
Even without using additional inputs, I did consider the frequency
approach since that is a common heuristic a human uses to solve a
substitution or shift cipher. However, the dictionary we were using
returns different frequencies than I expected to see based on my
experience with cryptanalysis (namely, watching "Wheel of Fortune" for
20+ years), so I abandoned this approach.
After thinking about it, I concluded we're getting essentially the same
effect from a different angle using word patterns.
-Michael