Collective Complicity vs. Analytics

It is Halloween and I want to highlight one of my favorite spooky stories: The Legend of Sleepy Hollow, published by Washington Irving in 1820 (wikipedia article).

Growing up, I remember watching the Disney version of the story around this time of year.  The penultimate scene had the lanky schoolteacher Ichabod Crane being chased on horseback through the woods with the spectral Headless Horseman chasing after him and ultimately besting him.  The Headless Horseman as a character has been reinforced through popular culture as a character on cartoon shows like Scooby Doo, the film Sleepy Hollow, and most recently in the inexplicably-popular Fox show of the same name.

However, in the original story, Irving strongly implies that there was no Headless Horseman that night and that Crane met his demise at the hands of his rival, Brom Bones.  The Headless Horseman was simply a legend developed by the townspeople in order to explain Crane’s disappearance and deflect the suggestion of wrongdoing away from Bones.

The story is scarier if you consider the implication that there was no spectral horseman.  Crane was not liked by the townspeople and the Headless Horseman represents a collective complicity (or at least willful ignorance) of the people of Sleepy Hollow to Crane’s disappearance.

Groups within companies and organizations develop similar collective myths to explain client/customer/user behavior (particularly negative behavior) by assigning a “causality story” to it.  It is up to us as Analytics professionals to be on the lookout for this type of shared mythology and use the tools at our disposal to dispel it.  It can be difficult to “bust” this type of myth, particularly when it benefits a collective narrative, than prove an everyday hypothesis, but these types of myths can be major hurdles to more accurate understanding of our users’ behavior.