In which Jill thinks about Big Data in Big Sur.

I’ve just returned from an inspiring week at Esalen Institute in Big Sur, California. People that know a little bit about Esalen understand it as the locus of the ‘60s counterculture, where Hunter S. Thompson got inspiration at the (clothing-optional) hot springs, Joan Baez sat at the fire pit and played her guitar, Timothy Leary lectured on the ecstatic experience, and everyone dropped acid. Esalen was a crossroads for encounter groups, Gestalt workshops, and body work, and it still is.  Indeed, people that really know Esalen recognize it as the birthplace of the human potential movement.

I had the pleasure of attending an informal gathering in which Rice University professor and Esalen historian Jeffrey J. Kripal explained some recent Esalen-inspired work, courtesy of The Esalen Center for Theory and Research. The Center connects big thinkers and their discoveries with the metaphysical sciences.

And that’s where Big Data comes in. Several active research studies involve Big Data applications that transcend what most of us know to be the traditional purviews of risk analysis, fraud detection, and customer churn prediction. Data on near-death experiences is being gathered and analyzed to help researchers understand the human survival of bodily death. And even that old standby, quantum physics, makes an appearance on the Big Data stage, where researchers are analyzing the causal effect of conscious effort on physical changes in the brain.

Since the inception of the Center for Theory and Research, world-class thinkers, scientists, and model-builders have been convening at Esalen to explore these and other topics. It’s not ideation—long a staple of the Esalen diet, fortified by the gentle crashing of waves, prolific organic garden, croaking of sea lions in the twilight—it’s scientific study, and it relies on huge volumes of often-unstructured and barely-tested data.

Government institutions have increasingly been leveraging Big Data—in both its granular form and in the technologies that enable it—to predict weather patterns, discover seismic patterns that can predict large earthquakes, or track the behaviors of rogue breast cancer cells.

The application of Big Data to humanity’s problems can be even more profound. Discovering the non-linear systems that could contribute to reversing climate change, improving patient health—at the bedside or even on the operating table—or driving economic stability are just the tip of the rapidly-melting iceberg.

As promising as it is for commercial businesses, Big Data can do more than help us understand our best customers or optimize our supply chains. It could do nothing less than save us.