This Mother’s Day, I enjoyed various online recollections of what some friends and colleagues had learned from their mothers. I’ve known for a while that my mother taught me more by what she did than what she said. No quippy little dictums about doing unto others or wearing clean underwear. No wise experiential aphorisms intended to help me navigate around the landmines she herself had tripped. Not my mother.

My mother was no coach. When I was seven years old, I emerged from my room on “Free Dress Day”—the one day our school allowed us to shed our uniforms and wear what we wanted. I was sporting a rainbow-colored ensemble that mixed florals and plaids. My mother looked me over briefly and pronounced my sartorial decision “a misguided effort at bohemianism.” So much for coaching.

When I entered the fifth grade, my school scheduled a Girls Health Day. We all recognized this as a euphemistic announcement that we were going to learn about sex. My classmates and I whispered and rolled our eyes, mustering all the worldly cynicism we could as fifth graders. The truth was we’d all been to slumber parties and knew the rumor that the man pees inside the woman and that’s how you made a baby. The quiet individual traumas that resulted reverberated as a collective snarkiness directed at anything having to do with reproduction.

On Girls Health Day, we somberly filed into the school auditorium, where a nurse would soon be showing us drawings of body parts that somehow interlocked. Each of us harbored an unspoken fear that we’d be confronted with images of appendages emitting the requisite urine that could not only rid the male body of toxins but also make us pregnant. Over Christmas break, we’d all chosen baby names for our firstborns, but the euphoria had been fleeting. We were about to enter puberty and we wanted nothing to do with it. The nurse had better be serious and quick so we could get back to our math homework. That’s how bad it was.

When the nurse walked onto the stage, she was indeed serious. Moreover she was my mother.

Soon after getting her nursing degree, my mother had moved into management, so I’d never seen her in her nursing uniform. But there she was, dressed in a starched white dress, a cross between a milkmaid and a Sister of Charity. Several of my classmates recognized her and began variously patting me on the head and kicking my chair. I hid my face as my mother delivered an anatomical treatise, giving both sexes their due in what seemed to me unnecessary detail. Every time she said the word “genitalia,” my classmates squealed and I died a little inside.

That night after I’d finished my homework, I marched downstairs and stood before my mother, who was sitting at the kitchen table sipping a glass of rosé and reading Redbook.

“H-how c-c-could you,” I stammered before bursting into tears. “Why did you come to my school? Why did you DO that? WHY?”

My mother put down her wine glass and smiled sympathetically. “Because you needed to learn it,” she said.

Had my mother given me a pop-quiz about the cruel transformations of adolescence I would have failed, so muddled was my memory of the day’s lecture. But learning how to be an effective and memorable teacher? It was the lesson of a lifetime.

The newest aphorism-du-jour in the big data world is this: “It’s not big data—it’s just data.” As debates on both the definition and value of big data continue, this argument has some validity. The struggle to find, assess, cleanse, annotate, integrate, standardize, and provision data predates not only the big data trend, but computing itself.

Information Week warned readers of the consequences of “infoglut” back in 1995 under the heading “New Tools that Can Help Tame an Ocean of Data.” Mixed metaphor aside, the article confirmed what had been keeping business and IT managers up at night: how to harness proliferating and ever more complex amounts of data.  The rise of Big Data means that these challenges will only get harder.

“Most of the complex problems we tackle should involve some sort of initial data exploration,” explains Bill Rand, Assistant Professor of Marketing at the University of Maryland and Director at the university’s Center for Complexity in Business.  Rand personifies the expanding role of the data scientist, professionals who not only explore diverse data sets but determine how the use of the data can help their companies compete.

Rand and his team have been applying analytical skills to examine diverse social media data to understand behavior patterns and propensities that could aid marketers. “Social media players aren’t a bunch of people working on a common problem,” he explains. ”They’re individuals working on separate problems. Data Scientists need to explore large volumes of detailed data to understand the realm of possible social media actions. Only after the initial analysis can they determine how to apply subsequent analytic models.”

The keyword here isn’t “analyze,” but “apply.” The people with the job title dubbed “The Sexiest Job of the 21^st Century” by authors Thomas H. Davenport, Ph.D, and D.J. Patil, are no longer expected to simply run mathematical models against diverse data sets. They’re now just as likely to suggest how to leverage the data to drive cross-selling techniques, suggest supply chain efficiencies, predict fraud, and determine a customer’s next likely purchase.

“Data scientists, by definition, combine business acumen with data acumen,” explains P.K. Kannan, Professor of Marketing Science and Marketing Department Chair at University of Maryland’s Smith Business School (and Rand’s boss). “From a knowledge perspective, a data scientist has keen insights into the business models driving the firm, its products and services, while simultaneously possessing mastery of data creation and data analysis. In that sense, they’re different from traditional statisticians not only in their business domain knowledge but also in terms of their broader scope.”

This is one lofty job description and one that, without the right set of guidelines, standards, and skills, is primed for failure. On the one hand, IT personnel are likely to have begun implementing data governance, establishing clear policies for the access, usage, and deployment of information from a variety of sources. They may have also adopted enabling technologies such as data quality, master data management, and metadata repository tools to help automate repeatable tasks. Depending on how it’s defined, the data scientist’s role could erode data governance policies, or worse, contradict them.

On the business side, the phenomenon of data hoarding is alive and well and making no apologies. Even in the age of big data, knowledge is (still) power, and line of business staff are loathe to share data that might bestow the sheen of indispensability. So the customer address data is shared, but the on-line behaviors are shielded from customer support reps. Or the electronic health record is shared with clinicians but the patient’s survey data is shared only with administrators. A data scientist (or business analyst or visualization tool user) can hardly deliver value if she can only access a portion of the data—however big—she needs to do her job.

Managers have to do the hard but sometimes unpleasant work of inventorying incumbent skills and even consolidating data management roles or functions.  Circumscribing role boundaries is key, not only to prevent duplication of effort, but to stem confusion among incumbent data experts.  Failing to do so can result in staff disaffection. “I guess I always assumed I was one of the firm’s data authorities,” an actuary at an insurance company confided recently. “Now I’m being ‘coached’ on how to do the job I’ve done for twelve years. Maybe if I called myself a data scientist I’d have more clout.”

With the increase of systems generating the data—both within and outside of the firewall—operationalizing the flow and usage of information is the biggest barrier to becoming a data–driven organization. At Baseline Consulting we called it “the data supply chain,” and it’s an apt term for big data’s interdisciplinary skill sets and cross-functional reach. Because no matter how big or complex the data is, the “it’s not the size, but how you use it” aphorism is as true as it ever was.

This post is an excerpt I wrote for Phil Simon’s new book, Too Big to Ignore: The Business Case for Big Data (John Wiley & Sons, 2013). The book is geared to business managers and executives seeking to understand big data’s value proposition, admonishing them to Think Big.

I grew up the curious product of parents with very different backgrounds. My father, a third-generation Californian, majored in animal husbandry at Cal Poly. For my eleventh birthday, he bought me a horse, which we kept in our suburban backyard to the chagrin of our middle-class neighbors. My mother, a Canadian with a master’s degree and a big job in health care, appreciated the finer things, tolerating my dad-decreed 4-H club membership with remarkable sangfroid. Dad and I would spend Saturday mornings with my troop studying the conformation of dairy cattle. Then I’d decamp with my mother to Bullock’s Wilshire for shopping and tea.

This peaceful coexistence took a turn when my father and some other 4-H dads decided to castrate my horse in our front yard. My mother and our long-suffering neighbors—never mind the new gelding—were not amused. After a heated exchange between my parents, I found my father solemnly hosing horse blood off the driveway. I made a smart-ass kid remark about the post-argument symbolism of that. Thereafter, I had fewer 4-H stars on my white cap and a lot more clothes for school.

Memories of my parents’ dueling child-rearing philosophies evoke past experiences helping companies align business and IT around analytics programs. Business people complain that IT doesn’t understand how to solve their problems. IT, in turn, laments the lack of business engagement. “They don’t know what they don’t know,” one says about the other. Then, in a familiar matrimonial dance, the parties retreat to their comfort zones and do what they’ve always done, flummoxed about how to collaborate.

Some visionary companies are now changing the game by redefining business and IT partnerships. This involves more than convincing people to attend requirements meetings or formalizing rules of engagement. These companies are letting business strategies drive their project portfolios, joining forces to refine delivery approaches and spur (sorry, old habits die hard) innovation. They’re not only talking, they’re coming together as blended teams, revamping budgeting and delivery models in the process.

“We’re moving away from our traditional funding model where you have a business case and you try to justify it,” an executive at a major property and casualty insurer explained to me. “IT and the business are now pooling budget money to support discovery efforts. Our objective is no longer about having all the answers up-front. Instead, we want to deliver jointly and quickly, fail fast if we need to, and move to the next business problem.”

Redefining collaboration between business and IT means bringing people together in more innovative ways. Managers more likely to dispense with protracted in-person meetings in favor of using on-line social communities and automated workflow software, letting different organizations and remote workers participate fully.  Job roles and work tasks are clearer and delivery times are tighter.  Lifting lessons from the agile playbook, these companies make sure that outcomes are well-understood by all, and those who are closest to delivery have a say in the cost-benefit conversation. Projects are not only being delivered at a faster pace, but a greater percentage of them meet requirements and are delivered on time.

The only constant in life is change. These days, Bullock’s Wilshire is a library and the 4-H Club has expanded its mission beyond livestock. My horse is long gone, but I imagine he’d have wanted a say in the cost-benefit conversation too, if he’d had a say.

In which Jill tests “the sexiest job of the 21^st century,” and finds herself alone. Oh, so alone.

It’s late fall in L.A. The Santa Ana winds have given way to crisp, sunny days and nights by the fire. Palm trees rustle with the suggestion of January’s arrival and the coming rains. It’s the last hour of light and I wait for my data scientist. And I wait.

Rrrriing!

My data scientist is running late. “I’m over halfway done documenting the semantic layer for the geospatial data,” he explains.

“But I’m already running the bath,” I protest.

“See you in a few hours,” he says hastily.

The ice in the Negroni will melt long before then, I want to tell him. The bath, the fondue, and I will all be cold. But it’s too late. He’s hung up.

Flash forward a few weeks. My data scientist and I are at a big data conference in San Francisco. I’m looking forward to a romantic dinner at La Folie, a Pisco Sour, and some classic rock. Maybe we’ll end up at that Tiki bar on Noriega Street. But my data scientist has other ideas.

“So do you really think R is still fringe?” he asks. I reapply my lipstick but he persists. “Do you think it can transcend the statistical community and stand on its own as a …”

I interrupt him. “When am I going to meet your parents?”

“It’s complicated,” he says perfunctorily, but at this point I’m not sure what he’s referring to.

We fly home. The data scientist works on the plane, muttering something about adaptive filters. It’s now twelve-hour days at the office and every conversation starts and ends with his job. The data scientist wants me to recommend a tool to capture business rules. He wants to run through optimal data validation techniques. He quizzes me on when unstructured data needs to be integrated with structured data. How to calculate the ROI of data federation. The merits of probabilistic versus deterministic matching. Whether I think data as a service has legs.

“I have legs. And they’re tanned. Here…take a look…”

But there’s no use. The data scientist is preoccupied. Preoccupied with finding, accessing, analyzing, validating, cleansing, integrating, provisioning, modeling, verifying, and explaining data to his management, colleagues, end-users, and friends. And to me. I’ve had enough.

I’ll give him an ultimatum tonight. It’s either me or the data, I’ll say. But he’s late again. His text message says only two words: “Stochastic processing.”

I pour Sambuca into a snifter and drop in two coffee beans. I watch them float in the glass. I fish one out and put it in my mouth. I bite down. Then I wait.

When people talk about big data they usually talk about the millions or billions of records that can be analyzed via a new crop of specialized technologies. But the focus is quickly shifting from how much data—technology is doing a fine job of keeping pace with growing data volumes—to how that data is used.

The combination of big data, in-memory computing, mobile devices, and the cloud promises the delivery of real-time insights, however you want them, wherever you are. Automobile companies are searching big data to predict hot spots in components before they fail. Government institutions have increasingly been leveraging big data—in both its granular form and in the technologies that enable it—to forecast long-term weather patterns and discover seismic activities that can predict large earthquakes. Economists are mining data to reveal how human sentiments can drive economic downturns and how human behaviors can stimulate economic growth.

The common denominator of these and other big data applications is this: Discovery. Unlike a traditional database inquiry that assumes a certain hypothesis level, mining big data reveals relationships and patterns in the data that we didn’t even know to look for. And the sooner executives support raw data exploration efforts, the sooner they’ll see payback from their big data investments.

I wrote about discovery twelve years ago in my book, e-Data (Addison Wesley, 2000). The pyramid below represented a taxonomy of analytics. The bottom layer represented the most common type of database inquiry, the standard business intelligence report, progressively evolving toward more advanced types of analytics with successively lower hypotheses:

The pyramid is capped by what I called Knowledge Discovery, the detection of patterns in data. As I wrote back then:

These patterns are too specific and seemingly arbitrary to specify, and the analyst would be playing a perpetual guessing-game trying to figure out all the possible patterns in the database. Instead, special knowledge discovery software tools find the patterns and tell the analyst what—and where—they are.

Hence, you could be mining data on breast cancer cells expecting to see trends in cell proliferation rates. But, to your surprise, you also discover that surrounding non-cancerous cells are also contributing to cancer cell growth. The Stanford University researchers who made this discovery didn’t know to look at the non-cancerous cells. But through low-hypothesis exploration, they found it.

Most companies have mastered the pyramid’s bottom two layers. Indeed many senior managers cite the third tier, representing predictive analytics, as the logical next step in their quest to be data-driven. But few companies possess the right combination of skills, technologies, and new delivery models to reach the pinnacle.

Executives assume there’s no time (let alone budget) for knowledge discovery. Indeed, the very term suggests an academic exercise with no tangible business payback. But as the above examples show, big data discovery efforts can result in startling and highly-actionable findings. A retailer we work with loaded twelve years’ worth of purchase transactions into a Hadoop cluster to uncover new relationships in the data that had gone unnoticed. The company discovered new correlations between products that ended up together in shoppers’ carts. The findings drove innovative product placement and shelf space management decisions. The revenue uplift per shopping cart was averaged at 16 percent in the first month of the trial. Executives were convinced. It’s the apocryphal “beer and diapers” legend writ large on the retailing bottom line.

Armed with newfound understanding of big data’s potential, business executives need to not only allow knowledge discovery efforts, they need to promote them. Fostering a culture of discovery means allotting budget money and resources for big data proofs-of-concept and surrendering expectations for their outcomes. It also means training the new batch of aptly-named data scientists to leverage the big data technologies that enable such discovery, and then translating the findings into business actions whose outcomes are then measured. In essence, running discovery trials on big data is a continuous loop, where the results may feed more traditional business intelligence, or drive additional discovery tests.

Sometimes this means isolating big data efforts from traditional analytics programs where delivery processes and organizational roles are already entrenched. Recently a commercial lines insurer reassigned senior data analysts from various lines of business to staff a temporary work effort to explore new attributes for fraudsters, mining hundreds of terabytes of social network interactions, customer profiles, and claims history. The team found that “loose affiliations” with low-income friends was an indicator a higher propensity to file fraudulent claims. The group of analysts evolved into an informal knowledge discovery SWAT team that reconvened whenever new data types or business processes invited fresh discovery efforts.

Knowledge discovery may force business executives to do an about-face, agreeing to revise team configurations or support discovery activities that will yield new business insights. Such activities, typically requiring quick-hit efforts of highly-skilled experts, have traditionally been prohibited by managers who considered them “skunkworks” projects. But these concentrated, intensive projects can reveal unknown customer behaviors, product affinities, financial risk patterns, and other findings that end up funding the initial discovery work many times over.

The sooner business executives understand the value of knowledge discovery, the more likely they can mobilize their organizations, introduce or revise analysis processes, and hire skilled resources that can ultimately differentiate them from their competitors. Indeed it’s through these low-hypothesis, high-reward surprises that companies can innovate and begin to thrive anew.

A shorter version of this article was originally posted on the Harvard Business Review site. Check out the original post, see what others are saying, and add your own comment there. This will honor the editorial work of HBR.

On a recent flight from Rio de Janeiro to Buenos Aires I found myself enjoying air travel for the first time in a long while. It was my second experience flying Emirates Airways. My first was a flight from London to Dubai in the 1990s. On that flight Emirates plied me with satay skewers and champagne, so my expectations were already as high as 35,000 feet.

The flight attendant greeted me as I stepped aboard and guided me to my seat. Then came the warm nuts and the even warmer (and lavender-scented!) washcloth. One flight attendant brought me a feather pillow. Another, after I’d declined lunch, urged me to try the blueberry cheesecake with my coffee. When I extracted my laptop from my briefcase she helpfully pointed out the power jack in my armrest. An hour into the flight she recalled the milk in my coffee and the cream in my seatmate’s. She admired my sandals, and slyly asked if I’d like to take some extra cheesecake with me to my destination. I liked her very, very much.

Of course I can’t help contrasting this experience with my usual routine with the U.S. carriers. I won’t name the airline I fly most. Anyone can find anything on the web these days, and that includes a martinet flight attendant out for some casual revenge. I’m selfish enough to want ice in my tepid Fresca and my bags to show up in my destination city. So let’s just say that the friendly skies aren’t as friendly as they used to be.

But this does suggest why commodity businesses—be they airlines or toothpaste purveyors—are one well-connected customer away from an avalanche of defections. These days social media sentiments go viral faster than a locomotive—which, not to put too fine a point on it, usually has a better on-time arrival score than my default airline. Witness the popularity of the #AmericanAirlinesSucks hashtag or the viral video about United breaking a guy’s guitar (over 12 million views on YouTube). Not that there’s anything wrong with United. I’m just saying.

Airlines can become positively smug about the loyalty their frequent-flier programs engender. But is it really loyalty or just a compromise to get a little back for what we’re enduring? Surveying my friends (yes, over cheesecake) the other night, everyone agreed that if they could fly anywhere for free once a year, their airline of choice would be completely different. As it is, with ever more rigid blackout dates and escalating service fees, redeeming points isn’t easy.

The best customer relationship management system and the most automated websites are no match for a tired customer community who might grudgingly do business with you but will discourage new prospective customers from jumping on board. It’s no coincidence that commodity businesses are the most interested in acquiring big data technologies to perform social media and sentiment analysis. They need to be one step ahead of the angry mob.

To be fair, I don’t know how liberal Emirates is doling out mileage rewards. (Full disclosure: my former consulting company did work for its Skywards program ten years ago.) All I know is that I got no points for flying Emirates. The flight attendants seem to like their work, there’s milk in my coffee, and the power jack actually works. Is that too much to ask?  In the meantime, my air miles are piling up faster than…well, you know.

Photo by davidwilson1949 via Flickr (Creative Commons license)

I’ve just returned from an inspiring week at Esalen Institute in Big Sur, California. People that know a little bit about Esalen understand it as the locus of the ‘60s counterculture, where Hunter S. Thompson got inspiration at the (clothing-optional) hot springs, Joan Baez sat at the fire pit and played her guitar, Timothy Leary lectured on the ecstatic experience, and everyone dropped acid. Esalen was a crossroads for encounter groups, Gestalt workshops, and body work, and it still is.  Indeed, people that really know Esalen recognize it as the birthplace of the human potential movement.

I had the pleasure of attending an informal gathering in which Rice University professor and Esalen historian Jeffrey J. Kripal explained some recent Esalen-inspired work, courtesy of The Esalen Center for Theory and Research. The Center connects big thinkers and their discoveries with the metaphysical sciences.

And that’s where Big Data comes in. Several active research studies involve Big Data applications that transcend what most of us know to be the traditional purviews of risk analysis, fraud detection, and customer churn prediction. Data on near-death experiences is being gathered and analyzed to help researchers understand the human survival of bodily death. And even that old standby, quantum physics, makes an appearance on the Big Data stage, where researchers are analyzing the causal effect of conscious effort on physical changes in the brain.

Since the inception of the Center for Theory and Research, world-class thinkers, scientists, and model-builders have been convening at Esalen to explore these and other topics. It’s not ideation—long a staple of the Esalen diet, fortified by the gentle crashing of waves, prolific organic garden, croaking of sea lions in the twilight—it’s scientific study, and it relies on huge volumes of often-unstructured and barely-tested data.

Government institutions have increasingly been leveraging Big Data—in both its granular form and in the technologies that enable it—to predict weather patterns, discover seismic patterns that can predict large earthquakes, or track the behaviors of rogue breast cancer cells.

The application of Big Data to humanity’s problems can be even more profound. Discovering the non-linear systems that could contribute to reversing climate change, improving patient health—at the bedside or even on the operating table—or driving economic stability are just the tip of the rapidly-melting iceberg.

As promising as it is for commercial businesses, Big Data can do more than help us understand our best customers or optimize our supply chains. It could do nothing less than save us.

I’ve always been fascinated by how managers brand their business initiatives. I’ve lost track of the number of companies where I’ve worked on a “Voice of the Customer” or “One [Company Name]” program. I once had four clients in a single year who’d launched a “Project Genesis,” all, of course, with vastly different objectives. Forget funding. These days getting your project noticed is its own challenge.

A well-crafted project name can draw attention to a business initiative that would otherwise be lost among so many activities competing for executive attention and budget. It resonates. It’s meaningful. It evokes the effort’s intent. It fosters community, instilling a sense of purpose. And it can get you noticed.

Some project names are so popular  they become cliché (“Project Phoenix,” anyone?). In his book What Clients Love, author Harry Beckwith emphasizes the importance of names in business, citing one company, internet.com, whose name change actually raised its stock price. “Names are golden,” he writes. It turns out naming a project is both art and science.

Managers increasingly shy away from using standard industry terms or acronyms for projects, particularly when the program goes south, as many Customer Relationship Management (CRM) projects did several years ago. My firm at the time was routinely asked to help re-brand moribund CRM efforts. “We can’t call it CRM anymore,” a V.P. of Marketing whispered, unwittingly representing her peers across industries and market segments. “It has too many bad connotations.” Irrespective of how hot the trend, no one wants their project linked with a failure statistic.

Thus many CRM projects have been recast as “Enterprise Customer Focus” (ECF) or “Know Thy Customer” initiatives. Likewise, “Web 2.0,” “enterprise data warehouse,” and even the classically-revered “process reengineering” are meeting similar fates. At best, these projects are re-named with softer-sounding synonyms—witness the rise of “Enterprise Information Management” as the de rigueur replacement for “data governance.” They’re often considered gimmicks by cynical business people. At worst otherwise-worthwhile projects risk being, in a word, “de-scoped.”

The best project names reflect the company’s brand AND the project goals themselves. They’re compelling, implying action. An automobile company’s “Customer-Driven!” project team created posters featuring the image of a laughing driver inside a late-model hybrid, her scarf trailing out the window. It not only illustrated the project’s desired outcome—a happy customer experience—it conveyed forward movement.

But the name is only one piece of the brand. An entertainment company in Los Angeles recently launched a corporate privacy program. The Chief Privacy Officer retained the services of the in-house cartoonists, who created superhero characters called “Captain Security” and “Privacy Man,” complete with unitards and chest logos. To roll out privacy guidelines, the studio hired actors and shot a commercial that was broadcast at employee events and on an internal website. Word about the privacy superheroes traveled around the studio faster than, well, a speeding bullet.

The CPO had recognized the need to fit in with the company’s culture in order to convince the studio’s employees that privacy was everyone’s issue. The superhero characters, and the narrative that accompanied them, were fresh and funny,  reflecting the corporate Zeitgeist.  As a result, privacy compliance has become embedded in the company’s business processes.

The fact is that if your project has clear objectives, realistic measurements, and valuable business outcomes, you can call it “Big-Budget Cluster du Jour,” for all anyone cares. Or simply go to the Project Name Generator at onlinegenerator.com. It’s way more fun than crafting something catchy that never sees the light of day.

Every large financial services company has instituted risk management, but that hasn’t prevented risky behavior in the form of office politics and personality conflict — as the JP Morgan trading debacle has demonstrated. Reports last week in the New York Times cited organizational discord at JPMorgan Chase as contributors to the company’s multi-billion dollar trading loss.

Risk management isn’t exclusive to banking. Pharmaceutical manufacturers, insurers, and retailers invest millions in risk avoidance. But in the era of Big Data and decision-making at the desktop, it’s vexing to think that any company with significant technology investments would allow the all-too-human traits of ownership debates and subjective opinion to override robust data analysis.

Put simply, risk management is the combination of business processes, technologies, and skills that allow companies to balance short-term and long-term risk exposure, understand the organizational tolerance for failure, pinpoint areas of vulnerability, gauge the costs of precarious business decisions, and forecast the outcomes of these decisions in advance of making them. But all too often, those safeguards are ignored.

Every business day, companies cede hard evidence to the political agendas of a willful manager or department, and these companies span geographies, industries, revenues, and market segments. As people from different departments with different ideas and goals work together, philosophical differences are almost guaranteed. You can imagine the closed-door debates between a cavalier merchandiser and a wary data analyst played out at a retailer:

“I know it’s a big inventory order, Charlie, but don’t worry. We can cover it.”

“But Dan, every supply chain model we’ve run shows that the quantities are way too large. And the pricing models….”

“Oh, your models, your models. Where do you guys get that data, anyway? By the time we build all the models our shelves will be empty!”

“It’s our own sales data, Dan. You know that. And we compare it with research, and third-party sources, climate data, and historical purchase trends….”

“Charlie, don’t talk to me about historical purchase trends, I’ve been doing this for thirty years. Go ahead and do your analysis. But we’re gonna pull the trigger.”

Substitute merchandiser in retail for the product manager at a pharma company or the brand manager at a packaged goods firm, and it’s tempting to dredge up the “man-versus-machine” metaphor. But it’s really the clash of two fundamental preferences: fact-based decision making versus gut-feel. And humans’ tendency to “trust their gut” is as old as the caveman and the spear.

The amount of data that companies collect every two years — 5 exabytes — now surpasses the digitization of anything anyone’s ever said. Investments in database software, servers, storage, and business intelligence tools continue apace. But optimizing the use of these and other technologies remains a challenge. It takes leadership to establish decision management and closed-loop measurement practices. But through decisive leadership and consistent measurement, plenty of companies have shown it can be done.

Savvy managers understand that weaving data-driven decisions into the fabric of corporate governance can obviate organizational infighting and drive progress. By establishing clear accountability measures, managers can determine whether and how corporate goals are being achieved, and hold people accountable for how they are achieving those goals. This motivates business people to rely less on hunches and more on hard data.

Of course, establishing strategic objectives and measuring their execution requires leaders to adopt deliberate planning and a clear strategic view. And linking employee performance to business results calls for transparency and expectations-setting. But faced with a rigorous reward system, business people will choose hard facts and numbers over anecdotes and guesswork, eliminating finger-in-the-wind guessing-games in favor of a data-to-decision cycle. This cycle, over time, becomes part of the company’s DNA. Until this happens, many companies continue to place their biggest bets on the fallout from turf wars and power plays. And the hard reality is that no one is too big to fail.

This article was originally posted on the Harvard Business Review site. Check out the original post, see what others are saying, and add your own comment there. This will honor the editorial work of HBR.

Early in my career I traveled to England to help Teradata land some early deals. The customers included Reuters, Royal Assurance, and Littlewoods, the latter two with their headquarters in Liverpool. Still a wide-eyed ingénue—yes, times have definitely changed—I adored the carbonated accents and fierce wit of the northerners. On Fridays British Rail would carry me back down to London, depositing me at Euston station wearing Liverpool red and sounding ridiculous in my attempts at Cockney rhyming slang.

A decade later I returned to London as my base, renting a flat in Surrey and traveling to far-flung places like Morocco and Dubai. I selected a neighborhood pub in Richmond, The Victoria, as my “local.” Americans compare British pubs to Starbucks, consistent hangouts where over time boundaries recede and friendships flourish. The Victoria was more like a communal living room where friends and neighbors—and even their children—could enjoy a refreshment, relax and discuss the events of the day. In turn, The Victoria selected me. When I’d leave for an extended business trip my pub friends would collect my mail and call to see when I’d be back. They were like my international family.

I’ve always believed that the beauty of global travel, and maybe of life itself, is just as much in the small slices of human color we experience as it is with our intimate relationships. We’re immersed in a lifestyle that’s not our own for a finite time, and we can either observe or participate.

One of the great successes of my life is that I’ve chosen to participate. Living in Paris in the early 1990s, I worked with a team of Americans who insisted on staying at a Marriott (yes, for the points) and eating out at The Hard Rock Café most nights. “You really can’t find a decent burger otherwise,” one of them explained, thus rebuffing the entire history of French cuisine. When my contract was extended I rented a flat on the river and bought some cute walking shoes. I took a subscription to Paris Match, and I took my French from adequate to excellent.

On my recent trip to the UK to keynote IRMUK, I took a little detour back to Richmond. The high street still overflowed with commuters rushing to and from the tube station. Sure, some of the local shops had been replaced by chain stores. But a soft rain fell as I scaled Richmond Hill and looked down on the Thames as it curled up toward London. At the steps of The Victoria I peered through the lead-glass windows. There were the mismatched wooden chairs and the banquettes upholstered in heavy brocade, the ancient wooden bar rising up like a pulpit with the barman presiding. Just like always.

I swear that gin and tonics taste better in England. Maybe it’s slice of lemon, or the tonic itself. But that day I think there was an extra shot of goodwill in my drink, mixed with the gratitude that comes from making good choices when those choices came along. It beats a burger any day.