jill-dyche

Hi.

Writer, classic rock lover, dog rescuer, company founder, software exec, and now independent management consultant--I speak, blog, and pester my friends about these topics. My current focus is getting IT and business organizations to collaborate more effectively and not kill each other. I also talk and write about big data, why analytics is fundamentally strategic, how to pitch business execs on IT projects, and why not to buy a dog from a pet store.

I’ve lived in London, Paris, and Sydney, but call L.A. home. #weatherwimp. I cultivate an organic vegetable garden and friends with issues. I’ve written three books, co-authored a fourth, and contributed to a bunch more. (I have another one in my head waiting to come out, but it’s crowded in there right now.) I prefer Def Leppard to Bon Jovi, mashed potatoes to brown rice, fly fishing to golf, Pinot Noir to Zinfandel, and nice people to assholes. I have a tattoo. I’m not telling you where. I feel guilty that I go hot and cold on social media, that I don’t spend enough face time with my friends, that my French is rusty, and that I ate that whole bag of Kirkland peanut butter cups in less than a week. I have to live with those things.

Data Ops: What Is It? Do We Need It? (Part 2 of 2)

Data Ops: What Is It? Do We Need It? (Part 2 of 2)

In which Jill shares six common pitfalls for busy data management teams.

In my previous article, reader Clayton wondered whether he and his enterprise data management team should be threatened by the pending launch of an adversary's new data ops team.

My answer was that Clayton had nothing to worry about if he and his team had every phase of the data life cycle covered. How to know for sure? I've identified six common pitfalls for busy data management teams. Use this as an abbreviated health check for your own team to see where you might be falling short.

Pitfall #1: Lack of project breadth

Many data teams in high demand work on only the most visible projects. Be careful not to put all your data skills into a few buckets. Your team should be contributing to a range of business initiatives, from supporting basic revenue reports to preparing and enriching data for machine learning. Focusing too narrowly on high-profile or complex projects can unfairly pigeonhole a data team, making room for interlopers.

Pitfall #2: Need for workflow

Workflow is the unsung hero of successful data management. It doesn't just formalize effective processes; it circumscribes inputs and outputs, clarifying roles and ownership boundaries. Ideally, it's automated via workflow management or data curation software. Absent workflow, data teams can become myopic and insular, ignoring their data's transcendent value.

Pitfall #3: Manual processes

Every time I decide data automation software has finally become a commodity, I meet a business professional working from a spreadsheet, inevitably emailed by a colleague who has tinkered with it before forwarding it. Sometimes these spreadsheets contain sensitive data. Too often they're considered authoritative sources. Manual and paper-based processes deter efficiencies, and they compromise data quality and jeopardize data security.

Pitfall #4: Missing standards

Data stewards, wranglers, and scientists who have been knee-deep in data tend to devise effective and repeatable processes, making it easier to quickly glean value from data. Once proven, these processes should be adopted more broadly, formalized, automated, and annotated, their artifacts organized for easy access. Standardizing data-specific processes is not only a data management best practice, it's a rallying cry of the data ops crusaders.

Pitfall #5: Lack of consistent quality improvement

By combining the collaboration endemic to agile development with deliberate delivery processes, data teams can, in theory, discover and remedy quality issues more quickly. Error detection in data and its associated metadata and business rules is a critical component of data ops (as well as its forebear, DevOps), never mind a data management best practice. Likewise, quality monitoring should be a discrete and repeatable exercise in a data ops environment.

Pitfall #6: Static cycle times and delivery bottlenecks

That time frames should improve with repeatable processes and optimized tools is a fundamental tenet of data ops. I've said it before and I'll say it again: maturity is not found in technology but in delivery. This is as true for data as it is for strategy, team effectiveness, or your blockchain technology du jour. If the lack of continuous -- agile, if you must -- delivery has impeded an important business program, your team is likely in the crosshairs of its rivals or worse, its constituents.

Additional Benefits

Did this list make you nervous? Maybe you have some holes to plug. Maybe, if you're lucky, Ken is still devising his business plan, giving you time to remedy your team's shortcomings and fix what might be broken.

However, let's give Ken the benefit of the doubt and assume his new data ops team is an earnest effort to enhance your company's data-driven culture. Consider that you and your team might actually benefit from Ken's effort.

Make a list of the tasks your team performs that are operationally focused. For instance, maybe data acceptance tests should be more routine. Perhaps contract negotiation with third-party data providers consumes half a headcount, or there's a backlog of data correction requests that never ends. These tasks could well fall under a data ops team, and you might not even miss them. Delegating them to Ken's team would free your team members to engage more with businesspeople who use the data in the context of their day jobs. As many have learned the hard way, the (data) expert most aligned to the (data) consumer always wins.

Aim high, Clayton, and let me know how it goes!

Original article published on TDWI.org.

Does Your AI Plan Account for Safety?

Does Your AI Plan Account for Safety?

Data Ops: What Is It? Do We Need It? (Part 1 of 2)

Data Ops: What Is It? Do We Need It? (Part 1 of 2)