In which Jill reconsiders her take on agile BI—for a second, anyway.
The theme of last week’s TDWI conference in San Diego was “Creating an Agile BI Environment—Delivering Data at the Speed of Thought.” I sat in on some workshops expecting to learn something new about agile BI, if not eat a little crow. You see, I’d received mixed responses on my assertion last week that agile BI attracted companies “that don’t have the discipline to enforce BI development in the first place…” Turns out I did a lot more of the former than I did of the latter.
Without fail each TDWI instructor acknowledged that agile was borne out of operational systems development and needed to be revised to support BI. Each emphasized time-to-delivery. But the messages diverged when it came to adapting agile development for BI.
In my cynical moments I’m convinced that agile is for extroverts who’d rather spend time talking than developing or, God forbid, documenting. They like the stand-up approach to requirements gathering. The efficacy of agile BI in practice is very much still up for debate. TDWI’s instructors focused mostly on how to wring as much productivity as possible out of BI development tasks, rightly citing data modeling and ETL as the main time-zapping culprits. But I was struck by how little content focused on sustained business engagement.
I didn’t get to stay in Larissa Moss’ workshop long enough to hear her formal take on agile BI but I suspect she’s got the right idea, which is (my words here): Think big, scope small. For years we’ve admonished clients that the success of BI programs is all about how the projects that comprise them are sliced. We tell them to embrace the reality of multiple releases of their BI applications. This keeps it warm with the business. It ensures regular delivery. It lets BI teams portion out critical data an app at a time. Users grow accustomed to seeing new data and using new reports. You get the idea.
So no new surprises here. Except for the sophistication of people who’d tried agile BI. These people know their sprints from their sandboxes. I was especially gratified by the smart comments coming from my trusty blog readers. Here’s a sampling of some of the more trenchant comments on last week’s blog post:
- One of the biggest challenges I have faced is getting people to actually think in 60-90 day chunks of work. Whilst it is easy for people to envisage their ultimate goal, defining smaller steps to get there seems to be a skill which is lacking, especially in those tasked with defining the scope and tasks for each “chunk”.—Rhonda B.
- Agile should not be used as an excuse to work without discipline. It is a different kind of discipline and it requires planning and thought like any other exercise but it should not be an excuse for bad behavior or short-sightedness.—Michael S.
- …the degree of unpredictability of the [BI] effort is in inverse proportion to the extent, degree, and clarity of the documentation…–David S.
- Please don’t bring usual operational system techniques and methodology into the BI world. …BI is all together a different ball game, and let it remain that way. –Rahul H.
- There is no EASY button for DW/BI. Effective data management requires hard work, rigorous engineering standards, and organizational discipline.—Bob Conway
- It will only be through adding more data, adding more functionality and even retiring outdated elements that ongoing value optimization can occur.—Kelly Lautt
- If the result of any BI development activity does not directly contribute to improving profitability, growing revenue, increasing market share and retaining customers, then it surely matters not a whit if the processes are “agile” or otherwise. —@PeterBeddows
Agile BI works when it’s sober, with managed expectations, realistic end-users, and experienced developers. But it breaks down when the goal of accelerated development time usurps delivering business value. From the messages last week it seemed that increased speed was the main motivator behind agile BI.
I haven’t changed my mind about agile. It seems to be embraced most ardently by IT people who have been unable to engage the right level of business executive in the strategic conversation. No one admitted it outright but I suspect some BI teams have spent more time honing their agile development approaches and researching BI “accelerator” tools than they’ve spent educating their business users about why BI is different in the first place.
But these users need to know that BI can be a strategic game-changer. That it can foster an early-mover advantage into new markets and increase competitive nimbleness. That it can streamline business processes, drive economies of scale, increase revenues, and delight customers. Sometimes these conversations require formal education, missionary work, debate, and even rigorous long-term planning. And that might just mean sitting down.
photo by http2007 via Flickr (Creative Commons license)
Jill:
Both of your blogs on this topic have provided a great forum and service. While my follow up here is way behind the 8 ball in timing, since you were kind enough to quote from my comment in your previous blog, I felt disposed to add some emphasis now to one of your comments here.
“No one admitted it outright but I suspect some BI teams have spent more time honing their agile development approaches and researching BI “accelerator” tools than they’ve spent educating their business users about why BI is different in the first place.”
It is my belief that you have hit a very thorny issue right on target in that statement and, in my opinion, it goes beyond just impugning the actual focus of just the BI teams, it includes impugning the focus of many IT teams in general.
Far too often, in my view, IT/IS is seen by CFO’s and CEO’s, and CIO’s and IT staff accordingly respond in acquiescence of their designated fate, as a “business overhead” when in fact, beyond the perhaps seemingly mundane activities of handling the mechanics of AP/AR/GL/PR etc, the products/output of IT should actually be oriented towards being another potential profit-center resource helping the business not only to be profitable but also to grow relative to the value that it can/could deliver through its efforts, knowledge and skills.
Reality, or at least perception (recent polls by CIO web), shows that the functioning and focus of most IT/IS groups appears to invariably be towards becoming even more expert in the latest .NET, SCRUM, Agile or other latest wiz-bang data manipulating, application developing and presentation technical technique rather than upon “how can we improve upon the relevance and value of information content and delivery that directly relates to what business needs to grow as distinct from what ensures self-preservation, awe and mystery of IT operations”.
I am fully aware that saying this is like waving a red flag before a bull and that not EVERY IT organization operates in this way but enough of them do that it is worth bringing this into the open as you have intimated in your statement about BI teams.
My conclusion is based upon many years of observation of having worked in a very broad range of businesses with a very broad range of clients: Finance, Health-Care, Manufacturing for example.
From before I even had a career I’ve always had a passion for building business and thus always been interested in exploring what helps to build business.
Perhaps ironic that I switched from immersion in a p/l responsible line management oriented career to an IT oriented career when I founded missi.com (Management Information Systems Solutions & Initiatives) as a consulting business in 1991 after handling all of the IT/IS needs of a Payroll Service Bureau startup in which, for the first time in my career, I not only had complete control of everything IT/IS from scratch, I had not even had any conventional IT career experience before that.
The motivation for starting the MISSI consulting business arose BECAUSE, in the preceding career, I had been frequently frustrated by the fact that, so often – albeit IT as we understand it today is much different than it was back then – most of the people then in IT understood everything there was to know about IT and absolutely nothing about what it took to provide the information that really could contribute to building a business. My feeling today is that not a whole lot has changed in that respect.
Hence I had the conviction that I could help translate the needs of business clients and produce intelligent, meaningful Information delivery in forms that actually served the business users needs as distinct from proving how clever IT was in handling data.
In the time this MISSI consulting business has been active, there has been an improvement in how IT relates to business and how business relates to IT but we are still a far cry from having IT act as, or be seen as, a direct potential contributor to business growth as distinct from being some geeky group that “gets our pc’s working again after they have crashed because we could not resist trolling the web on work time and they also fix network problems”
To quote, again for emphasis, from Matthew G’s comment “There is nothing more dangerous in the Business World than a hastily delivered ‘report’ containing ‘facts’ that are then used to make ‘informed decisions’. In the BI world, may I remind people of their granny’s favorite saying – More Haste, less Speed?”
I submit that Matthew’s comment, apart from being so true, perfectly relates to your last statement:
“But these users need to know that BI can be a strategic game-changer. That it can foster an early-mover advantage into new markets and increase competitive nimbleness. That it can streamline business processes, drive economies of scale, increase revenues, and delight customers.”
Every sentence in that last statement is so true in my view that it ought to be repeated in every meeting between IT and the business management team until the purpose of IT becomes fully aligned with the business purpose so that the result id accelerated growth of profitability and stakeholder equity.
“Sometimes these conversations require formal education, missionary work, debate, and even rigorous long-term planning. And that might just mean sitting down.” True that!
Great post. Not the only reason
Agile may fail, I think, but certainly one of the key challenges for the
Agile community to fess up to and face up to.
I wrote about some other challenges in my recent “State of Agile” post here http://www.agileexams.com/