The business intelligence debate: Who has the right skills?
Consultant editor David Carter recently published a tirade about business intelligence (BI) tools that produced summary totals, but wouldn't let him drill down to the underlying transactions that made up the figures.
His argument, triggered by a series of frustrating product demonstrations, touched on technology tools such as online analytical processing (OLAP), but took on a political hue as he called on accountants to rise up and take control of business intelligence systems from technologists who had no appreciation for what the numbers meant.
The article was deliberately provocative and spilled out into the wider blogosphere. The result was a stimulating, high-quality debate about purpose and processes of management reporting, and the respective roles of finance and technology people.
The nuts and bolts of BI
As Carter explained it, business intelligence takes the multitude of transactions generated by an organization, and summarizes them into monthly totals that can be delivered on screen to help managers get an immediate view of business performance.
To illustrate how such information is used, Carter likened typical BI reports to a profit and loss account. But accountants who have any doubts about P&Ls are usually able to go into the trial balance and see the account totals, or run a transaction report on each balance to see the individual general ledger transactions that make up the totals.
"If it’s an on-screen report, often I can drill down on a total to display the transactions," he wrote. "But the worlds of accountancy and BI seem to have different views here. In BI they don’t seem to feel that drill down on totals is important."
Management accountant Garry Twiss voiced a related complaint that many "designed IT systems" lacked flexibility. "These systems tend to dictate what you can and can't do, and what you can't do is always in the next version. My advice is go right to the heart of the information - in the database tables themselves.
"Use the IT department with what they excel at, which is systems structures, and understand how the tables are populated. Then use tools such as Microsoft Access and SQL to extract, analyze and summarize the data, present it in say Microsoft Excel, and if you use a pivot table you have a built-in drill down facility.
"Equip accountants with strong IT analysis skills, particularly database technologies and languages, and you will have one seriously skilled employee."
But not everyone agreed with the prosecution's case, and the debate moved on to a discussion about different approaches to management reporting systems.
Paul Crinson said BI had more to offer than Carter's piece suggested. "As with all reporting tools you only get out what you put in. All of the data [David Carter] wanted to see is available and is easily calculated if it is not. [Microsoft SQL Server's] Analysis Services and Integration Services combined can be used to filter out specific data and be made to calculate columns. And you can include as much or as little as you want in your data warehouse."
Robert Brook, himself a developer of management reporting tools, acknowledged that drill down to transactions was rare, though "open" OLAP systems did exist which provide access to the underlying database tables, he explained.
For smaller businesses, having a dedicated OLAP database was not essential. You could construct a management dashboard by querying the data contained in the transactional systems using the structured query language, SQL.
"When a figure does not look right, you can build another SQL with a report to give the transaction data, or investigate using the normal reporting tools," Brook suggested. Anyone wishing to take this approach would need to have suitable SQL skills to construct the queries, he added.
Who's going to fill the finance-IT skills gap?
Carter's comments did ring true for several contributors, including Isolist developer Jim Johnson, who noted an important skills gap identified in the article.
"I’ve long recognized this gap, because I invariably took on the role of filling it in all of the companies I worked for," Johnson said in his Accounting Mechanics blog. He then posed the question, who was best placed to fill the skills gap? An IT person willing to learn financial concepts and take on responsibility for the meaning of data, or an accountant not afraid to grapple with SQL, data normalization and storage technologies?
In his experience, "IT departments typically manage the storage and connectivity to large volumes of data without being accountable for the values.
"Conversely, accountants working with the same data are usually less concerned with the database but are accountable for the values and meanings contained. This seemingly small difference in the perspective of two sets of professionals looking at the same set of data makes a world of difference in practice."
Too many accountants shy away from technology and fail to realize that only they can provide the meaning," Johnson wrote. "The result is IT systems that fail to deliver what is truly required."
Ultimately, he endorsed the Carter BI manifesto: "At the end of the day, the accountant is accountable for the numbers, and if that means straddling the gap into IT skills to ensure they can be delivered and explained, then that needs to be done. No one else is going to do it."
Stephane-Robert Langer, a techie-turned management accountant, joined in on his Just another Knowledge Worker blog. While agreeing with the Carter-Johnson thesis, he warned accountants seeking to fill the BI skills gap to prepare themselves for several "positioning" issues:
- Old-school finance people might see you too much like an IT resource
- IT will most definitely see you as competition
- The added value is obvious once you’re part of an organization and start to deliver results, but it’s difficult to "sell" that particular skill set without sounding too much like “an IT guy”
- You run the risk of ending up maintaining systems instead of using them to answer the very questions you built them to find answers to in the first place.
Having dispensed such useful advice, Langer had to return to his day-to-day workload: "Back to my EOM ETL jobs, SQL-based cost allocation processes and OLAP cubes…"
Microsoft answers back - define the user's needs
Having discussed the potential for open OLAP systems, Robert Brook suggested that David Carter had encountered a "closed" OLAP system, or that the salespeople didn’t understand the full scope of their product. "That’s not unusual in my experience," he wrote.
The demonstration that set Carter off was presented by Microsoft, which he credits with having created a marvelous environment for creating and distributing management information. But the company's technical team gave him figures he couldn't check. There were further problems as the sales reports included additional costs such as delivery charges which undermined their integrity and threw doubt on any margin calculations.
Paul White, head of Microsoft Dynamics in the UK, tackled the issues head on. While accepting in principle the argument that accountants should own the data, White warned against oversimplifying the BI challenge.
"I don't think it's responsible to make such general assertions without defining the market you're talking about. Reporting needs and sales margins are unique to individual organizations," White said.
"While the principles are consistent, the needs of enterprise, mid-size and small businesses are radically different."
For a smaller organization, it would be feasible to design the margin calculation into the application, so that it was available as a standard sales report. But technical and commercial considerations made it difficult to support drill-down in bigger installations, White explained.
"Some Dynamics customers hold terabytes of data. If you attempted to bolt real-time drill-down capabilities and tools onto an installation of that size, it would be blown away." A 1.9 terabyte transactional database would nearly triple to 5.9 terabytes if it became a drill-down OLAP system, he added.
Ultimate responsibility for business intelligence lay not with the software vendors, he argued, but with the managers who commissioned and paid for the systems. "Technically, it might be possible to create the kind of corporate BI system you're looking for, but it's not commercially feasible in terms of the return on investment," he said.
Don't play politics with BI
Neil Conacher, an accountant whose firm successfully uses the BI tools within the APS practice management suite, urged participants to stop playing party politics with BI.
"OLAP is only one element of a reporting strategy, but a very powerful one. [It] should not be criticized for what it doesn’t do, but admired for what it can," he said.
"Microsoft should be congratulated for making this technology available to medium to small organizations at a price which represents extremely good value for money. However, it should be pointed out that the skill set required to deliver an effective OLAP solution is very different from that associated with a traditional transaction reporting system."
These skills needed to be identified along with the software product to ensure an effective solution - much as you would look for the correct skills needed to deliver effective tax or accounting advice, he added.
For any successful reporting solution to be delivered, whether it be OLAP or transactional, teamwork was essential between the users and developers. "What is vital is that both disciplines acknowledge each other’s potential contribution. IT developers are clearly not accountants and vice-versa. The best solution will come from the combination of the two skill sets," Conacher wrote
"It’s not about accountants knowing more than IT developers or the other way round, teamwork brings the best results in today’s business world."
He warned that the original article ran the risk of influencing views in a way that could discourage firms from enjoying the benefits of current technology. "BI is about using a set of tools and solutions to satisfy ongoing reporting requirements which will enable a firm to run efficiently and make not only the IT developers and accountants happy, but ultimately better serve the customers. Surely that is in everyone's interest."
By John Stokdyk
This article first appeared on our sister site, AccountingWEB.co.uk