White Paper: IT Compliance Benchmarks

Executive Summary

Key Findings
Frequency of audits, time allocated to compliance by IT and IT spending distinguish leaders from laggards in achieving compliance. The three major drivers of performance results in achieving IT Compliance are:

1. Frequency of internal audit and IT security monitoring. Leaders audit for compliance on a continuous basis, at least once a month.

2. Time allocated by IT to compliance: Leaders are spending 50 percent more time on compliance than laggards.

3. Spending on IT security. Leaders spend ten percent of the IT budget on IT security, while laggards spend less than seven percent on IT security.

While leadership in compliance costs more in time and money, laggards pay the price in decreased public trust in the company and brand, worry about the accuracy of quarterly financial statements, and concerns that regulatory issues may affect revenue and profit.

The major priorities for improving performance in achieving regulatory compliance include:
1. Identifying repeatable and more efficient methods to demonstrate compliance
2. Using technology to automate IT security, audit, and compliance procedures
3. Improving data and risk management practices for

Analysis Summary
The pressure to demonstrate compliance with regulatory mandates continues to increase, with some organizations now subject to five or more regulatory mandates. Most firms, however, are currently subject to three "most pressing" regulatory compliance mandates requiring that they demonstrate IT security through internal or external audits.

This report, covering the most recent research conducted by SecurityCompliance.com, indicates that the four prioritized pressures driving regulatory compliance among organizations are:
1. An increase in the scope of regulatory audit
2. An increase in the number of mandated audit reports
3. Public trust in the organization and its brands
4. Pressure exerted by boards of directors and senior management

Leaders Versus Laggards
Measuring performance in achieving compliance can initially appear counterintuitive since performance is measured by a declining number of significant and material deficiencies identified through the audit process. Thus, the fewer deficiencies an organization has, the better its performance in achieving IT compliance.

Although the pressures to demonstrate compliance are shared by all organizations, performance results vary significantly. Only 11 percent of organizations are achieving superior performance results and could be considered "leaders" when measured by audits that tally their overall "significant and material" deficiencies. In contrast, about twice as many survey respondents, 20 percent, can be considered "laggards," while more than two-thirds or 69 percent, performed at industry norms.

In this report, industry norms are based on regulatory audit results that show:

  • One in 10 firms with two significant and material deficiencies (leaders)

  • Seven out of 10 firms with six significant and material deficiencies (norm)

  • Two out of 10 firms with 35 significant and material deficiencies (laggards)

Performing as a leader obviously costs time and money, as well as the potential of lost opportunity when leaders focus IT resources on demonstrating compliance rather than on activities that would impact revenue and growth.

Performing as a laggard, however, also has its costs. According to this benchmark, these costs manifest themselves in terms of decreased public trust in the organization and its brands, concern about the accuracy of quarterly financial statements, and the potential impact of non-compliance sanctions or penalties on the organization and its executive management.

Actions that Make a Difference
Beyond simply quantifying regulatory pressures organizations are responding to, this benchmark quantifies the relationship between performance results and the actions that organizations are taking to achieve their results. This benchmark clearly demonstrates a link between performance results and a few critical success factors that illustrate what is working across many organizations.

As such, the report offers several suggestions for actions that have been identified as improving results for IT security and regulatory compliance.

The recommendations in this report are intended to assist organizations with making improvements for IT compliance.

Recommendations
Based on the findings of this benchmark, there are five key actions that can result in improved results for IT compliance of most organizations. The key actions include:

Conduct internal regulatory and IT security audits at least monthly
Industry leaders are conducting internal audit and IT security monitoring at least once a month. That's eight times more frequently than the industry laggards and five times more frequently than firms operating at the industry norm. As a result, leaders are experiencing significant and material deficiency levels that are 18 times better (lower) than the laggards. Frequency of audit by itself, however, is not the only factor responsible for achieving improved performance in regulatory compliance.

Spend 30 percent of IT staff time focused on regulatory compliance
Industry leaders are allocating 30 percent of IT staff time to regulatory compliance. In comparison, companies performing at industry norm are allocating 26 percent of staff time to regulatory compliance, while industry laggards are allocating only 20 percent to regulatory compliance. Based on 250 workdays a year, this translates to a little over six days per month among the leaders, five days per month among the norm, and about four days per month among the laggards.

Allocate 10 percent of the IT budget to IT security
Industry leaders are spending almost 50 percent more on IT security than are the laggards. For this increase in spending on IT security, the compliance leaders are experiencing 1750 percent fewer "significant and material" deficiencies than the industry laggards. Despite the link between spending and IT staff time devoted to compliance, how the money is spent on IT security and where the time is focused differentiate performance results among firms operating as leaders, at the industry norm, and as laggards.

Establish clear objectives and measure results at regular intervals
Leading organizations are consistently measuring results monthly and managing compliance deficiencies to achieve and sustain compliance. Roles and responsibilities for compliance are clearly defined, and objectives are measurable. In contrast, procedures for demonstrating regulatory compliance among laggards and normative firms are rarely defined and are mostly manual. Among the laggards, data and knowledge about compliance is rarely managed.

Automate compliance and IT security controls and procedures
with IT technology tools

Nearly all IT security technology controls and procedures are now automated among the organizations performing as leaders in compliance. Although most firms are improving IT security policies, standards, and documentation, the leaders are singularly focused on documenting procedures, making changes to both business and technical procedures, and automating these processes as much as possible.

Where to Concentrate Now
For industry leaders
The opportunity for improvement among the leaders includes rationalizing multiple compliance mandates (performance leaders are subject to four or more pressing regulatory mandates) to reduce the time being spent and costs associated with complying with regulatory mandates. This analysis and rationalization may have to be undertaken on an ongoing basis to enable the organization to reallocate resources to other pressing portions of the IT portfolio mix.

For the industry norm
The biggest opportunity to improve compliance for firms performing at the industry norm would be to increase the frequency of internal audit and security monitoring to monthly intervals. Secondarily, these firms should increase the amount spent on IT security. The primary challenges facing firms operating at the industry norm include monitoring and sustaining compliance, making changes to IT security controls and procedures, allocating additional budget and resources for compliance, and resolving interpretations with auditors.

For industry laggards
Firms operating as laggards should increase the frequency of internal audits and IT security monitoring as much as possible, even if this means first increasing the frequency to quarterly before implementing them monthly. Laggards must also increase the amount of money being spent on IT security and focus it on specific areas instead of trying to improve too many areas at one time.

About the Survey
From December 2005 through March 2006, the Security Compliance Council sponsored an online benchmark survey designed to distinguish the performance results of organizations seeking to pass audits and demonstrate regulatory compliance. Research was conducted for this report through SecurityCompliance.com, featuring a closed-ended survey in which qualified candidates participated. Several key findings from the survey identify best practices among compliance leaders and suggest benchmarks that others may use to measure their own performance results.

Key Findings
Management commitment, sometimes referred to as "tone at the top," is often cited as one of the major reasons for differences in results for regulatory compliance among organizations. But, once committed to regulatory compliance, what actions result in better or worse performance results? This benchmark quantifies strategic and operational actions, activities, critical success factors, capabilities, and practices that are needed to improve performance results for regulatory compliance programs by organizations of all sizes.

Keep in mind that measuring performance in achieving compliance is counterintuitive since performance improvements result from a declining number of significant and material deficiencies identified through the audit process. Thus, the fewer deficiencies an organization has, the better its performance in achieving compliance.

Industry leaders
Organizations performing as industry leaders are experiencing median rates of 25 overall compliance deficiencies and 2 significant and material deficiencies. These firms are experiencing 30 percent fewer overall deficiencies and 66 percent fewer significant and material deficiencies than organizations operating at the industry norm. Compared with firms operating as laggards, industry leaders are experiencing 66 percent fewer overall deficiencies and 94 percent fewer significant and material deficiencies.

Industry norm
Firms performing at the industry norm are experiencing a median level of 36 overall compliance deficiencies, with 6 of these being considered significant and material. These organizations are experiencing 52 percent fewer deficiencies and 76 percent fewer significant and material deficiencies than are organizations performing as industry laggards.

Industry laggards
Companies operating as industry laggards are experiencing a median level of 75 overall deficiencies, with 35 of these being identified as significant and material. These organizations are experiencing 300 percent more deficiencies than industry leaders. Furthermore, the industry laggards are experiencing significant and material deficiency rates that are three times higher than the industry norm and 17.5 times higher than industry leaders.

Procedures and Controls Contributing to Compliance Deficiencies
With the exception of business procedures, the top 10 deficiencies are either related to or are exclusively the responsibility of IT functions in organizations. Following closely behind the top 10 are business procedures, training and education, and the separation of duties. However, these deficient controls and procedures are not equally the same for companies of different sizes and in different industries. Nor are deficient controls and procedures identical, based on differences in performance results.

Most deficiencies in procedure and controls are lower-across the board-among firms that are performing as industry leaders. Specific areas where industry leaders are experiencing higher levels of deficiencies include personnel security; application, systems, and server access controls; application development and maintenance; virtual private networking; remote access controls; and data archive procedures and controls.

In contrast, organizations operating at the industry norm are experiencing higher levels of deficiencies in most procedures and controls, while also experiencing deficiency levels that are lower than those of the industry laggards.

Lastly, industry laggards are leading-in the least desired measure-with the most deficiencies across most controls and procedures.

Compliance deficiencies among the industry leaders
Procedures and controls that are contributing the least number of deficiencies among industry performance leaders include asset classifications, network access controls, and cryptographic controls. At the opposite end of the spectrum, performance leaders are suffering the largest number of deficiencies in several areas, including user and application access controls; applications, systems, and server access controls; application development and maintenance; database access controls; and data archive procedures and controls.

Compliance deficiencies among the industry norm
The least deficient area for firms operating at the industry norm is cryptographic controls. The most deficient include configuration and change management; user and application access controls; information access controls; and audit, logging, and security monitoring.

Compliance deficiencies among the industry laggards
Firms performing as laggards are having the least trouble with VPN and remote access controls. However, these firms are experiencing the most problems across 16 other procedural and technical controls.

Critical Success Factors for Improving Regulatory Compliance Results
Size has little or no benefit when it comes to regulatory compliance. The benchmark shows a very small-and insignificant-advantage among small businesses with revenues that are less than $50 million. The results confirm that when it comes to demonstrating compliance with regulatory compliance, large, midsize, and small organizations currently share an equal chance of being industry laggards as well as being industry leaders.

Firms performing as industry leaders are engaged in very specific activities—more frequently—while also allocating more resources to the task of demonstrating security compliance than others. The critical success factors that are correlated with the performance results of the industry leaders include:

  • Conducting internal audit and security monitoring at least monthly

  • Spending 30 percent of the time in IT on regulatory compliance

  • Spending 10 percent of the IT budget on IT security

Frequency of audit and security compliance measurements
The frequency with which internal audit and IT security monitoring are conducted is completely aligned with performance results. Among firms that score as leaders, all are conducting internal audit and monitoring IT security at least monthly. The only variation from the monthly frequency of internal audit among compliance leaders involves 30 percent of these organizations that are conducting IT security monitoring activities more frequently than monthly.

Industry leaders are conducting internal audit and IT security monitoring eight times more frequently than are the industry laggards and five times more frequently than are firms operating at the industry norm. Leaders are experiencing overall deficiency levels that are three times lower and significant and material deficiency levels that are seventeen times lower than the laggards.

In contrast, the leaders are experiencing overall deficiency levels that are one and one-half times lower and significant and material deficiency levels that are three times lower than the industry norm.

Although the frequency of internal audit and IT security monitoring is the dominant critical success factor, the frequency of audit among the industry norm is closer to that of the laggards, yet the industry norm are experiencing results that are closer to that of the industry leaders. As a result, the frequency of audit-by itself-is not the only factor responsible for contributing to lowered deficiency levels for IT compliance.

Time spent by IT to demonstrate compliance
Industry leaders are allocating 30 percent of the time in IT to compliance. In comparison, companies performing at industry norm are allocating 26 percent of the time in IT to regulatory compliance while industry laggards are allocating 20 percent of the time in IT to compliance. Based on 250 workdays in a year, this translates to a little over six days per month among the leaders, a little over five days per month among the norm, and a little over four days per month among the laggards.

Over the course of a year, the industry leaders are allocating 50 percent more time to demonstrating compliance than are the industry laggards. And, the leaders are allocating 15 percent more time than are the firms performing at industry norm. The amount of time spent by the IT function on compliance among the industry norm correlates closer to their performance results. In addition to the frequency of internal audit and IT security monitoring, the time spent by IT to demonstrate compliance is a second critical success factor for improving results for compliance.

Despite this, the value of the time spent on compliance is temporal in nature. It is only valid for the snapshot this benchmark represents. Time spent on compliance is likely to decrease as organizations further automate procedures and controls to demonstrate compliance.

Spending on IT security
The median spending on IT security by the industry leaders is 10 percent of the total IT budget. Firms performing at the industry norm are spending almost 7.5 percent on IT security, while industry laggards are spending 6.8 percent on IT security.

Industry leaders are spending 33 percent more on IT security and are experiencing 144 percent fewer overall deficiencies and 300 percent fewer significant and material deficiencies than are firms performing at the industry norm. In contrast, the leaders are spending 47 percent more on IT security and are experiencing 300 percent fewer overall deficiencies and 1,750 percent fewer significant and material deficiencies than are the firms performing as industry laggards.

Spending on the IT security function correlates well with the number of overall deficiencies, but diverges for significant and material deficiencies because the firms operating at industry norm are able to better align their results closer to the industry leaders. This indicates that firms operating at norm are taking additional actions that differ from those performing as laggards.

The frequency of internal audit and IT security monitoring is the one critical success factor that is primarily responsible for driving performance results the most. Despite the correlations of spending on IT security and time spent by IT on compliance, it is what is done with the money spent on IT security and what is done with the time spent on compliance that differentiates performance results between firms operating at the industry norm and those of the laggards.

Capabilities and best practices for regulatory compliance
What are industry-leading organizations doing with the additional time and money being spent on compliance? For organizations committed to monthly measurements, spending additional time on compliance and additional money (10% of the IT budget) on IT security, what kinds of activities made possible by these additional funds and time contribute most to declining, and consistently low, compliance deficiency levels?

This benchmark quantifies the defining capabilities and best practices being funded and focused on by organizations that are operating at three levels: as leaders, at the industry norm and as laggards. The capabilities quantified by the benchmark include maturity levels and practices from across the entire sample of the 671 organizations that participated in the benchmark.
These capabilities include:
a) procedures
b) management of data and knowledge
c) organizational structure and strategy
d) information and IT security technology
e) training, awareness and employee accountability programs, and f) risk management practices.

The practice levels measure the maturity of each capability and include consistency, frequency, and completeness across the 671 organizations participating in the benchmark. Although the effective practice maturity levels for any one capability differ from laggards to industry leaders, the worst practices of the industry leaders is on par with the best practices of the industry norm and the industry laggards.

Industry leaders are implementing enabling capabilities at best practice levels. By contrast, industry laggards are implementing practices below the norm. For example, performance leaders are consistently delivering employee training and accountability for compliance and ethics to all employees and most business partners and suppliers. In addition, performance-leading organizations are certifying and holding all employees accountable. Leaders are also employing interdisciplinary teams that operate across the organization, while managing the IT specific controls from within the IT organization.

Leaders are consistently classifying and managing risk for all of their information assets, not just the most important IT and information assets. Furthermore, leading organizations are consistently measuring results monthly, and managing compliance deficiencies to achieve and sustain compliance. Lastly, almost all IT security technology controls and procedures are automated among the organizations that are performing as leaders.

In contrast, procedures for demonstrating regulatory compliance among laggards and normative firms are rarely defined and are mostly manual. Among the laggards, data and knowledge about compliance are rarely managed. Among the normative firms, the IT organization is solely responsible for managing all aspects of compliance with little, if any, assistance, oversight, or interdisciplinary teaming among other functions in the organization.

Among the laggards and normative firms, few if any IT security, audit, and compliance procedures are automated. Only a few employees are provided training and certification for ethics and compliance policies, and only these few are held accountable. In addition, risks to the organizations' information and IT assets are rarely if ever classified and managed.

These are very significant differences in enabling capabilities and practice levels, all of which are contributing to higher levels of overall deficiencies and higher levels of significant and material deficiencies among firms that are performing at industry norm and as industry laggards.

Of the six enabling capabilities and practices measured by this benchmark, four stand out as
areas for prioritized improvement among firms that are performing at industry norm and among
the laggards. The four prioritized improvement areas are:

  • Management of data and knowledge

  • Risk management practices

  • Technology maturity

  • Procedural automation

Spending additional time and funds to improve the practice levels of these capabilities will enable laggards and normative performers to approach the performance levels being achieved by the industry leaders.

Additional actions of compliance performance leaders
In addition to increasing the frequency of audit and spending more time and money on regulatory audit and IT security, other actions being taken by the performance leaders distinguish their results. Actions taken by the leaders (that are not taken or emphasized as much by laggards and the norm) include: changing business procedures that are found to be non-compliant; automating change management procedures; documenting procedures, assets and controls; and establishing objectives and measurements.

For leaders, increasing the frequency of audit has resulted in automating the procedures and controls for: change management and remediation of deficiencies; role-based access to IT resources; and IT security controls. The actions undertaken by the leaders should be proof of what is working. Firms operating as laggards or at norm should be undertaking the following actions to improve IT compliance results:

  • Establishing objectives and measuring results

  • Documenting business procedures, assets, and IT controls

  • Automating procedures for change and remediation management

  • Automating role-based access to IT assets

  • Changing business procedures and controls

Firms operating as laggards are more prone to make changes to IT procedures and controls than to business procedures and controls. This may be due to a lack of awareness of the relevant business procedures, because only about half of all laggards are taking the time to document business procedures that can be changed.

The additional actions undertaken by compliance performance leaders are contributing to better performance results for regulatory compliance. However, not all firms have practices that align well with these actions.

Nonetheless, the actions taken by the compliance leaders can be used as a reliable gauge to determine actions that should be taken when prioritizing plans to improve regulatory compliance results for the organization.

Analysis and Recommendations
Despite stellar performance results, there is still room for improvement among the industry leaders. These companies are experiencing median rates of 25 overall deficiencies and 2 significant and material deficiencies.

Challenges and opportunity for the industry leaders
The benchmark results reveal that the primary challenges faced by industry-leading organizations include monitoring and sustaining compliance, analyzing IT security procedures and controls against multiple regulations, and managing delays to more important IT projects. Moreover, the large amount of time being dedicated to compliance by the IT function in these organizations is impacting other IT projects.

The opportunity for improvement among the leaders includes rationalizing multiple compliance mandates (performance leaders are subject to four or more pressing regulatory mandates) to reduce the time being spent and costs associated with complying with regulatory mandates. This analysis and rationalization may have to be undertaken on an ongoing basis to enable the organization to reallocate resources to other pressing portions of the IT portfolio mix.

Lastly, sustaining compliance is all about operational excellence. The capabilities and practices most in need of improvement among the industry leaders include the automation of procedures and the use of technology to achieve this automation. Automating additional procedures with the use of technology and closing the gaps between policies, training, measurements, data, knowledge, procedures, and reporting should enable the performance leaders to approach "zero defects" on a consistent basis.

Challenges and opportunity for the industry norm
Whatever else is done, the first priority of firms performing at the industry norm should be increasing the frequency of internal audit and security monitoring to monthly. Secondarily, firms operating at the industry norm should increase the amount spent on IT security. The primary challenges facing these firms include monitoring and sustaining compliance, making changes to IT security controls and procedures, allocating budget and resources for compliance, and resolving interpretations with auditors.

The opportunity for improvement among the industry norm includes delivering operational excellence around maintaining and sustaining compliance. The capabilities most in need of improvement among the industry norm are the automation of procedures through the use of technology and improvements to the firms' risk management practices. Automating additional procedures with technology and closing the gaps between policies, measurements, change management, and reporting should enable the firms performing at industry norm to approach or exceed the current performance results of the leaders.

Taking these actions will require organizations to spend more money on IT security along with additional cross-functional assistance to resolve outstanding issues with auditors.

Challenges and opportunity for the industry laggards
There is only one direction for industry laggards to go: performance can only improve by lowering the levels of overall and significant and material deficiencies. Whatever else is done, firms operating as laggards should increase the frequency of audit, as much as possible to approach once per month, even if this means first increasing the frequency to quarterly before jumping to monthly internal audits and IT security monitoring. Secondarily, firms operating as industry laggards must increase the amount of money being spent on IT security.

The primary challenges facing the laggards include poor procedures, little insight into and management of data that can be turned into actionable knowledge, weak or nonexistent technology automation, and poor risk management practices.

Industry laggards could try to focus on improving many capabilities simultaneously. However, this is more likely to lead to distraction, worse performance results, and setbacks by trying to accomplish too much, too rapidly. Based on the performance profile of the industry norm, it may be better for firms performing as laggards to change their organizational structure to improve compliance results first. After this, the prioritized improvement capability would be improving the management of data and knowledge.

Improvements in technology automation, procedures, and risk management practices might be best considered, especially as these are connected to specific objectives and measurements and the firm believes that staffing levels and organizational changes are already under way.

You may like these other stories...

Whenever I speak to accountants about creating a cloud practice, the most common question is, “How do I charge my clients?” Ten years ago, maybe even five years ago, if I would’ve posed this question...
While reputational risk is the No. 1 nonfinancial concern among corporate directors, cybersecurity/IT risk is gaining steam. In fact, both private companies and organizations with more than $1 billion in revenue felt they...
Accountants who specialize in forensic and valuation services point to electronic data analysis, or big data, as the most pressing issue they’ll face in the coming months, according to results of a new survey released...

Upcoming CPE Webinars

Jul 31
In this session Excel expert David Ringstrom helps beginners get up to speed in Microsoft Excel. However, even experienced Excel users will learn some new tricks, particularly when David discusses under-utilized aspects of Excel.
Aug 5
This webcast will focus on accounting and disclosure policies for various types of consolidations and business combinations.
Aug 20
In this session we'll review best practices for how to generate interest in your firm’s services.
Aug 21
Meet budgets and client expectations using project management skills geared toward the unique challenges faced by CPAs. Kristen Rampe will share how knowing the keys to structuring and executing a successful project can make the difference between success and repeated failures.