Is peer review all it's cracked up to be?


Some call it “auditing the auditors.” On a regular basis, professionals cross check each other. That sounds like a good thing, and it can be. But like most things, the process is both fallible and corruptible if the wrong motives are in play or the bar is set too low. That’s why some participants and observers have grown skeptical about the merits of peer review, not just in accounting, but in most fields.  Is a report that earns the label “peer-reviewed” really worthy of a Good Housekeeping Seal of Approval?
Those skeptics raise some good questions:  Are there partisan special interest groups involved? Is there a political motivation pushing the outcome of the peer review? Is the depth and breadth of the peer review adequate to render a good opinion?  Are enough reviewers utilized to do a thorough examination? Can the results of the peer review be duplicated by another reviewer?
Here’s what author John Moore wrote in Nature about the corruptible process of peer review:
"It's been peer-reviewed so it must be right, right? Wrong! Not everything in the peer-reviewed literature is correct. Indeed, some of it is downright bad science. Professional scientists usually know how to rate papers within their own fields of expertise (all too often very narrow ones nowadays). We realize that some journals are more stringent than others in what they will accept, and that peer review standards can unfortunately be too flexible. A lust for profits has arguably led to the appearance of too many journals, and so it can be all too easy to find somewhere that will publish poor-quality work."
It’s true that, in any field, peer-reviewed work is more likely to have fewer errors than work that hasn’t had other eyes pore over it. Still, just the label “peer-reviewed” is no guarantee of quality.   A report from the Journal of the American Medical Association estimated that one third of the studies published in peer-reviewed journals cannot hold up under closer scrutiny.
Jack Dini is a Texas accountant who writes a blog called Skeptical CPA, as well as other materials. He wrote:
“As bad as peer review is in the hard sciences, it's worse in the CPA business. About 12 years ago Florida considered making peer review mandatory for CPA firms. As luck would have it, one member of Florida's House was a CPA. His fellow legislators were horrified when he told them large CPA firms had been peer reviewed since about 1977 and that peer review has and would do nothing to protect Floridians from crooked or incompetent CPAs.”
Dini worries that professionals, and more importantly, the public, tend to equate peer review with a “gold standard.”  Then, says Dini, people who are politically motivated, or just plain lazy may use an inadequate or non-independent peer review process to push a point of view or secure a recommendation that isn’t earned. 
Look at the well-publicized case of Bernard L. Madoff Investment Securities LLC. Many accounting firms inspected the books and did not see the fraud.  Neither did representatives of the feeder funds that channeled client money into Madoff’s company.  Legal and accounting experts say, those firms may now be legally vulnerable.  Former chief accountant for the SEC, Lynn E. Turner seems to doubt that the feeder fund auditors looked at the books at all. “If they didn’t,” said Turner, “then investors will hold the auditor accountable.”  
Four years ago, in 2005, CalCPA (the California state society for CPAs) expressed its own skepticism when it decided not to embrace the AICPA policy of mandatory peer review. The CalCPA board made this decision because they said the process did not provide enough transparency and they had unresolved questions about the scope of the peer reviews.  In addition, CalCPA believed the policy of exempting small firms from mandatory peer review would result in a disservice to the public. The board concluded by saying they would reconsider their decision this year, in 2009.  
Change does seem to be in the air.  In 2008 the AICPA upgraded its peer review requirements to increase transparency, understandability, and clarity. Now firms that are undergoing peer review will receive reports designated “pass,”  “pass with deficiencies” or “fail.”  Peer reviews will not include comment letters, but will include a documentation of findings which will not affect the report but will be retained with it.  
New York recently passed sweeping reforms of the accounting profession in their state, designed to ensure that the public’s trust in accountants is justified.  One of those reforms is mandatory participation in peer review for all but the smallest CPA firms. 
The skepticism about peer review may be well-earned by some bad players. But this doesn’t seem to be a baby that should be thrown out with the bathwater. If used as intended, it can still be a great measure of quality.  By taking another, more intense look at peer review,  groups like the AICPA and the New York State Society of CPAs may be able to put some muscle back into the process so that it can once again be the “gold standard” it was meant to be.

You may like these other stories...

The Auditing Standards Board of the American Institute of CPAs (AICPA) has issued two new audit interpretations intended to help auditors navigate the pension accounting guidelines for state and local governments that were...
The responsibilities of the CFO and other finance professionals have moved way beyond just corporate accounting and financial reporting. Finance teams must now be well-versed in such areas as strategic business planning, IT...
The number of CPAs and financial executives who are optimistic about the US economy has surpassed the 50 percent mark for the first time since the recession, according to results of a new survey released by the American...

Upcoming CPE Webinars

Jul 16
Hand off work to others with finesse and success. Kristen Rampe, CPA will share how to ensure delegated work is properly handled from start to finish in this content-rich one hour webinar.
Jul 17
This webcast will cover the preparation of the statement of cash flows and focus on accounting and disclosure policies for other important issues described below.
Jul 23
We can’t deny a great divide exists between the expectations and workplace needs of Baby Boomers and Millennials. To create thriving organizational performance, we need to shift the way in which we groom future leaders.
Jul 24
In this presentation Excel expert David Ringstrom, CPA revisits the Excel feature you should be using, but probably aren't. The Table feature offers the ability to both boost the integrity of your spreadsheets, but reduce maintenance as well.