Mobilizing the PCI Resistance, Part II: First Let's Re-Examine The SOX-404 Problem...

(Reprinted from personal blog entry.)

Previously, I wrote about my blog post "Upset about the subjectivity and ambiguity in the PCI DSS compliance standards? My #BSides submission on the answer..."  In that article, I suggested that in order to improve the state of the practice for PCI, we should look at the similar symptomology that happened in Year 1 and Year 2 for the IT portions of SOX-404.

Last week, during one of the working calls I had with my PCI Scoping SIG team, I dug out some of the early presentations I did as we were launching the GAIT project at the Institute of Internal Auditors.

The approximate timeline of the project began in July 2005 when we held our first summit, to February 2007 when the GAIT guidance was officially announced.

We held the first summit in July 2005, where we assembled internal auditors and security executives from publicly held companies (i.e., SEC registrants), external auditors from the Big Four and probably most importantly, Bill Powers from the Public Company Accounting Oversight Board (PCAOB, created by SOX-404 to audit the auditors).

Instead of showing you slides from that summit, I'm going to show you slides from the January 2006 summit, because we were much better talking about the problem by then.

The Problem

When I showed the slide above to the PCI Scoping SIG team, most everyone seemed startled at how similar the SOX-404 IT problem statements are to the current PCI problem statements.

Let's talk about each one of these in turn in the SOX-404 context:

• No well-established guidance for scoping IT work results in inconsistency and the process being overly subjective
  
  One of the problems was that auditors were often guilty of auditing anything, just because it looked important.  Sure, all the applications that talk to SAP may look important, but is there anything in those apps that could result an undetected material error?
  
  The problem is that in a risk-averse environment, management and internal auditors could rarely effectively challenge the external auditor.  In other words, the external auditor could say, "Well, I can't give a clean, unqualified opinion on your 10-K unless we audit everything."
  
  So, these were always lonely battles against the external auditor.
  
  (It sort of feels like a Mafia protection racket.  "Sure is a nice, clean 10-K financial statement you got there. Shame if something bad were to happen to it. (wink)")
  
  
• Significant key controls reside inside IT and IT processes as well as in the business processes
  
  Incidentally, the real control where reliance is placed may not even be an IT control, instead it's some manual reconciliation process!  So in that case, auditing that IT systems would be totally inappropriate.
  
  
• Sometimes result in overly broad scope and excessive testing costs
  
  If you're auditing areas where there is little risk, you're wasting resources, time and money.
  
  I remember vividly talking to a controller at a Fortune 20 company who was furious that his compliance costs for the IT and financial portions was the same. I think he said something like, "The Enron failure was not caused by a DBA. So, why am I paying so much for IT control compliance, when that's not where the risk is?"
  
  So this is when scope is overly broad: we're testing stuff that doesn't matter.
  
• Significant risks to financial assertions may be left unaddressed
  
  This is the case when scope is too small: we never test something that's actually really important, that could cause an undetected material error in the financial statement!
  
• Suboptimal use of scarce resources
  
  This doesn't require any explanation.

I will describe in my next post why there was a problem.  This is one of my favorites, because it shows the real gap that existed between COSO and COBIT.  I'm hoping you'll find that as interesting as I do!

In the meantime, do you see the similarities between the problem statements of SOX-404 and PCI?   Please comment.

(Also, if anyone interested in the slides, let me know -- I can post them on Slideshare or something...)