About Gene Kim

I've been researching high-performing technology organizations since 1999. I'm the multiple award-winning CTO, Tripwire founder, co-author of The DevOps Handbook, The Phoenix Project, and Visible Ops. I'm an DevOps Researcher, Theory of Constraints Jonah, a certified IS auditor and a rabid UX fan.

I am passionate about IT operations, security and compliance, and how IT organizations successfully transform from "good to great."

SEARCH BLOG
« Mobilizing The PCI Resistance, Part III: Quantifying The Huge SOX-404 Problem... | Main | Talk notes: Effective Compensation Committees (NACD Director Professionalism) »
Friday
Jun112010

Mobilizing the PCI Resistance, Part II: First Let's Re-Examine The SOX-404 Problem...

(Reprinted from personal blog entry.)

Previously, I wrote about my blog post "Upset about the subjectivity and ambiguity in the PCI DSS compliance standards? My #BSides submission on the answer..."  In that article, I suggested that in order to improve the state of the practice for PCI, we should look at the similar symptomology that happened in Year 1 and Year 2 for the IT portions of SOX-404.

Last week, during one of the working calls I had with my PCI Scoping SIG team, I dug out some of the early presentations I did as we were launching the GAIT project at the Institute of Internal Auditors.

The approximate timeline of the project began in July 2005 when we held our first summit, to February 2007 when the GAIT guidance was officially announced.

We held the first summit in July 2005, where we assembled internal auditors and security executives from publicly held companies (i.e., SEC registrants), external auditors from the Big Four and probably most importantly, Bill Powers from the Public Company Accounting Oversight Board (PCAOB, created by SOX-404 to audit the auditors).

Instead of showing you slides from that summit, I'm going to show you slides from the January 2006 summit, because we were much better talking about the problem by then.

GAIT2006-slide1.jpg

The Problem

NewImage.jpg

When I showed the slide above to the PCI Scoping SIG team, most everyone seemed startled at how similar the SOX-404 IT problem statements are to the current PCI problem statements.

Let's talk about each one of these in turn in the SOX-404 context:

  • No well-established guidance for scoping IT work results in inconsistency and the process being overly subjective

    One of the problems was that auditors were often guilty of auditing anything, just because it looked important.  Sure, all the applications that talk to SAP may look important, but is there anything in those apps that could result an undetected material error?

    The problem is that in a risk-averse environment, management and internal auditors could rarely effectively challenge the external auditor.  In other words, the external auditor could say, "Well, I can't give a clean, unqualified opinion on your 10-K unless we audit everything."

    So, these were always lonely battles against the external auditor.

    (It sort of feels like a Mafia protection racket.  "Sure is a nice, clean 10-K financial statement you got there. Shame if something bad were to happen to it. (wink)")

  • Significant key controls reside inside IT and IT processes as well as in the business processes

    Incidentally, the real control where reliance is placed may not even be an IT control, instead it's some manual reconciliation process!  So in that case, auditing that IT systems would be totally inappropriate.

  • Sometimes result in overly broad scope and excessive testing costs

    If you're auditing areas where there is little risk, you're wasting resources, time and money.

    I remember vividly talking to a controller at a Fortune 20 company who was furious that his compliance costs for the IT and financial portions was the same. I think he said something like, "The Enron failure was not caused by a DBA. So, why am I paying so much for IT control compliance, when that's not where the risk is?"

    So this is when scope is overly broad: we're testing stuff that doesn't matter.
  • Significant risks to financial assertions may be left unaddressed

    This is the case when scope is too small: we never test something that's actually really important, that could cause an undetected material error in the financial statement!
  • Suboptimal use of scarce resources

    This doesn't require any explanation.

 

I will describe in my next post why there was a problem.  This is one of my favorites, because it shows the real gap that existed between COSO and COBIT.  I'm hoping you'll find that as interesting as I do!

In the meantime, do you see the similarities between the problem statements of SOX-404 and PCI?   Please comment.

(Also, if anyone interested in the slides, let me know -- I can post them on Slideshare or something...)

 

 

References (55)

References allow you to track sources for this article, as well as articles that were written in response to this article.

Reader Comments (1)

thanks your posting very interesting for me please visit Berita Android | Informasi

Seputar Handphone Android

December 30, 2014 | Unregistered Commenterfajarimamm

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Post:
 
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>