Mobilizing the PCI Resistance, Part II: First Let's Re-Examine The SOX-404 Problem...

(Reprinted from personal blog entry.)

Previously, I wrote about my blog post "Upset about the subjectivity and ambiguity in the PCI DSS compliance standards? My #BSides submission on the answer..."  In that article, I suggested that in order to improve the state of the practice for PCI, we should look at the similar symptomology that happened in Year 1 and Year 2 for the IT portions of SOX-404.

Last week, during one of the working calls I had with my PCI Scoping SIG team, I dug out some of the early presentations I did as we were launching the GAIT project at the Institute of Internal Auditors.

The approximate timeline of the project began in July 2005 when we held our first summit, to February 2007 when the GAIT guidance was officially announced.

We held the first summit in July 2005, where we assembled internal auditors and security executives from publicly held companies (i.e., SEC registrants), external auditors from the Big Four and probably most importantly, Bill Powers from the Public Company Accounting Oversight Board (PCAOB, created by SOX-404 to audit the auditors).

Instead of showing you slides from that summit, I'm going to show you slides from the January 2006 summit, because we were much better talking about the problem by then.

The Problem

When I showed the slide above to the PCI Scoping SIG team, most everyone seemed startled at how similar the SOX-404 IT problem statements are to the current PCI problem statements.

Let's talk about each one of these in turn in the SOX-404 context:

• No well-established guidance for scoping IT work results in inconsistency and the process being overly subjective
  
  One of the problems was that auditors were often guilty of auditing anything, just because it looked important.  Sure, all the applications that talk to SAP may look important, but is there anything in those apps that could result an undetected material error?
  
  The problem is that in a risk-averse environment, management and internal auditors could rarely effectively challenge the external auditor.  In other words, the external auditor could say, "Well, I can't give a clean, unqualified opinion on your 10-K unless we audit everything."
  
  So, these were always lonely battles against the external auditor.
  
  (It sort of feels like a Mafia protection racket.  "Sure is a nice, clean 10-K financial statement you got there. Shame if something bad were to happen to it. (wink)")
  
  
• Significant key controls reside inside IT and IT processes as well as in the business processes
  
  Incidentally, the real control where reliance is placed may not even be an IT control, instead it's some manual reconciliation process!  So in that case, auditing that IT systems would be totally inappropriate.
  
  
• Sometimes result in overly broad scope and excessive testing costs
  
  If you're auditing areas where there is little risk, you're wasting resources, time and money.
  
  I remember vividly talking to a controller at a Fortune 20 company who was furious that his compliance costs for the IT and financial portions was the same. I think he said something like, "The Enron failure was not caused by a DBA. So, why am I paying so much for IT control compliance, when that's not where the risk is?"
  
  So this is when scope is overly broad: we're testing stuff that doesn't matter.
  
• Significant risks to financial assertions may be left unaddressed
  
  This is the case when scope is too small: we never test something that's actually really important, that could cause an undetected material error in the financial statement!
  
• Suboptimal use of scarce resources
  
  This doesn't require any explanation.

I will describe in my next post why there was a problem.  This is one of my favorites, because it shows the real gap that existed between COSO and COBIT.  I'm hoping you'll find that as interesting as I do!

In the meantime, do you see the similarities between the problem statements of SOX-404 and PCI?   Please comment.

(Also, if anyone interested in the slides, let me know -- I can post them on Slideshare or something...)

Upset about the subjectivity and ambiguity in the PCI DSS compliance standards? My #BSides submission on the answer...

(First a disclaimer: Although I am part of the leadership team of the PCI Scoping Special Interest Group, everything in this article are only my opinions, not anyone else’s, or an official position of the PCI Security Standards Council.)

Don’t get me wrong.  I think the mission behind the Payment Card Industry Data Security Standard (PCI DSS) is critical one: ““improve the security of global payment systems by protecting consumers, merchants and banks from credit information theft and loss and subsequent fraudulent activity.”

Given the fact that millions of cardholder records continue to be stolen show that there is a need for significantly increased discipline and rigor around the necessary controls required to protect cardholder data.

But as organizations mobilize to comply with the PCI Data Security Standard, they're finding that it’s a huge project.  Like really huge.  Many organizations are finding that complying with PCI DSS will require more project hours than the organization has!  Even if the only project they had to complete was "comply with PCI," even then, wouldn’t be able to complete it in one year!!

Even for organizations that don't have over ten-thousand project hours dedicated to PCI, PCI compliance is still sucking up all the air in the room, starving a gazillion important projects of necessary resources.

One of the most frustrating aspects of PCI, though, is the standoff between the organizations who have to comply with the PCI DSS, and the Qualified Security Assessors (QSAs) that audit them for compliance.

The interaction may sound like this:

• Organization: “We have isolated our sales order entry systems as best as we can, and believe we are still effectively protecting cardholder data. Due to an architectural decision, we can’t partition off these systems from the rest of the business processes.”
• QSA: “I understand. But, we’re still liable for our role. So, your entire 20,000 systems will be in the scope of the PCI assessment.”

Maybe it's not 20,000 systems that are being argued about.  Maybe it's the CEOs laptop, even though the CEO isn’t entering customer orders or able to retrieve cardholder records.

I think this is an important topic.  So, here's the topic that I’ll be submitting for this year’s Las Vegas #BSides conference.

"Properly Mobilizing the PCI Resistance: Lessons Learned From Fighting Prior Wars (SOX-404)"

I have noticed that there is a growing wave of discontent and disenchantment from information security and compliance practitioners around the PCI DSS.  Josh Corman has been an effective voice for these concerns, providing an intellectually honest and earnest analysis in his talk “Is PCI The No Child Left Behind Act For Infosec?”

The problem are well-known and significant: too much ambiguity in the PCI DSS, Qualified Security Assessors (QSAs) and consultant using subjective interpretations, existing guidance either too prescriptive or too vague, scope missing critical systems that could risk cardholder data, overly broad scope and excessive testing costs, excessive subjectivity and inconsistency, poor use of scarce resources, no meaningful reduction in risk of data breaches, and so forth.

For years, I have been studying the PCI DSS compliance problem, as well.  I have noticed many similarities to the PCI compliance challenges and the “SOX-404 Is The Biggest IT Time Waster” wars in 2005.  I was part of the leadership team at the Institute of Internal Auditors (IIA) where we did something about the it. We identified inability to accurately scope the IT portions of SOX-404 as the root cause of the billions of dollars of wasted time and effort, while not reducing the risk of financial misstatements.

I propose to present the two-year success story of the IIA GAIT project and how we changed the state of the IT audit practice in support of SOX-404 financial reporting audits.  We defined the four GAIT Principles, which could be used to correctly scope the IT portions of SOX-404.  We mobilized over 100K internal auditors, the SEC and PCAOB regulatory and enforcement bodies, as well as the external auditors from the 8 big CPA firms (e.g, Big Four and other firms doing SOX advisory work).  In short, we made a difference, in a highly political process that involved many constituencies.

I am attempting to do something similar with the PCI Security Standards Council, through my work as part one of the leaders of the PCI Scoping SIG (Special Interest Group).  My personal goal is to find a “third way” to better enable correct scoping of the PCI Cardholder Data Environment, and create a risk-based approach of substantiating the effective controls to ensure that cardholder data breaches can be prevented, and quickly detected and corrected when they do occur.

My desired outcome is to find fellow travelers who also see the pile of dead bodies in PCI compliance efforts, and work with those practitioners to catalyze a similar movement to achieve the spirit and intent of PCI DSS.

There is a better way...

I’ll be writing a lot more on this.  Here are some topics I’m hoping to cover in the next couple of weeks:

On GAIT and SOX-404:

• a history of the GAIT for SOX-404 project
• examples and analysis of inappropriate SOX-404 scoping
• the method behind the madness: why did GAIT work?

On it's application to PCI:

• what principles can be ported over to PCI DSS?
• conversation with Josh Corman on "inside baseball talk: how does the PCI SSC and the SIGs work?"

And some very exciting news on how the PCI Scoping SIG is doing:

• the thought process behind the solution
• desired outcomes and guidance
• a report on our progress and work in process on solving this problem

And most importantly, what can you do to help?

Of course, the last point is likely the most important one.  There are things you can do to help the movement.  Interested in learning more, or is this a hysterical person on a lonely crusade against an imagined problem?

Thanks, and looking forward to your comments!