My #BSidesLV slides: "Mobilizing the PCI Resistance: Lessons Learned From Fighting Prior Wars (SOX-404)"

I had a great time presenting the work I'm doing with the PCI Scoping SIG to the #BSidesLV community. It was especially gratifying to me that the practitioners in the room agreed so strongly with me on the importance of correct scoping.

Simply put, the success of any PCI DSS compliance initiative is very dependent on accurate definition and scoping of the Cardholder Data Environment and the scope of assessment.

Thank you Mike Dahn (@MikD) and for allowing me to present in your dojo. And I felt incredibly honored when you said that you're already telling your clients that the scoping tools that we're building will likely be the most significant change to the PCI SSC in years.

There is no doubt in my mind that Mike has been one of the top contributors to the PCI DSS body of knowledge, so that kind of endorsement means a lot to me!

Taking Bets On When The "Mike/Josh Hug" Will Occur!

I am also announce that I am taking up the challenge from Josh Corman (@joshcorman) and Mike Dahn (@MikD) to help them "hug it out."  Given both of their genuine passion for helping organizations be simultaneously secure and compliant, and their obvious talents, I think I'm up to the task!

I'm starting a pool: who can most accurately predict when "Mike/Josh Hug" will finally occur? The winner will get a framed picture of the hug, signed by both Mike and Josh.

Full Slideshare link below!

2010 07 BSidesLV Mobilizing The PCI Resistance 1c

View more presentations from realgenekim.

Talk abstract:

Properly Mobilizing the PCI Resistance: Lessons Learned From Fighting Prior Wars (SOX-404)"

I have noticed that there is a growing wave of discontent and disenchantment from information security and compliance practitioners around the PCI DSS.  Josh Corman has been an effective voice for these concerns, providing an intellectually honest and earnest analysis in his talk “Is PCI The No Child Left Behind Act For Infosec?”

The problem are well-known and significant: too much ambiguity in the PCI DSS, Qualified Security Assessors (QSAs) and consultant using subjective interpretations, existing guidance either too prescriptive or too vague, scope missing critical systems that could risk cardholder data, overly broad scope and excessive testing costs, excessive subjectivity and inconsistency, poor use of scarce resources, no meaningful reduction in risk of data breaches, and so forth.

For years, I have been studying the PCI DSS compliance problem, as well.  I have noticed many similarities to the PCI compliance challenges and the “SOX-404 Is The Biggest IT Time Waster” wars in 2005.  I was part of the leadership team at the Institute of Internal Auditors (IIA) where we did something about the it. We identified inability to accurately scope the IT portions of SOX-404 as the root cause of the billions of dollars of wasted time and effort, while not reducing the risk of financial misstatements.

I propose to present the two-year success story of the IIA GAIT project and how we changed the state of the IT audit practice in support of SOX-404 financial reporting audits.  We defined the four GAIT Principles, which could be used to correctly scope the IT portions of SOX-404.  We mobilized over 100K internal auditors, the SEC and PCAOBregulatory and enforcement bodies, as well as the external auditors from the 8 big CPA firms (e.g, Big Four and other firms doing SOX advisory work).  In short, we made a difference, in a highly political process that involved many constituencies.

I am attempting to do something similar with the PCI Security Standards Council, through my work as part one of the leaders of the PCI Scoping SIG (Special Interest Group).  My personal goal is to find a “third way” to better enable correct scoping of the PCI Cardholder Data Environment, and create a risk-based approach of substantiating the effective controls to ensure that cardholder data breaches can be prevented, and quickly detected and corrected when they do occur.

My desired outcome is to find fellow travelers who also see the pile of dead bodies in PCI compliance efforts, and work with those practitioners to catalyze a similar movement to achieve the spirit and intent of PCI DSS.

The Reprint Of Internal Tripwire Departure Announcement

Yesterday, I posted my blog article about "An Exciting Day! Leaving Tripwire To Begin My Next Chapter In Life".  As I described in that article, it was a tough decision.

Shown below is the email announcement I sent out after a quick meeting to announce my departure from the company.

Date: July 1, 2010
Subject: The toughest and most bittersweet email I’ve written in 13 years

Hello, all –

After my identical twin sons were born in March, I had the luxury of spending two months with my family.  Some of it was around the clock, and some of it was while working half-time.  But, in either case, against all my expectations, it was the two best months of my life.

At the end of this period, my wife asked me, “Do you really have to go back?”

Of course, I responded, “Of course I need to go back.  That was our plan.”

But, over the next couple weeks, my wife kept bringing up how much life had changed after I returned back to work, especially for our two-year old, Reid.  And each time, I had more and more difficulty looking her in the eye and telling her again and again, “Darling, it’s absolutely impossible that we can live just like in those two great months.”

In the middle of some night, I wondered if one day, years later, when I’m like Jim [Johnson, Tripwire, CEO] with far more years behind him than ahead of him, as he squints in the dusky sunset of his life, whether I’d be thinking, “Holy crap. Should I have done it differently?”  Which led to making one of the toughest decisions I’ve made in my life.

Thirteen years after I co-founded Tripwire, I’ve decided to leave the company.

I told Jim, “First, I want what’s best for the company and I want the IPO to be successful.  But I’d like to discuss when would be the best time for me to step down, in a way that doesn’t jeopardize the first goal. It could be a couple weeks, a couple months, or even a couple of years. You tell me what would work best for you and the company. Because no one wants you to screw up the IPO.”

And Jim has been absolutely terrific in every way, and I am grateful to him and the entire team for making this sensitive and (for me) difficult process so easy. My last day will be July 23. And there will be a party at the Nines. And booze! It’s July 20th at 4:00 pm PST.

You might be asking, “So, what’s next for you, Gene?”  Well, first off, not traveling, much to the relief of my wife. And I’ll be working on some projects that I’ve back-burnered for years, including finishing a book that I started five years ago.

From the very bottom of my heart, I am grateful to everyone for making Tripwire such a success and company that so many people admire, as well as for the amazing memories I’ll always treasure, accumulated over the last 13 years. And I genuinely wish that everyone will someday have the freedom and ability to make the same choice I’m making.

So, to everyone: if I don’t see you before July 23rd, see you then.  And again, thank you for everything.

PS: I noticed that my copy of the book “How Even Auditors Can Find Love And Happiness” seems to be missing.  I really need it back. If you have it, please return it, and I promise, no questions asked.  There might even be some drink tickets in it for you.

PPS: Feel free to contact me anytime at genek(at)realgenekim.me.

Cheers,
Gene

An Exciting Day! Leaving Tripwire To Begin My Next Chapter In Life

The evening of July 20 was a both joyous and bittersweet.  Why?  It was joyous because I spent the evening with so many Tripwire colleagues that I’ve loved working with, who were all congratulating me and wishing me well.  It was bittersweet because this was my farewell party at Tripwire: thirteen years after I founded Tripwire, I was leaving the company to start the next chapter in my life.

I had announced to the company on July 1 about my plans.  I'll be posting this letter tomorrow.

I am very proud of my contributions to the company. Looking back, I’ve achieved almost everything I set out to achieve at Tripwire.  Eighteen years ago, I wrote the original version of Tripwire in 1992 with Dr. Gene Spafford. Now, it is a company that has thousands of customers, booked over $80MM in 2009, and continues to be used as part of information security, compliance and IT operations programs worldwide.  And as widely reported, the company completed its S-1 filing in May.

I am very grateful to Jim Johnson, the Tripwire CEO, for making something that was so difficult (for me) so easy.  He is a genuinely great guy with unquestionable integrity. The company future has never been this bright, and I am deeply grateful to everyone who has helped make that happen, including our customers and investors.

For me, the time was right to take some time off to spend with my family and resume work in area of passion: to complete the study and enable the replication of what makes high performing IT organizations tick.

As many of you know, since 2000, I’ve been studying a group of IT organizations that simultaneously achieve the best IT service levels, the best posture of compliance, the best integration of information security into the software development lifecycle, and also have the highest release rates and project due date performance.

How these organizations made their “good to great” transformation is what my colleagues and I captured in the Visible Ops and Visible Ops Security Handbooks, why we created a non-profit research organization, which benchmarked over 1500 IT organizations to conclude which practices led to improved performance.

Along with some trusted collaborators and fellow travelers, I believe that the conditions are now very favorable to propose some new solutions, dramatically different than the status quo.

In addition to spending half-time with my family, here are the three things that I intend to complete in the next two years:

Project #1: Finish My Book: "When IT Fails: The Novel"

Finish the novel “When IT Fails: The Novel.” The novel describes the fall and eventual triumph of the CEO and VP IT Operations of a 100 year old, $4B/year company at the brink of existential failure.

The CEO must close the gap with the competition.  But the two most critical projects necessary to achieve this are years late and way over budget, mostly because of IT. Furthermore, the company is losing customers due to outages and fragile and insecure IT infrastructure, SOX-404 IT audit findings are jeopardizing their 10-K with disastrous footnotes, PCI compliance failures threaten to damage the company brand, and developers are taking dangerous shortcuts in order to meet external promises.

It starts to dawn on the CEO that his survival now depends upon the success of IT and information security. And while he believes that IT is not their core competency, he learns that the company cannot function without it, and is therefore a competency that they must develop.

You can learn more about the book here.

Project #2: Start An Exciting New Venture

During my thirteen years at Tripwire, I was very focused on the mechanics of how organizations can detect and manage configurations and changes.  But in reality, the problem actually starts far upstream, in how the business and IT organizations made decisions that necessitated those changes.

I am starting a new venture to develop the methods, procedures and enabling software tools needed to support the transformations described in “When IT Fails: The Novel.”

I am very excited to be working with some very talented and trusted colleagues, so stay tuned for more details.

Project #3: Continue Engaging With Kick-Ass Communities Of Practice

Work with the communities that I believe will be an instrumental part of creating the management movement to change how IT is managed.  These include: DevOps, PCI Security Standards Council, Service Management, the Institute of Internal Auditors, the Software Engineering Institute, and I know I've forgotten mention some others!

I’ve had tremendously productive collaborations with these groups, as well as forming lasting friendships.  And I believe bigger and better achievements are still to come.

So Stay Tuned!

Thank you again for all your support, and I look forward to collaborating with you in this new chapter my new story.  If you want information on my progress, follow me on Twitter or subscribe to my newsletter.

You can find my internal email announcement of my departure to the company here, as well as pictures from the amazing farewell party that they threw for me here.

Are You As Good At Scoping PCI As You Think? (Mobilizing The PCI Resistance, Part VII)

I'm getting ready for my BSides talk on Wednesday on "Mobilizing the PCI Resistance." And I made a slide that was designed to hint that defining the scope of PCI assessment may not always be as easy one might think.

In today's blog post, I pose some (possibly trick) questions.  But before I do that, to refresh your memory, here's what are the previous postings:

• Part I: "Upset about the subjectivity and ambiguity in the PCI DSS compliance standards? My #BSides submission on the answer..."
• Part II: "The problems that management and auditors faced in 2005 and 2006 for the IT portions of SOX-404."
• Part III: "Quantifying the huge amount of wasted IT audit effort in SOX-404"
• Part IV: "What goes wrong in a bottom-up SOX-404 audit: a cautionary tale..."
• Part V: "The GAIT Vision For Solving The SOX-404 IT Scoping Problem
• Part VI: The Politics Of SOX-404 And GAIT (And Implications With PCI)

PCI Scoping Quiz: Please Show Your Work

At one time, I believed that the Cardholder Data Environment being defined by where cardholder data "enters, is transmitted, processed, stored, displayed or printed" is sufficient to inform most scoping decisions.

Gosh, was I wrong.

Don't worry if the answers don't seem immediately obvious. I'd wager I would have gotten most of these wrong last year.

Question 1:  Is the Cardholder Data Environment (CDE) equivalent to the PCI Scope of Assessment?

Question 2: Is a domain controller (e.g., Windows Active Directory server) that is being relied upon by CDE applications for authentication and security services in the PCI Scope Of Assessment?

Question 3: How about a domain controller (e.g., Windows Active Directory server) that is not relied upon by any CDE applications?

Question 4: Is a network attached stapler that happens to be on the same network segment as a CDE system component always also in the CDE?

Question 5: Does it matter if a workstation that a customer service representative uses a thin- or thick-client?

Question 6: When should it be acceptable that if a virtualization hypervisor hosting  a production application in the CDE be also able to host another VM without it being part of the CDE, as well?

Question 7: If you have a domain controller that is not in the CDE, but in the scope of PCI assessment, is a print server on the same network segment as that domain controller also in the scope of PCI assessment?

Bonus Exercise: For each of the questions where you answered "in scope of the PCI assessment," describe a strategy to contain the scope, such that systems connected to that system are not in scope.  (See Michelle Klinger's great post on the "PCI Contagion Dilemma.")

(Image courtesy: Flickr: Zeligfilm)

The Answers And My Goal

The answers for all of the questions is actually, "It depends."  But just as problematic is the fact that people could arbitrarily disagree with your answer, with little ability to defend its validity.  The trick is being able to state, "Yes, it depends, but on what does it depend upon?"

Also, generating a consensus on scoping conclusions takes a lot of time. Prior to creating a structured method with agreed upon definitions, the Scoping SIG required over 40 hours to come to a scoping conclusion for one scenario.

With the proposed guidance under development, our team was able to generate a consensus on 15 scoping conclusions in less than 2 hours.

We believe there are three things needed to aid the scoping effort, and be able to defend your answers, so that another person can follow your reasoning.  Even if they don't agree with your scoping conclusion, at least they can scrutinize your assumptions.

My goal is to have the PCI Scoping SIG deliver;

• Define and deliver the following, in a manner that clarifies and supports the spirit and intent of protecting cardholder data:
  • Scoping principles and definitions (should be 200 words or less)
  • A structured scoping methodology (should be a decision tree, with fewer than 30 boxes)
  • A library of scoping scenarios demonstrating its usage for educational and clarification purposes (should be about 30 pages)
• Create useful tools and guidance that will assist in the scoping effort for both merchants and QSAs.

Interested?

Mobilizing The PCI Resistance, Part VI: The Politics Of SOX-404 And GAIT (And Exploring PCI As Well)

This is Part 6 of the "Mobilizing PCI Resistance" series.  Briefly, we've covered:

• Part I: "Upset about the subjectivity and ambiguity in the PCI DSS compliance standards? My #BSides submission on the answer..."
• Part II: "The problems that management and auditors faced in 2005 and 2006 for the IT portions of SOX-404."
• Part III: "Quantifying the huge amount of wasted IT audit effort in SOX-404"
• Part IV: "What goes wrong in a bottom-up SOX-404 audit: a cautionary tale..."
• Part V: "The GAIT Vision For Solving The SOX-404 IT Scoping Problem

The Politics Of SOX-404 And GAIT (And Implications With PCI)

The GAIT project was one of the most politically charged projects I've ever been involved with.  The diagram below shows each one of the constituencies, and the relationships between them.

As we go through them, I'll discuss the equivalents to the PCI universe.  I posit that while the PCI ecosystem is also very political, it's less so than for SOX-404.  This is good news.  :-)

My reason for this blog post is to describe how we analyzed the lay of the land, and created a winning strategy that allowed us to change the compliance landscape. By doing this with PCI, we can play to win.

(By the way, this type of diagram is called a "value network," and was taught to me by the famous Eileen Forrester at the Software Engineering Institute at Carnegie Mellon University.  Full slides available on SlideShare)

So, let's go through each of primary constituencies, and then I'll discuss how they relate to each other:

The Federal Regulatory Enforcement Bodies:

The Sarbanes-Oxley Act of 2002 was enacted as a response to accounting scandals at WorldCom, Enron, Tyco International, etc.

• 

  The Securities and Exchange Commission (SEC) created a new set of requirements that public companies (i.e., "SEC registrants") had to comply with.

• The Public Corporation Accounting Oversight Board (PCAOB) was created by SOX-404, charged with overseeing, regulating, inspecting and disciplining accounting firms in their roles as auditors of public companies.
  
  In other words, they audit the auditors.  For instance, on at least an annual basis, they would audit the work papers submitted by the Big Four firms to ensure the quality of their work, and that it's in compliance with the standards of independence. Each year, they would publish a report describing when the external auditors behaved in ways that was "out of line."

The PCI equivalent here is the PCI Security Standards Council and the card brands. The parallels between the PCAOB and the new PCI requirements for QSAs to submit their work papers and Reports on Compliance (ROC) is striking. PCI is now actively auditing the auditors.

The Public Companies:

• 

  CEOs and CFOs: These are the people specifically named in SOX-404 as personally accountable and responsible for the effectiveness of controls that support the accuracy of the company financial statements (e.g., 10-K statements submitted to the SEC).

  When people make jokes about "keep the executives out of orange jumpsuits" or "keep them out of jail," these are the people who would be wearing them.

• 

  CIOs and IT Management: We identified that one of the primary contributors around the wasted time/effort in support of SOX-404 was in the domain of IT controls. Business management owned the financial controls, and IT management owned the IT controls.

• Information security: More specifically, many of the IT controls were owned by information security.

The PCI equivalent here are the merchants, service providers, and anyone else who has custodianship of cardholder data. These are the people who are often complaining that the PCI DSS is too subjective, too arbitrary, that the cost of compliance is too high, etc.

And again, this is the likely the community we must mobilize first.

The Internal Auditors

Internal auditors are independent of Business Management and IT Management. Their goal is to ensure that the risks to the organization are understood and that adequate controls exist to prevent, detect and correct for those risks.

(Interestingly, in my experience, in most organizations, internal auditors are only recently showing up at the table for PCI compliance. But, they have a wealth of experience to bring in terms of managing external auditors. And it's my understanding that internal auditors can now submit PCI Reports On Compliance now, even though they're not technically a QSA.)

The External Auditors:

External auditors generate an opinion on the financial statements. They also have their work papers audited by the PCAOB.

The PCI equivalent here are the Qualified Security Assessors (QSAs). Instead of generating an opinion on the accuracy of financial statements, they file a ROC.

Professional Organizations:

• Institute of Internal Auditors (IIA): 118K+ due paying members: internal auditors
• Information Systems Audit And Control Association (ISACA): 80K+ due paying members: information security and IT auditors
• National Association Of Corporate Directors (NACD): professional organization of corporate board members.

The IIA was clearly at the forefront of this issue, and this is the community we mobilized.  (ISACA was peripherally involved in the beginning, but dropped out due to unbelievably bad relationships between them and IIA.)

(In the PCI world, I'm not sure if there is a professional organization that we can mobilize. Instead, I believe the best lever is the PCI Community of Participating Organizations.)

Our Resulting SOX-404 Strategy: Start With Internal Auditors, Where Collective Outrage Was Highest

The constituency who probably saw the problems the clearest were the internal auditors, who saw that the amount of investment in IT controls were way out of proportion to the risk. While other constituencies also saw the problem, we believed it was the internal auditors who could describe it most clearly, and could mobilize a response.

We then gathered the heads of audit (and sometimes IT audit) from some of the largest publicly traded companies. Our goal was to assemble the auditors from top Fortune 50 companies, and have them uniformly say, "we all see a common problem, and it is causing significant economic harm. And we have a joint proposal for what we should do about it."

As mentioned in a previous blog post, the initial companies involved included General Motors, Wal-Mart, Hewlett Packard, Intel, Microsoft, Marathon Oil, Chevron Phillips Chemical, Business Objects, etc.

When those companies start screaming, people listen.

Next, Rope In Their External Auditors

Now that we had an impressive list of publicly traded companies all saying we had a problem, the next step was to engage their audit firms.

We had each one of the audit executives reach out to their IT audit engagement partners, and invite them to the GAIT summit. We selected and organized the companies to ensure that we had coverage of each of the Big Four firms (i.e., PwC, KPMG, E&Y and D&T). This was framed as a strong request to the Big Four firm, along the lines of, "I believe that it is very important for the relationship between our two companies that you personally attend this summit. And I'd like you to bring someone from your national practice, as well."

That last request was important, because the goal was not to change the IT scoping practices of, say, E&Y and Microsoft. Instead, the goal was to affect all of the engagements that E&Y was involved in. The national practices at the Big Four firms are usually responsible for establishing the standards, practices and procedures for the entire organization. They're typically some of the most senior and experienced partners, as well as promising staff members likely to become partners. Why would they want to attend?

Lastly, How To Mitigate The Response Of "External Auditors Say, 'Up Yours'?"

This was probably one of the biggest obstacles to the GAIT initiative. How could encourage or compel the Big Four firms to participate? A cynical person would say that there was no real reason for them to get involved, as GAIT would do the following:

• Decrease billable hours in support of SOX-404 projects (i.e., "stop the SOX-404 gravy train")
• Increase the standards that they"™d have to adhere to (i.e., "create rope that they could get hung on")

Our solution was to make sure that Bill Powers from the PCAOB attended each of the GAIT Summits. The goal of GAIT transformed then to ensure that IT work performed in support of SOX-404 represented a continuation of the top-down, risk-based approach described in Auditing Standard 2, a publication published by the PCAOB. The implication is that the PCAOB would be very interested in any Big Four firms who did not want to participate in the GAIT process.

Voila: The GAIT Summits Began

Now we had assembled each of the constituencies required, each that could achieve a desired outcome out of a successful GAIT project.

• Internal auditors: accurately scope and substantiate IT controls in support of SOX-404
• Company management: ensure accurate financial statements and contain costs
• External auditors: stay out of trouble with the PCAOB
• Regulators: ensure that IT audit activities matched the spirit and intent of AS-2

Next up, I'll pose some thought-experiments on PCI scoping, and walk through the GAIT Principles.