Mobilizing The PCI Resistance, Part IV: When Bottom-Up SOX-404 Audits Go Bad. Really Bad.
Friday, June 18, 2010 at 11:13AM
This is Part 4 of the "Mobilizing PCI Resistance" series. Briefly, we've covered:
- Part I: "Upset about the subjectivity and ambiguity in the PCI DSS compliance standards? My #BSides submission on the answer..."
- Part II: "The problems that management and auditors faced in 2005 and 2006 for the IT portions of SOX-404."
- Part III: "Quantifying the huge amount of wasted IT audit effort in SOX-404"
In this article, I will share a cautionary tale of how the problems discussed in the previous articles result in such horrible outcomes. More specifically, how the inability to scope correctly the IT portions of SOX-404 led to tons of firefighting by information security and auditors and wasted effort, all at the expense of more important things that they should have been focusing on.
Like keeping the organization secure. As opposed to trying to achieve compliance of a misguided and mis-scoped audit. Which is at the heart of the PCI problem.
How Bottom-Up SOX-404 Auditing Happens: A Cautionary Tale
How do these audit horror stories happen? How do intelligent people in management and audit end up auditing things that don't matter?
We studied this problem when we wrote the "Visible Ops Security: Achieving Common Security and IT Operations Objectives in 4 Practical Steps" book.
This is an excerpt from the book, in a chapter called "A SOX-404 Cautionary Tale":
When external auditors started testing against SOX-404 in the first year, IT findings represented the largest category of findings, totaling more than the combined findings in the revenue, procure-to-pay, and tax categories. It’s estimated that as much as $3 billion was spent in the first year of SOX-404 to fix IT controls to remediate these findings. Ultimately, most of these findings were found not to be direct risks to accurate financial reports and did not result in a material weakness. This is because they followed a bottom up versus a top-down, risk-based approach.
Consider the following scenario: The SOX-404 team asks for an information security review of a WebSphere server that runs the materials management systems. The review shows that it’s a custom WebSphere application running on a cluster of servers that is connected to a clustered Oracle database. We then locate the firewall and determine the segment it’s on.
An information security review of the materials management system uncovers:
- Numerous ghost accounts
- A lack of password aging policies
- Critical vulnerabilities in the Java code, including cross-site scripting issues in the HTML
- Vulnerabilities in the Oracle database configuration
- Firewall rules that are suspect and need further investigation
Our task list keeps expanding and the internal auditors are showing up next week. We decide to focus on the operating system level, and our suspicions prove to be correct: The operating system is not running at the latest patch levels. We add this to our list of corrective actions that need to be taken right away, and start talking with the owners of the operating system, database, and application, and even the firewall team.
When the internal audit team comes in, we are candid and transparent about all the issues. Management is informed about the risks, and soon 50+ people are working on all these issues, dropping other high-priority projects to get these issues fixed in time. After all, the argument is made, these issues should be fixed eventually because they do represent risk.
But there just isn’t enough time. The external auditors come in and find all of these issues. They start preparing a management letter stating that the integrity of the IT general control process (ITGC) environment cannot be substantiated.
As a result, more high-level meetings take place, and the financial people start to argue that the ITGC issues really can’t lead to an undetected financial reporting error. They pull out the “nine firm document” and use something called “Chart Three” to make the case. Then management and the CPA firms argue back and forth about the linkage, and management starts bringing in all the business experts to show that a failure in the ITGCs for this system could not result in inaccurate financial reports.
Finally, the owner of the materials management business process determines that even if the application, database, operating system, and firewall were compromised by a person trying to perpetrate fraud, the attempt would be caught by a daily financial reconciliation between the materials management inventory report and another report from the ERP system.
Given this new evidence, everyone agrees that reliance is actually placed on the daily financial reconciliation, which would catch both fraud and errors. Furthermore, they agree that reliance is not placed on the IT system and the supporting ITGCs. So, the IT systems are out of scope, and no further IT testing is required.
Everyone is relieved. As the information security practitioners, however, we struggle with this unsettling question about why we went through all this trouble if our efforts were not required to substantiate the accuracy of financial statements. Furthermore, we wonder if all the “good hygiene habits” are actually important and can be justified.
To be clear, it’s not that the downstream manual financial reconciliation control is the best control possible. The point is that if the scoping of IT controls were done correctly in the first place, the only control weaknesses that we would have tested and found would be those that truly jeopardized accurate financial reporting. Instead, we found control weaknesses on systems that were out of scope, and then kept digging needlessly.
Next up, I'll discuss the GAIT vision that was realized in February 2007, when the GAIT guidance was finally published.
Reader Comments (4)
A blog that is so distinctive with well-analyzed information. I am actually in need for this sort of vital information. Thanks for providing me solutions. Functional blog.
thanks your posting very interesting for me please visit Berita Android | Informasi
Seputar Handphone Android
I would be very thankful if you continue with quality what you are serving right now with your blog...I really enjoyed it...and i really appreciate to you for this....its always pleasure to read so....Thanks for sharing!!!
I really enjoyed it...and i really appreciate to you for this....its always pleasure to read so....Thanks for sharing!!!
garageband for android
blackmart alpha download