“How I can communicate the value of an ISO implementation to non-security management?”

This question came to me after one of the posters on the ISO Google Group asked about KPIs for ISO implementation.  Got great responses in email, blog comments, and on Twitter from current/former CISO folks and consultants and analysts.  Some really great thought and effort, by the way - thank you.  It’s really great to be able to have these sorts of conversations online.

First, I have to point out some resources Brian Honan linked to from Gary Hinson, just because they’re so cool.  Gary has invested gobs of time and effort to become one of the defacto resources on the ISO (you might also want to read or re-read Gary’s web post on the 7 myths of metrics).   Brian links to an implementation guidance document(pdf) and a metrics example(pdf) document.

As full of awesomeness as they are, though, these are simply metrics “mapped” to the ISO (i.e. the ISO isn’t a pre-requisite for generating this information).  They are not KPI’s that express the value of ISO implementation.  Problem is the metrics created here still require some level of “translation” in order to create some value statement that data owners can understand.  As Myrcurial twittered me “27001 is orthoganal to process” meaning (I hope) that metrics have their foundation in events that are generated by processes.  27001 by itself was never meant to create metrics (see above), and so we’re asking a question the ISO can’t answer.  But the desire, the need to measure still exists.  To that extent we can google “ISO compliance” (whatever that means) and if something can be certifiable or deemed “compliant” we can and are “measuring”.  But does that have value? Rybolov (my favorite Guerilla CISO) wrote:

“Whatever you do, don’t start measuring percentage of compliance. Eventually, that’s what all metrics efforts around a framework devolve into.”

I have to agree.  Being ISO “compliant/certified” has little expressive business value prima facia. I find that one KPI that absolutely asserts value when expressed properly is risk - and similarly  Shrdlu wrote:

“I really have no idea. I personally wouldn’t try to justify an ISO implementation by itself. If I could show traceability on how it affected our overall security risk, then that’s what I’d do.”

And that’s a delightful answer.  That “traceability” (geeze-louise Shrdlu - what a word!) is absolutely what I’m after here.  How do I get that?  

If you’re going to do something with corporate budget (time, money - and goodness knows an ISO implementation is time & money) you better be able to communicate the value.  And while the zealotry for ISO implementation differs from person to person, I have yet to come across someone who says that ISO adoption is totally without value.  It’s just not apparent what that value of adoption is and how we can measure (metrics) and express it (KPIs).

Jenean Paschalidis wrote what he thought that value was in a very nice email in which he puts a qualitative name on the value of adoption:

“Transparency and accountability-this is what all executive/senior management (the company) is on the hook for. ISO provides that. If you want to understand and have confidence in your operations as supported by security (because you will know the who, what, where, when, why and how of a system (human, technical etc.) and you want to be able to trace back why a decision (risk-vetted) had been made - then adoption of this best international practice will assist in providing these answers.”

So working with our above thoughts a little here - if we agree with Shrdlu that the only value of an ISO implementation can only be expressed if we can say how said implementation affected our overall security risk - and we agree with Jenean that the primary benefit is an ability to have confidence in operations as supported by security, then….

The value of the ISO should be expressed as a KPI or set of KPIs that cleary explain how the confidence it generates helps us understand (and then reduce) our risk.

If risk is a probability issue,  ISO adoption helps generate confidence in our predictive analytics.  The dollar value the ISO generates (the ultimate KPI) is part of the cost of being able to make wise risk decisions.

So what is that (making wise risk decisions) worth to you?

SOME CONCLUDING THOUGHTS

First, it occurs to me that this is a real shame.  In a sense, an inability to generate a quantitative value statement for ISO use is simply more witch-doctory (“use it because we, the wise men of the tribe say you should”).  In some future version, the ISO should include some mechanism for measuring and expressing the worth of adoption to the organization (a better reason to use the ISO than “because we said so”).

Second, It should be noted that of Jack Jones’ 3 true value statements from which all metrics/KPIs should point to - we’re only talking about one of those value statements - the ability to reduce risk.  Using the ISO in an organization most certainly could create operational efficiencies (help us do more with less) - but the ISO isn’t a standard that creates operational efficiencies as a primary goal, nor does it give implicit direction on how to create operational efficincies.    The ISO folks do, however, play fast and loose with the idea of “risk” and “risk management” so it’s within this context that I interpreted our conversation.

Finally if you’re going to hire someone to help you with ISO adoption in your organization, the deliverables you ask for in your RFP/SOW/what-have-you should include quantitative (probability) statments about risk reduction and the creation of operational efficiencies.  If the firms answering can’t tell you what value their work will be to your company, then drop me a note and I’ll gladly point you to some friends of RMI’s that know FAIR & all our Risk Management frameworks and also do great ISO work.

Case in point is the jury trial of Julie Amero, a schoolteacher who was charged with felony for allegedly showing porn to her class–when in fact the porn sites were popups caused by malware on the classroom computers that popped up while she was teaching:

a series of incompetent computer experts and overzealous prosecutors tried to claim that the pornography that appeared on the school computer browser was deliberately viewed. In reality the computer was infected with a browser hijack or other form of malware nastiness that launched a flood of porn pop-ups. There was an outpouring of support and some technical folks like Alex Eckleberry, who led an effort to prove that Julie was innocent of the charges

After a long trial, Amero has finally been vindicated. But she has still lost those years of her life spent on the case, her teaching credential, and is being charged a $100 fine. While her trial might be over, her personal troubles aren’t.

So, my dear readers, imagine how amazed I was to find myself being truly inspired by my CEO,  for the first time in my working life! Philippe’s “no-B.S.” approach definitely works for me. I listened to his speech at a company meeting last week and – I am serious! – that was the most interesting, visionary AND inspiring speech that I’ve heard in a long time. It was clear what we’ve been doing, what worked, what didn’t and what we need to be doing and why it will work.

I already learned more than a few things from him just by listening to him  speak or conduct a meeting (or by watching him beat up a job candidate…). For example,  one CAN be “positive, but not marketing-ish,” even if situation is difficult. If one has an issue, one has to face it with no sugarcoating rather than ’play’ positive and pretend the issue is not there. One can have BOTH a driving vision AND be attentive to customers. One CAN release something when it is ready, not a year before :-) Etc, etc.

Finally, while some choose to lay people off, we at Qualys  ARE HIRING!  Come join us and help build the SaaS security platform that actually works! Specifically, we are looking for TAMs (kind like an SE, but better :-)), PMs and a lot of engineers.

I’ve often read people who say that we (security, risk management) need to “think like the attacker”.  And when you read this sort of article, that usually alludes to trying to anticipate the tactics an attacker might use to mess with your C, I, or A.  Smart stuff, that, and very useful when architecting security solutions.  But as I was training some folks Monday, I was thinking in the back of my head about Threat Capability (TCap) in FAIR.  As you might know, we like to estimate the capability of a threat to apply some level of “force” against our assets.  This ability to apply force is a byproduct of the attacker’s skills and resources.  And thinking of how an attacker applies skills and resources, I came across another way we might “think” like an attacker.

Traditionally, I’ve thought of “skills” as being a byproduct of the toolset an attacker has.  This mindset probably stems from my time with Penetration Testing teams, where in the process of scoping the  PenTest I would ask our clients to select the level of effort that they wanted us to throw at them.  If a client chose “high” we’d throw every ‘spoit we had at them.  If they chose “low” we’d limit ourselves to a more commonly available toolset.

But while the resources part of TCap is time & materials (money) - the skills are really more than just the toolset.  Skills would include the ability of the attacker to be creative and innovative.    As an example of that innovation from those PenTesting days - when we got a “high” effort request, we would always try to couple that with some “social engineering”-type of attack, or some unique means of delivering an existing exploit.  Our creativity was not necessarily a byproduct of a unique exploit or tool we had, but the process by which we might deliver pre-existing or commonly available exploits.  I remember when we first got ahold of a handful of 32mb thumb drives (hey, 32mb was huge back then) and “dropped” a few in the lobby of a client’s retail space.  The keystroke loggers and phone-home script weren’t new, but using the thumb drive as delivery vehicle certainly was.

So I’ve started to really think about this concept of innovation, and how if “thinking like an attacker” means to be innovative, we ought to do the same.  I’ve been thinking of two main categories of innovation this morning.

INNOVATION

The first I’ll call Technology Innovation.  And by Technology Innovation, I mean some new, unique, “ahead of the curve” technology that an attacker can use against us.  The obvious example of which is a zero-day.  It’s that “high” tool set our PenTesters would use against the clients.  For security departments, this might be the latest security product designed to enhance our ability to P, D, and/or R.

Alternately, we can be creative in the way we deliver (manage) existing technology.  I think of this as Process Innovation.  It’s doing more with what we already have, just like the PenTest team would be creative in the delivery of an existing exploit.

Unfortunately for us - attackers have traditionally had quite a leg up on us in terms of Process Innovation.  It is much easier fro them to be creative, as they are free of political constraints and bureaucracy.  In contrast, when the security industry tries Process Innovation, the results are checklists and “standards”.  It’s committees and consensus.  An extreme example of which might be something like SABSA - a great work if you want to understand some very smart people’s comprehensive understanding of organizational security  - but the “adoption”of which will do very little to help you be innovative in P/D/R.

It’s worth noting that ultimately, this is one reason I don’t like regulatory compliance efforts - they simply serve to prove how mundane your security department is,  wasting valuable resources that could be spent on creating ways to be more effective.

PROCESS INNOVATION AS A SUBSTITUTE FOR TECHNOLOGY INNOVATION

As we come to the close of 2009, some surveys suggest that security spending isn’t horribly impacted yet by the economy (the latest from E&Y points to only 5% of their respondents getting budget cuts).  But if this is a protracted downturn, and because InfoSec is an operational expense, I would expect cash to become more and more difficult to keep.  And regardless if technology spends do slow, I believe it makes sense to think about Process Innovation because I see Process Innovation as a means to increase effectiveness without significant capital expenditures (effectiveness increases because our ability to manage risk has a direct correlation to the amount of risk we have).

The bad news is, of course, that great innovation is hard.  It is R & D.  Failure is usually a pre-requisite to success.

The good news is, our current state is so bad that many of us don’t need to come up with a whizbang new way of reducing software defects in the SDLC as innovation.  Simply inserting a risk analyst into the PMO’s processes might count as a big enough victory. Be cautioned, though,  that if we’re substituting the risk reductions provided by technology acquisition - Process Innovation might actually be even more “expensive” as it requires us to expend political capital.   But there are (forgive the term) innovative ways to spend this political capital.

For example, by taking a second now and figuring out the 3 things that the rest of the organization can do to make your life easier, when that “I need to reduce your budget” talk comes, you can be prepared to negotiate.  Get a political capital “loan” or “investment” from the C-Suite reducing your budget.  Something to the effect of: “I expected this, and am happy to give up my budget.  But if our tolerance for risk hasn’t changed, what I’d like to do is get you to personally back my office on three projects I’ve identified that can reduce our risk without requiring significant capital expenditure.”

Be wary and warn the folks you know — this isn’t Windows Defender, an anti malware program.

F-secure has a screenshot so you know what to look for…and of course the requisite joke, hoping that future versions might promise an end to “maleware.”

Good luck with that, guys.

This is a quick and dirty video documentary of the things that when on around the talks and event at Phreaknic 12 (2008). Don't watch if you get sick at shaky cam movies like Blair Witch or Cloverfield. A rough timeline of the content in the video is as follows:

        Intro and leaving Louisville with Brian. Morgellon talks about hacking the Arduino micro controller platform. Sorteal talks about the LiVes Open Source video editor. AT&T Batman building by night. Mojo-JoJo soldering some stuff for the shooting range. The patron gods of hackerdom. Registration. Con swag overview. Morgellon  gets his discreet logic on. AK-47 building with HandGrip and Buttstock. Froggy talks up Notacon, which I plan to go to next year. Skydog explains the Jware chair toss event, and then we compete. Rootwars hacker wargames. I ask Int80 about using his nerdcore music in some of my videos. NotLarry explains rootwars. Some iPhone hacking with Lee Baird and John Skinner. I do a little Bluecaseing/Warnibbling with the Bluetooth on my Nokia n810. John, Lee, Brian and I go to the German restaurant. I blind DOSman with the light from my camera and check out what folks are doing with the Arduinos Droops brought for folks to play with. I check back in on R00tW4rz. I blind Droops. I talk Ettercap filters with operat0r. USB door key fun with the Arduino. More breadboard fun. Nokia n810 + Ettercap Filter + Lemon-part = win. Int80 gets down with his own bad self, and the rest of Phreaknic. I find an energy drink with protein. Folks play with the hardware keyloggers I brought, and we have some epic fail with the IBM Model M + USB adapter + Mac OS 10.5. Winn Schwartau joins in on the keylogger fun. DOSman and Zack use a directional antenna from the 9th floor to search downtown Nashville for WiFi access points. Zoom in on Al. John and Lee eat jerky. Daren and Shannon from Hak5 blind me this time. :) Then they do a quick interview. I interview TRiP about the legalities of wardriving, sniffing and leaving your access point open so you have plausible deniability of copyright infringement (most likely it won't hold water in court if you are a computer geek). I give Hak5 Daren beef jerky. Ziplock had more con badges than God. I meet up with Iridium. I talk with Nightcarnage about the audio/video setup at Phreaknic. As I predicted, the Potters won the WiFi Race. I say why this was the best Phreaknic ever. Using green lasers on crack dealers. Techno in the dark, the Aiptek action HD does not do well in low light. Nicodemius shows off his Minority Report like multi-touch table. Hula hoop contest. I check back in with Jeff Cotton and his USB keyed door. I strap on my gear to leave the con. Brian and I do a wrap up of our thoughts on Phreaknic 2008.

This is a quick and dirty video documentary of the things that when on around the talks and event at Phreaknic 12 (2008). Don't watch if you get sick at shaky cam movies like Blair Witch or Cloverfield. A rough timeline of the content in the video is as follows:

        Intro and leaving Louisville with Brian. Morgellon talks about hacking the Arduino micro controller platform. Sorteal talks about the LiVes Open Source video editor. AT&T Batman building by night. Mojo-JoJo soldering some stuff for the shooting range. The patron gods of hackerdom. Registration. Con swag overview. Morgellon  gets his discreet logic on. AK-47 building with HandGrip and Buttstock. Froggy talks up Notacon, which I plan to go to next year. Skydog explains the Jware chair toss event, and then we compete. Rootwars hacker wargames. I ask Int80 about using his nerdcore music in some of my videos. NotLarry explains rootwars. Some iPhone hacking with Lee Baird and John Skinner. I do a little Bluecaseing/Warnibbling with the Bluetooth on my Nokia n810. John, Lee, Brian and I go to the German restaurant. I blind DOSman with the light from my camera and check out what folks are doing with the Arduinos Droops brought for folks to play with. I check back in on R00tW4rz. I blind Droops. I talk Ettercap filters with operat0r. USB door key fun with the Arduino. More breadboard fun. Nokia n810 + Ettercap Filter + Lemon-part = win. Int80 gets down with his own bad self, and the rest of Phreaknic. I find an energy drink with protein. Folks play with the hardware keyloggers I brought, and we have some epic fail with the IBM Model M + USB adapter + Mac OS 10.5. Winn Schwartau joins in on the keylogger fun. DOSman and Zack use a directional antenna from the 9th floor to search downtown Nashville for WiFi access points. Zoom in on Al. John and Lee eat jerky. Daren and Shannon from Hak5 blind me this time. :) Then they do a quick interview. I interview TRiP about the legalities of wardriving, sniffing and leaving your access point open so you have plausible deniability of copyright infringement (most likely it won't hold water in court if you are a computer geek). I give Hak5 Daren beef jerky. Ziplock had more con badges than God. I meet up with Iridium. I talk with Nightcarnage about the audio/video setup at Phreaknic. As I predicted, the Potters won the WiFi Race. I say why this was the best Phreaknic ever. Using green lasers on crack dealers. Techno in the dark, the Aiptek action HD does not do well in low light. Nicodemius shows off his Minority Report like multi-touch table. Hula hoop contest. I check back in with Jeff Cotton and his USB keyed door. I strap on my gear to leave the con. Brian and I do a wrap up of our thoughts on Phreaknic 2008.