Over the past couple years of blogging, I’ve found that about once or twice a month I’ll write a really long blog post on a subject, only to scrap it before publication.   It might be because my thoughts on the subject are too emotional, because they’re just not terribly interesting, or as I found myself doing last week, it was just a really long-winded way of saying something that could be said more succinctly.   Last week I saw someone in a major magazine refer to something called a “PCI Budget”, and that an informal survey of risk managers suggested that these “PCI Budgets” weren’t going to suffer in 2009.

I proceeded to write a several paragraph post about the role of PCI compliance in Information Risk Management, and how, if you had a significant portion of your budget allocated to “buying more PCI”, you were doing it wrong.  My premise was that you should be investing in those things that lead to maturity in your IRM Program, and that you could count me among the many that believe that ideally PCI compliance is the result of a mature IRM program, not the goal of one (see postscript 1).  But the exercise of writing that article was not without benefit, as I started thinking more about that concept of maturity, and what it means, and how that maturity might be achieved.  I thought I’d share my thoughts on the subject with you today because they seem to be much more interesting than PCI for the sake of PCI.  In fact, I think we can build a couple of axioms or mantras to repeat about IRM programs and achieving maturity.  Think of these as my little contribution to the “New School” movement.

First, maturity does not mean “secure”.

Maturity actually means striving to achieve one of two goals (see postscript 2):

1.)  reducing the probable frequency and impact of loss events i.e., risk, and
2.)  the creation of operational efficiencies in the act of risk reduction.

In order to achieve those goals, we have to first understand how to do two key things, understand the risk tolerance of decision makers given the current business strategy (what we might call “aligning” IRM with business objectives), and figure out the “how’s” and “what’s” of measurement.  We could even say that the first is just a significant piece of the last - we need to figure out what and how to measure the risk tolerance of data owners.  But these two needs leads us deduce our second mantra:

Second, maturity can only be reached by measurement.

This one should be a real “duh” but it’s a lot tougher than it looks.  Press most people about what they are measuring and why, and they’ll have a tough time defending their metrics program.  I’ll give you an example.

Earlier this year, I heard from a well known CISO that he had “metrics under control”, that they were really way ahead of the curve and, basically, seemed to arrogantly imply that they had nothing to learn from the rest of the IRM community at large.  Later in the year, when their metrics person joined one of the projects I worked on, I was very interested to see the quality of what they were going to contribute.  I was keenly disappointed that that this organizations representative contributed to the discussion with much the same condescension as their CISO.  So imagine my surprise when it became knowledge that this wonderful metrics program apparently was built on the foundation of vulnerability scanner results, and not even a useful interpretation or translation of those results, just the straight output.  Needless to say, the arrogance quickly disappeared when others (not myself) confronted them on the usefulness of those metrics.

The moral of the story is - if you’re going to get mature, and if you’re going to measure - you’ll probably need help.  Out of all the projects and mailing lists and so forth that I’m a part of, I can think of about a half-dozen or so people that I would really entrust creating a metrics program for my IRM group.  <shamelessplug>Jack Jones, of course, is the primary person I’d seek to hire </shamelessplug>.

Third, the quality of your measurement will dictate the quality of your maturity (with limits).

Another real obvious point, but doing a good job at measuring enables a quality result.  It doesn’t necessarily mean you’ll execute, no, but it does mean that you’ll be in a position to execute competently.   The funny thing about that organization above that “had metrics all figured out” is that there are now entire groups of people that have an idea about the quality of that IRM program, primarily because the quality of their metrics and metrics interpretation were so faulty.  Whereas they thought they were in the 90th percentile or so of IRM program quality, turns out that they may not be *that* much better than everyone else.

It’s important to note that there are diminishing returns to measurement and how much a metrics program will benefit an organization.  Now I’ve got an idea that once you figure out the “how’s” and “why’s” that there are business and operational theory tools that can help divine the most return for time spent measuring (Lean IRM Programs!), but I’m not aware of *any* organizations that have reached that level of maturity yet.

Fourth, maturity via measurement is not a function of any one set of IRM standards that I’ve seen, but could be achieved using several different available tools.

Well, maybe SABSA is all encompassing but most of the world will never know because that thing is a beast.

This thought is based on my experience(small sample size warning!).  Most people are borrowing from here, from there, and creating an ad-hoc model of how the whole management of risk works, and how/what they need to measure.  As an example, I’m personally involved in a couple of efforts around various ISO standards and FAIR, for example, where FAIR provides the basis for adding rigor into 27k series documents.  Now I’m not out to denounce this experimentation around IRM theory, but it’s worth noting that few people are really satisfied with their results.  This mass creation of one-off models that aren’t shared means that we’re all sort of driving blind.  There are obvious problems with this, which leads to my final thought.

Fifth, visibility and co-operation in creating management models and metrics programs are critical.

If you really want a mature IRM program, if you really want standards compliance to be an after-thought rather than *the* business driver, then I’ll offer that it’s going to be really tough get mature in a vacuum.  If it didn’t work for one of the more well-known CISO’s in the industry, I have to think that the chances are against you doing it all by yourself.  I-4, CIS, Securitymetrics.org are just a few places that you can get involved and get started on the road to maturity.

In conclusion, let me state that mature IRM programs won’t be for everyone.  We are still beholden to the risk tolerance (stated or not) of the business owners.  If their tolerance for Information Risk is great - the resources to get mature might be few and far between.   But measurement and development of mature IRM program practices will at least help you calibrate both your and their expectations around organizational information risk.

====================================================================
Postscripts:
(1) - It always surprises me how my reputation regarding PCI proceeds me.  Let me state for the record that I’m pretty agnostic about PCI - it is simply a piece of the impact landscape CISO’s and IRM managers need to be aware of.  What does get my cackles up is the seriousness others (mainly vendors or those vested in the PCI cottage industry) attribute to it’s ability to make you “secure”.

(2) Recall Jack Jones explains that there are only 3 real value propositions for IRM; reduce risk, create operational efficiencies, and reduce loss.  I’ll rarely reference that last value proposition (reduce loss) because it requires having similar past incidents to compare to and proving that “you didn’t screw up as bad this last time”.   It’s just not a lot of fun, so I normally leave it out in discussion.