Hash functions are the most commonly used cryptographic primitive, and the most poorly understood. You can think of them as fingerprint functions: They take an arbitrary long data stream and return a fixed length, and effectively unique, string. The security comes from the fact that while it's easy to generate the fingerprint from a file, it's infeasible to go the other way and generate a file given a fingerprint.

Originally created to make digital signatures more efficient, hashes are now used to secure the very fundamentals of our information infrastructure: in password logins, secure web connections, encryption key management, virus and malware scanning, and almost every cryptographic protocol in current use. Without cryptographic hash functions, the internet would simply not work. At the same time, there isn't a good theory of hash functions. Unlike encryption algorithms, there are no secret keys involved; this makes it harder to mathematically define exactly what hash functions are.

The National Institute of Standards and Technology, NIST, is holding a competition to replace the SHA family of hash functions. "SHA" stands for "Secure Hash Algorithm." It was developed by the NSA in 1993 to replace the commercial MD4 and MD5 algorithms, and has been updated several times since then. All the SHA algorithms are very similar, and have been increasingly under attack, so NIST wants to replace them.

The competition is important because, unlike other technological standards, committee design — balancing the interests of diverse constituents — isn't conducive to good security. Security is best when it's designed by expert teams and then subjected to public review. And cryptography is best when it's chosen by competition.

In 1997, NIST held a competition for a block cipher to replace DES. Fifteen candidates and three-and-a-half years later, Rijndael became the new Advanced Encryption Standard — AES. NIST is doing the same thing for what it's calling SHA-3 (not, for some unexplained reason, the Advanced Hash Standard or AHS).

The deadline was October 31, and NIST received 64 submissions. This isn't surprising — I predicted 80 — as most of the 15 AES submitters were professors, whose students at the time have become professors themselves, with their own students. (If NIST does a stream cipher competition in another ten years, they should expect about 256 submissions.) These submissions came from academia, from industry, and from hobbyists. CIO magazine recently interviewed one of the submitters, who is 15. Twenty-eight submissions have been made public by the submitters, and six of those have been broken.

NIST is going through all the submissions right now, making sure they are complete and proper. Their goal is to publish all accepted submissions by the end of November, in advance of the First Hash Function Candidate Conference, to be held in Belgium right after the Fast Software Encryption workshop in February.

The group expects to quickly make a first cut of algorithms — hopefully to about a dozen — and give the community a year of cryptanalysis before making a second cut in 2010. After another year of cryptanalysis, NIST will choose a winner in 2011. Expect a final standard by 2012.

My advice for software developers is to let the process run its course. While it's tempting to use the new cool algorithms in your designs, it's far too soon to trust any of them. This process is likely to result in all sorts of new research results in hash function security, and some real cryptanalytic surprises. Give the community a few years to figure out which ones are good and which aren't.

I've previously called this sort of thing a cryptographic demolition derby: The last one left standing wins. But that's only partially true. Certainly all the groups will spend the next few years trying to cryptanalyze each other, but in the end there will be a bunch of unbroken algorithms. NIST will select one based on performance and features.

NIST has stated that the goal of this process is not to choose the best standard but to choose a good standard. I think that's smart; in this process, the best is the enemy of the good. While there's no rush to choose a new standard — the SHA-2 algorithms will remain secure for the foreseeable future — we don't want to analyze the candidates forever.

Personally, I was part of a group of eight cryptographers that submitted Skein to the competition. A decade ago, writing Twofish and participating in the AES process was the most fun I had ever had in cryptography. These next few years promise to be even more fun.

---

Bruce Schneier is chief security technology officer of BT. His new book is Schneier on Security.

Be sure to check it out soon (you can preview bits of each course right now for free), as we're offering a limited-time early adopter discount that's good for the life of your subscription. Our online courses are told by the authors themselves, with names that you'll recognize, as many are MSDN Magazine contributing editors and book authors on their topics.

Courses we now offer online include:

WCF Fundamentals by Aaron Skonnard

Windows Workflow Fundamentals, by Matt Milner

WPF Fundamentals, by Ian Griffiths

Silverlight Fundamentals, by Ian Griffiths

ASP.NET 3.5 Fundamentals, by Fritz Onion

ASP.NET Ajax Fundamentals, by Fritz Onion

LINQ Fundamentals by Scott Allen

BizTalk Fundamentals by Matt Milner

BizTalk Server 2006 R2 Fundamentals by Jon Flanders

We'll be expanding this library of content in the months to come, as we continue to grow this online resource. I plan on adding modules on the Geneva family of identity products (Geneva Server, Geneva Framework, Geneva CardSpace) announced at PDC this week.

I've learned a lot of interesting tidbits as I helped to develop the back end infrastructure for Pluralsight On-Demand, and now that I'm not so crammed for time, I'll be sharing those insights here on this blog.

Congrats to all who helped bring this incredible resource to the public!

Aaron has more details if you want to know about pricing, customer feedback, and so on.

ScienceLogic: What is “BSM Lite” and how is it different from “heavy” BSM?

Doug McClure: I think the concepts that Peter Sevcik from Net Forecast initially outlined in his blog post sum up what “BSM Lite” is all about: a simpler, less expensive, more responsive way of achieving the goals and objectives of Business Service Management (BSM).  He’s contrasted this nicely against what he termed “BSM Heavy” being the larger investments in time and resources to deploy domain specific tools and solutions each providing a view into the business service delivery with some aggregation and consolidation to tie up all of the disparate tool’s information into a concise end-to-end business service management story.

I’m pleased that he leveraged some of my thinking around a better working definition of what BSM really is from the BSM Defined page on my blog. Of course, these definitions are going to vary depending on whom you talk with and how they see the overall BSM Maturity Model.  I’ve created a BSM Maturity Model that aligns with the famous Gartner IT maturity model.  I’d like to think that a “BSM Lite” solution is one attacking the low hanging fruit, enabling one to achieve value quicker, and in a more tactical manner.  The “BSM Heavy” solutions are capable of the same, but span all along the BSM Maturity Model by adding additional point solutions, products and technologies from their broader portfolio. 

ScienceLogic: Does “BSM Lite” just refer to the tools, or can it refer to the process and methodology as well?

Doug McClure: I think that BSM is as much a philosophy as it is technology, process, people and methodology.  If we can get people to think, operate and respond differently than they do today with a focus on the business, customers, quality, revenue, or whatever else is most important to their business goals and objectives, than that is Business Service Management and could be “BSM Lite” if you will. 

Being that I work for IBM Tivoli, one of my personal objectives is to identify ways to use our key BSM enabling products in a more efficient, effective and BSM centric way. This was a huge driver for trying to hold DevCampTivoli focused on “Collaborative Development of End-to-End BSM Solutions”. 

In my opinion, we don’t make things very easy for our clients and the answer can’t be to “buy this product, module or widget” to fill in the gaps.  In my opinion, we must establish a BSM overlay within IBM Tivoli’s development and product management organization that ensures that we have clearly thought about how to enable BSM with the hundreds or products that we sell.  In my opinion, every product release must incorporate the fundamentals of enabling BSM in addition to the core domain specific functionality intended. I hope to keep this spirit alive and get our smartest IBMers and clients thinking about the best way to take a “BSM Heavy” solution and make it “lighter”. I hope to share more about my plans here and guidance for the industry in general soon.

That said, I am always interested in consulting with clients and collaborate with peers in the industry to figure out how to get the focus on the people, process and technology as key components of their BSM strategies.  I am absolutely convinced that without a documented BSM strategy, roadmap and top level sponsorship within the business and IT, the chances of BSM success greatly diminish.

ScienceLogic: Given the complexities involved in implementing a BSM strategy and dealing with the people and processes components of any business, how does “BSM Lite” really work? Should the expectations and outcomes be “lite” as well?

Doug McClure: Time will tell if “BSM Lite” will work.  I’m seeing emerging companies that are already breaking down some of the barriers to BSM success.  I do not expect that those choosing to begin with a “BSM Lite” approach should expect “lite” outcomes. 

The outcomes are the same regardless of the approach IF you’ve got a documented BSM strategy, roadmap and top level sponsorship in place before you begin. New features, capabilities and technologies will be needed as the needs of the business change and companies mature in BSM and fundamental IT management. This will likely force companies to move in more “BSM Heavy” directions to fill those gaps. 

In my opinion, this is the ideal scenario now as it gives “BSM Lite” vendors opportunities to grow their products and solutions. It also GREATLY improves the chances for success with a “BSM Heavy” solution because the organization would have already had matured enough to approach a “BSM Heavy” solution than if they hadn’t done a “BSM Lite” solution in the past.

ScienceLogic: Is “BSM Lite” more appropriate for a small or midsized organization, or does it apply equally to large companies? Is there an ideal profile for a company that can successfully implement a BSM strategy? Is there a different profile for “BSM Lite”?

Doug McClure: From an economic perspective, the concepts of “BSM Lite” are appropriate for all companies.  Remember, with “BSM Lite” we’re focused on identifying ways to make the goals and objectives of BSM easier to implement and in a more cost effective way.  Any company concerned about their IT cost overhead should care about this, especially when the risks of starting out with a “BSM Heavy” type deployment are much greater and the time to value generally much longer.

The “ideal” profile for any company is one where the BSM initiative begins by establishing top level buy in through creation of a formal BSM strategy for the company. This BSM strategy personalizes how the company defines what BSM is, what value the company expects from it, and how it will use BSM as a competitive differentiator for delivery of its business and IT services, products, etc.

The organizational “profile” I’ve seen most successful is when implementing a BSM strategy originates from within or actively includes a group that many companies have now that serves as a liaison or relationship management role between the various lines of business and IT. Sometimes this group is often seen as the gatekeeper to filter (and hinder) business driven requirements into the IT organization. In the ideal scenario, this group works very closely with the business and IT (usually staffed by business people and not IT people) to understand both the business side and IT side of complex business services and applications. 

Apart from the traditional IT components, what this group can do is help IT really understand the business perspective.  Analysis of the impact on the business in business terms is only possible by collaborating with a group such as this.  True value oriented BSM becomes attainable when we get to this level of IT and business alignment, cooperation, collaboration and communication.

If BSM is an IT only initiative, this will likely result in an IT centric perspective severely lacking in the necessary business perspective.  In these cases where IT doesn’t invest their BSM efforts with the business as an equal partner, the implementation ultimately becomes a “CYA” tool for IT and not achieve the desired value oriented expected.

To some degree “BSM Lite” may have an entirely different profile. If we see the price points, complexity and time to value change significantly we may see these types of deployments originate exclusively within the Line of Business. The possibility may exist where large enterprises operating in a shared IT services or IT outsourcing type model that the Line of Business brings in a “BSM Lite” solution to gain the visibility, checks and balances needed to ensure that the LoB’s needs are being met from the internal/external provider. I’d envision that “BSM Lite” may even be capable of operating within a “SaaS” model or other managed service type offering where the price points are below the signing levels triggering broader IT involvement and review.

To Be Continued…

ShareThis

Why? Well, the problem with layered security is that we tend to assume if Layer X isn’t providing a particular protection, Layer Y must be… and we all know what assuming does.

In the good ol’ days, we relied on firewalls- perhaps nested firewalls, or ones positioned strategically on the LAN as well as the WAN. Because of our network architecture at the time, that was the primary (and probably only required) protection. After years of de-perimeterization and the increase of threats from both remote-access and insiders, we have a much different landscape.

The addition of resources and availability in the network has lead to the addition of vulnerabilities and threats.

Now… our schools need to protect children from material online. Now… we need to stop Trojans from sneaking in with VoIP apps. We need to access our corporate network securely from Starbucks. Our corporations need to protect their network from users accessing or publishing illegal content on the Internet. We need to protect our email, make sure its virus-free and not allowing employees to send sensitive information to the outside world.

All these increased risks and threats lend to the need for more protection in the environment. There’s just no single silver bullet or cure-all for the problems we’re facing.

What does this mean? It means we’re adding security products to the network to address these issues. We need content filtering. We need layer-7 visibility on the WAN for inbound/outbound application control. We need data leakage prevention. We need email security. We SSL-VPNs for secure remote access… the list goes on.

So, what’s the problem? We’re living in a world of security buzzwords and ‘hot topic’ solutions. But the problem is 2-fold.

Problem 1- We forget to KISS IT. In the frenzy to understand and implement these hot new products, we’re losing sight of some basic security functions and overlooking some really important security fundamentals. Remember to KISS IT and keep your basic security solutions simple- then layer on top of that. Your hot new NAC or DLP solution won’t seem so impressive if your basic firewall rules haven’t been properly configured.

Problem 2- We forget thy layers. After you KISS IT, you need to start layering responsibly. That means having a CLEAR understanding of what each solution does- or does not- do. You wouldn’t believe how many customers call and want to hear about Widget A for a certain solution that Widget A is not designed to fix. I deal with it daily and I blame (for the most part) vendors for mis-advertising their product as a fix-all. Whether its hardware or software- know what each piece of your security solution is designed to do, what it’s actually doing, and keep that information documented. Documented- I’m going to say it again. Your firewall/UTM may offer content filtering and gateway AV, but are you using it? Are you using a WAN optimization product to stop prohibited applications, or is your web filter doing that? Do you even know?

Solving the Cube. Layered security is like solving a Rubik’s Cube. You may think you’re on the right track after you get one side solved… but the other 5 are just a huge mess. There are patterns and algorithms you must follow to solve all sides together. Your layered security solution is no different. Understand what each piece is doing, how it fits in, and when to twist one layer here to implement a solution as part of a different layer over there.

# # #

There is considerable value in separating out the two concepts: in explaining how the two are different, and understanding when we're referring to one and when the other. There is value as well in recognizing when the two converge, understanding why they diverge, and knowing how they can be made to converge again.

Some fundamentals first. Viewed from the perspective of economics, security is a trade-off. There's no such thing as absolute security, and any security you get has some cost: in money, in convenience, in capabilities, in insecurities somewhere else, whatever. Every time someone makes a decision about security -- computer security, community security, national security -- he makes a trade-off.

People make these trade-offs as individuals. We all get to decide, individually, if the expense and inconvenience of having a home burglar alarm is worth the security. We all get to decide if wearing a bulletproof vest is worth the cost and tacky appearance. We all get to decide if we're getting our money's worth from the billions of dollars we're spending combating terrorism, and if invading Iraq was the best use of our counterterrorism resources. We might not have the power to implement our opinion, but we get to decide if we think it's worth it.

Now we may or may not have the expertise to make those trade-offs intelligently, but we make them anyway. All of us. People have a natural intuition about security trade-offs, and we make them, large and small, dozens of times throughout the day. We can't help it: It's part of being alive.

Imagine a rabbit, sitting in a field eating grass. And he sees a fox. He's going to make a security trade-off: Should he stay or should he flee? Over time, the rabbits that are good at making that trade-off will tend to reproduce, while the rabbits that are bad at it will tend to get eaten or starve.

So, as a successful species on the planet, you'd expect that human beings would be really good at making security trade-offs. Yet, at the same time, we can be hopelessly bad at it. We spend more money on terrorism than the data warrants. We fear flying and choose to drive instead. Why?

The short answer is that people make most trade-offs based on the feeling of security and not the reality.

I've written a lot about how people get security trade-offs wrong, and the cognitive biases that cause us to make mistakes. Humans have developed these biases because they make evolutionary sense. And most of the time, they work.

Most of the time -- and this is important -- our feeling of security matches the reality of security. Certainly, this is true of prehistory. Modern times are harder. Blame technology, blame the media, blame whatever. Our brains are much better optimized for the security trade-offs endemic to living in small family groups in the East African highlands in 100,000 B.C. than to those endemic to living in 2008 New York.

If we make security trade-offs based on the feeling of security rather than the reality, we choose security that makes us feel more secure over security that actually makes us more secure. And that's what governments, companies, family members and everyone else provide. Of course, there are two ways to make people feel more secure. The first is to make people actually more secure and hope they notice. The second is to make people feel more secure without making them actually more secure, and hope they don't notice.

The key here is whether we notice. The feeling and reality of security tend to converge when we take notice, and diverge when we don't. People notice when 1) there are enough positive and negative examples to draw a conclusion, and 2) there isn't too much emotion clouding the issue.

Both elements are important. If someone tries to convince us to spend money on a new type of home burglar alarm, we as society will know pretty quickly if he's got a clever security device or if he's a charlatan; we can monitor crime rates. But if that same person advocates a new national antiterrorism system, and there weren't any terrorist attacks before it was implemented, and there weren't any after it was implemented, how do we know if his system was effective?

People are more likely to realistically assess these incidents if they don't contradict preconceived notions about how the world works. For example: It's obvious that a wall keeps people out, so arguing against building a wall across America's southern border to keep illegal immigrants out is harder to do.

The other thing that matters is agenda. There are lots of people, politicians, companies and so on who deliberately try to manipulate your feeling of security for their own gain. They try to cause fear. They invent threats. They take minor threats and make them major. And when they talk about rare risks with only a few incidents to base an assessment on -- terrorism is the big example here -- they are more likely to succeed.

Unfortunately, there's no obvious antidote. Information is important. We can't understand security unless we understand it. But that's not enough: Few of us really understand cancer, yet we regularly make security decisions based on its risk. What we do is accept that there are experts who understand the risks of cancer, and trust them to make the security trade-offs for us.

There are some complex feedback loops going on here, between emotion and reason, between reality and our knowledge of it, between feeling and familiarity, and between the understanding of how we reason and feel about security and our analyses and feelings. We're never going to stop making security trade-offs based on the feeling of security, and we're never going to completely prevent those with specific agendas from trying to take care of us. But the more we know, the better trade-offs we'll make.

There is considerable value in separating out the two concepts: in explaining how the two are different, and understanding when we're referring to one and when the other. There is value as well in recognizing when the two converge, understanding why they diverge, and knowing how they can be made to converge again.

Some fundamentals first. Viewed from the perspective of economics, security is a trade-off. There's no such thing as absolute security, and any security you get has some cost: in money, in convenience, in capabilities, in insecurities somewhere else, whatever. Every time someone makes a decision about security -- computer security, community security, national security -- he makes a trade-off.

People make these trade-offs as individuals. We all get to decide, individually, if the expense and inconvenience of having a home burglar alarm is worth the security. We all get to decide if wearing a bulletproof vest is worth the cost and tacky appearance. We all get to decide if we're getting our money's worth from the billions of dollars we're spending combating terrorism, and if invading Iraq was the best use of our counterterrorism resources. We might not have the power to implement our opinion, but we get to decide if we think it's worth it.

Now we may or may not have the expertise to make those trade-offs intelligently, but we make them anyway. All of us. People have a natural intuition about security trade-offs, and we make them, large and small, dozens of times throughout the day. We can't help it: It's part of being alive.

Imagine a rabbit, sitting in a field eating grass. And he sees a fox. He's going to make a security trade-off: Should he stay or should he flee? Over time, the rabbits that are good at making that trade-off will tend to reproduce, while the rabbits that are bad at it will tend to get eaten or starve.

So, as a successful species on the planet, you'd expect that human beings would be really good at making security trade-offs. Yet, at the same time, we can be hopelessly bad at it. We spend more money on terrorism than the data warrants. We fear flying and choose to drive instead. Why?

The short answer is that people make most trade-offs based on the feeling of security and not the reality.

I've written a lot about how people get security trade-offs wrong, and the cognitive biases that cause us to make mistakes. Humans have developed these biases because they make evolutionary sense. And most of the time, they work.

Most of the time -- and this is important -- our feeling of security matches the reality of security. Certainly, this is true of prehistory. Modern times are harder. Blame technology, blame the media, blame whatever. Our brains are much better optimized for the security trade-offs endemic to living in small family groups in the East African highlands in 100,000 B.C. than to those endemic to living in 2008 New York.

If we make security trade-offs based on the feeling of security rather than the reality, we choose security that makes us feel more secure over security that actually makes us more secure. And that's what governments, companies, family members and everyone else provide. Of course, there are two ways to make people feel more secure. The first is to make people actually more secure and hope they notice. The second is to make people feel more secure without making them actually more secure, and hope they don't notice.

The key here is whether we notice. The feeling and reality of security tend to converge when we take notice, and diverge when we don't. People notice when 1) there are enough positive and negative examples to draw a conclusion, and 2) there isn't too much emotion clouding the issue.

Both elements are important. If someone tries to convince us to spend money on a new type of home burglar alarm, we as society will know pretty quickly if he's got a clever security device or if he's a charlatan; we can monitor crime rates. But if that same person advocates a new national antiterrorism system, and there weren't any terrorist attacks before it was implemented, and there weren't any after it was implemented, how do we know if his system was effective?

People are more likely to realistically assess these incidents if they don't contradict preconceived notions about how the world works. For example: It's obvious that a wall keeps people out, so arguing against building a wall across America's southern border to keep illegal immigrants out is harder to do.

The other thing that matters is agenda. There are lots of people, politicians, companies and so on who deliberately try to manipulate your feeling of security for their own gain. They try to cause fear. They invent threats. They take minor threats and make them major. And when they talk about rare risks with only a few incidents to base an assessment on -- terrorism is the big example here -- they are more likely to succeed.

Unfortunately, there's no obvious antidote. Information is important. We can't understand security unless we understand it. But that's not enough: Few of us really understand cancer, yet we regularly make security decisions based on its risk. What we do is accept that there are experts who understand the risks of cancer, and trust them to make the security trade-offs for us.

There are some complex feedback loops going on here, between emotion and reason, between reality and our knowledge of it, between feeling and familiarity, and between the understanding of how we reason and feel about security and our analyses and feelings. We're never going to stop making security trade-offs based on the feeling of security, and we're never going to completely prevent those with specific agendas from trying to take care of us. But the more we know, the better trade-offs we'll make.

This article originally appeared on Wired.com.

On the 22nd episode of The Silver Bullet Security Podcast, Gary interviews Ed Amoroso, Chief Information Security Officer of AT&T. They discuss how Peter Neumann influenced Ed, the difference between bugs and flaws and whether bugs are getting too much attention, the propensity for confusion around how security actually works, privacy, security, and monitoring, and software correctness/quality vs software security. They also discuss the Hugh Thompson show now airing on AT&T’s Tech Channel.

• Cyber Security
• Fundamentals of Computer Security Technology
• Silver Bullet Interview with Peter Neumann
• AT&T’s Tech Channel
• Gary on The Hugh Thompson Show