Security Question of the Week, 1/8/10 Edition

---

That’s right folks! It’s time for another installment of Security Question of the Week¹! This week’s edition: an exciting blog post about the OWASP ESAPI secure storage API! Yay!

For those of you playing along at home, that link goes here — to a post titled Secure Storage using the OWASP ESAPI at the Security Ninja blog.

Disclaimer

So, before we begin, it’s worth being nice and pointing out that these criticisms are meant to be taken in a constructive spirit. In particular, I think we all ought to agree that the ESAPI project is a great idea whose time has come — and that posts like Security Ninja’s, which help programmers figure out how to use ESAPI in the real world, are equally necessary. I should also mention that there are almost certainly mistakes in what follows; if you have anything to add or correct, let me know.

That said, security is hard, and I think there are a bunch of problems in the API — and Security Ninja’s post exemplifies a few of them quite nicely. So…

OK, Let’s Go!

Let’s meet our first contestant, Marc. Welcome, Marc! Marc tweets:

Convincing answers! But whatever could they mean? Let’s investigate.

“Secure access to the master salt”

In the blog post, Security Ninja warns us:

“I cannot stress the importance of making this [master salt] a sufficiently strong/random value and securing access to the value itself wherever you choose to store it”

The ESAPI hash function JavaDoc also seems to suggest that salts need to be secret:

“Some good choices for a salt might be … [a] string that is known to the application but not to an attacker.”

And, finally, the Wikipedia article on salts agrees:

“For best security, the salt value is kept secret, separate from the password database. This provides an advantage when a database is stolen, but the salt is not. To determine a password from a stolen hash, an attacker cannot simply try common passwords (such as English language words or names). Rather, they must calculate the hashes of random characters (at least for the portion of the input they know is the salt), which is much slower.”

To which I say: hogwash. Total nonsense. Wrong.

The point of a salt is not secrecy. The point is to defend against precomputation (“rainbow table“) attacks. Salts achieve this by introducing a bit of extra variance into the mix: if an attacker wants to compute the hash values for the 1,000 most common passwords, and no salt has been included, then she only needs to compute 1,000 hashes. If there’s a 1 bit salt included, though, her work doubles: she’ll have to compute 1,000 hashes to cover the case where the salt is 0, and another 1,000 to cover the case where the salt is 1. With a 2-bit salt, 4,000 hashes are required; with a 1-byte salt, 256,000 hashes are required, and so on. A salt even a few bytes long will make computing (and storing) rainbow tables literally millions of times harder.

But it’s worth pointing out that salts have this property whether or not the salt is a secret. It really doesn’t matter if the attacker knows what the salts are: so long as the salt value varies independently from password to password, computing a general-purpose rainbow table will still require computing a hash for every possible combination of candidate password and possible salt. In fact it’s important, in at least an abstract sense, that the system not depend on the secrecy of salts: Kerckhoffs’ principle reminds us that the security of the system should reside only in the key (password), and that the system should not be impacted if the enemy learns the details. Security through obscurity and all that.

As long as we’re talking about this, it might be interesting to point out that most real-world cryptosystems make no effort to protect password salts. UNIX shadow password files embed the salt in the hash string, for example, and WPA-PSK, the fancy new(-ish) WiFi security standard, uses the SSID as the salt. Which is to say, it literally broadcasts the salt to everyone in range. Salts are not secrets.²

So, anyway, back to Marc’s answer. I’m going to give it 5 points as a nice reasonable starting point, but then add a “stupid Wikipedia advice” bonus multiplier, bringing the total for this answer up to, say, 10 points. A strong start for our first contestant! Let’s see what happens next!

“Two salts”

Yep. The ESAPI code distinguishes between a “master salt”, configured in a central .properties file, and the per-password salt passed to the hash() function. But this is just silly: having an extra salt adds nothing but fragility to the system. Bad idea for ESAPI — but another solid observation from Marc! 6 points!

“Write your own hashing code”

Ouch! This is the final nail in the ESAPI hash() coffin, and it’s a doozy. The fun part is that the javadoc for the method even refers us to the good Mr. Ptacek’s article on secure password storage — an article which happens to contain the none-too-subtle exhortation

No, really. Use someone else’s password system. Don’t build your own.

…and what did the ESAPI guys do? They built their own. Let’s take a look at it now:

Yep, that’s kind of exactly what you don’t want to do. All custom and hand-crafted and such. What’s wrong with PBKDF2 or bcrypt? The differences are notable as well: for example, this key-stretching code here doesn’t have an ‘iteration’ or ‘cost’ parameter to tune as computers get faster. Even if we stipulate that 1024 is a reasonable number of iterations on today’s hardware,³ it wont be 5 or 10 years from now. But given this code, there’s no way to change that constant at a later date — nor is there any way to tell what iteration constant was used for a given password hash. If you use this, you’re gonna be stuck with this.

It also seems odd that the various extra rounds of hashing don’t make any use of the password: I’d sort of expect that you’d want to make the computation depend on the password a bit more. Does that matter? No idea. I’m not a cryptographer. Which is why I don’t go around designing my own password hashing functions. This is precisely Ptacek’s point: the details of these things are hard to get right, and there are already plenty of good options. So why build your own?

This answer gets a solid 10 points, plus a 5 point “death-blow” bonus, for a total of 15 points. Marc takes an early lead! Can he hold on to it? Let’s find out!

“Hide configuration file naming the algo”

Security Ninja quotes Jeff Williams as saying,

“The SecurityConfiguration interface stores all configuration information that directs the behavior of the ESAPI implementation. Protection of this configuration information is critical to the secure operation of the application using the ESAPI. You should use operating system access controls to limit access to wherever the configuration information is stored.”

At first glance, this might seem to be another violation of Kerckhoffs’ principle, aka an example of “security through obscurity.” To the extent that “security configuration” means stuff like algorithm names and the like, it should be perfectly okay to broadcast that to the world: the security of the system should not rely on these parameters being secret.

However, I’m inclined to take a somewhat more charitable view: I suspect that Jeff meant that the configuration needs to be protected because that’s where you configure the MasterPassword and possibly other secrets. And these do need to be kept secret.

Of course, there’s the obvious point that configuring both secret things and not-very-secret things in the same place encourages fragility. You might want to track the generic ESAPI settings in source control, for example — but the password is in there, and so checking the file in wouldn’t be so great. I think this is a valid point, but it’s not terribly strong. 2 points.

“Overuse of ‘ninja’”

This would be a good point, but for Marc’s own checkered past with the word. 0 points.

At the close of our first round, Marc has an impressive 33 points!

Exciting! Thrilling! Etc! Stay tuned!

A Challenger Appears!

Our second contestant tonight is Coda, who hails from Berkeley, CA. Howdy, Coda! What can you tell us?

Solid answers here, folks. And notice the style: Coda is dropping acronyms and sentence fragments like nobody’s business: he’s not asking questions, he’s telling it like it is. That full-stop at the end is particularly final, showing real conviction. What confidence! What showmanship! What an exciting contest!

DES, MD5

A powerful first answer from Coda, as he points out that no one should be using these algorithms anymore. Seriously. DES can be brute-forced in a week, and MD5 collisions can be found in 15 minutes. These algorithms haven’t been good choices since the early 90s. 6 points each for these important observations; add in the one-two comma splice combo multiplier to get 18 points

No integrity checks

For his next move, Coda tacks hard into crypto-nerd territory, pointing out that the decrypt() method doesn’t do any message authentication. Which is to say, it doesn’t provide any native protection against chosen ciphertext attacks. If a bad guy can control what gets passed to decrypt() and see what comes out the other side, there are a whole bunch of fun attacks available to her.

What does a chosen-plaintext attack look like, you ask? That’s a great question, because it allows me to refer you once again to the inimitable Mr. Ptacek, whose prime-time AES melodrama features a nice explanation of a chosen-plaintext attack against PKCS#7. To prevent this, the encryption code should wrap the ciphertext (or plaintext) in a MAC would prevent this attack — or else use the block cipher in EAX mode — and the decryption code should check it. Alas, ESAPI does neither of these things.

This is a solid observation from our challenger: 10 points, plus a crypto-nerd bonus to total 18 points!

No desription of where the secrets are stored

I take it that Coda is here referring to the “master secret” used in encryption/decryption. The blog post doesn’t make this especially clear, but this secret is stored in the system-wide ESAPI.properties file, and the “Security Configuration” section at the bottom of the page does provide a bit of guidance on how to protect that file. (To wit: ‘you should protect that file,’ more or less.)

So there is, in fact, a little bit of a description about where the secrets are stored — and given the problem, the system-wide .properties file with restrictive permissions is about as good an answer as you’re going to get. I would probably score this observation as 0 points, but for the fact that Coda can teach us all a thing or two about secret storage. Instead, we’ll give him the benefit of the doubt, plus 2 points.

Coda pulls into the lead with 38 points!

Impressive! Astonishing! Exciting! Stay tuned!

Dénouement

So. We’ve seen an impressive array of problems and observations about the ESAPI storage API. Pulling them all together, it’s time for a bit of meta-critique. What have we learned?

ESAPI reinvents too many (crypto) wheels

From password storage to encryption routines, the ESAPI reference implementation uses custom crypto for just about everything. The algorithms used are, of course, implemented by the JCE provider — the ESAPI folks aren’t crazy — but the authors apply those algorithms in somewhat naive ways. And this leads to some drawbacks. So, for example, the hash() function implements its own key-stretching code which is less flexible and future-proof than more mainstream alternatives; or again, the encrypt() method doesn’t authenticate the cipher- or plaintext, opening a window to attacks.

The best strategy for dealing with these problems is to avoid them. Outside the context of research, it’s hard to see why even highly capable cryptographers would bother building a new crypto building blocks.⁴ And there really are plenty of good options already available — PBKDF2 and OpenPGP’s PBE would be my choices for these particular problems. Because the ESAPI code rolls its own routines, though, it’s very hard to evaluate whether or not it provides enough protection for a given threat model…

ESAPI has too many fiddly knobs to configure

In digging through the ESAPI code, I found quite a few different settings and secrets that get configured in ESAPI.properties — things like the master salt/secret and preferred encryption algorithm, but also internal regexes and PRNG algorithms. I don’t think that these things should be options at all: a security API should just “do the right thing” and be done with it, all the time. Making these things configurable is, at best, really confusing — and at worst, it’s really dangerous.

For example, I spotted this in the current SVN trunk version of ESAPI.properties:

Essentially, that boils down to: “Here is a knob. Its meaning and purpose are obscure. If you ever change it, you will be screwed.” Things like this are serious misfeatures of the ESAPI.

ESAPI is way too complicated

Similarly, the code required to set up, configure, and use ESAPI is way too complicated. The detail of Security Ninja’s post is evidence enough of this: ideally, using and understanding the API wouldn’t require reading such lengthy blog posts. I realize that the words “enterprise” and “Java” bring with them an expectation of configurability and flexibility, but I think this is a Bad Thing. Not only is it unlikely that the developers using ESAPI will be able to make informed decisions about fixed IVs and authenticated encryption options — it’s also unlikely that even the software security group or external auditors will be able to make the right call about this stuff.

Ideally, I think, a security API should expose very few, simple interfaces which provide a “batteries included” approach to security. So, for example, there should not be a hash() function which requires you to pass in a salt along with the plaintext — instead, there should be a hashCreate() (or whatever) method which generates a random, strong salt for you and embed it in the hash value along with any other parameters necessary (cost factors or algorithm identifiers, say). A separate hashCompare() method would then be responsible for checking a given plaintext value against the hash value, hiding all the details from the programmer. That’s a narrow point; the general one is: don’t make developers think about this stuff; instead, do the right thing for them, and make it hard to screw up.

ESAPI is improving

Digging through the code also shows that the ESAPI code in SVN has improved rather significantly from the v1.4 code we looked at in Security Ninja’s post. For example, the properties setting immediately following the above seems to control setting a ciphertext MAC,⁵ thus (presumably) remedying Coda’s second point above. It also sounds like the default encryption algorithm has been changed to the more modern AES-128, another sensible move. I haven’t looked through the v2.0 code extensively, but it’s good to see some of the problems identified have been fixed already.

Conclusion

So, at the end of all this, what are we left with? We’re left with the fact that Coda wins the prize! A round of beer, redeemable at next week’s BaySec, say? And the honorable second-place runner up mention award goes to Marc. Yay everyone!

I’ll post next week’s question on Twitter on Monday.