I am terrified that I do not consider myself competent enough to write a crypto library, and yet there isn't a single mention - in this article, nor at the time of writing the comments here on Hacker News - of many of the pitfalls I know to avoid when undertaking such an endeavour. There is even a list of "you have to do A, B, C, and that's about it" that is missing some major - well known, even! - items.
I know "don't roll your own crypto" comes across as dismissive or patronising. I'm not a huge fan of the phrasing. But as a first-order approximation, it is correct. If you want to roll your own crypto, that needs to be your _thing_. It's very unlikely you're going to be a fantastic full-stack web developer _and_ be able to do that. If it's really what you want to do then awesome! Go study it, learn it, practice it. But if you're doing it as a side-hobby, never put it into production.
Hey, like you I decided to dive into cryptography coming from a different background. Although this quote is not directly related to your problem it can be safely applied to it:
"Almost certainly you will get the urge to invent new cryptographic algorithms, and will believe that they are unbreakable. Don't resist the urge; this is one of the fun parts. But resist the belief; almost certainly your creations will be breakable, and almost certainly no one will spend the time breaking them for you. You can break them yourself as you get better."[1]
The problem that Schneier is referring to is that doing a careful analysis (that is required if you want to use your crypto in production) is tedious, time consuming, and requires expertise in the area to know most blank spots of the algorithms (heck a single one is hard enough). That's why he recommends you to be have enough experience breaking many algorithms before you make any serious claim about your the safety of your crypto.
So all in all it's great that you got the interest in the area, there is a lot of work needed in OSS. But be very careful about claiming that your lib is ready for production. What you got is a bunch of volunteers to glance at your code. Few cryptographers (if any) will seriously try to break it.
Note that the author is not inventing crypto algorithms, rather implementing them (although the part about XChaCha20 being a mix of ChaCha20 and XSalsa20 is IMHO dancing on the line). Still a risky business, and tricky to get right, but several orders of magnitude safer than "hey, what if we just XORed everything with a random number? Unbreakable, eh?"
> the part about XChaCha20 being a mix of ChaCha20 and XSalsa20 is IMHO dancing on the line
It was a few months ago. Then Libsodium danced on the same line, and I was able to compare the results (they behave the same, phew).
It's not inventing crypto either: it's a straightforward application of what was done to Salsa20. The security reduction that worked for XSalsa20 (guaranteed secure if Salsa20 is secure) applies to XChacha20 as well.
> Then Libsodium danced on the same line, and I was able to compare the results (they behave the same, phew).
Do they really? What about timing attacks? Does all your code run in constant time? If not, there's a whole class of vulnerability that basically will only be found when experienced attackers have a chance to poke at it. Timing attacks are a great example of why "don't roll your own crypto" is sound advice. No amount of testing your library against a reference library will uncover them.
Can't say for sure, but it look like they didn't have an independent implementation to compare to. Just like me a few months a go.
---
> What about timing attacks?
Monocypher is immune.
First, the primitive were chosen precisely because they are easily immunised against timing attacks.
Second, this is easily checked simply by looking at the source code: no secret-dependent branches, no secret-dependent indices, and that's about it. One doesn't need to be an expert to check that, a junior programmer could do it after being told what to look for.
What's wrong with xoring with a stream of random numbers? Isn't it how stream ciphers work? Get a good cryptographic RNG, initialize it properly with a long enough key and you should be fine.
You're right about the theory, but you're also wrong about how to implement - which is the point of telling people not to roll their own crypto.
What about reseeding or prediction resistance? How do you handle biased entropy input from that hairdryer someone is blowing on your chips? Oops, the quantum random number generator card doesn't work anymore after adding that extra GPGPU to the system.
You wouldn't want true random numbers anyways. If you're encrypting it, then you'll want to decrypt it later. That means you'll need the random data you used again. If you had a secure channel over which to send or store the random data, then you could just use that channel to send or store the plaintext itself (note that the random data will be the same length as the plaintext). Such a system could only be useful if the cryptographic key was shorter than the plaintext. So it would have to be a deterministic RNG, and you would only need the seed and the ciphertext in order to decrypt.
Also, if someone has acccess to the hardware, then you're screwed. No one would bother wasting time with a hairdryer. If the plaintext was on your harddrive, they would just grab that and run.
I think you missed the point of the comment, which wasn't about physical possession of a computer.
You also make assumptions, such as chips being inside the computer, when they can in fact be in an outside environment-based RNG. The hairdryer example is the canonical and prototypical weakness in that particular type of "rng". I'll try to be explicit about that in the future.
I didn't write anything about environment-based RNG. RNG must be deterministic and seeded by preshared encryption key. One point of using a cryptographically secure RNG is that it should not be possible to learn the seed by observing the output of the RNG without bruteforcing the key.
If you had a RNG that sampled some hardware noise but then you altered the environment by introducing extreme temperatures, maybe the RNG would end up with biased output. A simple example: you can use a camera as a good source of randomness, even taking a picture of a wall indoors. But if an attacker was able to flood that room with light and overwhelm your sensor, he could predict the image would be 100% white pixels or near to it and guess the random output.
But the comment is exaggerating. Encryption algorithms do not handle the case of your entropy source being compromised. Good RNG algorithms like Yarrow try to mitigate this somewhat.
Even if you need a real entropy-based RNG, e.g. for key generation, you need to gather entropy once at the beginning to seed the RNG. Then running RNG does not need any more entropy, so your hairdryer trick won't work. A common myth I hear is that "entropy" gets "depleted" during RNG operation, but in fact it does not. You only need enough bits of entropy once at the beginning, equal to the internal RNG state length.
Strong RNG designs re-seed to deal with compromise or poor seeding. That way if an attacker has temporary read access they do not permanently compromise the system going forward. Or if on boot you are in a compromised or low entropy environment the system will improve over time. Perhaps the utility of this is arguable. If an attacker is able to see your private RNG state then it is most likely game over. But it is a cheap operation with little downside and again, designs like Yarrow put considerable time into thinking it through.
Well, to be fair, a good implementation should throw some simple statistical sanity checks at RNG input during self-test and throw an error when it doesn't pass. Passing such tests doesn't indicate that the RNG is cryptographically secure, but failing them indicates that it is insecure.
Nothing wrong with what you wrote - but note the subtle difference between "XORing a stream of random numbers" (potentially viable, even unbreakable when using a OTP) and "XORing a random number" (essentially a Caesar cipher, kid-sister encryption).
> note the subtle difference between "XORing a stream of random numbers" (potentially viable, even unbreakable when using a OTP) and "XORing a random number" (essentially a Caesar cipher, kid-sister encryption)
No, that's not a difference. A "stream of numbers" is just one very large number. What you're worrying about is how large the numbers are, not whether there's one or more than one.
Original point was certainly to point out the most common mistake of typical homegrown cryptosystems. Even programmers who know that XORing stuff with constant byte is not viable encryption can produce systems that do essentially the same thing (usually by XORing everything with RC4 output with long lived or even constant key)
And instead of saying nothing those who harp on "C's pitfalls" could politely enumerate those relevant to the issue at hand, that way we actually have something discrete to discuss.
I apologize if that sounds dismissive (it's not intended to) but your post is not more than the usual vague FUD. Take a look at so-called 'professional' cryptographers and their libraries, and you'll find that practically all of their code had severe bugs. Professionals make errors like everyone else, 'professional' just means you're being paid for it. You will have a hard time finding a professional and widely used crypto library that wasn't essentially compromised in one of its functions at one time or another. Not only that, professionals from the closed-source department have a long history of coming up with compromised or bogus, sometimes even ridiculous cryptographic algorithms and implementations. Two typical examples: CMEA cell phone encryption, and the Crypto AG backdoor debacle.
Surely people who invent and implement cryptographic algorithms for a living can be expected to be better than your run-off-the-mill programmer, but as I've said elsewhere implementing crypto is also not a black art. It requires about the same level of skill as e.g. implementing a production-ready low-level network protocol or a file system. That rules out many programmers but not all. That concerns implementations. As for algorithms, their development requires extensive experience in cryptanalysis (not just applied cryptography!). Some pros have that, others don't.
Last but not least, many popular and widely used crypto libraries started as a side hobby. For example, libtomcrypt started that way. In fact, many widely respected cryptographers started as hobbyists. For example, Bruce Schneier. Of course, the ones who are widely accepted as peers are also those who are in the 'hire me as a consultant, not the other guy' business, and they will naturally advise everyone to not roll their own crypto but to rely on their expertise...
As I said I'm not a fan of the phrase "don't roll your own crypto". I don't disagree that that (obviously!) prevents new crypto libraries being written. But if you're going to do it, you damn well do it right, and these days that means an amount of work that would likely total millions of dollars. If you aren't going to do it right, don't cut a 1.0 and use it in production.
Sidenote: how we started these things historically is not relevant anymore. We have learned a lot since then. Failing to "stand on the shoulders of giants" and learn from history is one the biggest factors that makes home-rolled crypto so easy to break - they are doomed to repeat the mistakes of history. Mistakes that are nowadays well documented and well known.
Having implemented crypto algorithms for educational and enjoyment purposes, I've always looked at "don't roll your own crypto" as less having to do with the actual writing of code which, as you state, isn't really that hard. Instead, what that statement is saying is that when you write your own code, you'll almost never get enough attention from the white-hat community to have any confidence that the code can be used in production.
You're right that the code written by professionals has lots of bugs. Bug is a term that describes a problem in code that has been found. With crypto software, the danger isn't the problems that get found (and then addressed), it's the problems that aren't found. "Roll your own" solutions have problems that never get found. That's the danger and it has nothing to do with the skill of the programmer.
The thing that normally nobody says, yet most the people telling you to not roll your own crypto assume you will understand is that you should rely only on well vetted libraries.
Rolling a usable crypto library is not a single person endeavor. Your own crypto won't be peer reviewed, thus you should not trust it.
> Take a look at so-called 'professional' cryptographers and their libraries, and you'll find that practically all of their code had severe bugs. Professionals make errors like everyone else, 'professional' just means you're being paid for it. You will have a hard time finding a professional and widely used crypto library that wasn't essentially compromised in one of its functions at one time or another. Not only that, professionals from the closed-source department have a long history of coming up with compromised or bogus, sometimes even ridiculous cryptographic algorithms and implementations. Two typical examples: CMEA cell phone encryption, and the Crypto AG backdoor debacle.
This doesn't diminish the claim that the modal individual should not attempt to "roll their own crypto." You're pointing out that professionals make mistakes, but of course they do - that indicates nothing about the propensity for amateurs to make mistakes.
> It requires about the same level of skill as e.g. implementing a production-ready low-level network protocol or a file system.
I have never implemented a low level network protocol or a file system (although at least the former sounds like a fun project) - so I cannot claim you're wrong here. However, I will point out that a network protocol is not designed to be error-proof in an intentionally antagonistic environment, which is a difficulty unique to cryptographic software. There are incentives involved in breaking crypto code that do not present themselves in other areas of software development, even if the programming difficulty itself is not entirely dissimilar.
> Last but not least, many popular and widely used crypto libraries started as a side hobby. For example, libtomcrypt started that way. In fact, many widely respected cryptographers started as hobbyists. For example, Bruce Schneier. Of course, the ones who are widely accepted as peers are also those who are in the 'hire me as a consultant, not the other guy' business, and they will naturally advise everyone to not roll their own crypto but to rely on their expertise...
They can start as side hobbies, but they are not going to remain that way for any meaningful amount of time if they are to be widely deployed and safe. You can only choose two out of those three.
More importantly, there is no cabal of cryptographers spreading FUD to line their own pockets with consulting fees. The cryptography consulting industry is very small compared to the actual security consulting industry. In the application security consulting industry I do believe there are firms that try to secure business this way, but that industry is much larger (and has firms which try to misrepresent cryptographic competency). I have had significant interactions with engineers at Riscure and NCC Crypto, which is a sizable portion of all the "real" cryptanalytic consulting work that occurs (at least in the United States) - they never struck me as being the sort to suggest what you're saying. That leaves Rambus/CR and a select few other firms doing real crypto consulting, which leads me to believe the industry is, on the whole, very legitimate.
> a network protocol is not designed to be error-proof in an intentionally antagonistic environment
Err, really? That may be the case sometimes. Also these days many things can be antagonistic: anything online is exposed to adversaries. So is anything that routinely reads untrusted input on everyone's computer (like a video player, or a pdf reader).
Thinking of the sheer amount of potentially vulnerable code out there makes me a little scared.
But is the code you're speaking of the security-specific code, or the network-specific code? If you attack a networking protocol, what are you attacking if not the associated cryptography? A networking protocol without any cryptography doesn't really need to be attacked because it's already wide open.
You introduce the confidentiality, authentication and integrity (which is really just going to be part of the authentication) through cryptography and security-specific code.
I'm not saying people don't attack networking protocols, or that there is no incentive for doing so. I'm saying that the portion that plays the defensive role is in the security software, not the overlying networking protocol.
Anything that accepts data from potentially-malicious sources is security-critical if it's itself privileged or produces data/events that go to something else privileged. Such analysis is usually used to determine the "Trusted Computing Base" (TCB) of a system that encompasses everything that might be used to break the security policy. All people who do real, security engineering keep it in mind, minimize it, strengthen it, and document what's left.
Here's an example from high-assurance field of what that looks like where they prototype an architecture for VPN's that minimizes TCB of key components:
Also shows their TCB is a lot bigger than their tiny kernel. Fortunately, there's been high-assurance developments to solve most of those things. I've seen robust, networking stacks but don't think they're high-assurance yet. If they're complex and done in C, here be dragons.
I was thinking of attacking the implementation of whatever processes attacker controlled input. If sending 100 pings in a row triggers a stack overflow, this will interest the attacker.
I would say that security specific code would be any code that directly processes the untrusted inputs. Parsers and decoders, mostly. You may want to isolate them from the rest of the program (ideally a separate address space), and make sure the interface between them and the rest of the program is much simpler than the input they are processing.
At least, you want to ensure such code either produces sound output for the rest of the program or fails cleanly. The attacker will have a harder time exploiting a bug deeper in to the program, I think —hope.
While I actually mostly agree with you, I feel I need to point this out... You write:
> I will point out that a network protocol is not designed to be error-proof in an intentionally antagonistic environment, which is a difficulty unique to cryptographic software.
But that's actually not true, at least not anymore. People attack protocols on the internet all the time, and it's a difficult task to harden them against that.
That said, it doesn't really take aware from your point, I think; but rather confirms it: Think twice before rolling your own network protocols (at least if they are meant to be used in large scale on the open internet), be aware of all the complicated pitfalls.
> This doesn't diminish the claim that the modal individual should not attempt to "roll their own crypto."
Just wanted to point out that "individual-should-not-attempt-to-roll-their-own-crypto" is ambiguous and unnecessarily antagonistic while the paragraph quoted above from Schneier is clear and difficult to argue with.
"As simple as possible but no simpler" applies quite well to summaries, esp. when they regard cryptography.
Okay, what exactly is the claim you'd like me to argue with? That Bruce Schneier was a hobbyist when he developed any of his cryptographic algorithms? He wasn't, and more importantly that misses the point since those algorithms were widely reviewed in competition settings.
I'm not sure what you're getting at here. Yes, someone has to develop cryptography, just like someone has to work on designing rockets. Theoretically everyone begins an aspiration as a hobby, but that's pedantic. A cryptographic design or implementation is not going to survive as a side hobby regardless of how it begins.
There is even a list of "you have to do A, B, C, and that's about it" that is missing some major - well known, even! - items.
Our "field" of programming is so weak in its preservation of institutional knowledge, that we have to occasionally fight the recurrence of bad ideas about very basic things, like using string filtering to avoid SQL injection. Yes, this happens, even in "hip" new parts of the field, like the communities around newer languages. (I have a specific example in mind, but I don't want to deal with the resulting war.)
I would have to read their source code to answer that question (I know almost nothing about 1Password). Some password managers just call a PGP executable. Others are assembling crypto primitives into larger pieces and making choices like "let's use CBC mode" themselves.
Anyone who can write code can write a crypto library.
The trick is to have a second person helping you, who has all your mathematics notes, a copy of your source code, physical access to your hardware, and the ability to run your program inside a debugger. You don't even get to know whether they put cream in their coffee.
If you can keep a secret from them, you're doing okay. So then you rent a smarter person to do the same thing to make sure all the tricks and shortcuts have been attempted.
If someone manages to break it in the wild after that, you're still doing about as well as some of the highest-paid professionals in the industry.
I generally also think that it's a bad idea to do, but I feel like it's a discussion that needs to be had with governments around the world wanting to backdoor everything.
It means that normal citizens won't be completely helpless against tyrannic governments and it helps to educate that writing crypto from scratch is most definitely something that criminals can do. The government making it illegal or backdooring it will only help against lowest-effort criminals, not against organized terrorist groups. Sure, those probably won't write unbreakable encryption, but it'll be enough to bypass a government that's expecting everything to be in plain text.
> but I feel like it's a discussion that needs to be had with governments around the world wanting to backdoor everything.
On the contrary, this is -exactly- why you don't want to roll your own crypto. Because they can and will break it. Good crypto takes serious thought and effort from people who put a lot of thought and effort into it.
You can do crypto as a side-hobby and safely put it into production. Of course, you can also do it wrong, but it is with little work possible to do it correctly.
Normally I wouldn't speak against what you said because the critique is only very minor, but I heard this too often. Your warning is too strong. People should try their own crypto and with little care it is not unsafer. Maybe giving a list of do's for that would be a good start.
> You can do crypto as a side-hobby and safely put it into production.
No, you really can't, unless you stretch the definition of "side-hobby" to the point of breaking at the seams. I'd be shocked if a single person who professionally works in security or cryptography is going to agree with you in this thread. You need far more than a "little care" to assure that a new cryptographic library is safe. It's an endeavor suited to a collaborative academic or corporate environment, with robust human capital and strong oversight.
I'm willing to accept that someone can safely implement a primitive or a construction in a new library with significant time, effort and feedback from others. But that level of effort doesn't really qualify as a side-hobby, and until the implementation was formally checked by professional cryptographers I'd consider it suspect.
This is to say nothing of designing novel primitives or constructions, which I would consider as far removed from a "side-hobby" as a local 5K is to the Olympics.
However, I absolutely think people should attempt to implement their own cryptography if they're curious about it and want to learn. Just don't use it in production and assume it's unsafe. People who work in the industry don't make these claims because we think it's fun, we do it because we've all seen the consequences.
We are talking in vague terms, but assume you want to encrypt some offline data.
- You can just use any recommended (by whom?) tool.
- Or you encrypt it with recommended tool with it's own dedicated key. Then you encrypt it again by whatever way you made up in five minutes, and you need only to ensure you didn't use the key material of the recommended tool. Then there you go, you just rolled your own not-unsafer crypto.
That is not rolling your own crypto. Rolling your own would be relying on your custom construction, not double encrypting and relying on the known construction for safety.
...Furthermore I'm confused as to why someone would do that. If you already have a safe way to encrypt your plaintext and you're the only one with both keys, why would you bother double encrypting? You gain absolutely nothing because it's redundant, and you lose performance.
What you're doing is tantamount to a weird sort of encoding that doesn't offer any benefit.
If a vulnerability is found for the outer encryption method, additional encryption of the inner message by a different method may provide some defence.
If that is the goal, though, it would be better to use two well studied encryption methods rather than something homemade.
Right, that observation is correct on the surface. But the reason why that's almost never done is because the goal of a cryptographic algorithm is to contribute enough safety margin on its own. Instead of encrypting twice, it's better to encrypt with a greater number of rounds, or to come up with a superior algorithm. In practice you sacrifice an unreasonable amount of performance double encrypting in a production environment for a threat model that is fantastically unrealistic.
Depending on the algorithm: yes. I'm not saying all algorihms that are widely used and well known are extremely unlikely to be broken.
I would happily bet $10,000 that AES will not be broken in the next ten years. I'd make the same bet for several hash functions.
And as a corollary to this point: I think it's incredibly foolish to try and combat the threat of an encryption algorithm being broken by double encrypting a plaintext, including with that algorithm.
I wouldn't take that bet either as I agree that is a very unlikely thing to happen.
Note, though, that you are using the specific word 'algorithm', where as I am talking about 'methods'. Most cryptographic failures are in how the algorithms are implemented or applied, not a problem in the underlying maths.
I would not be nearly so confident about a similar bet that applied to the actual code being widely used for encryption.
You'd lose the bet so hard. Don't take anything for granted. Even the most supposedly secure and widely used primitives should be scrutinized and are subjects to constant attacks.
Read that paper again. Cache timing attacks (and more generally timing attacks) are a subset of side channel attacks, which are not a break in the fundamental design of a cryptographic algorithm. When Bernstein mentions that he considers it a design flaw, that's a misnomer in the mathematical sense: he means that AES is antagonistic to a software implementation that can resist a timing attack, not that the design of the algorithm is fundamentally unsound. This is essentially true for all encryption schemes that utilize lookup tables and S-boxes, which he even mentions in the paper.
This is similar to using acoustic analysis or fault injection for key retrieval - yes, you've done it, but it's not really fair to say you've broken AES whatsoever. That remains a pipe dream. You've merely broken an AES implementation on a particular platform that was not hardened for this threat. Just a few months ago a team came up with a way to retrieve an AES key through acoustic analysis in a paper that was published here on HN. That's not actually the same as a cryptanalysis, the best of which is this: https://eprint.iacr.org/2009/317
There are effective CPU mitigations that allow you to safely implement algorithms in order to avoid this issue - for example, AES-NI theoretically mitigates cache timing attacks. See: http://cseweb.ucsd.edu/~hovav/dist/aes_cache.pdf
I know this is a side channel attack. I know people have used acoustic analysis to retrieve 4096 bit RSA keys, but the attack outlined in the paper is more feasible in real life situations, and systems do get compromised due to implementation details. AES might not be broken as a whole but that doesn't mean you can't attack specific usages.
AES isn't proven to be secure. I don't know about $10k, but I would not bet my life on any hash function not being broken.
That's not an (algorithmic) break, but a (side-channel) attack on some implementation. "Break" is precise jargon here; you're right that a particular real-world implementation of a strong algorithm may be weak, but that doesn't contradict anything dsacco said.
Doing your own encryption does clearly not exclude also using other encryption. Or is implementing AES already not anymore doing your own encryption, because AES wasn't invented by you? You _always_ use work of others.
Doing your own crypto can be used as way to safeguard against unknown vulnerabilities in the tools you would use otherwise. If the recommended library is safe, then fine, you are good to go. How do we know it is safe?
My idea above is a way to increase the safety margin in a way that is orthogonal to the increase of the key space of the recommended tool. That sounds like a very good thing to me.
I'm sorry, I don't understand what you're trying to say. Can you clarify?
What I am trying to say is that you gain nothing by, say, implementing your own encryption algorithm, using it on a plaintext, then encrypting the resulting ciphertext with a known good implementation of a known good algorithm like AES. That is how I interpreted your previous comment.
So - you don't gain anything from that, if that's what you meant. If you mean just implementing a known good algorithm on your own (again, like AES) - why? You are overwhelmingly more likely to shoot yourself in the foot, and there are plenty of open source implementations that you can inspect if you don't trust the code.
Finally, if you distrust the design of a known good algorithm, you're out of luck, because (again) you're almost certainly not going to come up with something better. If you do, by all means use it, but it's simply not likely whatsoever.
And again: you're not increasing the safety margin by double encrypting, unless you know both algorithms and implementations are solid. And even if they both are, the safety margin improvement will be negligible compared to the decrease in performance.
> "If you mean just implementing a known good algorithm on your own (again, like AES) - why? You are overwhelmingly more likely to shoot yourself in the foot"
I've never understood this argument. Encryption algorithms are deterministic. If you implement a known good algorithm on your own, and it gets the same outputs given the same inputs, then you are highly unlikely to shoot yourself in the foot. This is not what people are talking about when they warn not to roll your own crypto.
In cryptography, the output of an algorithm is far less important than how that output was generated (with very precise specificity). Crypto code requires far more heavy lifting in the "how" of the output than other software does. This is not intuitive, because ordinarily there are several ways to do things and you can compare on output, and it's probably fine to develop something in a functional but unorthodox way (minus performance consequences, maybe).
As the parent comment here mentions, there are extremely bad consequences to subtle mistakes in how the algorithm and its implementation arrives at the output, even if the output is correct.
If Alice sends Bob an encrypted message and the custom crypto provides an incorrect output, Bob can't decrypt the message. But if Alice sends Bob an encrypted message with the correct output generated in an unsafe way, there are many ways by which the secret key could theoretically be recovered, which is far worse than Bob not being able to read the message in the first place.
> "Just because an AES implementation matches the test vectors does not make it correct or safe."
Actually, this does make it correct. Whether or not it is safe depends on the application. For example, I sent my friend Bob some cyphertext that I calculated by hand with a pencil and paper using the AES algorithm. I sent it via my trusted courier Eve. It took four hours for me to do the calculation. At the last minute I started to second guess my math so I double checked it against a respected crypto library. It was fine, so I handed it off to Eve. I am pretty sure that in this application my choice to calculate the answer by hand was exactly as safe as if I had just used the library. In fact, I am so sure (given that the answers were identical), that just as Eve was walking out the door, enroute to Bob, I pulled her close and whispered in her ear: "Eve, be very careful with this cyphertext, it took me four hours to create it."
...what? Yes, if you compute by hand and compare with a known implementation, than it's likely you computation is correct. But this has nothing to do with test vectors. You could match all test vectors while still giving incorrect results for values not in the test vectors.
But there's more to cryptography than just inputs and outputs.
Like, let's say we have two functions which validate a password.
Function A takes a user-defined parameter, compares it to a stored value byte by byte, and as soon as the strings differ, it exits.
Function B takes a user-defined parameter, compares it to a stored value byte by byte, sets a flag if the strings differ, and exits when all bytes have been compared.
Both of those functions take the same input, and both of them return the same output. Are you prepared to tell me that the two functions are equally secure?
> Encryption algorithms are deterministic. If you implement a known good algorithm on your own, and it gets the same outputs given the same inputs, then you are highly unlikely to shoot yourself in the foot. This is not what people are talking about when they warn not to roll your own crypto.
That is exactly what people mean by "don't roll your own crypto." If your belief is that you only need to match the same output per input, then your implementation might be an oracle that leaks information.
Crypto people don't say this for job security or to feel important. They say it, because they, themselves, have made mistakes, and they constantly see examples of programs that trivialize both security and cryptography while simultaneously introducing foreseeable bugs and obviously weak crypto.
They are deterministic, but timing attacks are a big deal. There were lots of practical key extraction attacks on RSA with nothing but timing, even on a network.
You can also do things like forget input validation in ways that reveal key material, or reuse a nonce or IV in a way that breaks the whole system.
Sorry, but this is not good advice. Widely-used crypto implementations have had the benefit of years of analysis by dozens of high-expertise stakeholders who have a lot to lose should the crypto fail. Even that isn't always enough to catch all weaknesses and vulnerabilities.
This cowboy-programming attitude being extended to security is no small part of why we are so vulnerable as a society to attacks on our computer systems that can compromise our core infrastructure[1], our secrets[2], and our economic security[3].
[1] Think weeks-long country-wide power outages
[2] Think juicy blackmail material on anyone with their hands on levers of power. Look at the damage that North Korea's attacks did to Sony Pictures, for example.
[3] Think small-scale attacks on services that even a single major company, or many small companies, rely on.
> Of course, you can also do it wrong, but it is with little work possible to do it correctly.
GCHQ loves people who think this.
Have a look at some of the game console cryptography. These are companies with strong financial incentive to get it right. They're large well funded multinational organisations. They still get it wrong.
Game companies are also a great example of a situation where a large group of people with time on their hands (teenagers still in high school) are motivated to look for vulnerabilities in the method used.
Have any studies been done on relative security of crypto algorithms that are either:
a) Well known, well studied but also attractive targets for attackers to study
b) Unknown (aside from the developer) until an attacker encounters a specific piece of encrypted data
It is a common assumption that well-known methods are better (and it is the assumption I work under) but does empirical data on security breaches back that up? There are plenty of examples of security breaches where 'standard' methods were being used. Are there similar examples where people using previously unknown methods have been compromised?
GCHQ and others invest a huge amount of resources in finding vulnerabilities in well known encryption methods. When they find one, everyone who used that method is vulnerable.
I have no doubt that if they really wanted a piece of data that I had encrypted with a homemade method, they would be able to break it.
However, are they going to invest the resources to do that if I am not being specifically targeted? Are they going to invest the resources to crack hundreds of different people's home-made encryption methods? Thousands? Hundreds of thousands?
If am being specifically targeted by something like GCHQ, they will get what they want one way or another.
I'm not aware of academic studies on the subject (I think that would be hard to do, because it's virtually impossible to know how many proprietary algorithms exist, how many are trivially breakable and how many have been broken).
However, this point is featured as 101 material in basically every cryptographic textbook. To put it very succinctly: there are conditions in which it can be beneficial to use proprietary cryptography, especially when you require very unique interoperability constraints. However it is almost never a benefit for the safety of the algorithm.
I've come across a proprietary algorithm and successfully broken it, in a black box setting, with differential cryptanalysis. This algorithm was deployed to disguise the sequential order numbers for a very large delivery company. It took me about a month, but it was done. The challenge in proprietary algorithms is shifted to figuring out what's going on because it's unrecognizable. That is a significantly easier challenge that identifying a vulnerability in an algorithm like AES, which has never had a meaningful vulnerability in a decade and a half of cryptanalysis.
If you use a proprietary algorithm it might be safer than a known unsafe open algorithm, but it's virtually guaranteed to be worse than widely studied algorithms, and most likely in a trivially breakable way. They can be safe, but that still means you're going to be working with professional cryptographers at a company like Riscure to assure it's safe.
I am aware of the usual (and strong in my opinion) argument.
Every time this discussion does the rounds, though, I do wonder whether the hypothesis could be tested.
Most vulnerabilities do not come from breaking the core algorithm but rather from a flaw in how they are implemented or applied. Standardisation can lead to monocultures that become tempting targets for those with plenty of resources to throw at them.
That's what leads me to being reasonably sure the hypothesis is valid.
As a scientist, though, I'm always going to wonder whether there is a way to subject it to a proper test rather than just relying on opinion (no matter how much I respect those opinions)
It's not thought policing. Would you consider us adamantly telling you not to develop and deploy your own rockets or medical software without significant expertise and third party review to be "thought policing"?
Furthermore, your argument is fallacious. That expert professionals make mistakes does not tell you anything about the likelihood of an amateur to make a mistake. You can't draw any logical conclusion from that statement on its own.
That's great, but that's not what I'm telling you. I'm telling you not to develop and deploy them with human passengers, in a production environment, on a trip to orbit. If you get collaboration from professionals with significant expertise and audit the design and development extensively, then sure, go for it.
Even were that as accessible as developing and deploying your own cryptography, I'd still advise against it.
If you roll your own crypto and say, "that was a fun learning experience" and write a blog post about what you learned implementing AES in some novel way, nobody is going to show up and tell you not to roll your own crypto. If you roll your own crypto and then deploy it in a production system, then you are putting actual users at risk.
> There is cult around "Don't roll your own crypto" that is thought-policing
You know who else gets "thought policed"? The people who actually need to rely on cryptography. In their case it isn't grumpy people on Internet message boards telling them not to do something, it's agents of the state killing them or imprisoning them for many years.
I know "don't roll your own crypto" comes across as dismissive or patronising. I'm not a huge fan of the phrasing. But as a first-order approximation, it is correct. If you want to roll your own crypto, that needs to be your _thing_. It's very unlikely you're going to be a fantastic full-stack web developer _and_ be able to do that. If it's really what you want to do then awesome! Go study it, learn it, practice it. But if you're doing it as a side-hobby, never put it into production.