Regulation can fix security, except you can't regulate security

Every time I start a discussion about how we can solve some of our security problems it seems like the topics of professional organizations and regulation are where things end up. I think regulations and professional organizations can fix a lot of problems in an industry, I'm not sure they work for security. First let's talk about why regulation usually works, then, why it won't work for security.

What is regulation?
You may not know it, but you deal with regulated industries every day. The food we eat, the cars we drive, the buildings we use, the roads, our water, products we buy, phones, internet, banks; there are literally too many to list. The reasons for the regulation vary greatly, but at the end of the day it's a nice way to use laws to protect society. It doesn't always directly protect people, sometimes it protects the government, or maybe even a giant corporation, but the basic idea is because of the regulation society is a better place. There are plenty of corner cases but for now let's just assume the goal is to make the world a better place.

Refrigerator regulation
One of my favorite stories about regulation involves refrigerator doors. A long time ago the door to a refrigerator would lock from the outside. If someone found themselves on the inside with a closed door, they couldn't get out. Given a refrigerator is designed to be air tight, one wouldn't last very long on the inside. The government decided to do something about this and told the companies that made refrigerators there had to be a way to get out if you're stuck inside. Of course this was seen as impossible and it was expected most companies would have to go out of business or stop making refrigerators. Given a substantial percentage of the population now owns refrigerators, it's safe to say that didn't happen. The solution was to use magnets to hold the door shut. Now the thought of using a locking door seems pretty silly especially when the solution was elegant and better in nearly every way.

Can we regulate cybersecurity?
The short answer is no. It can't be done. I do hate claiming something can't be done, someday I might be wrong. I imagine there will be some form of regulation eventually, it probably won't really work though. Let's use the last financial crisis to explain this. The financial industry has a lot of regulation, but it also has a lot of possibility. What I mean by this is the existing regulation mostly covers bad things that were done in the past, it's nearly impossible to really regulate the future due to the nature of regulation. So here's the thing. How many people went to jail from the last financial crisis? Not many. I'd bet in a lot of cases while some people were certainly horrible humans, they weren't breaking any laws. This will be the story of security regulation. We can create rules to dictate what happened in the past, but technology, bad guys, and people move very quickly in this space. If you regulated the industry to prevent a famous breach from a few years ago (there are many to choose from), by now the whole technology landscape has changed so much many of those rules wouldn't even apply today. This gets even crazier when you think about the brand new technology being invented every day.

Modern computer systems are Turing complete
A refrigerator has one door. One door that the industry didn't think they could fix. A modern IT system can do an infinite number of operations. You can't regulate a machine that can literally do anything. This would be like saying the front fridge door can't lock when you have a fridge with infinite area on the inside. If you can't find the door, and there are millions of other doors, some which don't open, it's not a useful regulation.

This is our challenge. We have machines that can literally do anything, and we have to make them secure. If there are infinite operations, there are by definitions infinite security problems. I know that's a bit over dramatic, but the numbers are big enough they're basically infinity.

The things that generally come up revolve around having security professionals, or training staff, or getting tools to lock things down, or better defaults. None of this things will hurt, but none really work either. even if you have the best staff in the world, you have to work with vendors who don't. Even if you have the best policies and tools, your developers and sysadmins will make silly mistakes. Even with the best possible defaults, one little error can undo everything.

What can we do?
I'm not suggesting we should curl up in the corner and weep (I'm also not saying not to). Weeping can be less dangerous than letting the new guy configure the server, it's not very helpful long term. I'm not suggesting that tools and training and staff are wastes of time and money, they have value to a certain point. It's sort of like taking a CPR course. You can't do brain surgery, but you can possibly save a life in an emergency. The real fix is going to be from technology and process that don't exist yet. Cybersecurity is a new concept that we can't use old models to understand. We need new models, tools, and ideas. They don't exist yet, but they will someday. Go invent them, I'm impatient and don't want to wait.

If you have any ideas, let me know: @joshbressers

Thoughts on our security bubble

Last week I spent time with a lot of normal people. Well, they were all computer folks, but not the sort one would find in a typical security circle. It really got me thinking about the bubble we live in as the security people.

There are a lot of things we take for granted. I can reference Dunning Kruger and "turtles all the way down" and not have to explain myself. If I talk about a buffer overflow, or most any security term I never have to explain what's going on. Even some of the more obscure technologies like container scanners and SCAP don't need but a few words to explain what happens. It's easy to talk to security people, at least it's easy for security people to talk to other security people.

Sometimes it's good to get out of your comfort zone though. Last week I spent a lot of the week well outside groups I was comfortable with. It's a good thing for us to do this though. I really do think this is a big problem the security universe suffers from. There are a lot of us who don't really get out there and see what it's really like. I know I always assume everyone else knows a lot about security. They don't know a lot about security. They usually don't even know a little about security. This puts us in a place where we think everyone else is dumb, and they think we're idiots. Do you listen to someone who appears to be a smug jerk? Of course not, nobody does. This is one of the reasons it can be hard to get our messages across.

If we want people to listen to us, they have to trust us. If we want people to trust us, we have to make them understand us. If we want people to understand us, we have to understand them first. That bit of circular Yoda logic sounds insane, but it really is true. There's nothing worse than trying to help someone only to have them ignore you, or worse, do the opposite because they can.

So here's what I want to do. I have some homework for you, assuming you made it this far, which you probably did if you're reading this. Go talk to some non security people. Don't try to educate them on anything, just listen to what they have to say, even if they're wrong, especially if they're wrong, don't correct them. Just listen. Listen and see what you can learn. I bet it will be something amazing.

Let me know what you learn: @joshbressers

Security will fix itself, eventually

If you're in the security industry these days things often don't look very good. Everywhere you look it sometimes feels like everything is on fire. The joke is there are two types of companies, those that know they've been hacked and those that don't. The world of devices looks even worse. They're all running old software, most will never see updates, most of the people building the things don't know or care about proper security, most people buying them don't know this is a problem.

I heard a TED talk by Al Gore called The case for optimism on climate change. This made me think of security in some ways. The basics of the talk are that things are getting better, we're surpassing many goals set for things like renewable energy. A few years ago the idea of renewable energy beating out something like coal seemed far fetched.

That reminded me of the current state of security. It's hard to see a future that's very bright sometimes. For ever problem that gets fixed, at least two new ones show up. The thing that gives me optimism though is the same basic idea as climate change. It has to get better because there is no alternative.

If we look back at renewable energy, the biggest force keeping it out of the market even five years ago was cost. It was really expensive to build and deploy things like solar panels. Today it's the same price or cheaper in some instances.

What happened?

The market happened. As new technology emerges and develops, it gets cheaper. This is one of the amazing things about emerging technology. Entrenched technology generally doesn't change price drastically just due to its nature. Solar power is getting better, it's not done yet, it will continue to get better for less cost. The day will come when we think about current power generation the way we think about using horses for transportation.

Now let's think about security.

If you want secure devices and a secure infrastructure it's going to cost a fortune. You're talking about very high skilled staff and extremely expensive hardware and software (assuming you can even get it in some cases). Today security is added cost in many cases, so lots of producers skip it. Bad security has cost too though. Today bad security is generally cheaper than good security. We need to flip this around, good security needs to be cheaper than bad security.

The future.

Here's my prediction though. In the future, good security will be cheaper to build, deploy, and run that bad security. This sounds completely insane with today's technology. A statement like is some kook ten years ago telling everyone solar power is our future. Ten years ago solar wasn't a serious thing, today it is. Our challenge is figuring out what the new security future will look like. We don't really know yet. We know we can't train our way out of this, most existing technology is a band-aid at best. If I had to guess I'll use the worn out "Artificial Intelligence will save us all", but who knows what the future will bring. Thanks to Al Gore, I'm now more optimistic things will get better. I'm impatient though, I don't want to wait for the future, I want it now! So all you smart folks do me a favor and start inventing the future.

What do you think? Leave your comments on twitter: @joshbressers

Security isn't a feature, it's a part of everything

Almost every industry goes through a time when new novel features are sold as some sort of add on or extra product. Remember needing a TCP stack? What about having to buy a sound card for your computer, or a CD drive? (Does anyone even know what a CD is anymore?) Did you know that web browsers used to cost money? Times were crazy.

Let's think about security now. There is a lot of security that's some sort of add on, or maybe a separate product. Some of this is because it's a clever idea, some things exist because people are willing to pay for it even if it should be included. No matter what we're talking about, there is always a march toward commoditization. This is how Linux took over the universe, the operating system is a commodity now, it's all about how you put things together using things like containers and devops and cloud.

Now let's think about security. Of all the things going on, all the products out there, all the methodologies, security is always the special snowflake. For being so special you'd think we could get more right. If everything was fine, the Red Team wouldn't win. every. single. time.

The reality is that until we stop treating security like some sort of special add on, we're not going to see things make any real improvements. Think about any product you use, there are always things that are just an expected part of it. Security should fall under this category. Imagine if your car didn't come with locks. Or if it had locks, but you had to cut your own keys before you could use them. What if every safe shipped with the same combination, if you wanted a new one you had to pay for it? There are a lot of things we just expect because they make sense.

I'm sure you get the idea I'm shooting for here. Today we treat security like something special. You have to buy a security solution if you want to be secure. Or you have to configure your product a certain way if you want it secure. If we want to really start solving security problems, we have to make sure security isn't something special we talk about later, or plan to add in version two. It has to just be a part of everything. There aren't secure options, all the options need to be what we would call "secure" today. The days of security as an optional requirement are long gone. Remember when we thought those old SSL algorithms could just stick around forever? Nobody thinks that anymore.

How are we going to fix this? That's the real trick. It's easy to talk about demanding security and voting with your pocketbook, but the reality is this isn't very possible today. Security isn't usually a big differentiator. If we expect security to just be part of everything, we also can't expect anyone to see security as a feature they look for. How do we ensure there is a demand for something that is by definition a secondary requirement? How do we get developers to care about something that isn't part of a requirement? How do we get organizations to pay for something that doesn't generate revenue?

There are some groups trying to do the right thing here. I think almost everyone is starting to understand security isn't a feature. Of course just because there's some interest and people are beginning to understand doesn't mean everything will be fixed quickly or easily. We have a long way to go still. It won't be easy, it won't be quick. It's possible everything could go off the rails. The only thing harder than security is planning for security :)

Do you think you know how to fix this mess? Impress me with your ideas: @joshbressers

Trusting, Trusting Trust

A long time ago Ken Thompson wrote something called Reflections on Trusting Trust. If you've never read this, go read it right now. It's short and it's something everyone needs to understand. The paper basically explains how Ken backdoored the compiler on a UNIX system in such a way it was extremely hard to get rid of the backdoors (yes, more than one). His conclusion was you can only trust code you wrote. Given the nature of the world today, that's no longer an option.

Every now and then I have someone ask me about Debian's Reproducible Builds. There are other groups working on similar things, but these guys seem to be the furthest along. I want to make clear right away that this work being done is really cool and super important, but not exactly for the reasons people assume. The Debian page is good about explaining what's going on but I think it's easy to jump to some false conclusions on this one.

Firstly, the point of a reproducible build is to allow two different systems to build the exact same binary. This tells us that the resulting binary was not tampered with. It does not tell us the compiler is trustworthy or the thing we built is trustworthy. Just that the system used to build it was clean and the binary wasn't meddled with before it got to you.

A lot of people assume a reproducible build means there can't be a backdoor in the binary. There can due to how the supply chain works. Let's break this down into a few stages. In the universe of software creation and distribution there are literally thousands to millions of steps happening. From each commit, to releases, to builds, to consumption. It's pretty wild. We'll keep it high level.

Here are the places I will talk about. Each one of these could be a book, but I'll keep it short on purpose.

1. Development: Creation of the code in question
2. Release: Sending the code out into the world
3. Build: Turning the code into a binary
4. Compose: Including the binary in some larger project
5. Consumption: Using the binary to do something useful

Development

The development stage of anything is possibly the hardest to control. We have reached a point in how we build software that development is now really fast. I would expect any healthy project to have hundreds or thousands of commits every day. Even with code reviews and sign offs, bugs can sneak in. A properly managed project will catch egregious attempts to insert a backdoor.

Release

This is the stage where the project in question cuts a release and puts it somewhere it can be downloaded. A good project will include a detached signature which almost nobody checks. This stage of the trust chain has been attacked in the past. There are many instances of hacked mirrors serving up backdoored content. The detached signature ensures the release is trustworthy. We mostly have trust here solved which is why those signatures are so important.

Build

This is the stage where we take the source code and turn it into a binary. This the step that a reproducible build project has injected trust into. Without a reproducible build stage, there was no real trust here. It's still sort of complicated though. If you've ever looked at the rules that trigger these builds, it wouldn't be very hard to violate trust there, so it's not bullet proof. It is a step in the right direction though.

Compose

This step is where we put a bunch of binaries together to make something useful. It's pretty rare for a single build to output the end result. I won't say it never happens, but it's a bit outside what we're worried about, so let's not dwell on it. The threat we see during this stage is the various libraries you bundle with your application. Do you know where they came from? Do they have some level of trust built in? At this point you could have a totally trustworthy chain of trust, but if you include a single bad library, it can undo everything. If you want to be as diligent as possible you won't ship things built by any 3rd parties. If you build it all yourself, you can ensure some level of trust up to this point then. Of course building everything yourself generally isn't practical. I think this is the next stage that we'll end up adding more trust. Various code scanners are trying to help here.

Consumption

Here is where whatever you put together is used. In general nobody is looking for software, they want a solution to a problem they have. This stage can be the most complex and dangerous though. Even if you have done everything perfectly up to here, if whoever does the deployment makes a mistake it can open up substantial security problems. Better management tools can help this step a lot.

The point of this article isn't to try to scare anyone (even though it is pretty scary if you really think about it). The real point to this is to stress nobody can do this alone. There was once a time a single group could plausibly try to own their entire development stack, those times are long gone now though. What you need to do is look a the above steps and decide where you want to draw your line. Do you have a supplier you can trust all the way to consumption? Do you only trust them for development and release? If you can't draw that line, you shouldn't be using that supplier. In most cases you have to draw the line at compose. If you don't trust what your supplier does beneath that stage, you need a new supplier. Demanding they give you reproducible builds isn't going to help you, they could backdoor things during development or release. It's the old saying: Turst, but verify.

Let me know what you think. I'm @joshbressers on Twitter.