The recently publicized “Heartbleed” SSL/TLS bug has received a tremendous amount of media coverage and, deservedly, a significant amount of concern amongst the IT security community. Rather than rehash the same information that has been shared repeatedly, I would like to offer some philosophical commentary, concise guidance, and additional resources to the IT community as a whole.
The first major worm was discovered on Nov. 2, 1988. The Morris worm infected BSD Unix machines and spread via a buffer overrun. In 1999 we had Melissa, Happy99, and by 2001 networks were taken to their knees by Sadmind and Code Red. Later that year Nimda broke new ground in both damage and difficulty to clean. The network disruptions these worms (and later, SQL Slammer, Blaster, Welchia, etc.) caused taught us that we HAD TO PATCH.
We didn’t just HAVE TO PATCH, we had to patch RIGHT NOW. If we didn’t patch, Bad Things would happen. Microsoft, for all the beatings it took in the early 2000s, has become very adept at both writing more secure code as well as designing simple and scalable ways to deliver code updates. It’s not just easy to stay up to date on Microsoft patches… it’s easy to know what you have to patch.
One of my colleagues told me that the above stories reminded him of all too many 24-hour days running around with network taps, sniffers, and patch/remediation kits. I was right there with him. But you know what is frankly encouraging? Back then it took the network going down to effect change.
The sheer amount of concern and furor that Heartbleed has caused, and the sensitivity to the possibility of data loss, means that each and every one of you reading this post is not just taking this seriously… you have the organizational support necessary, and are prepared to do whatever it takes to address the problem. And that is something that most technologists that went to war with the early worms never would have believed. We’ve come a long way.
Unfortunately, Heartbleed is going to be a long and difficult road. The vulnerability exists in OpenSSL which, for better or worse, is an ”under-the-hood” building block that is commonly used. Right now technology industry leaders are being very proactive about identifying vulnerable systems and releasing updates. But how long is it going to take for your industrial machinery supplier to realize there is a vulnerable version of OpenSSL running underneath the embedded web interface they bolted onto their core product?
When CDW’s security team performs security assessments for customers, we find architectural issues. We find people and process issues. We find configuration issues. And we find vulnerable software. In some cases, the vulnerabilities are in systems that customers didn’t know they had. “What? My HVAC has a web interface?! And you can access it from the guest Wi-Fi?”
We’re going to be tripping over Heartbleed for a long time to come.
So what should I do?
1. Identify vulnerable systems.
Your asset management and software inventory will give you the first sign post. Internet-facing services are certainly the most critical to check first. In the long run, once you’re past the immediately critical issues, you will need to include this in your annual vulnerability assessment. Top priority is Internet-facing SSL/TLS; including web servers, SSL VPNs, and some VPN tunnels.
So which systems are vulnerable? This isn’t an easy answer. The technically precise answer is: The affected versions of OpenSSL are OpenSSL 1.0.1 through 1.0.1f. Later versions (1.0.1g and newer) and previous versions (1.0.0 and older) are not vulnerable.
Still, this is challenging to answer comprehensively as OpenSSL is built into many other products, software, and appliances. Effectively, anything with a web interface is likely to include an SSL implementation, possibly OpenSSL, and possibly a vulnerable version of OpenSSL.
A number of CDW’s partners have published Heartbleed statements, in some cases listing vulnerable products, in other cases stating no vulnerabilities. A few examples include: Cisco, VMware, F5, IBM & Citrix
Affected services should be updated to remove the vulnerability. In many cases this will be applying a vendor supplied patch or software/firmware release, though for directly managed OpenSSL instances it will mean selecting a more recent version and upgrading/migrating.
3. Revoke certificates, then reissue to rekey.
Any certificate present on an affected system was vulnerable to the loss of its private key. You will need to reissue the certificate. You need to change the locks and issue new keys.
4. Change passwords that could have been exposed.
This applies to your personal life as well. This is a good opportunity to change your own Internet passwords, and recommend friends and families do the same.
If you are a CDW Professional Services Customer…
If you would like assistance applying patches, or rekeying certificates please contact your Account Executive.
If you are a CDW Security Assessment Customer…
All assessments from this point forward will include the identification of Heartbleed vulnerable systems, and include properly tailored recommendations for those systems.
If you would like help identifying vulnerable systems…
The CDW Security Assessment team has also created a rapid and lightweight engagement limited in scope to the identification of vulnerable systems. Please contact your CDW Account Manager.
If you purchased Symantec/VeriSign, Thawte, GeoTrust, or RapidSSL certificates from CDW…
The CDW SSL team is standing by to rapidly revoke and reissue certificates at no cost for our customers. Please contact us at email@example.com
If you didn’t buy your certificate from CDW, but need a new one fast…
The CDW SSL team is standing by. We know this is a priority for you, and we want you to be able to close these holes as quickly as possible. Please contact us at firstname.lastname@example.org
What are some of the better resources available online?
Codenomicon (one of the two parties credited with discovering this bug) maintains a very informative site.
CSO Magazine online had a very well written piece covering “How you need to respond”
The current Wikipedia entry is not bad, and perhaps more importantly contains a very comprehensive list of sources.
And last, but certainly not least, the web comic XKCD explains the flaw itself in a unique way.
Where can you go to learn more?
CDW’s most senior security engineers are hosting weekly webinars starting on 4/17/14. Here are the links to register:
At this session our experts will discuss this issue in more detail, provide meatier recommendations, and host a detailed and extended Q/A for the audience.