At X41, we find bugs and some of them are security bugs in publicly used software. There are various reasons why and how we find them. Some of them are identified while performing penetration tests or security audits for customers, some are the result of our team looking for them in their spare time or internal research projects, and some just jump at us while we are using the software.
The last months, we identified various security issues in medical software.
The bugs identified range from bad password storage (passwords in cleartext or only obfuscated) to DLL side loading, memory corruption issues and backdoor passwords. The responses from the vendors are as diverse as the issues, ranging from curious interest and a helpful attitude up to us talking to lawyers or non-technical persons that are more interested in mitigating any public fallout than fixing the bugs. For some, the issues are patched quickly and users are notified, others skip this step and try to hide the security fixes in normal updates. Some organizations have proper online update infrastructure, others rely on DVDs sent by snail-mail.
The varying responses we get when we are alerting vendors about these issues is what encouraged us to write this blog post.
Identifying a security bug raises the question: how should we proceed? The easiest option would be to just ignore the issue (non-disclosure) and go on with our lives. The bug will not be fixed and people will remain vulnerable which will potentially have a serious impact on their data and systems. So this is not an option we want to pursue at X41, even if some of the security issues might be helpful to compromise customer systems in later engagements.
For some of the bugs, there is even a gray or black market which is not an option for X41 for various reasons, be they ethical or legal.
Depending on the context and circumstances, the choice on how to disclose can be different. Deciding factors may include who is affected, if a fix is easy, who are the affected groups, and many more.
Disclosure can be done openly (full disclosure), by posting the issue somewhere for the public and reporting to the vendor at the same time. This option makes the same (full) information available equally to everyone. While users, blue teams, and other entities can use this information to detect attacks or patch the vulnerability via a workaround, persons or organizations with malicious intent can use the information as well. This approach has advantages and drawbacks and a decision to do full disclosure may depend on factors such as whether the vulnerability is actively exploited in the wild, or if working patches from a vendor cannot be expected in time.
Another option is what is called coordinate disclosure and is the approach we follow at X41 by default. The Bundesamt für Sicherheit in der Informationstechnik recommends this approach as well and even publishes a guideline for it.
The idea is to confidentially transfer the details about the security issue to the vendor, so the vendor is able to create a proper fix and prepare a release. The release of the fix is accompanied by a changelog entry about the issue, a CVE ID and an advisory explaining the technical details. These three should be released at the same time, so users of the software get all the information they need at the same time as possible attackers to decide whether they should install the update or apply workarounds. Confidentiality, vendor trust and context are crucial for coordinated disclosure to work. A situation where information about a vulnerability may leak to restricted malicious circles without the public knowing should be avoided. Therefore, such disclosure is also always time limited to give the vendor a reasonable time to fix and not hold back the information from the general public.
The trouble often starts when X41 tries to contact the vendors securely. Many vendors still do not have a dedicated security contact. In these cases, researchers usually start writing on public channels (info email account, contact forms, chatbots) and ask for a security contact and secure communication channel until they get a response. This can already eat up several hours of valuable time, which in some cases is more time than we spent identifying the issue.
A lot of vendors are challenged by trying to setup an encrypted communication channel by GPG or S/MIME and might even forward us to the wrong contact persons. Instead of a person that knows IT security or technical details about the products, researchers sometimes are referred to the business management or even lawyers. Some even have the impression that security researchers are responsible for the security issue because they discovered it. That is not the case; the bug is already there and it’s the vendor’s responsibility for putting it there, not spotting it earlier and now fixing it. It will not magically go away if the vendor ignores it.
If you are a vendor, the following helps security researchers help you:
This makes it easier for researchers to inform you about security issues in your products. A good and professional handling of security vulnerabilities will improve your public image and reinforce users’ trust.
When we send the technical details to a vendor, we tell them that this information will be made public after 30 days to ensure the users and general public are able to properly respond to unfixed vulnerabilities. In case there are valid reasons to delay disclosure, we are happy to extend that time period. These reasons should be of technical nature and marketing reasons are not considered valid. Sometimes this results in a bargaining discussion, therefore X41 became quite strict with this, since we don’t want to spend more time reporting an issue for free than it takes to identify it.
The main point about the coordinated release is that the patch and all information about it is released at the same time so that users and attackers have the same information. If a patch is released before the detailed information, dedicated attackers may look at the differences between software versions and abuse the security issues before users will install the patch. It is a common misconception that reversing binary patches or updates is hard. While it can be, a dedicated reverse engineer can build custom tools to automate much of the work. On the other hand, users will usually not reverse engineer patches to identify whether it’s important to install them quickly or not. Having isolated security releases that only fix security issues will make the decision whether to install them or not easier, since these are less likely to introduce failures in comparison to updates where more changes are mixed in. In case of serious issues, a heads up to users that an important patch will be released on a certain day is useful to reduce the time between the release of the patch and users installing it.
Of course, we are always happy about a thank-you for a bug we reported, but we don’t feel that any vendor is obligated to do so in a material or financial way. We think a proper reference to the person who spotted the bug is something each vendor should do. For individuals we know that some goodies helped to spark interest into looking deeper into some of the products for security issues. Nevertheless, vendors should be aware that bug bounties do not replace a proper audit. Especially when the prices are low, bug hunters will only report easy to identify issues whereas attackers will dig a bit deeper and still find options to exploit the software.
X41 D-Sec GmbH is an expert provider of application security services. With extensive experience and expertise in the information security industry and a strong core security team of world-class experts, X41 can provide premium security services. Their fields of expertise in the area of application security are security-centered code reviews, binary reverse engineering, and vulnerability discovery. Custom research and IT security consulting and support services are the core competencies of X41.