The Community is Our Security Research Team

August 13, 2018


Reading time

Today we welcome the first contributions from Microsoft to the open-source repository of security analyses managed by Semmle.

Semmle’s team of security researchers and language experts have a long history of working closely with Semmle’s customers, including Microsoft and Google, to find vulnerabilities and include feedback and ideas for new security analyses that are part of Semmle’s products. However, with the contributions now happening in the open, we have reached a major milestone and I’d like to share our vision for community-based security research, and why it is so important that security analyses are freely shared in open source.

No omniscience. Most security tool vendors, especially for deep code analysis, have closed proprietary implementations of their analyses. The rationale is that the analyses are their unique, differentiated IP and should therefore not be shared. The problem with this approach is that no team of experts can be omniscient - no single firm can hire enough security researchers to keep up with the increasing torrent of vulnerability disclosures.

Security expertise is scarce. There is now a shortage of security experts in general - narrow that down to security researchers who also have expertise in code analysis, and the pool of candidates becomes very small indeed. For the task of finding specific vulnerabilities in a given application, an answer to this shortage is to use a bug bounty program, as well as outside pentesting services. This, however, still doesn’t scale enough as the amount of software to examine grows too fast. We therefore need to automate security expertise where possible, so security professionals can focus their skills on identifying new attacks, whereas all known patterns are automatically identified and eliminated during the development process.

Obscurity is not a defense. One could argue that it’s dangerous to publish the principles behind known attacks, as that enables bad actors to develop new exploits. This is just another instance of Security through Obscurity, which was discredited for the case of physical locks in 1851. What held true then holds true today: we must share knowledge to become more secure.

How to share analyses. Sharing knowledge of deep, accurate code analyses is not easy. Implementing a single new security analysis is often measured in weeks not minutes, and requires PhD-level knowledge of complex APIs. To easily share key ideas, we need modular, concise ways to create new analyses that can be whipped up quickly. Furthermore, to make such analyses generally useful, you need the capability of testing them on tens of thousands of codebases. Without such extensive testing facilities, accuracy is impossible to achieve.

Join the community! To solve these problems, Semmle created QL: an object-oriented query language that greatly simplifies the implementation of deep analyses, and which also allows easy sharing of libraries of useful queries. All our queries, including those from customers such as Microsoft, are now publicly available on Semmle also created LGTM, a platform for running QL analyses at massive scale. is free to use on open source projects.

Join us in spreading security expertise.