It has been roughly three years since the ball began rolling on Brakeman Pro (the commercial version of the Brakeman security tool for Ruby on Rails), and it has been a little over a year since Brakeman Pro actually went on sale. I have learned a ton in that time (and I am still having lessons beaten into me). There have been a ton of challenges going from an OSS project that was never meant to be a paid product to one people are actually buying. Here are just a couple I have much time thinking about:

The “Free” Version

Clearly, what makes building a commercial product on top of an OSS project different from just selling some software is the existing OSS project itself.

With an OSS project, there is an opportunity to acquire a large number of people testing the software in many different environments. For a static analysis tool like Brakeman, testing on a wide variety of codebases is incredibly valuable. OSS with easy bug reporting and contributing (e.g. a GitHub repo) is not only very likely to receive bug reports and patches, but also suggestions for features and improvements. Brakeman does not receive a large number of code contributions, but bug reports and suggestions for new rules have driven a large chunk of Brakeman’s development.

Being free and open source also makes it easier to advertise your project. People are more willing to promote free software and you can share it around social media with little fear of backlash. Not to mention it is vastly easier to give conference talks about open source tools!

Personally, I will forever be grateful to the OSS community. Being the main author of widely-used (within a small niche) software has propelled most of my career, led to me speaking all over the world, and has brought me acquaintances and friends I would not have otherwise. I am very glad Brakeman is open source and I would never want to change that.

However, the existence of a “free” version, especially a successful one, has a serious drawback for a business. In particular, the “paid” version must now not only justify its utility, but also the incremental advantage over the free version. It has become abundantly clear the biggest competitor to Brakeman Pro is Brakeman OSS!

The number one question I receive regarding Brakeman Pro is “What is the difference between Pro and the open source version?” Among other things, one very simple, easy-to-explain difference is the existence of a GUI. However, most people want to know if it will “find more things.” This leads to considerable hedging from me because Pro probably will find different vulnerabilities while also reducing some false positives - the net outcome of which may be more or fewer overall warnings. Explaining why Pro may report different vulnerabilities quickly gets me lost in fine details of how the two tools work - at which point people’s eyes tend to glaze over.

Trying to quantify the differences between the OSS and Pro versions is a losing battle for me. Potential customers try to add up all these little details and see if it comes out to enough of a difference to begin paying for software they are used to having for free. But, as a technical person, papering over the differences with hyperbolic qualitative statements can seem dishonest. I have yet to arrive at a good solution for this problem.

Existing User Base

With a well-established OSS project comes another big advantage: the existing user base. These users already like the project and have found it useful! In a way, they have validated a market exists for the product. In the case of Brakeman, I have also felt a tremendous amount of goodwill from the community (for which, again, I am incredibly thankful).

These users are going to be the very first people in line to try the commercial product.* They will already be familiar with the OSS version - therefore communicating and justifying the additional value of the commercial version will be critical. The good news is they already know what the product does and have found it valuable. In some cases (but not very many, I’ve found) they may even purchase the product just to support the OSS version. In most cases, though, people need to justify why they are spending budget on this particular software instead of using the free version.

If you are like me, you may also find this existing user base to be a source of stress. Marketing to OSS users often feels scummy, but it also makes no sense not to promote the commercial tool to the people already using the free version! For quite a while I did not want to take advantage of the existing audience at all. I have only made very small steps in that direction, preceded by a lot of thought. The last thing I want to do is alienate the community or burn any of the goodwill Brakeman has.

One way to push customers towards the commercial version is to make the OSS version obviously worse. But while it would make selling the commercial version easier, not working on or supporting the OSS version is unthinkable. Even the appearance of doing so could turn a community against you. When your potential customers are mostly developers the support of the developer community has extreme value. Besides the business aspect, I personally would have a hard time dealing with loss of the community when the community has done so much for me.

That leaves making the commercial version so much better than the OSS version the additional value is ridiculously obvious and people happily pay for it. Sadly (gladly?), many people have let me know “the free version of Brakeman is really good and already does all I need.” Making the Pro version extra awesome without hurting the OSS version is an ongoing struggle which I continue to hope will resolve itself over time as we continue to improve Pro.

Like many things, the existing user base for an OSS project has both advantages and disadvantages which need to be considered and kept in mind if one is going to turn the project into a commercial product.

As a Security Tool…

This probably does not apply to very many projects, but as a security tool Brakeman has additional issues related to those above. With every feature that might be exclusive to Pro, I must consider - “Am I making the world less safe by not adding this feature to the OSS version?” The answers to this question likely lead me to make terrible business decisions. In the end I can live without Brakeman Pro being a successful business, but consciously compromising my integrity and potentially the security of applications is not something I could personally handle.

As a result, the features that tend to go into Pro but not the OSS version are noisier, slower, or focused on ease of use and not actual vulnerability discovery. I believe more false positives (but potentially more true positives) are acceptable in the Pro version because we make it easy to triage and ignore them. Slower features are also much more acceptable in the Pro version - the OSS version needs to be fast and lean. (Sometimes these features also end up in OSS, just off by default. “Off by default” means they might as well not exist for most users.)

Conclusions

If you are considering taking an open source project and building a commercial tool on top of it, I hope this little post has given you some (perhaps less obvious?) issues to ponder. For Brakeman users, I hope this explains a little bit of the thinking I have done while trying to balance between OSS and Pro.

Note that this blog post is actually an example of the first two issues above: I had to tell you about the “free” version to talk about the Pro version and at the same time you probably feel like this is a bit of an advertisement for the Pro version!

(I think I have to plug my product here now? Brakeman Pro is a static analysis security tool for Ruby on Rails applications. Try it out for free.)

* One of the early mistakes I made with Brakeman Pro was not realizing who the first customers would be. I thought the people most willing to *buy* a tool would be security auditors, and so the tool and pricing were targeted at *security professionals*. Unfortunately, the much larger market and initial user base for Brakeman are developers. Brakeman Pro should have made developers our top priority from the beginning just like Brakeman OSS does.

Backstory

I recently decided to distribute the Brakeman gem with all its dependencies included. This was the culmination of a lot of frustration with Bundler, version conflicts, RubyGem bugs, and trying to maintain compatibility with older versions of Ruby while libraries did not.

Brakeman is most often used as an application, not a library. Yet most Rubyists are used to including all dependencies in a Gemfile for use with Bundler. Doing so causes Brakeman’s dependencies to be mixed in with users’ application dependencies, which doesn’t make sense and causes a lot of anguish.

I liken it to having to worry about whether or not your Rails application’s dependencies conflict with your browser’s. It shouldn’t matter.

However, Bundler does not have a way to isolate dependencies for applications like Brakeman, and Bundler is the best way to manage dependencies so we are stuck with it.

Since Brakeman is not normally loaded into users’ applications (and I recommend against doing so), its dependencies are separate and should not really matter to the end user. To this end, I wanted to distribute Brakeman with all its dependencies already inside the gem.

Bundling Dependencies

Conveniently, Bundler already has a way to do this: bundle install --standalone. This generates a bundle directory with two subdirectories bundler and ruby.

The bundler directory just has one file: setup.rb. This file adds the bundled gems to the load path. We’ll come back to this file later.

The ruby directory has everything you need to run Bundler, along with all of the bundled gems and their executables. The path to the gems looks something like ruby/2.3.0/gems/rake-10.1.1/. Note this includes the Ruby version and the gem’s version. When setup.rb sets up the library paths, it chooses dynamically based on the running Ruby implementation and version (which is not what we want, see below).

Adding Dependencies

All the dependencies are now there in the bundle/ directory, but it’s still assumed you will be using Bundler. I would prefer to just load the dependencies myself.

To do so, the Brakeman build script removes the bundle/bundler/setup.rb file and generates its own load.rb using similar logic. However, it does not build paths dependent on the running Ruby version because we don’t know what the end user will be using. Instead, it just globs the paths as they are and loads those.

In Brakeman itself, it loads bundle/load.rb lazily if the file exists. I do not use it in normal testing or development. In general, all that is needed is to require the load.rb file inside your code somewhere.

Building the Gem

All that is left to do is add the bundled gems to the Brakeman gem itself.

Note that Brakeman’s Gemfile relies on its gemspec, but the gemspec needs to rely on the bundled gems, leading to a circular dependency.

This simple code is all that is required in the gemspec:

if File.exist? 'bundle/load.rb'
  s.files += Dir['bundle/ruby/*/gems/**/*'] + ['bundle/load.rb']
else
  # add dependencies as normal
end

Pros

The main advantage of this approach is not polluting application dependencies! No more version conflicts! No more worries that weird Bundler or gem bugs will break users’ installs.

In theory it also makes it easier to distribute Brakeman as a standalone application, if someone were interested in that.

Cons

The main problem, of course, is that this hides the dependencies. If you add Brakeman as a dependency and then either load it programmatically or run it with Rake, you may get mysterious library conflicts. To avoid this, use the “brakeman-lib” gem, which is the same as the main Brakeman gem but does not bundle dependencies.

It also locks dependencies to a specific version such that updating dependencies requires a new release. This can be good (avoid breaking with new versions) but it can also be bad if a library has a bug or vulnerability.

Code

The script I use to build the main Brakeman gem is here.

Here’s the annotated version:

#!/usr/bin/env ruby
puts 'Packaging Brakeman gem...'

# Clean up any existing build artifacts
system 'rm -rf bundle Gemfile.lock brakeman-*.gem' and

# Generate gem bundle in ./bundle
system 'BM_PACKAGE=true bundle install --standalone'

abort "No bundle installed" unless Dir.exist? 'bundle'

# Remove the setup.rb file we don't use
File.delete "bundle/bundler/setup.rb"
Dir.delete "bundle/bundler"

# Generate new file to set load paths
# Code below is a little confusing because it is generating code
File.open "bundle/load.rb", "w" do |f|

  # Set path at runtime
  f.puts "path = File.expand_path('../..', __FILE__)"

  # Add each gem's lib/ directory to the load path (again at runtime)
  Dir["bundle/ruby/**/lib"].each do |dir|
    f.puts %Q[$:.unshift "\#{path}/#{dir}"]
  end
end

# Build the gem
system "BM_PACKAGE=true gem build brakeman.gemspec"

When bundling gems and building the gem, the script sets the BM_PACKAGE variable so that development dependencies are not included in the bundled gems.

I am not sure this is a problem everyone has, but I grew tired of people commenting on old, resolved GitHub issues. Almost every time someone would comment “I have this problem, too” it would actually be a different issue. Then I’d go through the routine of asking them to open a new issue with details about their specific problem. Sometimes they would, and sometimes they’d never come back.

Fortunately, right around the time I decided I should do something about this annoyance, GitHub released an API to lock issues. (Locking issues or pull requests prevents any new comments except from repo collaborators.)

So I put together a little gem called github-auto-locker to fetch and lock old, closed issues.

To install it (requires Ruby):

gem install github-auto-locker

Then run:

github-auto-locker USER REPO TOKEN [age in days]

For example, I run this to lock resolved issues over 60 days old:

github-auto-locker presidentbeef brakeman N0TM1R34L70K3N 60

The default is 120 days.

I’ve been running it periodically myself since February without any complaints. Perhaps it will be useful to you!