ArturT

I've created a knapsack ruby gem for CI parallelisation that has over 122 million downloads. Primarily due to the fact, Gitlab is using it.

I spin off https://knapsackpro.com from the knapsack gem and we are helping our customers run fast CI builds.

I posted my add-on in Heroku Marketplace over a year ago https://elements.heroku.com/addons/knapsack-pro and did not see much organic leads from the marketplace itself.

Anyone using Heroku Marketplace as well? Any ideas how did you approach promoting your add-on? Thanks.

Sidekiq Pro version was inspiration for me to create a pro version for my Ruby gem called knapsack that automatically splits tests in parallel on CI nodes/jobs.

With the knapsack_pro version I focused on common needs of users and there was simple validation method to verify if particular feature was needed. People just pay if it brings value.

In case of free open source tool, people tend to want something but you don't know if they actually will you use it and sometimes people demand a thing while you have to put the hard work.

When you do paid version of your tool then somehow people are polite. :)

Nowadays I have also support for running parallel tests in JS.

More at https://knapsackpro.com and here is detailed technical explanation how it works https://docs.knapsackpro.com/2020/how-to-speed-up-ruby-and-j...

Maybe because of running Rails on Heroku is simpler than with Docker.

If you have no tests at all I would start with simple E2E tests to ensure your critical parts of app are working like sign up page and billing. You can add later other important sections of your app to ensure happy paths for your features are working fine.

When I started my own product for Ruby & JS developers https://knapsackpro.com I was doing mostly unit testing and later relayed on E2E tests for testing user dashboard to ensure happy paths are covered.

It's very easy to introduce weird bugs and it's much faster to run CI build to test your app than manually verify if it works correctly in your browser when you make some changes.

knapsack_pro gem uses Knapsack Pro API which is paid. See more at https://knapsackpro.com

It works similarly like tools for exceptions tracking, i.e https://www.honeybadger.io is paid but has MIT client https://github.com/honeybadger-io/honeybadger-ruby

Other than that I do support paying customers.

I started with writing open source Ruby gem https://github.com/ArturT/knapsack for running tests in parallel on CI and later I gave a talk on local meetup event for Ruby developers.

With time I wrote more blog posts https://docs.knapsackpro.com and shared on Reddit, Twitter, LinkedIn profile or Ruby groups. From time to time posts landed in RubyWeekly newsletter.

Also asking users for recommendation might help.

I connected with each person on LinkedIn I talked to or who signed up and I followed them on Twitter. People who like your product can easily see you publish new content and can like or share it.

End-to-end tests tend to take a lot of time when project grows. Some tricks like recording API calls with Puffing Billy can help save some time but still majority time is spend in browser.

This problem triggered me to work on knapsack ruby gem to solve problem how to split tests across parallel CI servers and run my whole test suite faster. E2E tests can vary in time execution so I decided to split them in a dynamic way between CI nodes to auto balance tests speed between nodes.

Recently someone let me know about fix for puffing-billy gem to work with my knapsack_pro Queue Mode. https://github.com/KnapsackPro/knapsack_pro-ruby#how-to-conf...

For annoying flaky features tests, I use rspec-retry gem to repeat the test a few times before marking it as failed. It helped for integration tests with external sandbox API.

I noticed discourse had a lot of flaky tests while using their repo to test my knapsack_pro ruby gem to run test suite with CI parallelisation. A few articles with CI examples of parallelisation can be found here https://docs.knapsackpro.com

I need to try the latest version of discourse code, maybe now it will be more stable to run tests in parallel.

Back in 2014 I work on a project where we had ~15min test suite running on CI and this was painful so I started working on open source solution knapsack for CI parallelisation for Ruby tests.

Later on developed more advanced way with dynamic tests allocation across parallel CI nodes not only for Ruby to get CI build as fast as possible. To give you some idea how it works check https://docs.knapsackpro.com/2017/auto-balancing-7-hours-tes...

or watch video https://www.youtube.com/watch?v=hUEB1XDKEFY

I work on knapsack gem to split tests across parallel CI nodes. To do optimal balancing I developed a dynamic test suite split with Queue approach.

Here is example for Travis CI https://docs.knapsackpro.com/2018/how-to-run-travis-ci-paral...

And this is the graph showing the difference if you compare knapsack_pro dynamic tests split with what CircleCI does which is basically only time-based split. https://docs.knapsackpro.com/2018/improve-circleci-paralleli...

Setup deployment takes some time so I prepared blog post and an example Phoenix app how to do it with edeliver, distillery and Ansible playbooks http://blog.lunarlogic.io/phoenix-app-deployment-with-ansibl...

We spent some time with friend doing research and trying to connect all pieces to make it work.

It's easy to lose yourself as a developer in problem-solving mode instead of looking at the big picture of a project. It's great to have people around you that reminds you the code is just a part of the more complex story - how to help your client solve a business problem.

Hi HN,

I turned my open source ruby gem knapsack into indie SaaS product. I was inspired by sidekiq gem. Here is the short story:

In 2014 I had a problem with large test suite in my customer Ruby on Rails project. I developed the open source gem knapsack to split tests among CI nodes in parallel.

In 2015 I had an idea to turn it into small SaaS product and by the end of the year, I had a first early adopter Buildkite team.

This year I released a new solution to dynamically split RSpec tests among CI nodes and I tested it with a few companies like Pivotal, VTS, Namely.

On one hand would be great to live in a world with small and fast test suites but on the other hand, we know how it is in the reality. There are large projects with dozen minutes or several hours test suite time execution.

I also wrote the article inspired by work we've done with one of my customers: http://docs.knapsackpro.com/2017/auto-balancing-7-hours-test...

I'm curious if there are similar problems with getting optimal time execution for large test suite across multiple CI nodes in projects done in other languages than Ruby. Have you experienced that problem? I'm thinking about building my tool in a different language and would be nice to test solution on the real project.

I'd love to hear your story about working with big test suites and what languages & tools you use. :)

I created ruby gem knapsack to speed up testing and I built a Pro version which is SaaS https://knapsackpro.com

I definitely learned a lot during last year while emailing with people interested in the product. Thanks to that I improved my tool iteratively while the early adopters discovered new areas or edge cases about testing I didn't even think about.

This week I published success story about running tests across 100 parallel CI nodes with my tool: http://docs.knapsackpro.com/2017/auto-balancing-7-hours-test...