I tried buildbot, and found that the documentation didn't match observed behaviour, and their insistence on not providing any examples (combined with some in-house jargon) because "everyone is different" made the whole process particularly painful. Buildbot required a lot of attention and time to get working for us, and after all that, wasn't particularly end-user friendly. You're going to get exactly the same '2 people with tribal knowledge' if you use buildbot.