Aren't the layouts the Hard part?

It depends on where your data's coming from, but I tend to avoid doing layout in JS, and instead pre-lay-out the data using Gephi or graphviz (possibly in a cron job if the data changes), loading the hardcoded initial layout in JS and using sigma.js only for display/interaction. In the non-cron case, also lets me tweak the layout a bit through trial and error or trying out different algorithms, to get something I like. Obviously doesn't work for every possible usage, though.

It is one hard part. But displaying thousands of elements in a web page is another one.

Also, sigma does provide in a plugin a force-directed layout, ForceAtlas2, initially developed for Gephi.