News for hackers. Stuff that matters.
Re:AWS S3 open source alternative written in Go by Ixiaus on Tuesday August 30, 2016 @03:10PM (#12392987)
Or, run Riak with their S3 compatibility layer. Riak is extremely stable and the work Basho has done to make a truly robust distributed database is significant.

http://docs.basho.com/riak/cs/2.1.1/

Re:AWS S3 open source alternative written in Go by viraptor on Tuesday August 30, 2016 @05:42PM (#12394173)
Re:AWS S3 open source alternative written in Go by theanalyst on Tuesday August 30, 2016 @05:50PM (#12394231)
Also ceph (& swift) are known to scale well in prod. clusters with over 30+ PB of data (at least looking at CERN's cluster) and the latest version of RGW does support geographic redundancy for S3 like apis
Re:AWS S3 open source alternative written in Go by shinydevops on Tuesday August 30, 2016 @07:54PM (#12394900)
+1 for Ceph. We're running several ~3.5 PB clusters in production. We've not taken advantage of the new RGW features in Jewel, but it works well as an object storage solution.
Re:AWS S3 open source alternative written in Go by dekobon on Tuesday August 30, 2016 @05:55PM (#12394265)
Don't forget:

manta - https://www.joyent.com/manta

Re:AWS S3 open source alternative written in Go by dc2447 on Wednesday August 31, 2016 @02:25AM (#12396361)
CEPH is a volume service not an object storage service.

SWIFT is indeed analogous to S3.

Re:AWS S3 open source alternative written in Go by viraptor on Wednesday August 31, 2016 @04:33AM (#12396831)
Come on, I literally linked to a website describing "CEPH OBJECT GATEWAY S3 API"
Re:AWS S3 open source alternative written in Go by XorNot on Wednesday August 31, 2016 @03:36AM (#12396596)
Ceph is object storage first. The volume service is implemented on top of that.
Re:AWS S3 open source alternative written in Go by Natales on Tuesday August 30, 2016 @11:00PM (#12395704)
+1 on RiakCS. They now call it RiakS2 for kicks. The scalability and reliability of their server is insane. You just can't beat Erlang software in that regard.

Unfortunately, Basho has been so successful with their TSDB and KV products that they have basically put S2 on maintenance mode. They are still "supporting" it, but no new features. I was hoping this Minio tool could do something similar, but with a single daemon is a single point of failure. Unacceptable for serious deployments.

Re:AWS S3 open source alternative written in Go by bramd on Wednesday August 31, 2016 @05:56AM (#12397174)
Another interesting project written in Erlang is LeoFS: http://leo-project.net/leofs/
Re:AWS S3 open source alternative written in Go by bandris on Wednesday August 31, 2016 @12:33PM (#12399547)
Worth adding that LeoFS is being used in production by Rakuten for years now. Still not a widely known project for some reason.

http://www.slideshare.net/rakutentech/scaling-and-high-perfo...

Re:AWS S3 open source alternative written in Go by jdboyd on Wednesday August 31, 2016 @01:39AM (#12396244)
Considering how many serious deployments still use non-clustered NASs, a single node object store seems equally reasonable.
Re:AWS S3 open source alternative written in Go by owyn on Wednesday August 31, 2016 @01:42AM (#12396252)
That sounds pretty nice. If it works does it need new features? :)
Re:AWS S3 open source alternative written in Go by corobo on Tuesday August 30, 2016 @05:47PM (#12394202)
There's also Skylable's SX Cluster if you use the libres3 daemon with it. Been using it for over a year with no problems. Set, forget, add more nodes when I need more disk.

Everyone's got their s3 of choice, always good to have more options on the table.

https://www.skylable.com/products/sx/

https://www.skylable.com/products/libres3/

Re:AWS S3 open source alternative written in Go by ranman on Tuesday August 30, 2016 @08:41PM (#12395100)
Ran Riak CS in production and had constant issues. It's not terrible but it's also not ideal. I would caution against anyone depending on it for mission critical systems. Many of the failure modes are undocumented.
Re:AWS S3 open source alternative written in Go by 0xmohit on Tuesday August 30, 2016 @10:18PM (#12395496)
Could you elaborate on some of the specific issues you ran into?
Re:AWS S3 open source alternative written in Go by mprev on Tuesday August 30, 2016 @06:42PM (#12394549)
There's also Pithos from Exoscale. Runs on top of Cassandra. Code is Clojure and open source. Http://Pithos.io
Re:AWS S3 open source alternative written in Go by merb on Tuesday August 30, 2016 @05:39PM (#12394150)
guess it's possible, but riak is not designed to run on a single node. Guess even basho suggests using at least a 5 node cluster.
Re:AWS S3 open source alternative written in Go by unlocksmith on Wednesday August 31, 2016 @12:42AM (#12396082)
Minio is deliberately designed this way. Cloud native applications require strict multitenancy. Minio's approach is to build just enough to meet a single tenant's requirement. Deploy one minio server per tenant or user or customer .. whichever fits you the best. This will allow you to upgrade, customize or bug fix in isolation. To replicate for HA, use "mc mirror -watch SOURCE TARGET" command to pair them up. If you have multiple drives (JBOD), you can eliminate RAID or ZFS and use Minio's erasure code to pool them up. Distributed version is also in testing at the moment. It should be out in a month.
Re:AWS S3 open source alternative written in Go by merb on Wednesday August 31, 2016 @03:48AM (#12396641)
I know and that. And that's why I find minio interesting. Start with a single node. Raise up to 16.
Re:AWS S3 open source alternative written in Go by dc2447 on Wednesday August 31, 2016 @02:24AM (#12396360)
Dont you have to pay for an enterprise licence if you want multi region/datacentre/AZ?