Re:AWS S3 open source alternative written in Go
by viraptor on Tuesday August 30, 2016 @05:42PM
(#12394173)
Other alternatives:
ceph - http://docs.ceph.com/docs/master/radosgw/s3/
swift - https://wiki.openstack.org/wiki/Swift/APIFeatureComparison#A...
Re:AWS S3 open source alternative written in Go
by theanalyst on Tuesday August 30, 2016 @05:50PM
(#12394231)
Also ceph (& swift) are known to scale well in prod. clusters with over 30+ PB of data (at least looking at CERN's cluster) and the latest version of RGW does support geographic redundancy for S3 like apis
Re:AWS S3 open source alternative written in Go
by shinydevops on Tuesday August 30, 2016 @07:54PM
(#12394900)
+1 for Ceph. We're running several ~3.5 PB clusters in production. We've not taken advantage of the new RGW features in Jewel, but it works well as an object storage solution.
Re:AWS S3 open source alternative written in Go
by dekobon on Tuesday August 30, 2016 @05:55PM
(#12394265)
Don't forget:
manta - https://www.joyent.com/manta
Re:AWS S3 open source alternative written in Go
by dc2447 on Wednesday August 31, 2016 @02:25AM
(#12396361)
CEPH is a volume service not an object storage service.
SWIFT is indeed analogous to S3.
Re:AWS S3 open source alternative written in Go
by viraptor on Wednesday August 31, 2016 @04:33AM
(#12396831)
Come on, I literally linked to a website describing "CEPH OBJECT GATEWAY S3 API"
Re:AWS S3 open source alternative written in Go
by XorNot on Wednesday August 31, 2016 @03:36AM
(#12396596)
Ceph is object storage first. The volume service is implemented on top of that.
Re:AWS S3 open source alternative written in Go
by Natales on Tuesday August 30, 2016 @11:00PM
(#12395704)
+1 on RiakCS. They now call it RiakS2 for kicks. The scalability and reliability of their server is insane. You just can't beat Erlang software in that regard.
Unfortunately, Basho has been so successful with their TSDB and KV products that they have basically put S2 on maintenance mode. They are still "supporting" it, but no new features. I was hoping this Minio tool could do something similar, but with a single daemon is a single point of failure. Unacceptable for serious deployments.
Re:AWS S3 open source alternative written in Go
by bramd on Wednesday August 31, 2016 @05:56AM
(#12397174)
Another interesting project written in Erlang is LeoFS: http://leo-project.net/leofs/
Re:AWS S3 open source alternative written in Go
by bandris on Wednesday August 31, 2016 @12:33PM
(#12399547)
Worth adding that LeoFS is being used in production by Rakuten for years now. Still not a widely known project for some reason.
http://www.slideshare.net/rakutentech/scaling-and-high-perfo...
Re:AWS S3 open source alternative written in Go
by jdboyd on Wednesday August 31, 2016 @01:39AM
(#12396244)
Considering how many serious deployments still use non-clustered NASs, a single node object store seems equally reasonable.
Re:AWS S3 open source alternative written in Go
by owyn on Wednesday August 31, 2016 @01:42AM
(#12396252)
That sounds pretty nice. If it works does it need new features? :)
Re:AWS S3 open source alternative written in Go
by corobo on Tuesday August 30, 2016 @05:47PM
(#12394202)
There's also Skylable's SX Cluster if you use the libres3 daemon with it. Been using it for over a year with no problems. Set, forget, add more nodes when I need more disk.
Everyone's got their s3 of choice, always good to have more options on the table.
Re:AWS S3 open source alternative written in Go
by ranman on Tuesday August 30, 2016 @08:41PM
(#12395100)
Ran Riak CS in production and had constant issues. It's not terrible but it's also not ideal. I would caution against anyone depending on it for mission critical systems. Many of the failure modes are undocumented.
Re:AWS S3 open source alternative written in Go
by 0xmohit on Tuesday August 30, 2016 @10:18PM
(#12395496)
Could you elaborate on some of the specific issues you ran into?
Re:AWS S3 open source alternative written in Go
by mprev on Tuesday August 30, 2016 @06:42PM
(#12394549)
There's also Pithos from Exoscale. Runs on top of Cassandra. Code is Clojure and open source. Http://Pithos.io
Re:AWS S3 open source alternative written in Go
by merb on Tuesday August 30, 2016 @05:39PM
(#12394150)
guess it's possible, but riak is not designed to run on a single node. Guess even basho suggests using at least a 5 node cluster.
Re:AWS S3 open source alternative written in Go
by unlocksmith on Wednesday August 31, 2016 @12:42AM
(#12396082)
Minio is deliberately designed this way. Cloud native applications require strict multitenancy. Minio's approach is to build just enough to meet a single tenant's requirement. Deploy one minio server per tenant or user or customer .. whichever fits you the best. This will allow you to upgrade, customize or bug fix in isolation.
To replicate for HA, use "mc mirror -watch SOURCE TARGET" command to pair them up. If you have multiple drives (JBOD), you can eliminate RAID or ZFS and use Minio's erasure code to pool them up. Distributed version is also in testing at the moment. It should be out in a month.
http://docs.basho.com/riak/cs/2.1.1/