notmyname

Reminds me of the work Jonathan Oxer presented at LCA 2019 about the journey of reverse engineering and building a similar (and more capable) breakout board for wheelchairs. https://youtu.be/-9Rjh8qJk68

https://www.superhouse.tv/product/wheelchair-control-breakou... is the result

Here's a small script you can run to mitigate the issues described in the article: https://gist.github.com/notmyname/824db39350e3d39496de2ea930...

No, because path-style bucket names weren't originally required to conform to dns naming limitations. I don't know how they're going to migrate those older non-conforming buckets to the host-style form.

seems an obvious response to https://www.mongodb.com/press/mongodb-issues-new-server-side...

You can play with simulations based on this data at http://adrift.org.au

Here's the update: https://01.org/mcu-path-license-2018

found via https://twitter.com/imadsousou/status/1032680311753072640

I would be extremely interested in any tools that have been developed to maintain a Common Pool Resource community, open source software or otherwise.

That's great to hear that you're forcing yourself to keep communication in the open, when at all possible. That's one of the best ways to keep a community healthy.

In my experience with a particular open source project, more than half of new contributors started by communicating privately with myself or some other leader in the project. This is only bad if the communication never moves out of private conversation. I've found it normally takes a few nudges to get people to build their confidence and participate publicly.

Interestingly, it's not only new contributors who suffer from this. I've been told by some of the most prolific contributors to our project that they are still intimidated to speak out publicly. Many times these feelings come from deep-seated cultural norms that don't really fit with traditional Western/American ways of communicating. And that's ok! We all learn together how to best communicate with each other. And we get a healthier community and better code out of it.

I'd encourage you to do it anyway. We're on the same journey at my company. Some of our product is based on a fairly large and active open source project. Others are much more limited and simply at the "here's the code, have fun" stage.

There's different levels of "open source" (everything from one-way code dumps to full-on maintained-as-a-whole-distributed-global-team). In my experience, it's easier to start with a simple "here's the code, bug reports welcome". This starting point is generally an easier sell to management who's worried about project management taking too much time away from other work.

Like all things, practice, start small, and grow from there. Get's easier as you go. But yeah--licenses/legal, written policies, governance, marketing, time prioritization, and more all take a lot of time to figure out.

NB: My own background in open source biases me to thinking open source is "easy". It's not; it takes a lot of work. The good news is that there's a lot of tools and help available for anyone wanting to start.

I use (and contribute to) OpenStack Swift.

It's an object storage engine (think S3, but it's open source and you can put it in your own data center) that's excellent at storing unstructured data.

It's completely deployable and usable without any other OpenStack projects.

There's S3 API compatibility for it. It supports globally distributed clusters. It supports multiple storage polices that can be either replicated or use erasure coding. It's designed for very high availability, very high durability, and high aggregate throughput.

One of my favorite features is being able to create sharable, expiring signed URLs to any object in the cluster.

Some of the common uses for Swift include storing user-generated content (eg images, videos, game saves), static web assets, movies, scientific data sets, backups, document sharing, VM and container images, etc.

API docs: - https://developer.openstack.org/api-ref/object-storage/

Docs: - http://swift.openstack.org

Vagrant All-In-One setup: - https://github.com/swiftstack/vagrant-swift-all-in-one

Come say hi! - #openstack-swift on freenode IRC (I'm notmyname)

Yeah the lack of further work on ZeroVM is unfortunate. You may be interested in https://github.com/openstack/storlets which is conceptually similar and is being actively developed.

As a quick answer, the name comes from being able to recover data when some of it is "erased".

The only way to durably store data so that it survives a hardware failure (e.g. drive dying) is to store more than one copy. Full replicas are the simplest way to do this, but you've got a relatively high overhead (e.g. Store 1GB of data with 3x replicas, and you store 3GB of data). Erasure codes are a way to effectively store fractional replicas, so you only use 1.5x or 1.7x of the original data.

Erasure codes are great when you've got a lot of data and you need high durability but don't want to pay for the storage space required for full replicas.

Why don't we always use erasure codes for everything? EC isn't great when you've got small bits of data, and since there's a bit of math involved in reading and writing the EC data, EC has higher latency than simple replicas.

https://www.swiftstack.com/blog/2015/04/20/the-foundations-o... is a great into to how erasure codes work.

PyECLib upstream is now at https://github.com/openstack/pyeclib

The Rackspace API is OpenStack Swift, a project used by many more public and private cloud providers than just Rackspace.

I've been working on Swift for the past 6 years, and I'd be happy to help with your OpenStack Swift API integration. See my profile for contact info.

It's great to hear how you are using Swift. I work on that project; feel free to reach out if you have any questions. Contact info is in my profile.

I've used ISA-L as the backend implementation for erasure codes in OpenStack Swift.

There are two versions of ISA-L. The open-source one includes some optimized erasure code libraries. The other version is free (as in beer) AFAICT but it requires signing a license agreement with Intel. This version also includes some other optimized libraries for hashing, crypto, and stuff like that.

https://github.com/openstack/liberasurecode is a plugable C library that you can use for erasure coding. It supports some simple EC algorithms itself and works with jerasure and ISA-L.

https://github.com/openstack/pyeclib is a Python wrapper for liberasurecode.

Great stuff! Thanks for publishing it. I'm looking forward to future posts.

I'm curious about how you're managing the data on a drive itself. Are you storing the blocks as individual files on a filesystem? Are you doing direct management of the block device itself? Something else?

Or try it yourself...

http://adrift.org.au

SwiftStack | swiftstack.com | San Francisco, CA

We build tools to help our customers build object storage based on OpenStack Swift. Our customers have big storage needs and need to store their data easily, safely, and cheaply in their own data centers. We're active contributors to OpenStack Swift, and everything we put into Swift we push upstream.

One reason I love working at SwiftStack is because it lets me see a big range of storage use cases. We've got customers using SwiftStack to store everything from backups to game assets to genomics data for cancer research.

Check out one of the things we've recently been working on: adding Erasure Codes to OpenStack Swift: https://swiftstack.com/blog/2015/04/24/behind-the-scenes-“un...

Reach out to us if you have any of the following:

  * Experience writing high-performance server software (Python/C/C++/Go)
  * A deep and abiding interest in distributed (storage) systems 
  * Vision that simplifies complex user workflows  (front end/UI/UX)

Apply at http://swiftstack.workable.com/jobs/27475, or email [email protected] if you have questions.

Yes, I find that unfortunate too.

But if you're interested in ACLs in OpenStack Swift, http://swift.openstack.org/overview_auth.html is a good place to start.

(all tongue in cheek)

SwiftStack - http://swiftstack.com/jobs - San Francisco, CA (preferred, will relocate)

We're a storage company focused on solving a lot of the hard problems around deploying and managing scalable storage systems. We build a management controller [0] around the OpenStack Swift [1] storage engine, and we sell that controller to customers. Some of our customers include eBay, PAC-12, and HP [2]. Our focus is on ensuring that the OpenStack Swift storage engine is well integrated into enterprise IT infrastructure and processes. This means we spend a lot of time automating day-to-day operations tasks, providing good visualization of monitoring data, integrating with auth systems, selecting hardware, building utilization tools, and on and on.

But we also get to build against new technologies[3], actively contribute to open source[4], and partner with major companies to build cool new features into Swift[5].

Interested? Send us an email at [email protected]. Tell us who you are, why you're interested in SwiftStack, and why we'll be interested in you. We'll get in touch with you shortly to continue the conversation.

[0] https://www.youtube.com/watch?v=7Aue0eoMGCA

[1] https://wiki.openstack.org/wiki/Swift

[2] https://swiftstack.com/blog/2014/06/25/ebay-pac12-and-hp-hel...

[3] https://swiftstack.com/blog/2013/10/22/kinetic-for-openstack...

[4] http://stackalytics.com/?release=all&company=swiftstack

[5] https://swiftstack.com/blog/2013/07/17/erasure-codes-with-op...

SwiftStack - http://swiftstack.com/jobs - San Francisco, CA (preferred)

SwiftStack is a storage company building and deploying object storage for our customers (think S3 in your own datacenter). Our product is a management controller that works with OpenStack Swift. Swift gives deployers the ability to create scalable storage pools for modern applications, and SwiftStack provides the management tools for day-to-day operation of the cluster. We charge our customers a recurring license fee for the management software based on how much storage capacity they have deployed.

SwiftStack (the company) is about 2.5 years old. We've got about 30 employees, and we're actively hiring developers. Here's a short list of some things we're currently working on:

    - New features and bugfixes in OpenStack Swift
    - Making clean, responsive UIs to simply communicate information about large storage clusters
    - Using metrics to diagnose errors and proactively fix them for customers

Interested in knowing more? Send us an email at [email protected]. Send us your github profile, your LinkedIn account, a link to your website – whatever will best display the work that you’ve done. Tell us in a few lines of text why you’re interested in SwiftStack, and why we’ll be interested in you. We’ll be back in touch shortly to get the conversation started.

SwiftStack - http://swiftstack.com/jobs - San Francisco, CA (preferred)

Everyone has data, and it's always growing. This growth in data means that we need an effective way to store massive amounts of data. OpenStack Swift is designed to effectively store massive amounts of data while providing high durability, high availability, and support massive concurrency across the entire data set. SwiftStack is building management, monitoring, and integration tools to turn the Swift storage engine into an effective storage solution that is used by millions, daily.

We're looking for engineers of all experience levels to join us. We do most of our work in Python, but we've got some Ruby, Javascript, and a few other things mixed into our stack.

So far this year, we've built out global clusters for a major expense reporting and travel support site[1], provided the storage system for Disney Interactive's gaming platform[2], and integrated Swift into radical new hard drive technology from Seagate[3]. We're currently working with companies like Intel and Box to build support for erasure codes into Swift[4].

My vision for OpenStack Swift is that it will be used by everyone, every day, even if they don't realize it. Swift is storage that is built for the web and already powers some of the world's largest storage clouds. I would like to work with you at SwiftStack to see this vision become a reality.

Interested? Send us an email at [email protected]. Send us your github profile, your LinkedIn account, a link to your website – whatever will best display the work that you’ve done. Tell us in a few lines of text why you’re interested in SwiftStack, and why we’ll be interested in you. We’ll be back in touch shortly to get the conversation started.

[1] Concur video: http://www.youtube.com/watch?v=qQmGfzewWQw&t=29m23s

[2] Disney + SwiftStack: http://venturebeat.com/2013/08/30/how-disney-built-a-giant-c...

[3] SwiftStack + Seagate: http://swiftstack.com/blog/2013/10/22/kinetic-for-openstack-...

[4] Building erasure codes into OpenStack Swift: http://swiftstack.com/blog/2013/07/17/erasure-codes-with-ope...

SwiftStack - http://swiftstack.com/jobs - San Francisco, CA (preferred)

SwiftStack builds and delivers on-premise object storage to companies looking for highly available storage that scales to millions of users. Our product is built around the OpenStack Swift[1] object storage engine and provides management, monitoring, and integration for the entire storage system.

Working at SwiftStack, you'll work with new technology[2], participate in open-source community work, and build a storage engine that is used by millions. We're solving all kinds of interesting problems, and I'd love to work with you to solve them. Three things we're working on right now are:

- building erasure codes into OpenStack Swift[3]

- making clean responsive UIs to simply communicate information about large storage clusters

- analyzing system metrics to intelligently respond to and predict system behavior

Interested in knowing more? Send us an email at [email protected]. Send us your github profile, your LinkedIn account, a link to your website – whatever will best display the work that you’ve done. Tell us in a few lines of text why you’re interested in SwiftStack, and why we’ll be interested in you. We’ll be back in touch shortly to get the conversation started.

[1] OpenStack Swift dev docs: http://swift.openstack.org

[2] SwiftStack + Seagate: http://swiftstack.com/blog/2013/10/22/kinetic-for-openstack-...

[3] Erasure codes in Swift: http://swiftstack.com/blog/2013/07/10/erasure-codes-with-ope...

SwiftStack - http://swiftstack.com/jobs - San Francisco, CA (preferred)

Everyone has data, and it's always growing. This growth in data means that we need an effective way to store massive amounts of data. OpenStack Swift is designed to effectively store massive amounts of data while providing high durability, high availability, and support massive concurrency across the entire data set. SwiftStack is building management, monitoring, and integration tools to turn the Swift storage engine into an effective storage solution that is used by millions, daily.

We're looking for engineers of all experience levels to join us. We do most of our work in Python, but we've got some Ruby, Javascript, and a few other things mixed into our stack.

So far this year, we've built out global clusters for a major expense reporting and travel support site[1], provided the storage system for Disney Interactive's gaming platform[2], and integrated Swift into radical new hard drive technology from Seagate[3]. We're currently working with companies like Intel and Box to build support for erasure codes into Swift[4].

My vision for OpenStack Swift is that it will be used by everyone, every day, even if they don't realize it. Swift is storage that is built for the web and already powers some of the world's largest storage clouds. I would like to work with you at SwiftStack to see this vision become a reality.

Interested? Send us an email at [email protected]. Send us your github profile, your LinkedIn account, a link to your website – whatever will best display the work that you’ve done. Tell us in a few lines of text why you’re interested in SwiftStack, and why we’ll be interested in you. We’ll be back in touch shortly to get the conversation started.

[1] Concur video: http://www.youtube.com/watch?v=qQmGfzewWQw&t=29m23s

[2] Disney + SwiftStack: http://venturebeat.com/2013/08/30/how-disney-built-a-giant-c...

[3] SwiftStack + Seagate: http://swiftstack.com/blog/2013/10/22/kinetic-for-openstack-...

[4] Building erasure codes into OpenStack Swift: http://swiftstack.com/blog/2013/07/17/erasure-codes-with-ope...