Re:Sliceapp.com - Bootstrap Slicehost Servers Automatically
by superjared on Tuesday August 11, 2009 @07:41PM
(#756802)
Looks like you have to give your root password, which, no offense to Clever Collie, doesn't seem like a good idea. I'd prefer to see a shell script generator, or something like that.
Re:Sliceapp.com - Bootstrap Slicehost Servers Automatically
by melito on Tuesday August 11, 2009 @08:14PM
(#756883)
Yea, as of now there's no way around that.
I'm working on adding 'sudo' logic into the script, but that doesn't seem any less risky security wise.
Users are welcome to use something like whowatch or a tty spy to view the progress of the script. What you see in the browser is what is going is happening on the machine.
Users are also welcome to set their passwords to something temporary and then change them when the deployer is complete.
In the end though, the only thing I can offer to put user's minds at ease is my word that I don't store any of their login info.
For the more technically curious I'm not even using a database right now. I just log the job id, the ip address you requested, and whether or not an email was successfully sent to the one you provided. Login information is only held in memory until the bootstrapper successfully connects to the machine. After that its gone.
As for the automated shell script, as of now it would be incredibly tedious to replicate something like this using standard shell scripting (for me at least). I've thought about adding a "copy to clipboard" button above the terminal, that would copy all the executed commands in the textarea. This could be pasted into a script, but it would still need some interaction from the user.
This is all very good feedback and I'm open to anymore that would help improve the project and make a better overall user experience.
Re:Sliceapp.com - Bootstrap Slicehost Servers Automatically
by clemesha on Tuesday August 11, 2009 @06:32PM
(#756613)
Cool! Can you say a little about how you are loading each instance with the chosen packages? For example, are you using Capistrano or Fabric to script the installs? Will you be offering contextualizing functionality, based on some sort of user data (ala EC2 user-data + script execution)? Anyways, nice project.
Re:Sliceapp.com - Bootstrap Slicehost Servers Automatically
by goodkarma on Tuesday August 11, 2009 @06:36PM
(#756626)
I love the idea!
I've always found Slicehost and other DIY hosting solutions to be difficult to set up and work with. Which is why a solution like Heroku is so tempting (though various limitations have precluded my using it so far).
I'm excited about the potential for something like this.
Re:Sliceapp.com - Bootstrap Slicehost Servers Automatically
by NoBSWebDesign on Wednesday August 12, 2009 @11:13AM
(#758256)
This is great. I have already setup my couple-page-long capistrano scripts to setup my slices, but this would have definitely prevented me from needing to do that.
I'd also like to see a munin package install (with Rails-specific plugins). Does this use Passenger for Rails installation?
Re:Sliceapp.com - Bootstrap Slicehost Servers Automatically
by rythie on Tuesday August 11, 2009 @08:05PM
(#756863)
Interesting. I wrote a shell script a couple months ago for our slicehost servers - I can build a machine in about 5 mins with it. It basically does yum installs (centos) and rsync and starts some services. Ready to scale in a hurry if I need it and avoids needing backup for every slice.
Re:Sliceapp.com - Bootstrap Slicehost Servers Automatically
by rythie on Wednesday August 12, 2009 @06:11AM
(#757685)
It's really something that is specific to what we do and I had/have no intention of making it generic or let alone designing the type of thing that helps people who don't have experience doing sys admin - because I want to stay focused. For those that are interested these are the main techniques I use:
First I go to slicehost and create a machine, they send me a email when it's done (about 5 mins or so).
I have one install script and a syncing script, The install scripts one calls the sync script. After the machine is live I just use the sync one to update it. $hostname is the name or IP address of the new machine.
First I copy SSH authorization over, so I can login without a password (assumes you have created SSH keys already - which is one time event):
My script took me a few hours to write and I did so that I can scale up (and down) in a hurry by adding a machine rather than taking the existing machines (there are 2) down whilst slicehost resize it - which is not something I wanted do while there is a lot of traffic coming in. We don't really need multiple machines, but buying one big slice is expensive to have running the whole time.
Re:Sliceapp.com - Bootstrap Slicehost Servers Automatically
by kirubakaran on Wednesday August 12, 2009 @12:18AM
(#757269)
If it just displayed the script that it generates instead of asking for password to run the script itself, it will be much simpler. OS installation is anyway just a single step with Slicehost webapp.
Re:Sliceapp.com - Bootstrap Slicehost Servers Automatically
by bmelton on Tuesday August 11, 2009 @06:29PM
(#756601)
Not to take anything away from melito, because really, this is cool as beans, but what I _really_ want out of something like this is a place where I can set my configuration up with variables.
Better than booting a semi-configured server would be booting a fully-configured server. If I could host my app on sliceapp.com, along with changes from a base bootstrapped instance (e.g., apache settings, hostnames, etc) -- and be able to have multiple images to launch (one for db, one for apache/mod-python/django, one for load balancer) -- then this would be the best thing that ever happened, pretty much.
Re:Sliceapp.com - Bootstrap Slicehost Servers Automatically
by melito on Tuesday August 11, 2009 @07:14PM
(#756719)
I have plans for that. I released the app, as is, to gauge how much interest there is in something like this.
The app originally started off as kind of a 'cloud management' console that would allow you to create multiple slices on the fly, have packages installed on them, and then have a central place to monitor their health and what was running.
The problem was, I was doing this all on my own.
By the time the original project reached a 'first draft', it was had become really complex and a pain to make progress on easily. The other downside was this complex project was tied to just one hosting provider. Integrating features and support for other platforms would have taken a great deal of work and increased the complexity of the app exponentially.
So I scratched everything and started off with something simple.
I figured at its essence I needed something that would connect to a machine and setup a bunch of software.
So I put together support for a limited number of packages and only one version of Ubuntu.
By doing so I was able to piece together something that is a lot easier for me to expand upon. A pseudo-framework. Something I'm comfortable working in and can change with little effort.
Future goals include:
Adding support for more software
Adding support for more operating systems
Testing against other hosting providers (this is a cloud!)
A management console
The management console will allow you to save 'recipes', so you can have a custom deploy package.
I also have the ability for making the system interactive (ie, a user is prompted for input via the web and their response is relayed to the server). This hasn't been tested as fully as I'd like, and doesn't serve much of a purpose at this point so I left it out of this release.
As for the architecture of the site, NONE of this was built using Capistrano, Vlad, or Fabric. I think all of those projects are great, but they are very cumbersome and don't allow for the flexibility that I get from what I currently have in place.
Judging by the interest this has stirred up over twitter and now Hacker News, I feel it'd be a good idea to move forward.
That being said, once I get a more fleshed out application in place, I plan on extracting the bits I'm using to bootstrap and releasing it as an open source tool for deployment.
Re:Sliceapp.com - Bootstrap Slicehost Servers Automatically
by melito on Tuesday August 11, 2009 @07:24PM
(#756753)
Also, if you do use this app and find it useful I am VERY open to feedback.
If you encounter any errors or wish that the app did something that it currently doesn't do, PLEASE email me via either [email protected] OR [email protected]
Full disclosure: I'm a developer at Slicehost.
I'm working on adding 'sudo' logic into the script, but that doesn't seem any less risky security wise.
Users are welcome to use something like whowatch or a tty spy to view the progress of the script. What you see in the browser is what is going is happening on the machine.
Users are also welcome to set their passwords to something temporary and then change them when the deployer is complete.
In the end though, the only thing I can offer to put user's minds at ease is my word that I don't store any of their login info.
For the more technically curious I'm not even using a database right now. I just log the job id, the ip address you requested, and whether or not an email was successfully sent to the one you provided. Login information is only held in memory until the bootstrapper successfully connects to the machine. After that its gone.
As for the automated shell script, as of now it would be incredibly tedious to replicate something like this using standard shell scripting (for me at least). I've thought about adding a "copy to clipboard" button above the terminal, that would copy all the executed commands in the textarea. This could be pasted into a script, but it would still need some interaction from the user.
This is all very good feedback and I'm open to anymore that would help improve the project and make a better overall user experience.
I've always found Slicehost and other DIY hosting solutions to be difficult to set up and work with. Which is why a solution like Heroku is so tempting (though various limitations have precluded my using it so far).
I'm excited about the potential for something like this.
I'd also like to see a munin package install (with Rails-specific plugins). Does this use Passenger for Rails installation?
First I go to slicehost and create a machine, they send me a email when it's done (about 5 mins or so).
I have one install script and a syncing script, The install scripts one calls the sync script. After the machine is live I just use the sync one to update it. $hostname is the name or IP address of the new machine.
First I copy SSH authorization over, so I can login without a password (assumes you have created SSH keys already - which is one time event):
ssh root@$hostname "mkdir /root/.ssh; echo \"`cat /root/.ssh/id_dsa.pub`\" > /root/.ssh/authorized_keys2"
Then run various commands on the new machine. I put these in a file "commands.fragment". which contains things like
and so on, then run it: The I run the syncing script (which I'll rerun over the life of the machine) Then I sync my code across: and programs I've complied: and so on.Now we have everything in place we can start the services for whatever the box does, for example to start a web server:
And that's it.My script took me a few hours to write and I did so that I can scale up (and down) in a hurry by adding a machine rather than taking the existing machines (there are 2) down whilst slicehost resize it - which is not something I wanted do while there is a lot of traffic coming in. We don't really need multiple machines, but buying one big slice is expensive to have running the whole time.
Hope that helps.
Better than booting a semi-configured server would be booting a fully-configured server. If I could host my app on sliceapp.com, along with changes from a base bootstrapped instance (e.g., apache settings, hostnames, etc) -- and be able to have multiple images to launch (one for db, one for apache/mod-python/django, one for load balancer) -- then this would be the best thing that ever happened, pretty much.
The app originally started off as kind of a 'cloud management' console that would allow you to create multiple slices on the fly, have packages installed on them, and then have a central place to monitor their health and what was running.
The problem was, I was doing this all on my own.
By the time the original project reached a 'first draft', it was had become really complex and a pain to make progress on easily. The other downside was this complex project was tied to just one hosting provider. Integrating features and support for other platforms would have taken a great deal of work and increased the complexity of the app exponentially.
So I scratched everything and started off with something simple. I figured at its essence I needed something that would connect to a machine and setup a bunch of software.
So I put together support for a limited number of packages and only one version of Ubuntu.
By doing so I was able to piece together something that is a lot easier for me to expand upon. A pseudo-framework. Something I'm comfortable working in and can change with little effort.
Future goals include: Adding support for more software Adding support for more operating systems Testing against other hosting providers (this is a cloud!) A management console
The management console will allow you to save 'recipes', so you can have a custom deploy package.
I also have the ability for making the system interactive (ie, a user is prompted for input via the web and their response is relayed to the server). This hasn't been tested as fully as I'd like, and doesn't serve much of a purpose at this point so I left it out of this release.
As for the architecture of the site, NONE of this was built using Capistrano, Vlad, or Fabric. I think all of those projects are great, but they are very cumbersome and don't allow for the flexibility that I get from what I currently have in place.
Judging by the interest this has stirred up over twitter and now Hacker News, I feel it'd be a good idea to move forward.
That being said, once I get a more fleshed out application in place, I plan on extracting the bits I'm using to bootstrap and releasing it as an open source tool for deployment.
If you encounter any errors or wish that the app did something that it currently doesn't do, PLEASE email me via either [email protected] OR [email protected]
I'll probably give it a shot this weekend, if nothing else than to test how quickly this tool enables me to migrate to a new slice.