News for hackers. Stuff that matters.
Re:Why I Won't Use Rimuhosting by apotheon on Thursday August 27, 2009 @05:28PM (#790105)
Gaining root in such an underhanded manner against the client's wishes is bordering on criminal behavior. It doesn't get much worse than that.

Of course, I write about security professionally. Maybe I have a different perspective on what constitutes unacceptable behavior than you have.

Re:Why I Won't Use Rimuhosting by SwellJoe on Thursday August 27, 2009 @05:55PM (#790187)
Gaining root in such an underhanded manner against the client's wishes is bordering on criminal behavior.

No. Not even close. Rimuhosting owns the box in question. You can't possibly break into your own box.

Of course, I write about security professionally. Maybe I have a different perspective on what constitutes unacceptable behavior than you have.

I work in the hosting industry professionally. I've never been or worked at a service provider, but I've worked for the industry for 12 years as a vendor.

As a security professional, perhaps you can take the other side's view for a moment...

What would you, as a security professional who was dumped into the role of running a hosting data center, do when a box on your network (which has thousands of other boxes) has been rooted and is sending out spam at a rate of a million messages per day, even if you know that your paying customer is not responsible for sending those messages? This much traffic is impacting network performance for others, is spewing filth into the mailboxes of innocent people all over the world, and your customer is unable to stop it or cannot be reached in a reasonable time frame. What do you believe your responsibility would be in such a case?

How about if it is (innocently) DoSing other systems on the network? This happens a lot more than you'd think.

Many people here are suggesting that simply applying resource limits would solve this problem...but that exhibits a lack of understanding of how such resource limits work, and the likely end result for the customer of such resource limiting in the event something has gone wrong. To the customer, a limit on resource usage could very well also result in services becoming unavailable. If you cap CPU at 10%, and your website gets enough traffic to need 90% of the CPU, you'll only be able to serve a small percentage of your clients (and they will wait a long time). The same is true of nearly every resource limit in a hosting environment. If you are using more than your fair share (which I know Aaron has explained he was not; again, probably poor judgment and problem assessment on the part of the tech) then when the limits are imposed, you will be "turned off" for some of your users. You needed those resources to serve all of your requests...you don't have those resources anymore, so you can't serve all of those requests anymore, or if you do, you do so very slowly.

As programmers, you all know there are no silver bullet for solving hard, complex problems. Resource usage in a virtualized environment with untrusted users is a hard, complex problem. No amount of hand-waving about your "rights" makes that less true.

Re:Why I Won't Use Rimuhosting by apotheon on Thursday August 27, 2009 @06:14PM (#790238)
> No. Not even close. Rimuhosting owns the box in question. You can't possibly break into your own box.

I don't think you're familiar with the common uses of "bordering on".

> What would you, as a security professional who was dumped into the role of running a hosting data center, do when a box on your network (which has thousands of other boxes) has been rooted and is sending out spam at a rate of a million messages per day, even if you know that your paying customer is not responsible for sending those messages?

I'd stop the server without gaining root against the client's wishes, immediately inform the client, and try to get things cleaned up to everyone's satisfaction (except the security cracker who rooted the box, of course). I wouldn't gain root on the box in violation of the client's obvious wishes, particularly after articulating specific policy saying that you can remove my initial root access to the box if you like, and shut down specific services that are making my life difficult for reasons entirely unrelated to the client's running of the site, then send an email later pointing out my policy that fails to mention I might do this.

Privacy is security, after all: http://blogs.techrepublic.com.com/security/?p=293

Nothing you say in your hypothetical examples in any way justifies accessing the customer's data, particularly by breaking in to the system. Shut it down? Sure, if circumstances call for that. Start meddling with server configuration and establish what amounts to a rootkit on the system (in effect, if not in technical truth) without the client's permission? That's just shady.

> No amount of hand-waving about your "rights" makes that less true.

Perhaps you should stop waving your hands about that straw man, then.

Re:Why I Won't Use Rimuhosting by SwellJoe on Thursday August 27, 2009 @07:23PM (#790369)
I don't think you're familiar with the common uses of "bordering on".

I'll promise to read up on it, if you'll learn the meaning of "not even close".

Nothing you say in your hypothetical examples in any way justifies accessing the customer's data, particularly by breaking in to the system. Shut it down? Sure, if circumstances call for that. Start meddling with server configuration and establish what amounts to a rootkit on the system (in effect, if not in technical truth) without the client's permission? That's just shady.

I will merely mention that most hosting provider customers not only accept this kind of thing, they demand it. If you spend a little time on web hosting forums (as I must because it is my industry) you will notice a very strong tendency for complaints to be system administration related. The customer expected more involvement than the hosting provider offered, and thus things went horribly awry.

I agree with you, entirely, that if you ask the host to never login to your system, they should respect that wish. But, I can also state without hesitation that you and I (and most people here at HN) are thoroughly in the minority in wanting our hosting provider to never login to our hosting systems. The default mode for hosting providers is to drop in on the box within a couple of comments in their ticketing system...if it can't be solved with one or two replies, then it's safest to simply drop in and fix it. For most hosting customers this is not an invasion of privacy or "breaking in", it is "great support".

Finally, as a security professional, I'm sure you're also aware that with access to the hardware, your host has root all the time. There is nothing you or I can do about it. Even more interestingly, the host also has the ability to login, poke around, and never tell you about it (and not leave a trail...just boot up a live CD, mount up the disk read only, and poke around til their heart's content). Also nothing you or I can do about that. With someone else having access to the hardware, you have nothing but good faith on the part of the hosting provider.

Re:Why I Won't Use Rimuhosting by apotheon on Thursday August 27, 2009 @08:37PM (#790468)
> I'll promise to read up on it, if you'll learn the meaning of "not even close".

I don't see how "I know you are, but what am I?" is a very strong argument.

> I will merely mention that most hosting provider customers not only accept this kind of thing, they demand it.

Give it to customers who want it. Don't ask someone if he wants it, get a "no" answer, then turn around and do it anyway. The former is good customer service. The latter is shady and underhanded.

> The default mode for hosting providers is to drop in on the box within a couple of comments in their ticketing system...if it can't be solved with one or two replies, then it's safest to simply drop in and fix it. For most hosting customers this is not an invasion of privacy or "breaking in", it is "great support".

Most hosting customers don't have explicit suggestions from the host that if they don't want the hosting provider logging in to the system they can remove their SSH keys -- and, more to the point, most hosting customers don't do that then find out the hosting provider's support personnel have been rooting around (pardon the pun) in their data anyway.

It's "breaking in" in this case only because the SSH key for access was removed, with the hosting provider's blessing, and they basically leveraged a local access vulnerability to give themselves root access.

If the hosting provider had a policy that forbade customers from obstructing host administrators from logging in to the machine, articulated in the terms and conditions of use, I'd say go for it -- but that's not the situation in this case at all.

> Finally, as a security professional, I'm sure you're also aware that with access to the hardware, your host has root all the time.

In principle, sure -- but when there's a clearly encouraged expectation of privacy, it's really bad form to break in to the system against the customer's wishes by virtue of having access to the hardware. That's a betrayal of trust.

> Even more interestingly, the host also has the ability to login, poke around, and never tell you about it (and not leave a trail...just boot up a live CD, mount up the disk read only, and poke around til their heart's content).

Indeed. The fact it wasn't kept secret in this case just shows how little they value the customer's request that they don't log in to the system with root privileges and muck about with the customer's data. I didn't say they were necessarily malicious about it -- but that doesn't mean it's not bad.

> With someone else having access to the hardware, you have nothing but good faith on the part of the hosting provider.

. . . or, in this case, bad faith.

Re:Why I Won't Use Rimuhosting by lsc on Thursday August 27, 2009 @08:29PM (#790456)
xm sched-credit -d username -w 10

would have solved the CPU problem without doing anything more than slowing down the heavy user. Giving each guest a single CPU rather than all CPUs on the box also helps a lot. In fact, if your 'bad' user bought the same percentage of the box as everyone else, if you left the weights at their default levels, that user would get approximately their fair share of CPU.

Tip: if you don't dedicate a core to the Dom0, you will have problems if anyone else uses much cpu, disk, or network.

If you want to do the same for network, simply use tc: http://wiki.xen.prgmr.com/xenophilia/2008/06/traffic-shaping... - if your problem is disk, use ionice: http://wiki.xen.prgmr.com/xenophobia/2008/06/using-ionice1-t...

The whole point of providing Xen instances is that I don't want to muck around in your instance. It pisses you off, and it wastes my time. That's why I'm using xen, and not OpenVZ or some other glorified chroot. From the point of view of a provider, handling everything within the Dom0 not only makes customers more comfortable, it's also less work for you, the provider.

Re:Why I Won't Use Rimuhosting by astine on Thursday August 27, 2009 @07:12PM (#790353)
You can't possibly break into your own box.

Yes you can; I've broken into my own house before.

Personally, I'd like it if they gave you notice before doing anything to a VPS you may be using so that you have the opportunity to shut things down gracefully.

Even a sudden CPU throttle would be better than shutting down services on the customer's VPS.

Edit: spelling

Re:Why I Won't Use Rimuhosting by ErrantX on Thursday August 27, 2009 @06:08PM (#790223)
as another security professional I'm inclined to agree with Swelljoe.

(mostly because it's clearly going to be in the T&C's)

Re:Why I Won't Use Rimuhosting by apotheon on Thursday August 27, 2009 @06:20PM (#790249)
If there's something in the terms and conditions that says they reserve the right to change the configuration on your system because their explicitly identified policies for dealing with high CPU usage aren't working, I might back off from the "borderline criminal" descriptor -- but I'd still say that's a great way to make customers feel violated and run like the wind to other service providers. As many others have pointed out, a much less invasive approach would have been to simply pause the server and get in touch with the client -- especially after the service provider asked for permission to get root access, and was turned down.
Re:Why I Won't Use Rimuhosting by ErrantX on Thursday August 27, 2009 @06:58PM (#790329)
yep, I'd agree entirely with that :)
Re:Why I Won't Use Rimuhosting by mbreese on Thursday August 27, 2009 @09:42PM (#790537)
In this case their posted terms and conditions aren't very clear on the extent that they can muck around with a customer's VM. It's quite clear that they can suspend or kill it, but it's unclear how far they can go to "restrict" the customer's VM.
Re:Why I Won't Use Rimuhosting by rbanffy on Thursday August 27, 2009 @09:44PM (#790539)
Rimuhostinh owns the box. The client rents it, but the owner never gives up the right to step in and fix something if it is impacting the service they provide for other clients.

If I require this level of privacy, I will go with a co-location contract, not a virtual machine. Those are different products with different features. A VM on a shared chassis is hardly a private computer.

Oddly, I have some experience with virtual servers and never had much trouble throttling down an instance that wanted too much resources.