News for hackers. Stuff that matters.
Re:CDE: Easily run a program on any linux without thinking a... by sfk on Saturday November 13, 2010 @04:50PM (#1901807)
Reposting the comment of chbarts, which has been killed despite being entirely appropriate. Moderating has gone completely out of hand lately:

chbarts wrote:

" I think the potted example on the web page is somewhat poor, as it seems to be a problem existing package management systems already solve.

The more interesting aspect is the sandboxing, and how this is great for one-off software runs where it isn't worth it to install the needed packages on your base system because they're logically just a unit with the software you're testing. You mention this, but I think you'd do well to make it more obvious.

This is also, and this is something you mention, a way of preserving historic software and data. Bundle a full CDE package with an emulator and you can keep running that program for even longer; in fact, it seems a CDE package could gradually grow layers as time demands, from moving a (user-mode) kernel in, to adding device emulation, to going the final step and making the package more-or-less a ship in a bottle running in full hardware emulation.

This could also grow into a way to run untrusted (or minimally-trusted) software in an iron box, not just a sandbox. But that is harder and probably not your preferred direction."

Re:CDE: Easily run a program on any linux without thinking a... by pgbovine on Saturday November 13, 2010 @05:39PM (#1901892)
thanks for reposting, sfk. i don't know why the original comment would've been killed ... chbarts brought up a great critique about my presentation of cde.
Re:CDE: Easily run a program on any linux without thinking a... by blasdel on Saturday November 13, 2010 @07:51PM (#1902131)
It looks like they got capriciously hellbanned for comments in this thread: http://news.ycombinator.com/item?id=1320949 — everything after that was auto-killed, several dozen productive comments over the course of six months.

Items can be fully deleted or [dead]-ed, and users can be silently hellbanned so that all of their contributions are [dead] on arrival for everyone but themselves. AFAIK there are no other metamoderation mechanisms.

Re:CDE: Easily run a program on any linux without thinking a... by rbanffy on Saturday November 13, 2010 @01:49PM (#1901370)
Common Desktop Environment? I have a couple old RISC boxes running it...

;-)

Sorry. Couldn't resist...

Now, seriously, this goes a bit beyond package management by bringing along what seemed to be the specific versions of libraries. The goal is to do reproducible results for scientific data but, in this case, I would prefer not to have the original libraries packed along but having the dependencies flagged in a way that any version-specific bug could be easily pointed out.

Re:CDE: Easily run a program on any linux without thinking a... by pyre on Saturday November 13, 2010 @06:39PM (#1902000)
I also though Common Desktop Environment at first. At least from a 'marketing' perspective, it might be a good idea to not use an acronym that is already in use in the Linux/Unix world.
Re:CDE: Easily run a program on any linux without thinking a... by mattew on Sunday November 14, 2010 @10:21AM (#1903334)
This is a very good point. I can't see CDE and not think about all the environment on all the Sun workstations that I have had in the past.
Re:CDE: Easily run a program on any linux without thinking a... by darkandbrooding on Saturday November 13, 2010 @07:26PM (#1902071)
That was my first thought as well. My initial exposure to nix was a purchased copy of Red Hat Linux 4.2, running on an HP Vectra tower. and I remember paying extra for the CDE install CDs. An absolute noob when I started, by the time I'd learned enough to make my modem work I felt like I'd climbed a mountain. sigh* Good times.
Re:CDE: Easily run a program on any linux without thinking a... by js4all on Saturday November 13, 2010 @02:19PM (#1901444)
Common Desktop Environment. That was also my first thought. We had it on our Sun E250.
Re:CDE: Easily run a program on any linux without thinking a... by rbanffy on Saturday November 13, 2010 @02:27PM (#1901458)
Nice machine. Wicked fast on its day. I bet it still can hold its own despite the 2GB RAM limit.
Re:CDE: Easily run a program on any linux without thinking a... by js4all on Monday November 15, 2010 @02:12PM (#1907434)
It was and served well. But now it's not in use anymore.
Re:CDE: Easily run a program on any linux without thinking a... by pgbovine on Saturday November 13, 2010 @02:02PM (#1901399)
hi everyone, i'm the author of CDE, so please feel free to ping me if you have questions, complaints, or bug reports. i'm in the midst of fixing a bunch of silly bugs now and improving the documentation.
Re:CDE: Easily run a program on any linux without thinking a... by rbanffy on Saturday November 13, 2010 @02:38PM (#1901488)
I find the dependency-capture mechanism very interesting. I would love to see it used to help generating dependency lists for package management systems too.

I will probably have to build my first .rpm packages (I am yet to read the first docs) in the near future and CDE could be a lovely addition to my toolbox.

Thanks for doing it.

Re:CDE: Easily run a program on any linux without thinking a... by Goladus on Saturday November 13, 2010 @05:46PM (#1901905)
This looks awesome. My biggest initial fear (not having used it yet), is that I would inadvertently package up something like my private SSH key.
Re:CDE: Easily run a program on any linux without thinking a... by pgbovine on Saturday November 13, 2010 @05:54PM (#1901917)
oh good point. the contents of the cde-package is simply a directory hierarchy, so you can feel free to use 'find' to skim thru it. also, i'm now adding a feature where you can tell cde to ignore certain directories (e.g., ~/.ssh/) so that it won't pick up those files.
Re:CDE: Easily run a program on any linux without thinking a... by bostonvaulter2 on Saturday November 13, 2010 @07:24PM (#1902069)
Maybe you could output a warning on files with perms like 600?
Re:CDE: Easily run a program on any linux without thinking a... by partition on Sunday November 14, 2010 @12:28AM (#1902607)
Something like this has been sorely needed. I'm wondering if it's possible to generalize this over different OSes, if the code itself is multi-platform, or there is some way to specify hints for it (or should there be). I also go to Stanford so would be easy to discuss this in depth if you want.
Re:CDE: Easily run a program on any linux without thinking a... by pgbovine on Sunday November 14, 2010 @12:42AM (#1902632)
please email me at the address listed on the website. looking forward to chatting more about this!
Re:CDE: Easily run a program on any linux without thinking a... by sandGorgon on Sunday November 14, 2010 @02:43AM (#1902797)
could you put up a doc or something explaining the difference to chroot jails ?

because you mention: . CDE runs packaged programs in a sandbox similar to a chroot jail, but truly malicious applications can circumvent this sort of isolation attempt.

Do you use similar techniques to chroot - essentially I am asking whether your code is basically chroot (in terms of its behavior AS WELL AS its internals) with some security features removed ?

Re:CDE: Easily run a program on any linux without thinking a... by pgbovine on Sunday November 14, 2010 @02:49AM (#1902803)
no, it's not using the same mechanism as a real chroot (since you need to be root user to create a chroot). cde uses strace to intercept all the system calls that a monitored child process makes, and re-writes all file access system calls to re-route their paths inside of cde-root/.
Re:CDE: Easily run a program on any linux without thinking a... by mayanklahiri on Saturday November 13, 2010 @01:50PM (#1901374)
This is fantastic. I haven't tried it out yet, but I'll give it a shot for the paper I'm writing.

BTW, even with a "uniform" Linux environment, I've been finding so many differences in the behavior of libraries that it's driving me nuts. This will be a nice way to get past, for example, inconsistencies in wxWidgets.

Re:CDE: Easily run a program on any linux without thinking a... by pgbovine on Saturday November 13, 2010 @02:06PM (#1901410)
what research field are you working in? please email me personally if you'd like to chat more about how cde can help your workflow. i'm interested in collecting real-life case studies and anecdotes
Re:CDE: Easily run a program on any linux without thinking a... by mayanklahiri on Saturday November 13, 2010 @02:39PM (#1901494)
I replied to you via email, but thought I'd post it here as well in case anyone else had suggestions.

I've been thinking about a general purpose experimentation framework for research. Over the last 5 years, I've been re-writing parts of this framework for different projects and wish I had spent some time generalizing and packaging it.

The workflow is:

dataset -> experimental parameters -> code+libraries (this is where the trouble often is) -> result files (plain text, hundreds!) -> analysis (generally in R) -> selectively plotting results

Re:CDE: Easily run a program on any linux without thinking a... by pierrefar on Saturday November 13, 2010 @01:56PM (#1901386)
From my superficial knowledge, this is quite similar to the PC-BSD PBI format: http://www.pcbsd.org/content/view/20/26/ . They both allow you to distribute software in an easy non-hellish way on Linux.
Re:CDE: Easily run a program on any linux without thinking a... by bbulkow on Saturday November 13, 2010 @08:48PM (#1902256)
Wouldn't this cause severe problems with copywrite and licensing? Just because you have a copy of some library on a machine doesn't mean you have the ability to duplicate it to another machine, let alone archive it for future researchers.

I could also see the possible issue of pulling in GPL licensed libraries, and the general theory is if you ship a package with GPL licensed code in it, your code must be GPL. There's a fine line between "sending along someone else's libraries" and becoming a monolithic executable containing someone else's libraries - and between those points is where GPL gets you.

Re:CDE: Easily run a program on any linux without thinking a... by pilif on Sunday November 14, 2010 @01:46AM (#1902726)
There needs to be some kind of solution for the problem of really confidential files to be packaged. The second example in the video packaged the machines /etc/passwd for example (granted, it doesn't contain the passwords, but still).

The third example was showing some real bloat in that it also packaged the kernel and the initramd. I really don't see how these two can be of any use on the target machine.

Still. Very interesting idea and impressively easy to use.

Re:CDE: Easily run a program on any linux without thinking a... by pixdamix on Saturday November 13, 2010 @05:44PM (#1901903)
It reminds me of the Ermine Statifier. http://www.magicermine.com/sesl.html. This tools is so useful when you have to for example run tcpdump or a QT app on an embedded system without installing a TON of libraries or messing with LD_LIBRARY_PATH.
Re:CDE: Easily run a program on any linux without thinking a... by nitrogen on Saturday November 13, 2010 @04:22PM (#1901747)
Cool idea, and a worthy cause. Is there any provision for copyleft license compliance? Perhaps a distant-future version could use dpkg -S or the rpm equivalent to find which package owns a file, then apt-get source or something to optionally link or bundle a source package with the sandbox.
Re:CDE: Easily run a program on any linux without thinking a... by mkramlich on Saturday November 13, 2010 @03:43PM (#1901671)
awesome tool, great job!

no like the name though. sounds like a Unix desktop system.

Re:CDE: Easily run a program on any linux without thinking a... by InclinedPlane on Saturday November 13, 2010 @02:41PM (#1901500)
If anyone wonders why C/C++ programming is still so popular, here's a perfect case study. Most new fangled languages don't offer the option of static linking. This single piece of uncompromising idiological doctrine has led to billions of dollars in effort expended in engineering systems to workaround that choice and to effectively reproduce static linking in different form.
Re:CDE: Easily run a program on any linux without thinking a... by j_baker on Saturday November 13, 2010 @03:16PM (#1901592)
Most "newfangled" languages don't use linking in the same way as C and C++ do anyway (unless of course you're interfacing with C or C++). For instance, you can't statically link to a python source file. That's not because of ideology. It's because static linking doesn't even have any meaning.
Re:CDE: Easily run a program on any linux without thinking a... by d0mine on Saturday November 13, 2010 @05:34PM (#1901883)
Analog of static linking for a pure python library is including it within the library package. Another example: deploying python application in its own virtualenv environment.
Re:CDE: Easily run a program on any linux without thinking a... by mkramlich on Saturday November 13, 2010 @03:41PM (#1901666)
dynamic linking is generally a functional superset of static linking because you can bundle the needed libraries with the app, as say loose files in the app's dir tree, for example. This lets you avoid Shared DLL Hell while still allowing you to surgically deploy a critical bugfix to one of those libraries.