News for hackers. Stuff that matters.
Re:Torvalds: git vs CVS by nuclear_eclipse on Friday April 16, 2010 @05:57PM (#1272062)
Poor or inefficient support for large binary assets, such as compiled objects, images, or video. Especially in the context of video game development, binary assets that constantly change are highly painful to deal with in Git or Mercurial.
Re:Torvalds: git vs CVS by lnguyen on Friday April 16, 2010 @06:34PM (#1272149)
It's not limited to DVCS or open source. Commercial tools such as ClearCase, AllFusion/Harvest, and Dimensions still handle large binary assets poorly.

I don't think you'll find a satisfactory tool that has as its primary use the management of texts and deltas of texts. You'd probably need to use a combination of tools which has its own set of frustrations.

Re:Torvalds: git vs CVS by smallblacksun on Friday April 16, 2010 @07:05PM (#1272209)
Perforce works well with large binaries and text (costs a fortune, though).
Re:Torvalds: git vs CVS by sandGorgon on Friday April 16, 2010 @10:37PM (#1272489)
Bitkeeper (the SCM that git was "inspired" from), claims good support for large binary assets

BAM solves this problem with a hybrid approach. BAM adds the concept of one or more BAM servers. Each BAM server contain all BAM data, but clones of the server (work spaces) contain only that data that the user requests. Typically this is the most recent version of each binary but in some cases it may even be less than that.

BAM has been successfully deployed in the game development space with great success. Game developers continue to enjoy the benefits of distributed work flow without the penalty of carrying all versions in every workspace.

Re:Torvalds: git vs CVS by naner on Friday April 16, 2010 @06:08PM (#1272095)
How is it possible that nobody has dealt with this problem yet?
Re:Torvalds: git vs CVS by pmjordan on Friday April 16, 2010 @06:13PM (#1272109)
Active open source development rarely meets "AAA" game development.
Re:Torvalds: git vs CVS by sshumaker on Friday April 16, 2010 @07:45PM (#1272272)
One could argue that the traditional source control model (where everyone has a copy on their own machine) isn't a great way to deal with huge binary assets. I've worked on game projects where the binary churn was on the order of gigabytes a day. Even though everyone was physically co-located, when everyone arrived at 10am and started to update to head, a hundred computers all trying to download the days worth of 2 gigs of assets over the network didn't make for a pretty picture.

You can try to cronjob the updates, staggered around 4am. Or you can realize that the majority of people are only modifying a subset of these assets, and roll your own link-based asset manager for large binaries.

Re:Torvalds: git vs CVS by pmjordan on Saturday April 17, 2010 @06:11AM (#1272864)
Yes, absolutely, you wouldn't want everyone to keep pulling all data, git-style. The programmers usually only need the assets in the game-ready format, not the source models, textures, etc. which are typically much larger, and only needed by the artists/level designers working with them.
Re:Torvalds: git vs CVS by blasdel on Saturday April 17, 2010 @03:53AM (#1272779)
Because anyone cleverer than a doorknob uses the right tool for the job: rsync. They're almost always artifacts, not assets.

You handle versioning either through filesystem snapshots, or by just using different paths for each version.

Re:Torvalds: git vs CVS by ableal on Saturday April 17, 2010 @04:30AM (#1272812)
So, git/hg/etc are doorknobs ?

Of course not, it's not their problem. What annoys me is that the "You must use DVCS" memo is being sent around uncritically, and anyone who does not fit the use case is left out.

I think that people who deal with images/spreadsheets/data would also benefit from a good VCS tool. If the coders, having scratched their itch, did not declare the problem solved and move on ...

P.S. thanks for the polite answer. So, there is either no problem or no solution ... I think we're done here.

Re:Torvalds: git vs CVS by blasdel on Saturday April 17, 2010 @04:45AM (#1272823)
git/hg/etc are not doorknobs. My simile was comparing you, and people like you, to doorknobs. http://www.google.com/search?q=dumber+than+a+doorknob

No version control system, centralized or distributed, actually handles blobs better than just treating them as artifacts and using rsync.

The only way to actually solve the problem is to make the data not be blobs anymore, where the diff/merge/serialization code all understands at least the structural container format and can render it usefully. There's never going to be a general purpose tool that does that outside of a live-in smalltalk image (or similar).

The best you're going to get is tools vertically-integrated with the application, and all the ones I've seen to date (MS Office, Adobe Version Cue, etc.) all do a terrible job of even doing diffs/RCS, much less actually implementing a real VCS.

Re:Torvalds: git vs CVS by stevelosh on Saturday April 17, 2010 @10:30AM (#1273071)
Greg Ward and some others are working on it with the bfiles extension for Mercurial. It's a work in progress but at least it's getting the ball rolling: http://mercurial.selenic.com/wiki/BfilesExtension
Re:Torvalds: git vs CVS by ableal on Sunday April 18, 2010 @02:04AM (#1274452)
Thank you very much. I liked what I read in the design doc (http://vc.gerg.ca/hg/hg-bfiles/raw-file/tip/design.txt), which seems to be trying to automatically do the best thing possible under the hg constraints.

It's interesting that the names are "Distributed Version Control Systems", or "Revision Control", but the ghost of the word "source" (meaning code text, in 1972's SCCS) is still read into it.

Surprisingly, when DVCS are shouted as the bees' knees for everybody, people who have images, documents and data may think they were included. Apparently not, those benighted heathens should use rsync or whatever manually, and not soil the "source control" systems with their binaries. Or go find not-included plug-ins, if they can.

(Those people are apparently too dumb to understand that a binary "artifact" produced with an image editor, spreadsheet, or data analysis tool is fundamentally different from a blessed code file written with a text editor, and thus is not entitled to be under real version control, or have a DCVS tool automatically do something smart about it.)

The Perforce guys must be laughing their asses off.

Re:Torvalds: git vs CVS by ableal on Friday April 16, 2010 @06:23PM (#1272122)
I have been trying to point it out [1], without much luck. Today I got in early by chance, and decided on the Socratic approach ;-)

[1] http://news.ycombinator.com/item?id=1201559, http://news.ycombinator.com/item?id=1219082, http://news.ycombinator.com/item?id=1222905, http://news.ycombinator.com/item?id=1242374

Re:Torvalds: git vs CVS by prog on Friday April 16, 2010 @10:31PM (#1272482)
This is my pet peeve with all DVCSes. I tried this with bzr, hg and git and all fail. Not sure if any other DVCSes handle it.

I don't see it happening anytime soon as long as it justified using "its _source_ control". There are programming activities that require large binary files to be versioned along with source and this is a clearly a limitation of DVCSes IMO.

Apart from that I am quite happy moving to bzr from svn. Life is much easier now.

Re:Torvalds: git vs CVS by ajb on Saturday April 17, 2010 @07:34AM (#1272931)
Apparently vesta (http://www.vestasys.org) can handle large binaries (it's from the EDA world, so this is plausible). It's open source, but the extent to which it's distributed is debatable - you need a server, but anyone can install a server and they form a distributed network like a dvcs.
Re:Torvalds: git vs CVS by ableal on Sunday April 18, 2010 @12:15PM (#1275009)
Thanks, looks interesting, and it came from DEC via Compaq. It's billed as a Configuration Management System (i.e. managing builds). They do talk of managing tens of GB of derived data, which is also important.

The hardware part seems limited: "the Alpha group ... hardware description language files into Vesta's source code control", so not schematics or layout binaries as 'source'.

That site seems to be resting since 2006, but there is some activity at http://sourceforge.net/projects/vesta/ (releases in 2009, mailing lists with 2010 messages).

Re:Torvalds: git vs CVS by ajb on Sunday April 18, 2010 @02:42PM (#1275243)
Last time I looked, the IRC channel was somewhat more active than the list.
Re:Torvalds: git vs CVS by sshumaker on Friday April 16, 2010 @07:55PM (#1272281)
With binary files, you really want an explicit checkout (exclusive lock) system - since there's typically no way to merge them. This makes the commit/merge style SCSes a bad fit for binary assets (and partially explains the popularity of Perforce, which is a lock-based SCS).
Re:Torvalds: git vs CVS by dalore on Friday April 16, 2010 @08:56PM (#1272364)
That and Perforce is a commercial company that provides support with their product. By default perforce doesn't lock, but it knows when a user has a file checked out for "edit".

By default all files are set read only. This was a big downside when you had no internet connection.

Re:Torvalds: git vs CVS by koenigdavidmj on Saturday April 17, 2010 @03:17AM (#1272750)
Yeah, but 'chmod u+w filename' and you're back in business. The GUI has a "resolve offline work" feature that will find everything.

Of course, if you don't mark it writable (and just use :w! in Vim, for instance), then a 'p4 sync' may eat your work if anyone's changed the file.

Re:Torvalds: git vs CVS by dalore on Monday April 26, 2010 @06:43AM (#1294553)
Ahh the resolve offline work feature must be new. It wasn't around when I was using perforce.
Re:Torvalds: git vs CVS by koenigdavidmj on Monday April 26, 2010 @10:41PM (#1297026)
Sadly, it has no direct CLI equivalent. You may find http://p4delta.sourceforge.net/ useful, however; it's like `svn up` for Perforce.