Re:Fork Your Application's Data (Not Just Your Code)
by eggbrain on Thursday November 08, 2012 @02:49PM
(#4759496)
One of the biggest problems I have is that I'll branch out to a new feature branch, modify (add/rename columns) in the db, then get sidetracked by something where I have to switch back to master, making everything explode because the db is still modified, while my application is calling the old values.
I'd love it if they found a way to link up your database to a repo, so that it could track/rollback changes easily as switching branches (I guess an sqlite db might work, but it's not perfect)
Re:Fork Your Application's Data (Not Just Your Code)
by amatix on Thursday November 08, 2012 @05:35PM
(#4760270)
I've read several times about github's use of branches in deployment/production. How are DB schema/data migrations handled (and kept safe/sane)? Presumably hubot and friends prevent Bad Things from happening by deploying a branch that doesn't match the current DB state?
Re:Fork Your Application's Data (Not Just Your Code)
by wahnfrieden on Thursday November 08, 2012 @09:19PM
(#4760932)
I didn't say one way is strictly better. There are tradeoffs to consider and that's all that really matters. I'm not sure it occurred to the OP that there are benefits to forgoing remote branches, so I mentioned it.
Re:Fork Your Application's Data (Not Just Your Code)
by grey-area on Friday November 09, 2012 @03:00AM
(#4761844)
Keep your database config for the app in version control with the code, and change to another database per branch.
This means your data in that db will diverge from the main branch slightly, but if that is too much of a problem and you need constant updates from live, you could look into migrations like those used by rails, which store your db changes so that they can be replayed on top of newer data from master and bring it up to the new schema. It's quite easy to setup a bare bones migration system using plain SQL migration files and some scripts if this is a problem you see often.
I often work with several branches of code in rails, and it's not a big problem, you just need a system for keeping the data in sync with the code.
Re:Fork Your Application's Data (Not Just Your Code)
by wahnfrieden on Thursday November 08, 2012 @09:22PM
(#4760940)
Consider forgoing long-lived/significant local branches too then. Branching has its downsides, and avoiding it has some upsides particularly for small teams. As an alternative, you might start using feature flags instead. Look into "continuous deployment" as well for more insight.
Re:Fork Your Application's Data (Not Just Your Code)
by atomical on Thursday November 08, 2012 @03:21PM
(#4759645)
I would disagree with this strategy as it adds more complexity than the original problem. It's much easier to rollback the migrations on the feature branch and switch to master. Avoiding situations like this is the best policy though as unexpected things will happen and you can be less sure that things are working properly.
Re:Fork Your Application's Data (Not Just Your Code)
by eggbrain on Thursday November 08, 2012 @03:41PM
(#4759723)
"Avoiding situations like this is the best policy though as unexpected things will happen"
Yes, but they happen. This seems like replying to someone drowning "well, you should have avoided swimming!"
You also make the assumption that we are using Rails (or some framework with migration support) -- many people won't have that luxury.
Finally, while the setup might be challenging (as in, figuring out a way to implement db tracking in git), I find it very curious that it would be much easier in anyones mind to do two tasks, doing all the rollbacks and switching branches, vs just switching the branch.
Re:Fork Your Application's Data (Not Just Your Code)
by eggbrain on Thursday November 08, 2012 @05:53PM
(#4760328)
Git checkout -b "feature-1"
Git commit -am "Added feature 1, which included a migration"
Git checkout -b "feature-2-that-builds-off-feature-1"
Git commit -am "Added feature 2, which includes another migration"
Git checkout master
Broken (and with multiple migration files, even though I branched off for each new feature.)
Re:Fork Your Application's Data (Not Just Your Code)
by robertfw on Thursday November 08, 2012 @03:22PM
(#4759649)
One approach is to use multiple local databases, one for each schema in use. When you switch branches, update your app config to point at the appropriate database.
It's not perfect and we still have to do work keeping in sync within our team, but it does the job locally.
Re:Fork Your Application's Data (Not Just Your Code)
by ubercore on Thursday November 08, 2012 @11:03PM
(#4761215)
We've been using a db-follows-branch system with Fabric. Devs can run "fab update_db" to get a new dump of production-based testing data, and create a database named after the current branch. Still a bit manual, but it really helps in situations like that.
Re:Fork Your Application's Data (Not Just Your Code)
by arscan on Thursday November 08, 2012 @04:06PM
(#4759833)
Personally I find this interesting from a marketing perspective... its not like cloning a DB is a new concept, but by calling it "forking your data" they make it seem quite a bit sexier.
I'm not sure if technically its the best term to use, but it does sound cool.
Re:Fork Your Application's Data (Not Just Your Code)
by TheMakeA on Thursday November 08, 2012 @02:37PM
(#4759453)
Interesting. I would take it a step further and say that everything related to your project should be forkable and branchable. Your source, your assets, your databases, etc. I would love to be able to go back to any moment in the projects history and get a full, working build.
Re:Fork Your Application's Data (Not Just Your Code)
by Q6T46nT668w6i3m on Thursday November 08, 2012 @03:11PM
(#4759595)
I would take it a step further and say that everything related to your project should be forkable and branchable. Your source, your assets, your databases, etc.
Re:Fork Your Application's Data (Not Just Your Code)
by TheMakeA on Thursday November 08, 2012 @03:14PM
(#4759609)
It is possible with Perforce, but it is far from ideal. Compare branching in Git to branching in Perforce. There's still a lot of room for improvement.
Re:Fork Your Application's Data (Not Just Your Code)
by CookWithMe on Thursday November 08, 2012 @04:32PM
(#4759963)
What about images? Should one save them in the DB? Is Postgres able to handle high-res images?
Last time I checked (for MySQL) it was possible, but not really recommended/wide-spread. Does anyone have practical experience with Postgres (and Heroku)?
Re:Fork Your Application's Data (Not Just Your Code)
by 5vforest on Thursday November 08, 2012 @02:55PM
(#4759522)
This is neat. I've always used Amazon RDS in the past because of its ease of import/export, but this may be enough to switch me over to Heroku's offerings.
Re:Fork Your Application's Data (Not Just Your Code)
by sterling312 on Thursday November 08, 2012 @05:41PM
(#4760292)
I've been wanting people to post their eigenvector/value on facial recognition training for a while. Imagine a crowdsource trained facial recognition database!
I'd love it if they found a way to link up your database to a repo, so that it could track/rollback changes easily as switching branches (I guess an sqlite db might work, but it's not perfect)
GitHub is a documented example (and I happen to deploy branches daily too :-):
https://github.com/blog/1241-deploying-at-github
This means your data in that db will diverge from the main branch slightly, but if that is too much of a problem and you need constant updates from live, you could look into migrations like those used by rails, which store your db changes so that they can be replayed on top of newer data from master and bring it up to the new schema. It's quite easy to setup a bare bones migration system using plain SQL migration files and some scripts if this is a problem you see often.
I often work with several branches of code in rails, and it's not a big problem, you just need a system for keeping the data in sync with the code.
http://mislav.uniqpath.com/rails/branching-the-database-alon...
This way you can enable a branch-specific database (if needed) and switch back and forth during the day if you need it.
One could somehow easily adapt that for another development framework, though!
Yes, but they happen. This seems like replying to someone drowning "well, you should have avoided swimming!"
You also make the assumption that we are using Rails (or some framework with migration support) -- many people won't have that luxury.
Finally, while the setup might be challenging (as in, figuring out a way to implement db tracking in git), I find it very curious that it would be much easier in anyones mind to do two tasks, doing all the rollbacks and switching branches, vs just switching the branch.
"Sorry, I can't look into your emergency production problem, I'm adding features to a blog."
It's not perfect and we still have to do work keeping in sync within our team, but it does the job locally.
Any thoughts about which DB would be best suited for this? I wonder if it could be integrated with git somehow and make the whole process seamless.
Yep, look like not as easy I think at the start of the reply...
I'm not sure if technically its the best term to use, but it does sound cool.
Without the ability to merge, shouldn't it just be called 'copying'
https://postgres.heroku.com/dataclips
I'll have to wait for them to become SafeHarbor compliant before using them, though (in progress!).
Isn’t that the advantage of Perforce?
http://www.perforce.com/product/product_features/perforce_st...
Last time I checked (for MySQL) it was possible, but not really recommended/wide-spread. Does anyone have practical experience with Postgres (and Heroku)?