mycelium
I would say that you should pilot with a few ETL vendors. We currently use Fivetran, they're fine but we've had enough burps that I cannot cold recommend them over other vendors. I cannot for the life of me remember the details, but I think we went with them over Stitch for pricing reasons.
If people can't get off their asses to stop working with a company that has worked with organizations responsible for thousands of deaths, I don't think support of Gab is going to drive anyone's payment platform decisions.
Thinking of it as inspection alone is a disservice to the cultural value of code review. You want a process that can teach a team member to contribute with lesser inspection? Code review.
I've typically found that with a new team or team member, initially there are a lot of patches returned for modification. After 3-9 months, it tends towards 80% of patches being one-shot LGTMs, the remaining 20% having spec issues or substantive style issues (ie. this module structure will bite us in the butt because...)
This idea of communicating shared knowledge also points at more efficient ways to do that, if CR is a bottleneck:
- Technical onboarding - Google does a great job of this. Taking time to explain how to work with your technologies, and what the expected code style is.
- Linting + style guides - Arguing over style is dumb.
Beyond the "this will likely deadlock without a timeout", "this does not clean up after itself on exception", "if the cache is not hot, this operation you assume is 1 sec will take 1 min" type thing — one of the strongest points of code review is having a conversation about "this code is hard to understand" and turning code that's correct if you think about it really hard into obviously correct code.
It takes a reasonably strong reviewer to get there though, and that's not always possible.
Draw from that what you may, but I bet that dynamic is happening in other places.
I'd be more concerned in Google's case about "legitimate" but still problematic uses of data: disclosure to law enforcement, crazy specific ad targeting, machine learning for military applications, etc.
I'm going to echo my sibling commenter, would love to hear it if you have any pointers on where to start reading.
I'm a software engineer but data work comes up all the damn time. Seeing coworkers approach causality in a hand-wavy way without knowing a more rigorous approach to suggest is frustrating! I had hoped Pearl's book was exactly that.
http://discovermagazine.com/1987/may/02-the-worst-mistake-in...
His podcast Managager Tools is also excellent, and has deep archives addressing pretty much every question under the sun: https://www.manager-tools.com/map-universe/hall-fame-highly-...
@slap_shot and anyone else — I'm curious if you have thoughts on, or even have heard of the Ballerina language? It's a programming language for doing data integration work, built by the ESB/integration consultancy WSO2. It seems to have a lot of eng resources sunk into it but surprisingly little fanfare.
The CEO's interview with the Software Engineering Daily podcast was great: https://softwareengineeringdaily.com/2018/07/12/ballerina-la...
The language site tends toward buzzword-salad, but clearly has had a lot of love and thought put into it: https://ballerina.io/philosophy/
They ended up being pretty weak, having trouble shipping, eating large amounts of onboarding and management attention, as well as other engineers patching up their work to make it serviceable. They were just good enough to make it seem worth the slog too, so they sucked a bunch of extra labor out of everyone else for the better part of a year before they left.
In retrospect, I made the right decision with what I knew in the moment — but with 20/20 hindsight I should have waited.
Once you do tutorial and learn the rules for going between shell and python, it's super easy to slice and dice.
I have always disliked that moment of "yes, a shell script is probably the best tool for this job" that then leads me to banging my head against bash syntax for an hour. Python libraries like `sh` are great, but still a little high overhead. Now the twice a month I have to shell script I find to be pretty fun, and I'd highly recommend xonsh.
Great PyCon 2016 talk on it here: https://www.youtube.com/watch?v=uaje5I22kgE
The main critique I have is their push to monetize as of late has been clumsily executed in my experience of the product. The expanded "pro" features have shifted around without enough heads up. The chatroom ownership system is not at all clear in how it interacts with paid accounts. We abandoned a few rooms used widely in our org until we figured out whose random account we had used to own a room earlier (from a prior push by Appear for room ownership) and got ownership transferred to a paid account.
Do you have any resources you've found helpful improving your skill at this?
To be fair to the grandfather comment, that was the point.
> gunicorn
Any particular reason you use it over uwsgi? I last did real research in 2013 and have not revisited since then.
Frontend frameworks: https://i.imgur.com/Sd1g5zw.png
State management frameworks: https://i.imgur.com/HCD32bo.png
I'd highly recommend his channel to anyone interested in science and fabrication.
This seems like a bad idea from a business value perspective. What value is delivered by translating mountains of code? Probably none, being honest.
As I am also partially Java guy, I came to conclusion that migration of project into JRuby and then embedding Clojure parts gradually might ease the transition.
Don't transition everything, giant waste of time. Run ruby/rails on jruby, then package the core Clojure logic up as a library, and call into it from rails where needed. Clojure is your core logic, ruby/rails is the web front end to that core.
edit: And to my sibling commenter's point, DRF has a funding model, so it's being actively worked on full time.