mvermaat

WeTransfer | Data Engineer | Amsterdam | Full-time | On-site

We're a fast growing profitable scaleup based in Amsterdam and Los Angeles. Our mission is to provide the effortless transfer of creative ideas, which results in transferring more than 1 billion files per month between our users. To help support our strategies and ideas with crystal clear data, we're expanding our growth and analytics team and are looking for an engineer with experience in dealing with large (>billion rows) amounts of data and strong programming skills. Our stack:

- AWS (heavy users of EC2, S3, RDS, RedShift, DMS, Lambda, ...)

- Datastores are MySQL, Redis, PostgreSQL, DynamoDB, InfluxDB, ElasticSearch, RedShift, BigQuery

- Other tools we use are Airflow, Metabase, Snowplow

https://wetransfer.homerun.co/

Apply there or send me an email at [email protected]

(We're also looking for React and Ruby wizards, and other roles)

WeTransfer | Data Engineer | Amsterdam | Full-time | On-site

We're a fast growing profitable scaleup based in Amsterdam and Los Angeles. Our mission is to provide the effortless transfer of creative ideas, which results in transferring more than 1 billion files per month between our users. To help support our strategies and ideas with crystal clear data, we're expanding our growth and analytics team and are looking for an engineer with experience in dealing with large (>billion rows) amounts of data and strong programming skills. Our stack:

- AWS (heavy users of EC2, S3, RDS, RedShift, DMS, Lambda, ...)

- Datastores are MySQL, Redis, PostgreSQL, DynamoDB, InfluxDB, ElasticSearch, RedShift, BigQuery

- Other tools we use are Airflow, Metabase, Snowplow

https://wetransfer.homerun.co/

Apply there or send me an email at [email protected]

(We're also looking for React and Ruby wizards, and other roles)

WeTransfer | Data Engineer | Amsterdam | Full-time | On-site

We're a fast growing profitable scaleup based in Amsterdam and Los Angeles. Our mission is to provide the effortless transfer of creative ideas, which results in transferring more than 1 billion files per month between our users. To help support our strategies and ideas with crystal clear data, we're expanding our growth and analytics team and are looking for an engineer with experience in dealing with large (>billion rows) amounts of data and strong programming skills. Our stack:

- AWS (heavy users of EC2, S3, RDS, RedShift, DMS, Lambda, ...)

- Datastores are MySQL, Redis, PostgreSQL, DynamoDB, InfluxDB, ElasticSearch, RedShift, BigQuery

- Other tools we use are Airflow, Metabase, Snowplow

https://wetransfer.homerun.co/

Apply there or send me an email at [email protected]

(We're also looking for React and Ruby wizards, and other roles)

WeTransfer | Platform Engineer | Amsterdam | Full-time | On-site

We're a fast growing profitable scaleup based in Amsterdam and Los Angeles. Our mission is to provide the effortless transfer of creative ideas, which results in transferring more than 1 billion files per month between our users. To keep up with that, we're expanding our platform team and are looking for an engineer with experience in AWS infrastructure-as-code and continuous deployment. Our stack:

- AWS (heavy users of EC2, S3, RDS, CloudFront, RedShift, SQS, ...)

- Terraform for infrastructure, Ansible for provisioning

- Datastores are MySQL, Redis, PostgreSQL, DynamoDB, InfluxDB, ElasticSearch

- Applications are written in Ruby, Elixir, and Go

https://wetransfer.homerun.co/

Apply there or send me an email at [email protected]

(We're also looking for Ruby wizards)

WeTransfer | Platform Engineer | Amsterdam | Full-time | On-site

We're a fast growing profitable scaleup based in Amsterdam and Los Angeles. Our mission is to provide the effortless transfer of creative ideas, which results in transferring more than 1 billion files per month between our users. To keep up with that, we're expanding our platform team and are looking for an engineer with experience in AWS infrastructure-as-code and continuous deployment. Our stack:

- AWS (heavy users of EC2, S3, RDS, CloudFront, RedShift, SQS, ...)

- Terraform for infrastructure, Ansible for provisioning

- Datastores are MySQL, Redis, PostgreSQL, DynamoDB, InfluxDB, ElasticSearch

- Applications are written in Ruby, Elixir, and Go

https://wetransfer.homerun.co/

Apply there or send me an email at [email protected]

(We're also looking for a Data Engineer and Ruby wizards)

WeTransfer | Data Engineer | Amsterdam | Full-time | On-site

We're a fast growing profitable scaleup based in Amsterdam and Los Angeles. Our mission is to provide the effortless transfer of creative ideas, which results in transferring more than 1 billion files per month between our users. As a part of the Engineering department, you will be responsible for the design, development, implementation and support of our analytical tools.

The position is flexible: the exact role will depend on the candidate's experience and ambitions, future hiring, and needs of the company.

https://wetransfer.homerun.co/

Apply there or send me an email at [email protected]

(We're also looking for a Platform Engineer and Ruby wizards)

WeTransfer | Platform Engineer | Amsterdam | Full-time | On-site

We're a fast growing profitable scaleup based in Amsterdam and Los Angeles. Our mission is to provide the effortless transfer of creative ideas, which results in transferring more than 1 billion files per month between our users. To keep up with that, we're expanding our platform team and are looking for an engineer with experience in AWS infrastructure-as-code and continuous deployment. Our stack:

- AWS (heavy users of EC2, S3, RDS, CloudFront, RedShift, SQS, ...)

- Terraform for infrastructure, Ansible for provisioning

- Datastores are MySQL, Redis, PostgreSQL, DynamoDB, InfluxDB, ElasticSearch

- Applications are written in Ruby, Elixir, and Go

https://wetransfer.homerun.co/

Apply there or send me an email at [email protected]

WeTransfer | Platform Engineer | Amsterdam | Full-time | On-site

We're a fast growing profitable scaleup based in Amsterdam and Los Angeles. Our mission is to provide the effortless transfer of creative ideas, which results in transferring more than 1 billion files per month between our users. To keep up with that, we're expanding our platform team and are looking for an engineer with experience in AWS infrastructure-as-code and continuous deployment. Our stack:

- AWS (heavy users of EC2, S3, RDS, CloudFront, RedShift, SQS, ...)

- Terraform for infrastructure, Ansible for provisioning

- Datastores are MySQL, Redis, PostgreSQL, DynamoDB, InfluxDB, ElasticSearch

- Applications are written in Ruby, Elixir, and Go

https://wetransfer.homerun.co/

Apply there or send me an email at [email protected]

(We're also looking for a Data Engineer and React and Ruby wizards)

I'd say that .ignore is too generic a name. What is to be ignored by what?

But I like the idea of standardizing this. Perhaps the cache directory tagging standard gives some inspiration.

http://www.brynosaurus.com/cachedir/spec.html

Thanks for the link, I didn't know about construct!

A colleague of mine recently developed a similar (from the looks of it) library, in Python and Javascript:

https://github.com/jfjlaros/bin-parser

Cool! We have a pretty similar setup implemented in Ansible for our Mutalyzer app:

https://github.com/mutalyzer/ansible-role-mutalyzer

healthchecks.io looks pretty nice, I might start using it!

This is great! We use IPython Notebook a lot in our group, for exploratory data analysis, teaching, sharing experimental results, and as an electronic lab notebook. Inline GitHub rendering is very nice for all our public work.

If you run a private GitLab server, you could have a look at my patch which adds similar functionality to GitLab (in a rather ad-hoc way): https://gist.github.com/martijnvermaat/6926070

That is true, but only when they're older than 2 weeks (or whatever you define with --prune). I don't know what GitHub's configuration is here.

In a way that's already true, since the commits are still there (and included when you clone). They're just not discoverable.

It would be neat though if GitHub added some UI magic to recover dangling commits, possibly using the knowledge of which branches previously pointed to them and/or push -f operations.

This is really cool, well done!

Be sure to send a message to the IPython-dev mailing list if you're happy to recieve contributions and/or feedback (also from IPython developers).

I've had an Asus Zenbook Prime (1920x1080, 256GB SSD) for about half a year now and am generally very happy with it. I run Debian testing (with Linux 3.7.1 at the moment) and as far as I know everything works.

Compared to my girlfriend's Mcbook Air I have to charge the battery much more often, but I'm not sure if that's the hardware or the OS. Another minor nitpick is the rather high fan speeds it prefers, even with low temperatures. This makes a noticeable noise in a quiet environment (I don't hear the Air).

Keyboard is not great but ok (better than that of the older Zenbook), I love the screen (this resolution is new for me) and wrt size/weight/form factor/price it's pretty much equivalent to a Macbook Air.

As I said, I'm happy with it, and would probably buy it again. The only other model that made me just a tiny bit jealous is my colleague's Lenovo X1 Carbon, but it's quite expensive and wasn't available half a year ago.

HTH.

Link to the original publication: http://www.pnas.org/content/early/2013/01/09/1211927110

(I'm sorry that it is behind a paywall.)

Irina Pugach, Frederick Delfin, Ellen Gunnarsdóttir, Manfred Kayser, and Mark Stoneking. Genome-wide data substantiate Holocene gene flow from India to Australia. PNAS 2013; published ahead of print January 14, 2013, doi:10.1073/pnas.1211927110

For the 30 y assumption they cite:

Fenner JN (2005) Cross-cultural estimation of the human generation interval for use in genetics-based population divergence studies. Am J Phys Anthropol 128(2):415–423.

I have a "pip install ipython" in ~/.virtualenvs/postmkvirtualenv to make that a bit easier.

My major problem I have is packages using cython or depending on external C header files in some other way. For example, matplotlib and the IPython qtconsole.

For one, I don't even have gcc on servers that I want to deploy to using a virtualenv. but the major pain is finding out what *-dev packages I need to have installed in order to install using pip (if it can be done at all).

I don't think I ever had the IPython qtconsole working from a virtualenv without using system site packages.

Except that doesn't work if the form posts to a HTTPS URL. You'd have to implement something at the browser level, e.g. installing a modified browser or a browser extension.

I've been using Delicious for storage and management of news links on http://checkuitcheckin.nl/ (a view on public opinion on the Dutch public transport card OV-chipkaart).

I developed a small HTML5 app a couple of weeks ago, optimized for the iPhone (in Dutch): http://c.phanatics.nl/

It converts between track and field performances and awarded points by the Dutch athletics competition. The conversions can be stored. Intended usage is on the field during competition day, or just for the curious.

Source: http://svn.vermaat.name/competitie-rekending/

Originally I also used a manifest file for caching, but decided I didn't need it (and Firefox popups warnings with it). (The manifest is still used in SVN rev 20.)

The biggest problem I encountered on the iPhone (and have not been able to solve) is that whenever you switch apps and return, Mobile Safari reloads the page (online, or from cache) and does so using the original url. There seems to be no way to stop this.

For me that meant that stored conversions where lost (they were stored in the DOM and url), so I had to store them explicitely using localStorage. And still then, it is annoying you have to wait a few seconds for the page to reload before you can do anything.

The only info I found on this problem suggested that it might be related to Mobile Safari's memory management and as such wouldn't occur (as often) on iPhones with more memory than the one I tested with.