psb31

Prolific (YC S19)| UK (Remote) | Full-time | Multiple roles | https://www.prolific.co/careers

Fast growing startup of 90+ people. Remote team, mainly based in the UK. YC alumni (summer 2019). Prolific is a marketplace platform that makes it easy to collect high quality data from people online. We manage a diverse pool of 100,000+ verified participants and provide access to an ecosystem of tools to run research on them for AI training, behavioural research, user testing and more.

Our focus is

1) building a community of trustworthy, engaged participants who are treated fairly as we believe this leads to higher-quality data.

2) Build a scalable API infrastructure to integrate with a wide range of data collection tools to empower a diverse range of people-powered applications.

Open positions:

- Mid-Senior Software Engineers

Our stack is currently Vue.js, Typescript (FE), Python (BE) and NoSQL databases and we’re moving towards Event-Driven Architecture on AWS

- Engineering Managers

You will be responsible for leading a cross functional team of up to 5 engineers, working closely with the Product Management and Design team Experience with building greenfield teams or coaching engineers through a period of architecture modernisation is a bonus!

- Senior Product Analyst

As one of the first embedded Product Analysts at Prolific, you’ll be a bridge between the Data and Product teams: working closely with product managers, designers, and user researchers to inform product development, optimise performance, and measure the impact of new initiatives using data.

More information here: https://www.prolific.co/careers

I think yes, since that’s what it roughly did normally if you didn’t balance. This was a change in the normal behaviour, so if you ran a study from one week the next you wouldn’t have known that your data would change dramatically without additional screening.

Prolific (YC S19) | Software Engineers Back End, Front End, Full Stack, Dev Ops and Engineering manager | Remote, UK based, £50-110k | https://www.prolific.co/careers/

Prolific connects people to facilitate world-changing research. We're a mission driven remote team of ~50 looking to change how online research is done. We're a profitable remote-first company - we were doing it before covid and work hard to ensure being remote is successful for everyone.

We primarily use Python, Django and MongoDB, with Vue.js on the front-end. We're looking for product focused engineers to join our cross-functional squads that own specific problem areas.

Learn more @ https://www.prolific.co/careers/ or email me phelim @ prolific.co (CTO & Founder).

Prolific (YC S19) | Software Engineers Back End, Front End, Full Stack, Dev Ops and Engineering manager | Remote, UK based, £50-110k | https://www.prolific.co/careers/

Prolific connects people to facilitate world-changing research. We're a mission driven remote team of ~50 looking to change how online research is done. We're a profitable remote-first company - we were doing it before covid and work hard to ensure being remote is successful for everyone.

We primarily use Python, Django and MongoDB, with Vue.js on the front-end. We're looking for product focused engineers to join our cross-functional squads that own specific problem areas.

Learn more @ https://www.prolific.co/careers/ or email me phelim @ prolific.co (CTO & Founder).

Prolific (YC S19) | Software Engineers Back End, Front End, Full Stack and Engineering manager | Remote, UK based, £50-90k + options | https://www.prolific.co/careers/

Prolific connects people to facilitate world-changing research. We're a mission driven remote team of ~50 looking to change how online research is done. We're a profitable remote-first company - we were doing it before covid and work hard to ensure being remote is successful for everyone.

We primarily use Python, Django and MongoDB, with Vue.js on the front-end. We're looking for product focused engineers to join our cross-functional squads that own specific problem areas.

Learn more @ https://www.prolific.co/careers/ or email me phelim @ prolific.co (CTO & Founder).

Prolific (YC S19) | Software Engineer Back End, Front End, Full Stack | Remote, UK/EU & London, UK | https://www.prolific.co/careers/ Prolific connects people to facilitate world-changing research. We're a mission driven remote team of ~40 looking to change how online research is done. We us Python, Django, Django Rest Framework, Celery, Redis, MongoDB and Postgres, with Vue.js on the front-end. We're looking for some ambitious product focused engineers to help us scale.

Learn more @ https://www.prolific.co/careers/ or email me phelim @ prolific.co (CTO & Founder).

Prolific (YC S19) | Software Engineer Back End, Front End, Full Stack | Remote, UK/EU & London, UK | https://www.prolific.co/careers/ Prolific connects people to facilitate world-changing research. We're a mission driven remote team of ~40 looking to change how online research is done.

We us Python, Django, Django Rest Framework, Celery, Redis, MongoDB and Postgres, with Vue.js on the front-end. We're looking for some ambitious product focused engineers to help us scale.

Learn more @ https://www.prolific.co/careers/ or email me phelim @ prolific.co (CTO & Founder)

Prolific (YC S19) | Software Engineer Back End, Front End, Full Stack | Remote, UK/EU & London, UK | https://www.prolific.co/ | https://prolific.co/careers/

Prolific is connecting people around the world to make trustworthy data more accessible and facilitate world-changing research. Come join a distributed, mission focused team team of ~40 looking to change how online research is done.

We use Python, Django, Django Rest Framework, Celery, Redis, MongoDB and Postgres, with Vue.js on the front-end. We're hiring for several roles so please take a look at https://prolific.co/careers/

Prolific (YC S19) | Software Engineer Back End, Front End, Full Stack | | Engineer Manager | Analytics Engineer | Remote, UK/EU & London, UK | https://www.prolific.co/ | https://prolific.breezy.hr/

Prolific connects people to facilitate world-changing research. We're a fast growing remote team of ~30 looking to change how online research is done.

We us Python, Django, Django Rest Framework, Celery, Redis, MongoDB and Postgres, with Vue.js on the front-end. We're looking for some ambitious product focused engineers to help us scale.

Learn more @ https://prolific.breezy.hr/ or email me phelim @ prolific.co (CTO & Founder)

Prolific (YC S19) | Europe | London, UK | Remote |https://prolific.co

Prolific is building a marketplace that connects researchers with instant, high quality, global research participants. We’re aiming to build the most powerful and trusted platform for behavioral research. We’re a team of 25 working remotely across the UK.

We’re hiring for product designers, product managers, engineers (Python and JavaScript), engineering managers and more. prolific.breezy.hr for an incomplete list of roles (more coming next week) or email your CV and a short note on why you’re interested in applying to [email protected].

This resonates strongly with me. I founded a startup as the technical co founder and now do a mix of product and engineering management. I agree with many of the others here: a small company / startup as an early engineer or a technical product manager sounds like it would fit well. Or, if you have the desire, founding your own company means you get to wear many hats.

Prolific (YC S19) | London, UK & Remote | Full-time | Software Engineer (Python), Product Manager, UX Designer & more | https://prolific.breezy.hr/

With Prolific, we're changing how research on the internet is done. We've started by building a marketplace that connects researchers (from both Academia and industry) with instant, high quality, global research participants. Now, as a growing team of 15, the bigger vision is to build the most powerful and trusted platform for behavioral research.

Python, Django, Django Rest Framework, Celery, Redis, MongoDB and Postgres, as well as Vue.js on the front-end. For infrastructure we use Kubernetes running on Google Cloud Platform.

Apply at https://prolific.breezy.hr/, questions to [email protected]

- Product Manager: https://prolific.breezy.hr/p/c23a0c793a9501-product-manager

- Backend Engineer (Experienced): https://prolific.breezy.hr/p/45e00c9c3c5e01-back-end-develop...

- Backend Engineer (Early Career): https://prolific.breezy.hr/p/4af2b5c5af8801-back-end-develop...

- UX/Product Designer: https://prolific.breezy.hr/p/0ac36250c9d601-user-experience-...

Thanks! We're actually cheaper than MTurk for the same participant reward. We take a 33% fee, while MTurk takes 40%.

1) You can run follow up studies through our platform using anonymous identifiers we provide.

2) ~40%

3) We have many more filters when you create an account. The "Number of children" filter is actually a followup to a "Do you have children?". If you select "No" you'll see we have ~ 38,755 active participants who are childless.

4) Good question, I don't have good stats on this right now. The process of setting up a study is mostly touchless for us, especially for 'expert' users who have run online research elsewhere. Support mostly consists of billing issues, custom filtering, and complex studies (we sell to universities, so billing can be complex). We're working on improving onboarding and making the whole process as self-serve as possible.

Yes, you could use Prolific for something like this. You would need to use our "opt-in" screeners and avoid collecting personal data as part of the study.

It would probably cost about $5-15 (it depends on how much you would pay the participant, I would recommend being generous for a study like this).

If you'd like to tell us a little more about your goals, we can get back to you with some more specific information: https://prolific2.typeform.com/to/aczPoI

The short answer is that with Prolific you can use any survey/experimental software you'd like. Behavioural researchers (our primary customers) don't tend to use SurveyMonkey, but instead use Qualtrics, Gorilla, and custom experimental software which have many features important for behavioural research (e.g. randomisation and reaction time recording).

In addition, SurveyMonkey audience uses a survey exchange provider CINT [1] and as a result you don't have any transparency of where exactly the participants are coming from, how they're vetted, the user experience they're having, and you can't communicate directly with participants or retarget them for longitudinal studies. All of which is important for research and are features that Prolific provide.

[1] https://www.cint.com/press-release/surveymonkey-audience-exp...

> what is your balance of private and university clients as of today?

Our clients are about 80-90% academic, and we find that there’s a strong move within academia towards fair payments. In the long term we think that this will show in the data quality and reliability of samples such that paying fairly will result in the best value. Regarding cost, we're about $2 total for a 15min survey right now.

> How large is your pool?

Lack of transparency about pool size is one of the most frustrating things about online panels. This is one of the reasons why we don’t report our total pool size, but only our ‘active and accessible’ pool (participants who have been active within the past 90days). As a result, you can expect > 50% of the numbers of eligible participants we report to actually take part in the next 24/48hours. We have ~70,000 active participants (20,000+ in the US). I expect we would be able to get a sample of 5,000 people in <48hours, but would only be about to repeat this for about 10weeks with unique participants. Our ‘pool’, as measured by traditional panel providers is approaching 500,000, but we don’t think this is a useful metric as the majority no longer respond to invites.

> a) how you recruit high income people into your pool;

To be honest, we haven’t cracked this nut yet and we expect our pool to be unrepresentative for high income people also right now. Ideas we have to help address this problem are to 1) introduce charitable donations for those who aren’t motivated by cash incentives 2) improve non-financial incentives (e.g. feedback on the impact your data is having) and 3) have highly targeted invites, we hypothesize high earners would be willing to help out if we need someone of their demographics in particular, and if this was communicated well. If you have suggestions, we’re all ears!

> and b) what percentage of your pool would be high income (say HHI > $125k a year or so).

According to self report we have ~2,000 participants from a HHI (>$100k/year on our screener), though it’s possible this suffers from slight inflation and we don’t (yet) have a way to verify income levels.

> Finally, what is your pool attrition rate?

Our annual retention rate is ~40% or so (e.g. if a participant takes part in a study, they’re about ~40% likely to do so again 12months later). There’s a balance between ‘refreshing’ our pool and keeping high engagement, and we’re working on keeping “naivety” high while allowing for studies over a long period!

It’s a great question and a tough problem. We have written a little bit about the problem of ‘slackers’ (participants with low attentiveness and engagement) [1,2]. The things we’re doing right now to address this problem is to 1) test for attentiveness and engagement before participants take part in real studies, 2) distribute surveys more broadly to reduce the % of “professional survey takers” in each study and 3) educate researchers about ways to use good attention and engagement tests [3] in their studies. We can then feed this data back into our system so we can iteratively improve on data quality.

I can’t go into too much detail about the attention and engagement checks we have built in, but if you sign up as a participant and pay attention you might spot some. In the long run, we think good feedback systems and high trust will be key so that our pool iteratively gets better and participants don’t feel as incentivized to cheat. It’s key to make sure that participants feel that their high effort responses get fairly rewarded (both financially and with social appreciation).

[1] https://blog.prolific.co/how-to-improve-your-data-quality/

[2] https://blog.prolific.co/bots-and-data-quality-on-crowdsourc...

[3] https://researcher-help.prolific.co/hc/en-gb/categories/3600...

No. All questions are optional and typically studies only filter on 1-3. We realise there’s a too many there by default and we’re working on staggering the demographics more.

I’m the CTO at an MTurk competitor (Prolific.ac). We’ve written up some thoughts on this below. It’s unlikely that these accounts are ‘true’ bots but rather low effort workers hiding location with a VPN and providing pseudo random responses.

https://blog.prolific.ac/bots-and-data-quality-on-crowdsourc... https://blog.prolific.ac/how-to-improve-your-data-quality/

We’ve done a lot of work on improving data quality on Prolific and filtering out these kind of accounts and heavily invest in making sure our pool are honest, attentive and engaged.

If anyone’s concerned about data quality in online samples please get feel free to get in touch.

Prolific (www.prolific.ac) | Oxford, UK + REMOTE | Full Time | Backend engineers + Data Analysts

At Prolific we bring people together to power the world’s research. It is the first scientific, highly scalable, ethical participant recruitment platform for researchers around the world.

https://prolific.breezy.hr/

Twitter’s VP of engineering has posted some additional information: https://twitter.com/michaelmontano/status/101024630798476902...

Prolific.ac – Oxford/London/Glasgow, UK | ONSITE OR REMOTE | £40-55k + options

Prolific (www.prolific.ac) brings people together to power the world’s research. Need to recruit participants for research? Prolific offers a scalable participant recruitment marketplace to rapidly access a pool of 40k people and growing.

We're hiring for Backend Engineer/ DevOps (Python/MongoDB/PostgreSQL/Redis/Kubernetes/GCP). Our stack: Python, Django, MongoDB, Redis, Kubernetes, Vue.js.

You’ll be joining a team of seven, with an exciting opportunity to scale a rapidly growing, bootstrapped startup.

More info: https://prolificac.workable.com/ or [email protected]

Prolific (www.prolific.ac) | Oxford, UK | £40,000-£50,000 GBP + options | REMOTE or Onsite

Prolific (www.prolific.ac) brings people together to power the world’s research. It is the first scientific, highly scalable, ethical participant recruitment platform for researchers around the world. Quantitative and qualitative researchers, data scientists, and entrepreneurs all need valid and reliable data sources to make predictions about human behaviour, social change, and global developments. Thanks to our online platform, researchers collect responses fast and participants earn cash rewards for themselves or for charity. In a world of unreliable results, Prolific makes data trustworthy and connects the public with science.

We're hiring for Backend Engineer/ DevOps (Python/MongoDB/PostgreSQL/Redis/Kubernetes/GCP). Our stack: Python, Django, MongoDB, Redis, Kubernetes, Vue.js.

You’ll be joining an enthusiastic team of seven (2 remote, 5 onsite) based in Oxford, UK.

More info: https://prolificac.workable.com/j/54995A4DF0

This research is highly debated and is by no means a sure thing. The research so far has been correlational, inconsistent, and weak. Twenge hugely overstates the confidence we should have in the findings.

See https://medium.com/@OrbenAmy/social-media-and-suicide-a-crit... and https://twitter.com/shuhbillskee/status/955070356091555840 for some critiques of the work.

Prolific (www.prolific.ac) | Oxford, UK / London, UK | Full time | ONSITE or REMOTE (UK prefered) | £40k+ with generous equity (1-2.5%)

Python backend web developer.

Stack includes: django mongodb azure aws docker travis git react

Prolific is a crowdsourcing platform for research. We've bootstrapped to first $1million in sales and we're looking for a motivated backend/fullstack python developer to join our small team (of 4) as a 2nd employee. Now is a great time to join if you're looking for a role that will allow you to really shape the trajectory of an exciting startup.

email: [email protected]

The tradeoff between verification/validation and openness is definitely a tricky one. I'm the co-founder of a crowdsourcing platform prolific.ac and we settled on system with minimal verification (via sms) with background screening and shadow banning. It's been effective so far at limiting bad actors though I think there are definitely some more effective ways to address this problem.