News for hackers. Stuff that matters.

acidity

Ask HN: How to partition tenant data in queue for predictable performance?

Hello HN,We use PubSubLite but the same can be applied to any queue system. The events are then processed by service running in K8S.I am looking for strategies/patterns on how do folks partition their data so that each worker unit can get consistent performance with optimal CPU/memory usage.Our data is global and temporal i.e. different regions will have peak/off peak data volume based on their...

( Read More... | no comments | on HN )
This is not a fly.io specific question but I always wonder how in these globally distributed system, how are databases handled?

I understand you can put your application server in any location but generally there is only one storage so are these application servers doing cross region database calls?

Having only worked with single cluster setup web apps, I am always curious about this part.

Is the answer always - use a managed replicated database and send read queries to one near your location and all write queries goes to the primary instance?

Thanks. My initial question arose where does products like SnowFlake/RedShift falls into this. They seem to be coined as datawarehouse but seems they do the same as OLAP work. But looking at Snowflake docs I read:

"""Currently there is no option similar to an SSAS cube in Snowflake. Once data is loaded into the databases Snowflake allows to query the data in a way similar to traditional OLTP databases. For data aggregations the Snowflake library offers rich sets of in-built functions. Together with UDFs, SPs and Materialized Views we can build custom solutions for precomputed data aggregations. For data analysis we still have to rely upon third party tools. Snowflake provides a variety of different connectors to access its database objects from other analytical tools. There are plans in near future to introduce an integrated tool for data aggregations, analysis and reporting."""

So they say, its not really an OLAP.

And how exactly it differs from OLTP? Is a web app OLTP?

Ask HN: Good starting point for OLAP/OLTP from RDBMS background

I have experience doing traditional web apps using PostgreSQL and some bit with SQL Server - back in 6.5 days.Lately, I have been hearing about OLAP/OLTP/OLAP cubes etc at various meetings.Doing a search does not seem to throw up a canonical reference on all these.What would be a good way to go about learning these things with my background - which would be more traditional RDBMS based...

( Read More... | 3 comments | 2 points | on HN )

Ask HN: Developing an easy to use Coursera like website for my dad

My father (based out of India) is a retired professional who wants to start an educational video site (e.g. KhanAcademy) but in specific subject (geared towards Accountants). He is what one would consider a non technical guy.Initially, the website/content would be free but maybe he wants to monetize it in future.For a start, I setup an account at wordpress.com and even that interface is kind of...

( Read More... | 2 comments | 3 points | on HN )
Re:Why is Kubernetes getting so popular? on Saturday May 30, 2020 @12:49AM (#23358564)
Can you expand on this? We are just starting on Helm so curious to know your thoughts.

Ask HN: How to start with developing distributed applications

This is similar to topics like: https://news.ycombinator.com/item?id=20765553I am a full stack and general software developer (have developed various Desktop apps using Win32 APIs).After move to web development (backend, frontend, infra etc) and using various distributed apps (Kafka, Redis etc), I am a bit curious how does one start developing such apps.I know you have to apply some sort...

( Read More... | 1 comment | 3 points | on HN )
Re:Let's Build a Compiler on Tuesday July 16, 2019 @07:09PM (#20454771)
As a side note, is there something similar for building a distributed application (could be a very simple NoSQL DB or maybe some stream processor).
Re:ReactXP – A library for building cross-platform apps on Thursday September 20, 2018 @04:19PM (#18035145)
Newbie question but will this help me also write desktop apps?
Re:The Declining Value of a CS Master's Degree (2013) on Friday March 23, 2018 @12:24AM (#16655063)
As an Indian who did MS in US. The parent is 100% correct.

Ask HN: How to build a Google Cloud Shell like feature?

Recently, I started playing with GCE and have been pretty impressed by the Cloud Shell implementation. They even make VI bindings work.We could use something like this at work too so I was wondering, what would be a good resource to start if I have to go on implementing something like this? Our front end app is in Angular and APIs are served by Python/Java services but we are language/library...

( Read More... | no comments | 2 points | on HN )
Re:RethinkDB versus PostgreSQL: my personal experience on Thursday February 09, 2017 @07:10PM (#13612241)
>>> Weird. OK, I tried it with some other parameters, and it suddenly took 15 seconds at 100% CPU, with PostgreSQL doing some linear scan through data. Using EXPLAIN I found that with full production data the query planner was doing something idiotic in some cases. I learned how to impact the query planner, and then this query went back to taking only a few milliseconds for any input. With this one change to influence the query planner (to actually always use an index I had properly made), things became dramatically faster. Basically the load on the database server went from 100% to well under >>> 5%.

I am actually interested in this part. Figuring out issues with EXPLAIN is one of my favorite things.

Re:Building Large Flask Apps in the Real World on Thursday February 18, 2016 @12:52AM (#11123963)
Another one that I have used in multiple of my projects: https://github.com/mattupstate/overholt
This is so cool. I am actually planning to start working on something similar but for wet shaving market. Basically, I was trying to setup something where user can choose their after shave pairing based on scent details.

Do you have plans to open source your algorithm?

Re:Announcing TypeScript 1.8 Beta on Friday January 29, 2016 @01:29AM (#10993829)
Seems like you used React for the project above. Did you build a view/controller framework in house or something more common?
Re:Having Won Over VCs, Y Combinator Turns to LPs on Tuesday August 25, 2015 @01:16AM (#10114716)
Slightly off-topic but is there any book or resource that a software engineer like me can read to understand the basics of how VC operate, what each round of funding mean, what one needs to know when one approaches a VC, difference between VC/LP?
Re:Ask HN: Who is hiring? (August 2015) on Monday August 03, 2015 @10:25PM (#10001296)
Niara | Full Time | Immigration Sponsorhip Available | San Francisco Bay Area

Niara is an early stage venture startup hiring core technology engineers and UI/UX/Product Designers in Sunnyvale, USA to build its next generation security intelligence platform. Check us out at: http://www.niara.com. You can apply through the website or send a mail to me at riteshn AT gmail

Full Stack Engineer (Member of Technical Staff)

This is the job for you if you like to work on every aspect of a product. As Niara's, MTS, you will be working on the frontend, middleware, and backend. You would work with frameworks like AngularJS backed by a Python backend. We prefer simple, elegant solution to complex problems, so bonus points for having a history of getting things done.

UI/UX, Product Designer

As a UI/UX designer, your defined user experience is critical to Niara's success. You will work on concepts, wireframes, story boards to final user interface. Your solutions will help visualize highly technical problems in an easy to understand manner for the customers.

Re:Who exactly is crawling my site? on Friday October 18, 2013 @02:12PM (#6572967)
Which tool/code you used to parse the logs and get those numbers?
Re:Don't use Hadoop when your data isn't that big on Wednesday September 18, 2013 @12:20AM (#6403460)
Okay. I already have read about the algorithms and even implemented them crudely while studying them.

Just not a big fan of NHI. But if in this case, thats the best way - let it be :)

Re:Don't use Hadoop when your data isn't that big on Tuesday September 17, 2013 @12:27PM (#6400230)
I have been looking into building simple recommendation engine (i have at max million data rows) using Python. I looked into Crab (https://github.com/muricoca/crab) and it seems to not have updated for 2 years.

Any suggestions for libraries or just use basic numpy/scipy and implement the algorithms?

Re:SoundCloud and Uploads on Wednesday September 11, 2013 @07:16PM (#6370836)
Even, I would be interested.
Re:SoundCloud and Uploads on Wednesday September 11, 2013 @07:16PM (#6370835)
Can we share mail address? Even I am working on my own music venture and would like to talk it out. Mail me at riteshn AT gmail.
Re:Show HN: Spotify for Independent Music on Friday August 30, 2013 @11:40AM (#6302306)
I cannot seem to figure out how an Indian startup has mostly foreign artists. I am asking because couple of years back, I tried to do the same thing and licensing was the biggest issue. Have you got funding in India?

Moreover, most of the artists and songs I see up there are just at bandcamp.com and no where else. Are you just scrapping Bandcamp and reselling their music. E.g. googling for the album "Creatures" on homepage (https://www.google.com/search?q=creatures+new+campaigns&oq=c...) loads up only from Bancamp and other albums too.

Re:Yeoman 1.0 Released on Friday August 23, 2013 @12:22PM (#6264519)
There are examples for CakePHP and Flask but none for Rails. Do you know of any? I have a simple traditional Rails which I am looking into porting to use Backbone and Rails as backend.
Re:Amazon launches Amazon Art marketplace on Tuesday August 06, 2013 @11:38AM (#6167171)
Oh wow. I have been thinking of exactly the same idea. To that end, I just started playing around with Raspberry PI to see if it can work out. Looking online, seems they also have color E-Ink available which should make the cost even less.

Would you like to chat?