acidity
acidity on Hacker News
Posted by
acidity on Saturday October 05, 2024 @08:15PM
from the ask-hn dept.
Hello HN,We use PubSubLite but the same can be applied to any queue system. The events are then processed by service running in K8S.I am looking for strategies/patterns on how do folks partition their data so that each worker unit can get consistent performance with optimal CPU/memory usage.Our data is global and temporal i.e. different regions will have peak/off peak data volume based on their...
This is not a fly.io specific question but I always wonder how in these globally distributed system, how are databases handled?
I understand you can put your application server in any location but generally there is only one storage so are these application servers doing cross region database calls?
Having only worked with single cluster setup web apps, I am always curious about this part.
Is the answer always - use a managed replicated database and send read queries to one near your location and all write queries goes to the primary instance?
Thanks. My initial question arose where does products like SnowFlake/RedShift falls into this. They seem to be coined as datawarehouse but seems they do the same as OLAP work. But looking at Snowflake docs I read:
"""Currently there is no option similar to an SSAS cube in Snowflake. Once data is loaded into the databases Snowflake allows to query the data in a way similar to traditional OLTP databases.
For data aggregations the Snowflake library offers rich sets of in-built functions. Together with UDFs, SPs and Materialized Views we can build custom solutions for precomputed data aggregations.
For data analysis we still have to rely upon third party tools. Snowflake provides a variety of different connectors to access its database objects from other analytical tools. There are plans in near future to introduce an integrated tool for data aggregations, analysis and reporting."""
So they say, its not really an OLAP.
And how exactly it differs from OLTP? Is a web app OLTP?
Posted by
acidity on Friday January 08, 2021 @01:46PM
from the ask-hn dept.
I have experience doing traditional web apps using PostgreSQL and some bit with SQL Server - back in 6.5 days.Lately, I have been hearing about OLAP/OLTP/OLAP cubes etc at various meetings.Doing a search does not seem to throw up a canonical reference on all these.What would be a good way to go about learning these things with my background - which would be more traditional RDBMS based...
Posted by
acidity on Saturday August 01, 2020 @06:37PM
from the ask-hn dept.
My father (based out of India) is a retired professional who wants to start an educational video site (e.g. KhanAcademy) but in specific subject (geared towards Accountants). He is what one would consider a non technical guy.Initially, the website/content would be free but maybe he wants to monetize it in future.For a start, I setup an account at wordpress.com and even that interface is kind of...
Can you expand on this? We are just starting on Helm so curious to know your thoughts.
Posted by
acidity on Wednesday October 09, 2019 @01:48AM
from the ask-hn dept.
This is similar to topics like: https://news.ycombinator.com/item?id=20765553I am a full stack and general software developer (have developed various Desktop apps using Win32 APIs).After move to web development (backend, frontend, infra etc) and using various distributed apps (Kafka, Redis etc), I am a bit curious how does one start developing such apps.I know you have to apply some sort...
As a side note, is there something similar for building a distributed application (could be a very simple NoSQL DB or maybe some stream processor).
Newbie question but will this help me also write desktop apps?
As an Indian who did MS in US. The parent is 100% correct.
Posted by
acidity on Monday March 13, 2017 @08:45PM
from the ask-hn dept.
Recently, I started playing with GCE and have been pretty impressed by the Cloud Shell implementation. They even make VI bindings work.We could use something like this at work too so I was wondering, what would be a good resource to start if I have to go on implementing something like this? Our front end app is in Angular and APIs are served by Python/Java services but we are language/library...
>>> Weird. OK, I tried it with some other parameters, and it suddenly took 15 seconds at 100% CPU, with PostgreSQL doing some linear scan through data. Using EXPLAIN I found that with full production data the query planner was doing something idiotic in some cases. I learned how to impact the query planner, and then this query went back to taking only a few milliseconds for any input. With this one change to influence the query planner (to actually always use an index I had properly made), things became dramatically faster. Basically the load on the database server went from 100% to well under >>> 5%.
I am actually interested in this part. Figuring out issues with EXPLAIN is one of my favorite things.
This is so cool. I am actually planning to start working on something similar but for wet shaving market. Basically, I was trying to setup something where user can choose their after shave pairing based on scent details.
Do you have plans to open source your algorithm?
Seems like you used React for the project above. Did you build a view/controller framework in house or something more common?
Slightly off-topic but is there any book or resource that a software engineer like me can read to understand the basics of how VC operate, what each round of funding mean, what one needs to know when one approaches a VC, difference between VC/LP?
Niara | Full Time | Immigration Sponsorhip Available | San Francisco Bay Area
Niara is an early stage venture startup hiring core technology engineers and UI/UX/Product Designers in
Sunnyvale, USA to build its next generation security intelligence platform. Check us out at: http://www.niara.com. You can apply through the website or send a mail to me at riteshn AT gmail
Full Stack Engineer (Member of Technical Staff)
This is the job for you if you like to work on every aspect of a product. As Niara's, MTS, you will be working on the frontend, middleware, and backend. You would work with frameworks like AngularJS backed by a Python backend.
We prefer simple, elegant solution to complex problems, so bonus points for having a history of getting things done.
UI/UX, Product Designer
As a UI/UX designer, your defined user experience is critical to Niara's success. You will work on concepts, wireframes, story boards to final user interface. Your solutions will help visualize highly technical problems in an easy to understand manner for the customers.
Which tool/code you used to parse the logs and get those numbers?
Okay. I already have read about the algorithms and even implemented them crudely while studying them.
Just not a big fan of NHI. But if in this case, thats the best way - let it be :)
I have been looking into building simple recommendation engine (i have at max million data rows) using Python. I looked into Crab (
https://github.com/muricoca/crab) and it seems to not have updated for 2 years.
Any suggestions for libraries or just use basic numpy/scipy and implement the algorithms?
Even, I would be interested.
Can we share mail address? Even I am working on my own music venture and would like to talk it out. Mail me at riteshn AT gmail.
I cannot seem to figure out how an Indian startup has mostly foreign artists. I am asking because couple of years back, I tried to do the same thing and licensing was the biggest issue. Have you got funding in India?
Moreover, most of the artists and songs I see up there are just at bandcamp.com and no where else. Are you just scrapping Bandcamp and reselling their music. E.g. googling for the album "Creatures" on homepage (https://www.google.com/search?q=creatures+new+campaigns&oq=c...) loads up only from Bancamp and other albums too.
There are examples for CakePHP and Flask but none for Rails. Do you know of any? I have a simple traditional Rails which I am looking into porting to use Backbone and Rails as backend.
Oh wow. I have been thinking of exactly the same idea. To that end, I just started playing around with Raspberry PI to see if it can work out. Looking online, seems they also have color E-Ink available which should make the cost even less.
Would you like to chat?
Posted by
acidity on Friday May 10, 2013 @05:23PM
from the latimes.com dept.