tessbi

thank you for your feedback! I thought I stress tested my server, but it wasn't handling the traffic load from this HN post, so the app stopped responding! Resizing the VM now.

thank you for your feedback! I thought I stress tested my server, but it wasn't handling the traffic load from this HN post, so the app stopped responding! Resizing the VM now.

Hi everyone, thank you for your feedback! I thought I stress tested my server, but it wasn't handling the traffic load from this HN post, so the app stopped responding! Resizing the VM now. I wish it could run fully locally in browser. I will keep optimizing it.

This is excellent! Good work!

Very cool!

Thanks for the tip! I will check it out!

I can add a button for that.

That would be in the future for sure, US data is just much easier to work with right now.

Thank you! Yes, but I am using their monthly dumps for now, because it was easier for data warehousing. I will put it in my plans to pull data daily incrementally in the future as a next phase.

Well the quarterly one is much smaller then say, the daily one. Select count(*) from superset.company_financials_quarterly; = 458,573. I have >6000 companies, with over 80 features/columns. I cannot post daily data through SuperSet, because people will query the whole dataset without filters, and it will be too slow. But I have it just in case people want the daily stock charts in the future. Cheers.

Yes, but they do not necessarily offer it as a dump right away. So I am still relying on monthly dumps.

I could try playing around with Grafana as well, thanks for the tip!

Yeah, I suppose I can remove that feature, I just thought if people wanted to add comments or blog on my tutorials. Just message me in Contact Us, I will remove you from WordPress.

Got it! I removed the phone number as necessary.

Thank you for the feedback! I dropped the phone number as being necessary for registration. If you still have issues registering, please email me: [email protected]

Oh no!!! I am so sorry to hear this, I will work on fixing this. If you are still interested in trying out the data, please email me: [email protected]. I can register you manually.

I am deeply offended that you do not trust me, a stranger on the internet! I am hearing that many people feel that I am asking for too much data. I removed the phone number being necessary for registration. But I did still want to keep the email verification step, so that I can know that the people using my database are actual people and not bots/trolls, and that I could reach them with new features, and establish a professional relationship going forward. I hope that simply giving me your name and email, is not too much to ask for going forward. When I meet people at any event, it is often customary to exchange some information to stay in contact. Cheers!

Thank you so much! This has been my project for the past 3 years.

I see. Well its not as slick as paid data visualization tools for sure.

I am not sure what you mean by 'trust it'? As in, does it display the data as it should? It is difficult to find free open source data visualization tools that can run in the browser, that anybody could log-in and use right away. For my personal analytics I use Tableau, connected to the same dataset.

For most questions, please look through the tutorial videos and documentation that I created! :)

https://tesseractanalytics.ai/data-tutorials/

https://tesseractanalytics.ai/finance-data-quarterly-data-do...

The data is refreshed nightly for stocks. And monthly from SEC once it is available to download. There is no guaranteed data delivery SLA that I know about from SEC, so I get it monthly once it arrives. I talk more about this in detail on the website, I will also update the data documentation tab, and in tutorials. https://tesseractanalytics.ai/finance-data-quarterly-data-do...

There is a registration form that automatically creates you SuperSet account. https://tesseractanalytics.ai/financial-database-registratio...

It also creates a Wordpress account for you if you wanted to blog about some discovery.

I used PostgreSQL as the backend database for ELT data pipelines, and used Citus Data cstore_fdw for columnar compression for the final dataset. All financial data is coming from SEC Edgar, https://www.sec.gov/developer. I used Python for downloading most of the data.

I also run the data load development locally on my home Ubuntu server that I built 5 years ago. I bought 4TB of M2 disks for best database performance, with PRIME B360M-A motherboard and Intel Chip Coffee Lake S.

I built the website simply using WordPress, and I run Apache Superset using gunicorn via Apache reverse proxy. The registration form I had to build myself with PHP and some JavaScript.

My main motivation was that I wanted to be able to drill down and filter across all the available stocks, look at the data for myself, and narrow down on the stocks I am interested in based on my own sets of criteria, and make data-driven analysis for my personal investment strategies.

Please don't use the database directly as an investment tool, as its in Beta, and the data still needs to undergo heavy data quality checks, please confirm all the numbers yourself, as I provide a link for every company to the SEC filings.