sanketsarang

I have a slightly different view of this. Here are the key reason why Indian's are succeeding:

1. Speaking the facts does not exist in India. Everyone sugarcoats. The successful see-through sugar coating. They are able to see through the true intentions of their peers, allowing them to be better leaders.

2. Indian's do Jugaad from birth. It is a need, as many don't even have stable electricity to light up a study lamp. They find creative ways to live their lives. This Jugaad at an early age gives them the "Power of Unlimited Thinking". A critical component in being a visionary leader. Finding solutions where no one thought it was possible.

This is a groundbreaking achievement by humankind. Even greater than sending people to the moon. Imagine, every time you ate pork, someone received a kidney transplant.

I did work on making a database myself, and I must say that querying 100TB fast, let alone storing 100TB of data, is a real problem. Some companies (very few) don't have much choice but to use a DB that works on 100TB. If you do have small data, then you have a lot of options. But if your data is large, then you have very few options. So it is correct to be competing on how fast a DB can query 100TB of data; while at the same time being slow if you have just 10GB of data. Some databases are designed only for large data, and should not be used if your data is small.

So what we understand from our study is the following:

1. You must have an Azure account with an API key for using their AutoML. An AutoML environment won't get created without a valid key. Both local and cloud runs mandate this.

2. Getting an Azure account requires a credit card on file and comes with a limited-time free trial. This is a big NO NO when it comes to software claimed to be for free use.

3. The free for life services do not call out AutoML anywhere. They do not claim the AutoML environment (a required step) to be free in any form. Check this: https://azure.microsoft.com/en-gb/free/

4. When they say "local" what are they referring to? Run locally on an Azure Notebook, or can I run this locally on my laptop? We have tried and failed ever to get this to run locally on a laptop. So it is not clear whether this is even possible, or the term "local" is misleading.

Have you managed to run Azure ML locally on a laptop, without requiring a connected Azure account?

Yes, if we run the AutoML from our laptop, it uses the API key to create a cloud instance. The data gets uploaded to the cloud and runs on the cloud. Results are thrown back to the local code. We would not call this a local run.

The question is, have you managed actually to use your computer's local resources for training? If so, please do share how this was possible. We would like to know how this was achieved.

Thanks Mark. Please do share your experience. Very keen on hearing how you find our project. We are still in beta, so lots of scope for improvement. We are open to any suggestions you might have.

That is not true actually. You must purchase an Azure VM for training a model. Deploying trained models on your own infrastructure is permitted. Reasonably speaking we would have to call this an enterprise solution, as there is no way to get a trained model without paying Azure fees. Most models will require GPU, and it is not like we can run AzureML on the free GPU offered by Google Colab.

Thanks for your question. Yes, we did research the space a lot before making AutoAI. Here is what we found:

PyCaret: Semi-automatic. You do the first run; then you figure the next set of runs. Ensemble models require manual configuration.

Tpot: Does a great job. Generates 4-5 lines of py code too. But does not support Neural Networks / DNN. So works only for problems where GOFAI works.

H2O.ai: They have an open-source flavor, but the best way to use it is the enterprise version on the H2O cloud. The interface is confusing, and the final output is black-box.

Now there are many in the enterprise category, such as DataRobot, AWS SageMaker, Azure etc. Most are unaffordable to Data Scientists unless your employer is sponsoring the platform.

AutoAI: This is 100% automated. Uses GOFAI, Neural Networks and DNN, all in one box. It is 100% White-box. It is the only AutoML framework that generates high-quality (1000s of lines) of Jupyter Notebook code. You can check some example codes here: https://cloud.blobcity.com

Hi HN, we have seen a lot of AutoML frameworks out there. As a Data Scientist myself, I have refrained from using these because at the end of the day, you have to submit complete source code to your clients, not just a functioning model. That is why we created AutoAI. Given data and target (value to predict), it can automatically discover and fully train the best performing AI solution. Still, most importantly, it also goes on to produce high-quality Jupyter Notebook code. AutoAI does Whitebox AutoML. A much-needed feature for Data Scientists. Do give it a try, and let me know what you think.

Hmm... not really found another way of doing this. Would be very keen on seeing what others are doing.

The first thing I would do is check if your Github repository is showing the traffic source. If your primary documentation is on Github, then you would expect most people to visit it in order to figure out how to use your software.

Yes absolutely. Jupyter Notebooks are a classic example of success in this space. Many Data Scientists use Jupyter over the browser for day to day work. Including me.

Advantages: Allows you to use server resources than be limited by your laptop compute capabilities. Log in anywhere, means I don't have to carry my laptop around. As long as I have a browser, I get the same environment to work. Mental peace that my work and data are always safe. I can drop my laptop in a swimming pool and yet I will not even lose the last character I typed on that presentation I was making.

Disadvantages: Yeah, I cannot access my system while on an aeroplane. Can't think of anything else honestly, unless you live in a zone where the internet is as bad as on aeroplanes.

All in all, prefer the browser-based virtual machine any day. The advantages outweigh the single disadvantage of the need for a stable internet connection.

Very interesting. Have you considered the possibility of monetising encrypted backups? Allow people to backup their node on a cloud service you provide. Keep the data encrypted in a manner that only the owner can decrypt. This would give users mental peace, and could be a monetisation source for you.

Got it. I see a classic launch problem coming your way. How do you attract the first set of content creators, when there aren't any content consumers? Have you given this a thought?

We offer Jupyter Notebooks on the cloud. We share infra across users, thereby allowing for unlimited runtime on GPUs at a starting price of $75/m.

I am confused. Does it only capture comments created on the tool, or it captures comments from anywhere on the internet?

Interesting concept, but I have a few questions:

1. How scalable is it?

2. What is the expected messaging latency?

3. Data loss protection?

We actually did work on this a few years ago but did not get enough takers for it. We created a one size fits all database, that leverages the full capability of the file system.

Try it here: https://github.com/blobcity/db

PS: I am the chief architect of the DB, and the project is no longer being actively maintained by us. But if you make a contribution, we will oblige to review and merge a PR.

Bottom line, nothing you do can make your database faster than the filesystem. So why not make a database that just uses the filesystem to the fullest, than creating a filesystem on top of a filesystem. BlobCity DB does not create a secondary filesystem. It dumps all data directly to the filesystem, thereby giving peak filesystem performance. This is scientifically really the best it gets from a performance standpoint. Not necessarily the most efficient in data storage / data-compression standpoint.

This means, we gain speed, while compromising on data-compression. We produce a larger storage footprint, but are insanely fast. Storage is cheap, compute isn't. So that should be okay I suppose.

You make a good point. But there is one more reason for advertising than just conversions. If Coca Cola were to stop advertising, it would free up substantial ad space allowing for their competition to advertise cheap. These large brands don't just advertise for branding/conversions. They plan their spend budget so as to actually influence the bid rate. This makes it harder for newer brands to compete and get any viable ad exposure at affordable prices.

Yeah, but that is not what I did. I tried on Safari on Mac.

On the same basis, it would also help if you could provide a comparison between GPUs commonly used for ML. Tesla k80, P100, T4, V100 and A100. How has the architecture evolved to make the A100 significantly faster? Is it just the 80GB RAM, or there is more to it from an architecture standpoint?

This is very strange. Just tried it, and I got a message flashed on the screen supposedly from the Ministry of Truth.

  This site contains information that is currently classified as Thought Crime in your region.
  If you are the owner of this site, please contact your local Ministry of Truth at your earliest convenience.

I am the owner! And I believe currently = 2046? This is hilarious, so contact now or in 2046?

Website: https://blobcity.com

Have you performance benchmarked this? Would be interested in giving it a try if performance is not compromised. Currently use XFS.

Since you are an active user of myBinder and Colab, I am tempted to ask. What are those features we can add that will make you let go of myBinder and Colab?

This is good feedback. Thank you for documenting your experience. Basis this we concluded on the following.

Adding features:

  - Login with Google & Github accounts
  - Changing default type setting to "Data Science Notebook"
  - A package library with one-click install

Changing features:

  - Simpler UX (reduced clicks required)
  - Removing the mandatory plan selection - Starting every user on free plan by default

Anything else you recommend we should do?

On related topic, what would you consider a good value proposition? Meaning what are the things you would expect out of BlobCity for you to become a registered and loyal member?

That's a brilliant suggestion. We are making a change to allow all public Cloudbooks to be run in guest mode (without registration) for 30 mins. The Cloudbook will auto shutdown in 30 mins to prevent any misuse. Would this be enough to assess the value prop?

Its free for sure, and comes with unlimited runtime. CPU & RAM will be limited by normal hardware capabilities of a single server. Storage is unlimited. Some of our free servers run up to 1TB RAM and 64 cores of compute power. This is extremely large unless you are analysing images from the Hubble telescope.

Please do give it a try. Only condition, your code & data must be open source if you want it on the free plan. Else you can take a paid plan. Unlimited part remains the same, but it won’t be free for private use.