TedPetrou

The tabula-py Python library, a wrapper around tabula(https://tabula.technology/), can extract tables within PDFs and convert them to Pandas DataFrames. It's a great little library and this is a blog showcasing an example.

Introduction to Statistical Learning by James, Witten, Hastie, and Tibshirani is one of the most popular books for gaining a fundamental understanding of machine learning. All of the code and exercises are presented with the R programming language. In this repo, all exercises are completed with Python, the most popular language today for machine learning. Recreations of some of the examples in the text are also provided. Each chapter is presented as a separate Jupyter Notebook.

Did you see all of the side effects from the vaccine vs the placebo? Here it is in percentages:

* Redness 5.8 vs 1.1

* Swelling 6.9 vs 1

* Pain 86 vs 23

* Fever 10 vs 1

* Fatigue 60 vs 46

* Headache 55 vs 35

* Chills 28 vs 10

* Vomiting 3 vs 1

* Use of pain med 37 vs 10

Only 1100 kids in the study and only 660 of them were followed up at least 2 months after the 2nd dose. These numbers seem unreasonably low.

This means that hundreds of kids had worse reactions than the placebo. All of this to prevent 11 symptomatic cases. I couldn't find the prognosis of these 11 symptomatic cases, but its highly likely that all of them were mild. To me, it seems like the overall health and well being of the vaccinated group was less than the placebo.

Seychelles has vaccinated 68.5k of its 98.5k population. [Around 25% of its total population](https://en.wikipedia.org/wiki/Demographics_of_Seychelles) are 0-17, so maybe 75k max 18+. Over 90% of adults are vaccinated and less than 7k remain unvaccinated.

Seychelles just reported 1800 cases for the last week or 1.8% of its total population. This equates to 6 million cases in the US or more than 800k per day, a number never approached.

> It looks like the people testing positive in the Seychelles are mostly unvaccinated

It stated that 1/3 were in fully vaccinated and the rest were in partially vaccinated or unvaccinated. There are now 60k that are fully vaccinated, leaving just 8.5k first-dose only. The article was written before the 1800 cases dropped a couple days ago.

More data is needed for hospitalizations and deaths, but as it stands now, it does not look convincing for the vaccine. It's likely more cases will continue to come over the next few weeks, even more than the record breaking prior week.

Seychelles and other island nations that had no major covid outbreaks before vaccine launch (only 500 cases prior and 7.5k since) are the best real life settings we have for vaccine efficacy. For much of the rest of the world, a huge number of people were already infected before vaccine launch, making it difficult to separate out the effectiveness of the vaccine.

I saw that you doubled down on [this comment](https://news.ycombinator.com/item?id=26181228). Week 5 excess mortality for 65+ in Israel is now up to 7.4 times above normal. It increased every update for the last three weeks. Unbelievable that people like you take 1 minute to read something, think you are an expert and try and proclaim victory. I'm 100% sure you will double down again instead of taking the honorable path of admitting you were wrong. People like you never get held accountable.

"Only 27% of the population received both doses". This is a misinterpretation. It's like saying 0.5% die of covid. The vaccinations (and fatality rates) are extremely stratified by age. Over 50% of the 70+ group received at least one vaccination by the third week of January and close to 85% have received both doses as of today.

You spoke too soon and I hope you issue a correction. The last week of euroMOMO isn't nearly fully updated. All countries see an enormous drop in the last week that is then pulled back up to the actual level in the following weeks of released data. Did you notice the yellow background and the "Corrected for delay in registration".

Edit: Here is an animation that shows the usual correction - https://twitter.com/jburnmurdoch/status/1350079943863115777

So you admit that total case numbers are not very good, particularly compared to a neighboring country that has not started vaccinating. Thats a good first step.

Scapegoating the orthodox jews? More than 92% of 70+ in Israel have received both doses of the vaccine. Are you blaming the high number of cases on this very small segment?

What happens when elderly all cause mortality is elevated substantially in 2021? This is the ultimate test for the vaccine.

Does it also make sense that they are currently ranked #2 in the world in cases per million in the last week for countries with more that 1 million population?

This is even after a down week in cases. Next door Lebanon is seeing cases fall at the same or faster pace and they peaked at the same time as Israel. Lebanon has yet to start vaccinating.

Case fatality rate has not fallen either. You would expect a huge drop in CFR if the elderly were protected, but this hasn't been the case.

Excess mortality for the elderly in Israel in the last 6 weeks is the highest it's been (by far) in the last 5 years. Check euroMOMO https://www.euromomo.eu/graphs-and-maps/

Israel is experiencing very high excess deaths with respect to its elderly in 2021 (see euroMOMO https://www.euromomo.eu/graphs-and-maps/)

This is despite over 80% of its 70+ age group having both doses of the vaccine. At least half of the 70+ were fully vaccinated by January 23rd.

It is the elderly all cause mortality that ultimately matters to me. If elderly are dying of other causes, then there are problems.

Regardless, 2021 elderly mortality in Israel will serve as a good test for the vaccine as the vast majority of 70+ will be fully vaccinated for nearly the entire year.

No one was deliberately inoculated with the virus and only symptomatic people were tested. 99.96% of the placebo group did not have a severe case of covid. There is no way to determine effectiveness from this trial.

I'm surprised so many people are excited about this. I have many concerns with the Pfizer vaccine

* 99.96% of the placebo group did not have a serious case of covid

* 99.25% of the placebo group did not get covid

* No one in the trial was deliberately inoculated - participants just lived their lives as normal.

* Only symptomatic participants were tested - this seems unbelievable to me - we have no idea what the actual incidence rate is because not everyone was tested

* Vaccines might just mask symptoms - since not everyone is getting tested, vaccine makers just have to make sure there are no symptoms. No symptoms equals no test.

* No trials done with two placebos - we need trials where both groups are in a placebo group. One gets a shot that gives a mild side effect and the other gives no side effect

* No trials done with unrelated immune boosters - we need to see how well this vaccine performs against other immune boosters. This could be a drugs or even supplements (vitamin D & C and exercise).

* Two shots were given in a trial lasting just 4 months - 4 months is an incredibly short amount of time to know whether it will be effective long term. It also gives Pfizer two chances to boost immune

* Long term health consequences of vaccine - we only have 4 months of data, which is way too short of time to see any longer term consequences

* There is a huge incentive to provide something that masks symptoms - billions of dollars are at stake. Big pharma is one of the very last companies I would trust with a novel vaccine

* No coronavirus vaccine in history - many coronaviruses currently circulate, but no vaccine has even been produced for them. It seems quite coincidental that humans finally put the pieces together for our current novel strand

Covid will be in our rear view mirror before a vaccine is safe, ready, mass produced and easily distributed. Herd immunity is not terribly far away in most countries and by the spring the threshold should be reached nearly everywhere. A vaccine won't be necessary and even if available isn't likely to be taken by most people, as covid has lower IFR than the flu for people under 50 (and magnitudes lower under 20).

There's not a single western industrialized nation or state that can be pointed to now as a mask success story. There were a few countries in eastern europe (specifically czechia) that were heralded as a primary reason for controlling covid in the spring.

We now know that this apparent good result was not because of masks, but because of geography. Czechia is ranked #1 in the world in cases / million in the last week and #3 in deaths / million (and its numbers are still climbing). Every single eastern european nation had low cases/deaths in the spring and now all of them are having their first large surge.

It's abundantly clear that cloth masks, as a whole, do not play a major role in how the virus spreads in a country.

Correct, I'd like dexplot to be a superset of seaborn first, making it much easier to use for those that don't want to dip into matplotlib for making minor adjustments that are necessary for most plots (figsize, ticklabels, etc..).

There should be a library to do exploratory data analysis quickly, without having to touch matplotlib, numpy, or pandas, and without installing something like pandas-profiling to make reports.

This is where the apps will come in to allow users to quickly generate reports on things like missing values, duplicate rows/columns, outliers/bad data, view different colors, etc...

Labels do not overlap and HTML tables are now fixed in the docs.

The data is the same. Dexplot automatically sorts the xtick labels alphabetically. Seaborn uses order of appearance. For the seaborn plot, the figure size and dpi have to be manually adjusted and there is no option to wrap the tick labels. They are a mess and overlap one another. The tick label wrapping is a huge win imo, otherwise you have to rotate them, which makes long labels look terrible.

Thanks! I'm focused on building the user-facing API, as this is where I believe I'm best suited to make improvements due to my experience teaching and writing.

I'm definitely open to looking at alternative backends in the future and will check out PyQtGraph, but am sticking to matplotlib for now.

Technically, there was a new "major" release with version 0.10, but it was just some bug fixes and the same as 0.9.1. The last release with anything new was in July of 2018. Given the rate of the last several releases, I don't expect much to happen for a while, thus "essentially dead".

You cannot control axes plot figure size from seaborn directly. You have to access the figure from the axes (which most people don't know how to do) or create the figure first by importing matplotlib. Really annoying for those that just want to analyze data quickly. Grid plots have the ability to adjust figure size, but return a seaborn object and not a matplotlib figure.

Agreed, docs need to get better. Better datasets, a gallery, etc... I've only spent a week on this, so there will be a lot of improvements in the future.

Thanks! The whole goal of dexplo and dexplot is to provide a much simpler and more consistent alternative to pandas/seaborn. I find pandas especially cumbersome to work with. I haven't had time to develop dexplo much, but am hoping for a first major release this year.

Hey everyone, I'm the author of Dexplot. I have written large sections of books on seaborn, have taught it in classes for years, and had many issues with it, some of which are outlined below:

• Not allowed to set figure size

• No wrapping of tick labels

• No strings for pandas aggregation functions

• No automatic ordering of x/y labels (dexplot provides several options)

• Having to use separate grid functions (catplot, lmplot) for multiple subplots

• Something like 5 different functions for scatterplots. Dexplot has one

• No relative frequency bar charts, which are a fantastic way to explore data. Dexplot provides normalization over any set of variables

• No stacked bar charts

• Seaborn docs have distribution plots (box, violin) in the "categorical" section. A major distinction needs to be made between plots that aggregate, those that show distributions, and those that plot raw data (like scatterplots)

• Returning of matplotlib axes or seaborn grid objects. Dexplot always returns the matplotlib figure

• Seaborn is essentially dead as far as I can tell with few changes in the last 2-3 years. There are even parameters that continue to be non-functional

In the future, Dexplot will add:

• Many more plotting functions

• Several apps (built from ipywidgets) to explore data. Currently, there is one for viewing colors

• Better automatic figure sizing (it exists now, but will be improved)

• Automatic DPI detection so that matplotlib inches correspond to actual screen inches

Dexplot aims to be very intuitive, easy to use, consistent, and allow easy exploration (the name is a smashing together of data exploration plotting).

Here is one example comparison between dexplot and seaborn. https://twitter.com/TedPetrou/status/1271436948721328129

Examples such as these are what drove me to create the library.

I'd love to get feedback and happy to take detailed criticism.

Hey there, I'm the author of Dexplot. There are many issues I had with seaborn

• Not allowed to set figure size

• No wrapping of tick labels

• No strings for pandas aggregation functions

• No automatic ordering of x/y labels (dexplot provides several options)

• Having to use separate grid functions (catplot, lmplot) for multiple subplots

• Something like 5 different functions for scatterplots. Dexplot has one

• No relative frequency bar charts, which are a fantastic way to explore data. Dexplot provides normalization over any set of variable

• No stacked bar charts

• Seaborn docs have distribution plots (box, violin) in the "categorical" section. A major distinction needs to be made between plots that aggregate, show distributions, and those that plot raw data (like scatterplots)

• Returning of matplotlib axes or seaborn grid objects. Dexplot always returns the matplotlib figure

• Seaborn is essentially dead as far as I can tell with few changes in the last 2-3 years. There are even parameters that continue to be non-functional

In the future, Dexplot will add:

• Many more plotting functions

• Several apps (built from ipywidgets) to explore data. Currently, there is one for viewing colors

• Better automatic figure sizing (it exists now, but will be improved)

• Automatic DPI detection so that matplotlib inches correspond to actual screen inches

Dexplot aims to be very intuitive, easy to use, consistent, and allow easy exploration (the name is a smashing together of data exploration plotting).