deeplstm

it’s not always wrong. some of it is wrong. the trick is figuring out what’s actually correct

humans are still very needed to verify AI outputs

it was kind of a hard part, but not the hardest

I don't think OpenAI and Google have no moat. The biggest moat they have is hardware-software integration, which allows them to serve the models at a very cheap price.

My detailed thoughts in a video format https://youtu.be/cIMlPYI3nz8

*

1 point by deeplstm 0 minutes ago | prev | next | edit | delete [–]

I don't think OpenAI and Google have no moat. The biggest moat they have is hardware-software integration, which allows them to serve the models at a very cheap price.

My detailed thoughts in a video format https://youtu.be/cIMlPYI3nz8

I don't think OpenAI and Google have no moat. The biggest moat they have is hardware-software integration, which allows them to serve the models at a very cheap price.

My detailed thoughts in a video format https://youtu.be/cIMlPYI3nz8

Here's the video summary https://youtu.be/uPV9Gk3IC-g

Blog: https://crfm.stanford.edu/2023/03/13/alpaca.html

Demo: https://alpaca-ai0.ngrok.io/

Video summary: https://youtu.be/6qdzsDSduww

Text summary

Alpaca is a solution to NLP centralization proposed by Stanford researchers

Alpaca learns from GPT-3.5/ChatGPT to generate data for training

Base model: Llama (7 billion parameters)

Training cost: 3 hours on an A100 GPU*8 (< $100) Can be used with various models (sequence-to-sequence)

Usage: Primarily for academic research, commercial use prohibited

Future directions: Scaling models, properties of base models and instruction data, alternatives to supervised training on ChatGPT data

paper Wav2CLIP: Learning Robust Audio Representations From CLIP https://arxiv.org/abs/2110.11499

Paper https://arxiv.org/abs/2110.08207

Code https://github.com/bigscience-workshop/promptsource/

paper https://arxiv.org/abs/2110.00560?context=cs

BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension https://arxiv.org/abs/1910.13461 (paper)

Code (Facebook) https://github.com/pytorch/fairseq/tree/main/examples/bart

Code (Hugginface) https://huggingface.co/transformers/model_doc/bart.html

Modelling long sequences has always been hard for transformer-based models. This paper proposes a super innovative way for the transformer to cache previously processed tokens. And it makes generation 9X faster. This is truly mind-blowing

Paper https://arxiv.org/abs/2012.15832

Code https://github.com/ofirpress/shortformer

This is super impressive!! Those generated images are quite accurate and realistic. Here are some of my thoughts and explanation about how they do use discrete vocabulary to describe an image. https://youtu.be/UfAE-1vdj_E

There are 300+ deep learning papers published every day. I find it very hard to keep up with. This paper introduces a cool way to summarize papers to extremely short summaries (TLDRs).

More interestingly, Semantic Scholar uses the proposed method to build its TLDR feature and it's available in beta for nearly 10 million papers now!

Code and data: https://github.com/allenai/scitldr

TLDR feature in Semantic Scholar: https://tldr.semanticscholar.org/

The AI system built by an MIT team can detect it with 97% accuracy. More interestingly, it's able to detect asymptomatic people 100% (sensitivity). The proposed model comprises 4 biomarkers (3 ResNet models and a Poisson mask). Each of them represents a hypothesis of the repository disease.

Caveat: More replication is needed. There are clinical trials ongoing in Mount Sinai and White Planes Hospitals in the US, Catalan Health Institute in Catalonia, Hospitales Civiles de Guadalajara in Mexico, and Ospedale Luigi Sacco in Italy.

Paper: COVID-19 Artificial Intelligence Diagnosis using only Cough Recordings https://ieeexplore.ieee.org/document/9208795

How to perform full end-to-end entity linking has always been a challenging problem in NLP. The typical approach for this is to use a model to detect entities and then employ another model to perform entity disambiguation. And this paper beautifully formulates these two steps into a single neural network model.

Paper https://arxiv.org/abs/2010.02413

Code https://github.com/facebookresearch/BLINK/tree/master/elq