skumar17

That’s a good observation. For this project, I found that while the base model could “read” the image, it didn’t really understand how to use it. GRPO allowed it to effectively search the solution space.

The attention demo code is in the /attention_demo directory if you want to try it on your own messages too :)

Unfortunately it might be a bit challenging as there’s a nontrivial amount of extra computation we do for the viz, but it’s probably possible?

Thanks! I really love the visualization too. We have a hosted demo you can try as well!

https://huggingface.co/spaces/Groundlight/grpo-vlm-decoder

Hi there! Last month at NeurIPS (an ML conference), I read an interesting paper "Human Expertise in Algorithmic Prediction" that describes a framework for determining where ML models are outperformed by human experts. I found the authors' work to be very interesting. Below, I explore their framework further and extend it to multiclass classification. My results are pretty surprising, showing that a group of modern model architectures have trouble with dogs and cats in CIFAR-10.

GitHub Link: https://github.com/sunildkumar/model_indistinguishability

Paper Link: https://arxiv.org/abs/2402.00793

I just setup this awesome way to easily deploy computer vision models using natural language, check it out: https://code.groundlight.ai/python-sdk/blog/getting-started

I built an application that automatically detects and publicly posts when my coworkers leave dirty dishes in the sink using computer vision.

Check it out: https://code.groundlight.ai/python-sdk/blog/grime-guardian

Source code: https://github.com/sunildkumar/GrimeGuardian

Hi there! I was interested in learning more about LoRA but I was having a hard time finding a good simple example of implementing LoRA, as most sources are training large models and use a combination of huggingface transformers and the loralib package the original LoRA authors wrote. As a result, I ended up writing a simple LoRA implementation from scratch in pytorch lightning, and I figured other people might find it helpful as a learning resource or springboard.