https://github.com/uipath/coder_eval
We needed a framework to write our evals in and easily update them, run A/B tests, set all kinds of constraints (ex: timeouts, number of turns), and configure the execution environment (ex: sandboxes, dependencies, how to handle AskQuestion). We also have an agent judge.It’s all in a YAML file now.We seem to be moving toward a world where companies rely more and more on evals to decide what to...
Remote: Yes (Worldwide)
Willing to relocate: No
Technologies: C# and .NET, SQL server, Javascript, Python, LLM & GenAI
Résumé/CV: https://drive.google.com/file/d/1Vjd6qrLAVpRuxtlzqhai-ke1DHI...
Email: please see the resume
Linkedin: https://www.linkedin.com/in/mihai-chirculescu-a65223b7
Github: https://github.com/Mihaiii
https://colab.research.google.com/github/Mihaiii/backtrack_sampler/blob/main/demo.ipynb
^You can find in the url a small colab demo.Below is the link to the actual framework. I take pride in making it as simple as possible so that others can contribute with ease: https://github.com/Mihaiii/backtrack_sampler
This means it gives you the ability to set rules by which the last tokens are considered incorrect and need to be regenerated.
I have included 2 demo algorithms.
It offers support for both GGUF models (llama.cpp) and models in Huggingface format (Transformers library).
Enjoy!
https://huggingface.co/datasets/Mihaiii/qa-assistant
I made a STSB alternatives, but with dialog/assistant samples.I couldn't find anything similar online (!), so I built it.The reason I did it was because I needed a very small model that would work well with my React component, and none of the existing 17M models performed adequately.The one I created with this dataset does.Embedding models, like other types of models, can be task-specific, and...
Another demo idea I had is to have an input field where the user can enter a GitHub username, retrieve all the starred repositories, and enable semantic search on the titles and descriptions of those repositories.
The main idea is that users will typically enter their own usernames, and therefore, they are familiar with the repositories they have starred, which provides a better search experience when testing the component.
Let me know if something else would work better for you.
This is a React component for searching/sorting by meaning (not by "characters included in a string", like standard search).
It uses a small ML model that runs on client side (inside the component!). When I say small, I mean ~20MB. The model will be downloaded only once (first time) and afterwards imported from browser's cache.
You can use this component to search and filter by meaning a dropdown list or an external list (like paragraphs of a webpage). You can search with sentences on sentences, not just with small words/substrings.
Here is a demo: https://mihaiii.github.io/semantic-autocomplete/
I believe this is super useful in the real world! :)
Let me know if you have any questions or feedback!
Thank you!
https://github.com/Mihaiii/semantic-autocomplete
If you've ever used CTRL+F on websites or documentation, you'll love this functionality.
https://github.com/Mihaiii/llm_steer
Hey, guys!I'm pretty amazed steering vectors work so well so I made a Python module to add steering vectors more easily.Google Colab link included in the repo.Let me know what you think.