News for hackers. Stuff that matters.

kavaivaleri

The most widely used benchmarks for evaluating LLMs

Commonsense Reasoning - HellaSwag - Winogrande - PIQA - SIQA - OpenBookQA - ARC - CommonsenseQALogical Reasoning - MMLU - BBHardMathematical Reasoning - GSM-8K - MATH - MGSM - DROPCode Generation - HumanEval - MBPPWorld Knowledge & QA - NaturalQuestions - TriviaQA - MMMU - TruthfulQAI collected their descriptions and links to their original papers here:...

( Read More... | 1 comment | on HN )
Great cover image! :)

We also write about LLMs! https://www.turingpost.com/t/FMOps

Re: on Wednesday February 14, 2024 @05:49AM (#39368497)
We explore Databricks' unusual history, its contributions to the generative AI field for Enterprise, and what its strategies reveal about the evolving landscape of GenAI and data intelligence platforms.

- Starting point of Databricks

- We are open-source and... poor. Changes in strategy that led to $$$

- Turning to Generative AI (and proving again open-source is cool)

- Generative AI's role in the next wave of enterprise data applications

- Product policy: “Cannibalize yourself before someone else does”

- Mission

- Financial situation and acquisitions

- How does the company make money?

- Vision of the future of the AI industry

- Bonus: Seven brilliant founders behind Databricks

Some pretty fascinating collaborators:

- Matei Zaharia, a CTO of Databricks - Pieter Abbeel Director of the Berkeley Robot Learning Lab, Co-Director of the Berkeley Artificial Intelligence Research (BAIR) lab - Two talented PhD students: Hao Liu, Wilson Yan

Re: on Tuesday February 13, 2024 @05:28AM (#39356331)
Since the bloom of social media, it’s been tough times for journalism as so many voices appeared and the cacophony was deafening.

AI-generated content adds even more infotrash. But surprisingly enough, I think that AI is here to bring us back to the quality of journalism, both as a risk factor and as an enabler.

Open-source tools for LLM monitoring and observability

- AllenNLP Interpret: A library for interpreting and visualizing LLM predictions, assisting in model explanation and debugging. It works for any model of your choice.- LangKit: An open-source toolkit for monitoring Large Language Models (LLMs). Features include assessing text quality and relevance, hallucinations check, sentiment and toxicity analysis.- BERTViz: Specifically designed for...

( Read More... | 9 comments | 2 points | on HN )

Recent AI developments across the globe

The UK: All about safety In November 2023, the UK made a significant leap in AI safety with Prime Minister Rishi Sunak announcing https://time.com/6330877/uk-ai-safety-summit/ the establishment of the UK AI Safety Institute at the AI Safety Summit. The initiative aims to become a global hub for AI safety, extending beyond testing to addressing risks ranging from social impacts to loss of...

( Read More... | no comments | on HN )

Books about AI&ML for your holiday reading

Classics- Deep Learning by Ian Goodfellow, Yoshua Bengio, and Aaron Courville- The Book of Why: The New Science of Cause and Effect by Judea Pearl, Dana Mackenzie- Designing Machine Learning Systems by Chip Huyen- The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World by Pedro DomingoFresh look (books published in 2023)- The Worlds I See: Curiosity,...

( Read More... | no comments | on HN )

What Is Vector Database's Role in FMOps?

The "Vector Database Market" research report forecasts significant growth in the sector from $1.5 billion in 2023 to $4.3 billion by 2028, a CAGR of 23.3%.We discuss why these databases matter for AI, how they work, and their role in handling complex data; we also explore alternative solutions and provide expert insight on security. Plus you get a curated list of open-sourced vector databases...

( Read More... | no comments | on HN )

One Year of ChatGPT – The Impact

In three days, ChatGPT, developed by OpenAI, will celebrate its first public launch anniversary. Its success has shown us just the tip of the iceberg of ML and deep learning capabilities. GPT models and other Foundation Models (FMs) have since ignited numerous debates about the future of AI development: Will the focus be on synthetic data, or perhaps a novel architecture beyond transformers? Or...

( Read More... | no comments | 5 points | on HN )

Real-world implementation of FMs and LLMs: bioengineering

We are happy to introduce a new series of interviews that highlights the real-world implementation of Foundation Models (FMs) and Large Language Models (LLMs). We start with an absolutely fascinating topic that can bring so many changes into our world: Bioengineering. In this insightful conversation with Barry Canton, CTO and co-founder of Ginkgo Bioworks, we discussed the evolution of...

( Read More... | no comments | 5 points | on HN )

What's Happening in Self-Driving Universe?

Editorial:Since the Tesla I was driving, equipped with Autopilot and Full Self-Driving, nearly drove me under a truck, my interest in self-driving cars has intensified. As if serving my curiosity, the last week was filled with news about self-driving, driverless, autonomous vehicles, and robotaxis – so, here's my Monday take.Let’s be clear, I don’t blame Tesla. In the same way, I can’t fault...

( Read More... | no comments | on HN )

Runway Chronicles: Melding Art and AI in a Transient Tech Terrain

There is almost nothing less stable than the landscape of Generative AI (GenAI). The last week's changes include a sudden $0.3 billion drop by Jasper. It’s not a surprise in a world where apps are built on someone else’s model/technology. But we will be talking about something ‘stable’ today: the Stable Diffusion model and Runway – whose researchers are behind the model development. It’s...

( Read More... | no comments | on HN )

The Quest for Self-Reproducing Automata

The concept of self-reproducing automata envisioned by John von Neumann in 1948 heralded a paradigm where machines could mimic biological self-reproducing. Alan Turing in 1950 posed a question if machines can think.Von Neumann never saw his self-reproducing machine come to life, but 75 years later this notion has resurfaced with contemporary advances in machine learning (ML), illuminating a...

( Read More... | no comments | on HN )

Lightricks: The Oldest Generative AI Unicorn

The landscape of Generative AI (GenAI) changes almost every week. Just like with Imbue the other day. The company, focused on reasoning in AI, joined the unicorn family overnight after raising an insane $200 million Series B round.In the breakneck race, let’s not forget about veterans: number 6 on our list is Lightricks, the oldest generative AI unicorn. Also, the first one on the list without...

( Read More... | no comments | on HN )

Apple's AI Moat

Several AI publications have expressed doubts that Apple has any moat. The lack of notable LLM moves, absence of AI mentions during their WWDC presentation in June, and the lack of eye-popping AI investments or partnerships have raised eyebrows.Well, Apple is playing its own chess game, probably maneuvering its queen in a strategic play that could checkmate the industry. Apple has been here for...

( Read More... | no comments | on HN )

Interview: OpenAI GC Che Chang on Managing Risks and Rewards of AI

We're pleased to launch 'Interviews with Innovators,' where we'll explore AI topics in-depth with experts from diverse backgrounds. Today, our guest is a person with lots of responsibilities on his shoulders: please meet Che Chang, General Counsel at OpenAIWe’ve discussed how the OpenAI legal team uses ChatGPT, how they navigate legal uncertainty around new technology, and how they operate...

( Read More... | no comments | on HN )

Cohere AI Chronicle – Challenging Tech Giants with AI for All

Cohere lost two positions in the ranking of AI unicorns after the sudden rise of Hugging Face and Inflection.Now, Cohere is #5, but does it mean the company loses its position in the market? Let's dive deeper and discover more about the birth of Cohere, its mission, financial situation, main products, and other crucial details! Things are not as simple as it...

( Read More... | no comments | on HN )

Democratizing AI: The Hugging Face Ethos of Accessible Machine Learning

In August 2023, Hugging Face secured $235 million in a Series D round, boasting a valuation of $4.5 billion. It surged ahead, not just of Cohere but surpassing Inflection, with its ~$4.0 billion valuation.According to our own Twitter sentiment analysis, Hugging Face is a widely admired player in both ML and AI. Everybody seems to love them.How did they achieve it?What makes this light-looking...

( Read More... | no comments | on HN )
Re:State of AI in Russia, 2023 on Monday August 28, 2023 @02:26PM (#37298470)
Thanks for your feedback. Just removed the subscriber break so anyone can read the article now.

State of AI in Russia, 2023

In 2023, Russia's relationship with AI has been multifaceted and complex. While the country has been somewhat isolated due to its exclusion from international platforms like ChatGPT, this isolation has not halted local developers.- Challenges The advancement of AI in Russia faces significant obstacles including brain drain, hardware limitations, and censorship constraints. It's the extremely...

( Read More... | 2 comments | on HN )

The New Roles of Prompt Engineer and AI Engineer

Have you heard about the fascinating new roles of Prompt Engineer and AI Engineer? Some useful resources• Andrew Ng’s editorial in The Batch says that prompt-based development using LLMs has accelerated the ML development cycle, reducing project timelines from months to...

( Read More... | no comments | on HN )

Older »