How do you systematically measure, optimize, and enhance the efficiency of LLM purposes (like these powered by RAG or software use)? Ragas is an open supply effort that has been making an attempt to reply this query comprehensively, and they’re selling a “Metrics Pushed Growth” strategy. Shahul from Ragas joins us to debate Ragas on this episode, and we dig into particular metrics, the distinction between benchmarking fashions and evaluating LLM apps, producing artificial take a look at knowledge and extra.
Changelog++ members save 5 minutes on this episode as a result of they made the adverts disappear. Be part of as we speak!
Sponsors:
- Meeting AI – Flip voice knowledge into summaries with AssemblyAI’s main Speech AI fashions. Constructed by AI consultants, their Speech AI fashions embrace correct speech-to-text for voice knowledge (reminiscent of calls, digital conferences, and podcasts), speaker detection, sentiment evaluation, chapter detection, PII redaction, and extra.
That includes:
Present Notes:
One thing lacking or damaged? PRs welcome!
