LLM evaluations for AI product teams

Free 7-day email course on LLM evaluations. A gentle introduction to evaluating LLM-powered products, no coding knowledge required.

Register now

About the course

  • 10+ byte-sized videos

    7 daily emails with byte-sized videos and key takeaways.

  • LLM evals explained in plain English

    It is a gentle introduction to LLM evals, no coding skills required.

  • Free

    This course is free. Let LLM evals be accessible to all!

Course syllabus

The course is organized as 7 daily emails that cover a particular topic.

  • Day 1: LLM evaluation benchmarks

  • Day 2: How to evaluate an LLM application

  • Day 3: Designing test datasets and LLM evaluation methods

  • Day 4: LLM-as-a-judge

  • Day 5: LLM safety and risks: hallucinations, jailbreaks, and how to protect

  • Day 6: LLM observability in production

  • Day 7: A business case for LLM evaluations

Meet your instructor

Elena Samuylova

Co-founder and CEO, Evidently AI

Elena Samuylova is a CEO and Co-founder at Evidently AI, a Y Combinator-backed startup developing AI observability tools to evaluate, test, and monitor the quality of AI systems. She has been active in the applied machine learning space since 2014. Previously, she co-founded and served as a CPO of an industrial AI startup. She worked with global metal and chemical companies to implement machine learning for production optimization. Prior to that, she led business development at Yandex Data Factory, an enterprise AI division of Yandex. She focused on delivering ML-based solutions to retail, banking, telecom, and other industries. In 2018, Elena was named 50 Women in Product Europe by Product Management Festival.

FAQ

  • Is the course free?

    Yes, the course is 100% free.

  • What is the course format?

    The course is organized into a series of 7 daily emails. Each day you will receive an email that consists of a byte-sized video covering a particular topic, a video summary, and recommendations for further reading.

  • Will I get a course certificate?

    You will receive a certificate if you successfully complete the final quiz by March 1, 2025. After this date, you can no longer take the quiz or receive a certificate.

  • Are there any course prerequisites?

    The course is designed for everyone interested in grasping the fundamentals of LLM evaluations. No previous knowledge of the subject or coding skills required.

  • What if I need help?

    Jump to our Discord #-llm-evaluations-course channel to chat with fellow learners and get support from the course team: https://discord.gg/PyAJuUD5mB.

  • Where can I find course materials?

    Follow our YouTube playlist to access all course videos: https://youtube.com/playlist?list=PL9omX6impEuMgDFCK_NleIB0sMzKs2boI&feature=shared