LLM evaluation for AI builders

Free 3-week video course with 10 hands-on code tutorials. From designing custom LLM judges to RAG evaluations and adversarial testing.

Register now

About the course

  • Free

    This course is free. Let LLM evals be accessible to all!

  • 10+ video tutorials

    Ptactical tutorials into your inbox.

  • End-to-end code examples

    Code examples on core LLM evaluation workflows.

  • Live Q&A sessions

    Live Q&A sessions with course instructors.

Course syllabus

This is a 3-week course that covers core LLM evaluation workflows.

  • LLM evaluation methods

  • Designing and evaluating LLM judges

  • Hands-on LLM app testing

  • RAG evaluation and testing

  • Adversarial testing

  • Organizing evaluation workflow in a team

Meet course instructors

Emeli Dral

Co-founder and CTO, Evidently AI

Emeli Dral is a Co-founder and CTO at Evidently AI, a startup developing open-source tools to evaluate, test, and monitor the performance of machine learning models. ‍ Earlier, she co-founded an industrial AI startup and served as the Chief Data Scientist at Yandex Data Factory. She led over 50 applied ML projects for various industries: from banking to manufacturing. Emeli is a data science lecturer at GSOM SpBU and Harbour.Space University. She is a co-author of the Machine Learning and Data Analysis curriculum at Coursera with over 150,000 students.

Elena Samuylova

Co-founder and CEO, Evidently AI

Elena Samuylova is a CEO and Co-founder at Evidently AI, a Y Combinator-backed startup developing AI observability tools to evaluate, test, and monitor the quality of AI systems. She has been active in the applied machine learning space since 2014. Previously, she co-founded and served as a CPO of an industrial AI startup. She worked with global metal and chemical companies to implement machine learning for production optimization. Prior to that, she led business development at Yandex Data Factory, an enterprise AI division of Yandex. She focused on delivering ML-based solutions to retail, banking, telecom, and other industries. In 2018, Elena was named 50 Women in Product Europe by Product Management Festival.

FAQ

  • Is the course free?

    Yes, the course is 100% free.

  • What is the course format?

    The course is organized into a series of emails. You will receive a few emails per week that consist of a video covering a particular topic and a link to practical code example to follow along.

  • Will I get a course certificate?

    You will receive a certificate if you successfully complete the final assignment by June 5, 2025. Important: After this date, you can no longer submit the assignment or receive a certificate.

  • Who the course is for?

    This course is for anyone building real-world AI applications - whether you're an AI/ML Engineer, data scientist, product manager, or researcher. If you’re working hands-on with LLMs and have basic Python skills, this course is for you.

  • What if I need help?

    Jump to our Discord #-llm-evaluations-course channel to chat with fellow learners and get support from the course team: https://discord.gg/PyAJuUD5mB.

  • How can I enroll?

    Enroll here to receive course materials: https://www.evidentlyai.com/llm-evaluation-course-practice