Catalog

With the newfound prevalence of applications built with large language models (LLMs) including features such as Retrieval Augmented Generation (RAG), agents, and guardrails, a responsibly-driven evaluation process is necessary to measure performance and mitigate risks. This session covers best practices for a responsible evaluation. Learn about open access libraries and AWS services that can be used in the evaluation process, and dive deep on the key steps of designing an evaluation plan including defining a use case, assessing potential risks, choosing metrics and release criteria, designing an evaluation dataset, and interpreting results for actionable risk mitigation.