Krakow, Poland, 22 - 24 June 2022
Sebastian Gębski - Principal Solutions Architect at Amazon Web Services. ex-CTO, ex-VP of Engineering. He has co-built online platforms since 2001: across all kinds of organizations - starting with startups and ending with international financial groups, on 3 continents. He was designing solutions for development factories up to 300 people and leading engineering organizations up to 70 people. Hyper-active blogger (No Kill Switch), co-host of CTO Morning Coffee, and a serial reader. Fan of deliberate system design (e.g. Semantic Spatial Maps), System's Theory, and practical application of DSLs.
Instead of a crystal ball - A/B testing in practice
Conference
The idea of A/B testing appears trivial: two versions of the same product are presented in parallel to the two groups of users, to determine (based on output metrics - e.g. conversion rate) which one is more beneficial. Thanks to a series of such experiments (we keep the better version and ditch the other one) one can iteratively improve the quality of the application. However, although many companies declare the willingness to try A/B testing in practice, very few do that. It turns out that the apparently simple idea is non-trivial to implement. It requires skillful usage of feature flags, telemetry for business metrics, and of course - a proper setup of analytic tools.
The goal of this session is to prove that nearly everyone can easily and quickly build an A/B testing rig for their Internet application - thanks to managed, ready-to-use cloud services on AWS. We'll start with custom metrics in CloudWatch and AppConfig, only to continue with features and experiments in Evidently. Then we'll have some fun while feeding analytical tools with A/B testing data to gain even more interesting product-related insights.