Alexey: Okay. So let's go to the questions. We have quite a few of them. The first question we have is, “What are the typical components of a machine learning system? And what percentage of it are machine learning algorithms?” ( 47:52) Valerii: Also, another good way would be just to sketch like what we've done right now. In five minutes, we almost finished a very, very, very basic design of a fraud detection system. Because we already spoke about loss function, the model, the feature interaction, the metrics. We even mentioned A/B tests. So now we could go, “Okay, we outlined it. Do you want me to focus on something else? I'll go step by step, diving deeper and deeper.” I would make a second iteration, a third iteration. Because usually, how I do that, I tell the interviewer, “I will build a baseline, and then once we have a baseline [we’ll move on]” because usually, what you do in real machine learning, is you either take a heuristic as a baseline, or you take a very simple model. ( 29:09) The tutorial approach has been tremendously successful in getting models off the ground. However, the business requirements change, and (3) data distributions constantly shift. Without an intentional design

Gradient boosted trees: Better performance than logistic regressions, can find non-linear interactions, typically doesn’t require much tuning.Hammer out timing SLAs (eg. we’ll incorporate user actions into recommendations within X seconds/minutes/hours) Modelling is one of the key skills for any ML practitioner, and you want to show your depth in this area. There’s so many techniques for modelling, it’s good to cover some breadth instead of naming one solution. Model Types Valerii: Let's do a mental exercise. Let's imagine that you have a computer vision, deep learning model. Very sophisticated – 175 layers. And then there is a classification model. And on top of this model, you have what? You have a linear classificator. What does it mean? It means that, actually, this model classifies with their linear model. And all that is done before is just representational learning, transforming the original features to the features, which might be fed to the linear model very successfully. See – features. Just with this mental exercise, you can see that. So that's why you can take embeddings, put them in whatever model you would like to, and you have a proper output. ( 49:57) Applying ML systems to real-world problems

Valerii: Let's be honest, the interviewer was a human, and humans are subjective. Maybe they had a bad day. However, to some extent, I'm surprised because it's hard to say the interview was nodding. Maybe, again, the way you remember it and the way it was – it's a natural thing for human beings to remember some things. There is even a saying “Lies like a witness.” So that's hard to say. However, usually, you could tell – you could try to secure yourself in an interview by asking “Do you want me to focus on that? Alright, let’s go.” ( 29:09) Machine Learning System design is now available, you can get the course on educative.io here and interviewquery.com SectionValerii: To some extent, it’s like cases for a consulting company. They train you to solve any case, even if you've never been working in their aircraft manufacturing company. But somehow, now you're an expert and you can suggest to the CEO of this company how to run his or her business. ( 39:13) The importance of defining a goal and ways of measuring it As a candidate, I’ve been interviewed at a dozen big companies and startups. I’ve got offers for machine learning roles at companies including Google, NVIDIA, Snap, Netflix, Primer AI, and Snorkel AI. I’ve also been rejected at many other companies. Valerii: Right, right. Like spending time, the attrition rate, the churn rate, retention, what else? There are many, many things. ( 43:53) What to do after you set a goal Alexey: Yeah, indeed. So, the original question I actually asked you is about the difference between system design and machine learning system design and I think it's very clear what machine learning system design is. It requires some domain knowledge, to some extent, or making some assumptions. Then you need to walk through the process of solving a particular problem. ( 22:05)

This is work-in-progress so any type of contribution is very much appreciated. Here are a few ways you can contribute: For example, if you are performing binary classification, you will use the following offline metrics: Area Under Curve (AUC), log loss, precision, recall, and F1-score. I’m a SWE, ML with 10 years of experience ( Linkedin profile). I had offers from Google, LinkedIn, Coupang, Snap and StichFix. Read my blog. Get Book Interaction Design for 3D User Interfaces by Francisco R. Ortega,Fatemeh Abyarjoo,Armando Barreto,Naphtali Rishe,Malek Adjouadi Pdf

Table of Contents

Alexey: I have an example from my personal experience of being interviewed at one of these companies on system design. I had the question to design a system for finding places of interest. So let's say I go to London – I go to whatever central square you have in London, and the system would need to give me all the points of interest, all the closest interesting places. ( 22:05) There is so much information one needs to know to be an effective machine learning engineer. It's hard to cut through the chaff to get the most relevant information, but Chip has done that admirably with this book. If you are serious about ML in production, and care about how to design and implement ML systems end to end, this book is essential." - Laurence Moroney, AI and ML Lead, Google Note that this is common for interview loops for ML generalists like myself. If you’re a researcher in NLP, image recognition or some other specialized field, you may get interview design questions focussed on that. Eg. If you’re coming from the Siri voice recognition team and interviewing at Alexis, you can probably expect some deeper ML questions on voice recognition. When you have nailed down all of your ML system’s requirements, you can proceed to building your model. This involves: