Flowchart of ML Project Phases
- Define business goals
- Stakeholders define the value, budget and success criteria
- Defining KPIs (Key Performance Indicators) is critical
- ML problem framing
- Convert the business problem into a ML problem
- Determine if ML is appropriate
- Data scientists, data engineers, ML architects and subject matter experts (SME) collaborate
- Data collection and preparation (preprocessing)
- Data collection and integration (make it centrally accessible)
- Exploratory Data Analysis
- Visualize data with graphs (makes it understandable)
- Correlation Matrix:
- Look at correlations (links) between variables
- Helps decide which features can be important
- Example
- Data processing
- Convert the data into a usable format
- Feature engineering: create, transform and extract variables from data
- Model development
- Model training, tuning, and evaluation
- Iterative process
- Additional feature engineering and tune model hyperparameters
- Retrain model
- Look at data and features to improve the model
- Adjust the model training hyperparameters
- Model Deployment
- If results are good, the model is deployed and ready to make inferences
- Select a deployment model (real-time, serverless, asynchronous, batch, on-premises…)
- Monitoring the model
- Deploy a system to check the desired level of performance
- Early detection and mitigation
- Debug issues and understand the model’s behavior
- Iterations on production model
- Model is continuously improved and refined as new data become available
- Requirements may change
- Iteration is important to keep the model accurate and relevant over time