Author: Max Frolov
How to build a project from scratch by using a latest, state-of-the-art AI technology stack?
After spending a few years working on data science – enriched products, we tried more than 10 different workflow concepts – adopted scrum & agile, waterfall, iterative kanban canvas etc.
As a result, we’ve drafted our own data science project development workflow that allows us to iterate over data science hypotheses, process and acquire data, build and deploy models.
1. Slicing business processes & ideas into data science hypotheses. H&T Specs.
Decompose existing business processes or draft new ones during brainstorm sessions with our Core AI team. Diving into existing data, we build dataset landscape used to generate data science hypotheses related to each business task. Created a Hypotheses & Test Cases (H&T) specifications we’re all set to start rapidly prototyping AI-Powered MVP’s.
2. Rapid prototyping over defined hypotheses. Build production-ready ML models.
A Dedicated team formed by our AI & HighLoad Labs, prioritize hypotheses list identifying key metrics – business values, KPIs and models accuracy. Data engineers prepare existing company’s and acquired data to fulfill a Data Warehouse in the cloud (Google Cloud, AWS, Microsoft Azure), forwarding it to data science team to train models and test hypotheses. Hypotheses become proved or not, during training & validation process, if they satisfy accuracy and boundary business KPIs, resulting in validated ML models – core components of any AI-Powered MVP.
3. ML Models deployment & real-world testing. Delivering AI-Powered MVP.
Delivering ML models to the real-world require an appropriate real-time infrastructure. We build such using the latest data engineering tech stack merging existing business logic & code together. Measurement of business KPIs for each model is a key impact factor during the entire development lifecycle. We ❤️ using Tableau, Power BI, Looker. By using various techniques to deliver AI-Powered predictions, scorings, recognized patterns, we complete business tasks and finalize our MVP development journey. Go to step 1 ⚡️
Originally posted here