Predicting the length of time, it will take to get a Machine Learning (ML) project into production can be tricky. If there is an issue, more often than not, it is likely related to a disconnect between engineering and the data science team. Collaboration between data science and engineering is critical for a Machine learning project, but it is often a challenge.
Although data scientists and engineers both work with code and machines – their roles and mindsets are different. Data scientists extract knowledge and insights from data, while software engineers build products and systems. Data scientists can spend considerable time creating and tweaking data models and algorithms to get an ideal result, which makes their work more experimental and iterative than software development. Engineers are responsible for building functionality around the ML data models and getting products into production within a set timeframe.
The model development portion of an ML project is considered the ‘research phase’ and is where many ML projects get stalled due to continual model adjustments. Therefore, it can be extremely beneficial for data scientists to think in engineering terms, which often leads to a faster production cycle.
When it comes to ML project management, one can separate the process into three stages: Proof of Concept (PoC) or the research phase, the Demo phase, and the Engineering phase. In this article, we examine these different phases and how one can handle them to ensure smooth and timely delivery of projects. The resulting protocol can also ensure better estimates of time for production deployment.
The Rule Book
Based on several years of experience handling various ML projects as part of a data science team, we have created a number of heuristic rules that one can follow to ensure smooth, predictable, and faster time to production.
PoC Phase
Almost all ML projects require a PoC phase. PoC ensures a reasonable performing model, apart from ensuring feasibility.
Rule 1: Time bound PoC efforts for machine learning project
Since the PoC is essentially a research effort, it can go on for an undetermined time for two main reasons: 1) data scientists are never done searching for a better model, and 2) ML models have a multitude of hyper-parameters to adjust and refine. Therefore, it is essential to set and stick to a pre-determined timeframe to complete the PoC. This reality also drives the need for Rule 2.
Rule 2: Set Expectations of PoC beforehand
Start by clearly defining the output of the PoC either in terms of metrics or a set of feature behavior. One could argue that by clearly defining Rule 2, Rule 1 is unnecessary. But, Rule 2 will only be operational if the problem can actually be solved. Therefore, Rule 1 ensures the team does not go beyond a certain number of retries before giving up.
So how do you estimate the appropriate amount of time to develop a PoC? This takes experience and can evolve, but as a rule of thumb:
5 months for problems involving classical learning techniques or problems involving transfer learning
3 months for problems involving proven deep learning techniques
Demo Phase
Once the viability of the ML project is ensured, demonstration of the work becomes important. This also sets the path for Minimum Viable Product (MVP).
Rule 3: Demonstrate PoC effort to all the stake-holders
Involving the stakeholders has a number of impacts for MVP:
- Defining future course of the product
- Defining or re-defining supported features
- Redefining ML metrics
- Defining how it fits into an existing product or in the case of a new product, what the final product will look like
Though stakeholders differ from project to project, the minimum stakeholders should include:
- Product Owners: Person(s) who defined the product at the first place (CTO/CEO)
- Data Science Team: Team involved in PoC and subsequently, person(s) taking it to production
- Engineering Team: This team helps define the feasibility of a product
- Dependents: Mostly UI/UX team which takes the product to end-users. In some case, UI/UX may be combined with the Engineering team
The quality of demonstration becomes important as this is the project buying phase: the better the demonstration, the higher the chance of the project being approved. Data science is all about creating stories and this is the phase where the stories should speak clearly. These data science stories, combined with business understandable visualizations, are direct indicators of a successful demo. In addition to the model demo, a snapshot of how the PoC would be taken to production should also be presented by the engineering team and other dependents.
The demo phase should not last more than a month and should be time-boxed. Delaying a demo will result in higher chances of the PoC landing in the scrap yard or being preempted by higher priority projects.
Engineering Phase
Once the project is in an approved phase, the next step is to take the PoC to production. Taking a PoC into production needs to be handled carefully since the underlying product sometimes becomes the face of the company.
Rule 4: Set the Requirements Clearly for machine learning project
Setting clear requirements is important as it not only defines the goals for the data science team, but also for all the team/parties on which the ML project is dependent. The following factors should be accounted for:
- Features that will be supported
- Business metrics to be met
- Engineering and/or UI/UX requirements
- Infrastructure needs and DevOps requirements
- Budget allocation
The requirements should also determine if the final model’s performance does not meet the expectations either due to data unavailability or unforeseen model limitations. In such situations, one can still deploy to a limited set of users to validate the feature, as discussed under Rule 7.
Rule 5: Define Clear Timelines and the Design of ML project
Defining timelines for the data science team ensures the project is being tracked and brought to closure within an estimated time. It also sets a product launch time and therefore, timelines should be set carefully, accounting for unknowns. Timelines should also be accordingly defined for dependents to ensure all the parties work in parallel. A regular, agile-type tracking is required to identify blockers early and bring them to closure – before they start to over-power the project.
The allocation of sufficient time for QA and code reviews is often ignored in timelines. Code reviews ensure quality and code coverage before QA takes over. QA defines the product stability and therefore, should be accounted for appropriately during the planning stage.
Timelines should include integration points clearly. In cases where a dedicated engineering team is available, at least one engineer from the product team should work together with the ML engineer to ensure smooth and faster integration with the system.
Design constitutes an integral part of the system and a well-thought-through system design ensures future changes, apart from system robustness. Timelines should allocate sufficient time for design which varies depending on whether it is entirely a new feature/product or an add-on feature. Various aspects to consider while designing:
- Modularity and Reusability
- Scalability
- Accommodability for improved ML models
- Ease of use by end users
- Most of the ML projects take roughly 6-8 months to go to production.
Rule 6: Pre-launch Demo with All Stake Holders
A pre-launch demo is a good way to make sure the final product is consistent with what was agreed upon during the start of the project. It also ensures the team accommodates minor changes resulting from final product observations and the resulting discussions. A pre-launch demo is also a counter-check on the business metrics defined earlier. Therefore, the pre-launch demo should be completed nearly a month before the launch.
Rule 7: Phase-wise Product Deployment
Deployment should be completed in phases to ensure user feedback is accounted for incrementally, thereby further ensuring product quality and stability. The specific phase-wise approach will differ depending on the type of ML project, but generally includes:
Selective User Deployment: Pre-define the users to whom the product will be available. Typically these are internal users, who are less risky to the business and will provide detailed feedback.
A/B Testing Deployment: This phase is used to show the feasibility of the ML solution against an existing solution, which in most cases, would be a heuristic or a rule-based approach. The product is exposed to end-users in a selective way to judge the performance of the ML model.
Final Deployment: In this phase, the product is exposed to all users and/or all organizations.
The final deployment may take 2-6 months, depending upon different deployment phases involved, but plan on a minimum of 2 months.
The Bottom Line about Machine Learning Project
After considering all the phases and steps, it’s clear that a machine learning project can take roughly 10-12 months from PoC to production. To ensure the project is delivered within the allotted timeframe, start with clear requirements and well-defined business metrics. Also, allow sufficient time for QA and a phased deployment schedule. By following the framework above, the probability of delivering your ML project on time can increase dramatically.
Written by Abhishek Gupta
Thanks for sharing this valuable article. Very effective for beginners.
Very informative. Thanks for sharing. visit Github.
The amount of the deposit depends on the car, usually the amount ranges from 1000 to 5000 AED.
car for rent dubai recently posted…car for rent dubai
We are proud to offer a public fleet of premium cars for rent today, all of which are available to rent for a period to suit your needs.
how much to rent a lamborghini in dubai recently posted…how much to rent a lamborghini in dubai
hey,
thanks to share great information. I am very interested in machine learning. and am currently working on data annotation. our company working on video annotation and labelling, and image annotation.
Maria Garcia recently posted…Video Annotating & Labeling
먹튀검증 사이트 추천: https://mtverify.com/
먹튀검증 사이트 추천: https://mtverify.com/
Thank you I have just been searching for information approximately this topic for a while and yours is the best I have found out so far However what in regards to the bottom line Are you certain concerning the supply
Its like you read my mind You appear to know so much about this like you wrote the book in it or something I think that you can do with a few pics to drive the message home a little bit but instead of that this is excellent blog A fantastic read Ill certainly be back
안전한 토토 검증 믿고 이용하세요! https://kraphtdesign.com/
I loved as much as youll receive carried out right here The sketch is attractive your authored material stylish nonetheless you command get bought an nervousness over that you wish be delivering the following unwell unquestionably come more formerly again as exactly the same nearly a lot often inside case you shield this hike
What i dont understood is in reality how youre now not really a lot more smartlyfavored than you might be now Youre very intelligent You understand therefore significantly in terms of this topic produced me personally believe it from a lot of numerous angles Its like women and men are not interested except it is one thing to accomplish with Woman gaga Your own stuffs outstanding Always care for it up
Your blog is a constant source of inspiration for me. Your passion for your subject matter is palpable, and it’s clear that you pour your heart and soul into every post. Keep up the incredible work!
Hi i think that i saw you visited my web site thus i came to Return the favore Im attempting to find things to enhance my siteI suppose its ok to use a few of your ideas
I have been surfing online more than 3 hours today yet I never found any interesting article like yours It is pretty worth enough for me In my opinion if all web owners and bloggers made good content as you did the web will be much more useful than ever before
Attractive section of content I just stumbled upon your blog and in accession capital to assert that I get actually enjoyed account your blog posts Anyway I will be subscribing to your augment and even I achievement you access consistently fast
Your blog is a breath of fresh air in the often mundane world of online content. Your unique perspective and engaging writing style never fail to leave a lasting impression. Thank you for sharing your insights with us.
Your writing has a way of resonating with me on a deep level. It’s clear that you put a lot of thought and effort into each piece, and it certainly doesn’t go unnoticed.
What i do not realize is in fact how you are no longer actually much more wellfavored than you might be right now Youre very intelligent You recognize thus considerably in relation to this topic made me in my view believe it from numerous numerous angles Its like men and women are not fascinated until it is one thing to do with Lady gaga Your own stuffs excellent All the time handle it up
I do trust all the ideas youve presented in your post They are really convincing and will definitely work Nonetheless the posts are too short for newbies May just you please lengthen them a bit from next time Thank you for the post
What i do not realize is in fact how you are no longer actually much more wellfavored than you might be right now Youre very intelligent You recognize thus considerably in relation to this topic made me in my view believe it from numerous numerous angles Its like men and women are not fascinated until it is one thing to do with Lady gaga Your own stuffs excellent All the time handle it up
My brother suggested I might like this blog He was totally right This post actually made my day You can not imagine simply how much time I had spent for this info Thanks
I loved as much as youll receive carried out right here The sketch is attractive your authored material stylish nonetheless you command get bought an nervousness over that you wish be delivering the following unwell unquestionably come more formerly again as exactly the same nearly a lot often inside case you shield this hike
I do believe all the ideas youve presented for your post They are really convincing and will certainly work Nonetheless the posts are too short for novices May just you please lengthen them a little from subsequent time Thanks for the post
Its like you read my mind You appear to know a lot about this like you wrote the book in it or something I think that you could do with some pics to drive the message home a little bit but instead of that this is fantastic blog An excellent read I will certainly be back
Your blog is a constant source of inspiration for me. Your passion for your subject matter is palpable, and it’s clear that you pour your heart and soul into every post. Keep up the incredible work!
Wow amazing blog layout How long have you been blogging for you made blogging look easy The overall look of your web site is magnificent as well as the content
Your blog is a constant source of inspiration for me. Your passion for your subject matter is palpable, and it’s clear that you pour your heart and soul into every post. Keep up the incredible work!
I wanted to take a moment to commend you on the outstanding quality of your blog. Your dedication to excellence is evident in every aspect of your writing. Truly impressive!
Your writing has a way of resonating with me on a deep level. It’s clear that you put a lot of thought and effort into each piece, and it certainly doesn’t go unnoticed.
Your blog is a true gem in the world of online content. I’m continually impressed by the depth of your research and the clarity of your writing. Thank you for sharing your wisdom with us.
토토 커뮤니티: https://kraphtdesign.com
토토 커뮤니티: https://kraphtdesign.com
My brother suggested I might like this blog He was totally right This post actually made my day You can not imagine simply how much time I had spent for this info Thanks
helloI like your writing very so much proportion we keep up a correspondence extra approximately your post on AOL I need an expert in this space to unravel my problem May be that is you Taking a look forward to see you
I loved as much as youll receive carried out right here The sketch is tasteful your authored material stylish nonetheless you command get bought an nervousness over that you wish be delivering the following unwell unquestionably come more formerly again since exactly the same nearly a lot often inside case you shield this hike
you are in reality a just right webmaster The site loading velocity is incredible It seems that you are doing any unique trick In addition The contents are masterwork you have performed a wonderful task on this topic
먹튀검증사이트: https://mtverify.com/
먹튀검증사이트: https://mtverify.com/
먹튀검증사이트: https://mtverify.com/
먹튀검증사이트: https://mtverify.com/
스포츠중계 국대티비: https://kuktv01.com/
My spouse and I stumbled over here from a different web page and
thought I should check things out. I like what I see
so i am just following you. Look forward to looking at your web page again.
официальное казино Money X recently posted…официальное казино Money X
먹튀검증, 안전하고 투명하게.
99.9% 안전 보장! 먹튀검증!
스포츠중계는 역시 국대티비! 고화질로 생생하게 즐기세요.
정확하고 신뢰할 수 있는 먹튀검증, 후회 없는 선택!
먹튀검증, 안전한 베팅, 최고의 선택!
인기 스포츠중계부터 비인기 종목까지, 국대티비.
믿을 수 있는 먹튀검증사이트로 피해를 예방하세요.
먹튀검증으로 믿음을 드리는 먹튀투데이.
먹튀 걱정 없는 베팅, 먹튀투데이와 함께하세요.
회원들의 안전을 최우선으로 하는 먹튀검증사이트, 지금 확인하세요.
먹튀 검증 사이트를 통해 안심하고 거래하세요.
먹튀 검증 사이트 먹튀오프, 안전을 보장합니다.
검증된 토토사이트만 모인 곳, 토토천국!
토토사이트 검증이 필요하다면 토토천국으로 오세요.
먹튀 걱정 없이 즐기려면 먹튀검증사이트를 통한 검증이 필요합니다.
Наша мастерская предлагает профессиональный ремонт стиральных машин адреса любых брендов и моделей. Мы осознаем, насколько важны для вас ваши устройства для стирки, и обеспечиваем ремонт наилучшего качества. Наши квалифицированные специалисты проводят ремонтные работы с высокой скоростью и точностью, используя только оригинальные запчасти, что обеспечивает надежность и долговечность проведенных ремонтов.
Наиболее общие проблемы, с которыми сталкиваются пользователи автоматических стиральных машин, включают проблемы с барабаном, проблемы с нагревом воды, ошибки ПО, проблемы с откачкой воды и повреждения корпуса. Для устранения этих неисправностей наши квалифицированные специалисты проводят ремонт барабанов, нагревательных элементов, ПО, насосов и механических компонентов. Обращаясь в наш сервисный центр, вы гарантируете себе надежный и долговечный официальный ремонт стиральной машины на выезде.
Подробная информация размещена на сайте: https://remont-stiralnyh-mashin-ace.ru
Наша мастерская предлагает надежный ремонт ноутбуков на выезде различных марок и моделей. Мы осознаем, насколько значимы для вас ваши ноутбуки, и стремимся предоставить услуги наилучшего качества. Наши опытные мастера проводят ремонтные работы с высокой скоростью и точностью, используя только качественные детали, что гарантирует длительную работу наших услуг.
Наиболее общие проблемы, с которыми сталкиваются пользователи лаптопов, включают проблемы с жестким диском, неисправности экрана, программные сбои, неработающие разъемы и перегрев. Для устранения этих поломок наши профессиональные техники оказывают ремонт жестких дисков, экранов, ПО, разъемов и систем охлаждения. Доверив ремонт нам, вы гарантируете себе долговечный и надежный ремонт ноутбука.
Подробная информация размещена на сайте: https://remont-noutbukov-first.ru
스포츠중계의 즐거움을 더하다, 국대티비.
스포츠중계의 모든 것, 국대티비에서 만나보세요.
Your blog is a treasure trove of knowledge! I’m constantly amazed by the depth of your insights and the clarity of your writing. Keep up the phenomenal work!
I have read some excellent stuff here Definitely value bookmarking for revisiting I wonder how much effort you put to make the sort of excellent informative website
you are in reality a good webmaster The website loading velocity is amazing It sort of feels that youre doing any distinctive trick Also The contents are masterwork you have done a fantastic job in this topic
I have read some excellent stuff here Definitely value bookmarking for revisiting I wonder how much effort you put to make the sort of excellent informative website
world of guns как открыть все оружие https://apk-smart.com/igry/simulyatory/1585-world-of-guns-gun-disassembly-vzlomannaja-mod-vse-otkryto.html world of guns как открыть все оружие
P.S Live ID: K89Io9blWX1UfZWv3ajv
P.S.S Программы и игры для Андроид телефона Программы и игры для Андроид телефона Программы и игры для Андроид телефона d934841
I just could not leave your web site before suggesting that I really enjoyed the standard information a person supply to your visitors Is gonna be again steadily in order to check up on new posts
Wow amazing blog layout How long have you been blogging for you made blogging look easy The overall look of your web site is magnificent as well as the content
안전한 게임 환경을 위해 믿을 수 있는 먹튀검증사이트를 이용하세요!
Your article helped me a lot, is there any more related content? Thanks!
Your articles never fail to captivate me. Each one is a testament to your expertise and dedication to your craft. Thank you for sharing your wisdom with the world.
What i dont understood is in reality how youre now not really a lot more smartlyfavored than you might be now Youre very intelligent You understand therefore significantly in terms of this topic produced me personally believe it from a lot of numerous angles Its like women and men are not interested except it is one thing to accomplish with Woman gaga Your own stuffs outstanding Always care for it up
Your point of view caught my eye and was very interesting. Thanks. I have a question for you.
Serving Iraq with pride, BWER supplies high-performance weighbridges designed to improve transport logistics, reduce inaccuracies, and optimize industrial processes across all sectors.
믿을 수 있는 토토 정보를 얻고 싶다면 토토커뮤니티 토토라이브!
안전한 토토사이트 찾기, 토토검증소가 도와드립니다.
Your passion for your subject matter shines through in every post. It’s clear that you genuinely care about sharing knowledge and making a positive impact on your readers. Kudos to you!
Your writing has a way of resonating with me on a deep level. I appreciate the honesty and authenticity you bring to every post. Thank you for sharing your journey with us.
I don’t think the title of your article matches the content lol. Just kidding, mainly because I had some doubts after reading the article.
Thanks I have recently been looking for info about this subject for a while and yours is the greatest I have discovered so far However what in regards to the bottom line Are you certain in regards to the supply
사용자가 안심하고 이용할 수 있도록, 토토랭커는 투명한 먹튀검증을 제공합니다.
I just wanted to express my gratitude for the valuable insights you provide through your blog. Your expertise shines through in every word, and I’m grateful for the opportunity to learn from you.
Your blog is a constant source of inspiration for me. Your passion for your subject matter is palpable, and it’s clear that you pour your heart and soul into every post. Keep up the incredible work!
hiI like your writing so much share we be in contact more approximately your article on AOL I need a specialist in this area to resolve my problem Maybe that is you Looking ahead to see you
Your article helped me a lot, is there any more related content? Thanks!
Usually I do not read article on blogs however I would like to say that this writeup very compelled me to take a look at and do it Your writing style has been amazed me Thank you very nice article
신뢰받는 토토사이트 추천, 전문가의 선택입니다.
Somebody essentially lend a hand to make significantly articles Id state That is the very first time I frequented your website page and up to now I surprised with the research you made to make this actual submit amazing Wonderful task
Noodlemagazine I’m often to blogging and i really appreciate your content. The article has actually peaks my interest. I’m going to bookmark your web site and maintain checking for brand spanking new information.
토토검증소의 추천 토토사이트로 안전하게 시작하세요.
Thank you for your sharing. I am worried that I lack creative ideas. It is your article that makes me full of hope. Thank you. But, I have a question, can you help me?
I do trust all the ideas youve presented in your post They are really convincing and will definitely work Nonetheless the posts are too short for newbies May just you please lengthen them a bit from next time Thank you for the post
I was suggested this web site by my cousin Im not sure whether this post is written by him as no one else know such detailed about my trouble You are incredible Thanks
Noodlemagazine I just like the helpful information you provide in your articles
Thanks for sharing. I read many of your blog posts, cool, your blog is very good.
I don’t think the title of your article matches the content lol. Just kidding, mainly because I had some doubts after reading the article.
I have been surfing online more than 3 hours today yet I never found any interesting article like yours It is pretty worth enough for me In my opinion if all web owners and bloggers made good content as you did the web will be much more useful than ever before
Thank you for the good writeup It in fact was a amusement account it Look advanced to far added agreeable from you However how could we communicate
I wanted to take a moment to commend you on the outstanding quality of your blog. Your dedication to excellence is evident in every aspect of your writing. Truly impressive!
Your blog is a beacon of light in the often murky waters of online content. Your thoughtful analysis and insightful commentary never fail to leave a lasting impression. Keep up the amazing work!