SlideShare a Scribd company logo
recommendation =
optimization(prediction)
Wit Jakuczun, PhD
Once upon a time Wit met a Customer that needed
demand forecasts to ...
Customer: I need better demand forecasts.
Me: I understand. Can I have a simple question?
Customer: Yes
Me: Imagine I created a demand forecasting model and provided you with
desired 5M forecasts (numbers). What are you going to do with them?
Customer: Well… I will take the forecasts and optimize my logistics decisions using
the numbers.
Me: I see. Why don’t we talk about the whole decision problem? Maybe the
inefficiency is not in demand forecasts but in optimization part?
Customer: Can you create a math model for such complex business problem with
many constraints and exceptions? I thought it was impossible.
Recommendations are calculated in “Sheet” and it is a bottleneck.
Me: It is possible to build decision support system that uses mathematical
optimization for your problem
Customer: Great, let’s talk about the details.
What has happened in the past?
What is an optimal course of
actions for the past?
What can happen in the future?
What is an optimal course of
actions for the future?
DataAnalyticsOptimization
Forecasts
Expert
(guts mainly)
Decisions
Forecasts
Recommendations
(optimization)
Decisions
Expert
(guts mainly)
Forecasts
Expert
(guts mainly)
Decisions
Forecasts
Recommendations
(optimization)
Decisions
Expert
(guts mainly)
Notreproducibleandnotscalable
Reproducibleandscalable
Forecasts/predictions are “just” a tool for better decisions.
And better decisions are based on right recommendations.
And right recommendations are result of optimizing business KPIs that are linked
to business decisions.
Optimization model deals helps to improve robustness of decision making
process robust.
Automation of the complex business process.
Transition to central/global planning.
Learning from best (optimal) decision.
I was selling vehicle routing solution to a logistics company.
I managed to persuade manager/owner to meet and talk with the team.
After 1 hour presentation of the solution to the company I got one question
How can I create an invoice in this solution?
This is real story that happened to me. Similar story can be read in “Being wrong with Clarke & Wright” by Robert E.D.Woolsey
recommendation = optimization(prediction)
There are three commercial highly efficient solvers for mixed integer problems.
Solutions have been on the market for 25+ years…. and are still in development.
Very resistant to parallelization and distributed computing techniques.
Very sensitive to data.
Tightly coupled with business.
Only “auto” for mip.
There is only one good library for mixed integer programming that is open-source.
Graphics taken from SCIP solver webpage. Benchmarks from Hans Mittelmann
Most common ML libs are open-source.
Open-source is very efficient compared to commercial solutions.
Easy to create distributed implementations
Fairly insensitive to data.
Less tightly coupled with business.
Quite a few AutoML solutions that work.
Problem is infeasible.
Explanation is extremely difficult.
Customers expect any-time feature.
Debugging is hell :)
recommendation = optimization(prediction)
Mixed-integer
programming
black-box
hard to customize
limited applicability
(still wide!)
Constraint
programming
white-box
easy to customize
not limited applicability
Metaheuristics
custom-box
easy to customize
not limited applicability
Easy Difficult
Mixed Integer Programming
IBM CPLEX
Gurobi
Fico
Local Solver
Constraint programming
Sicstus
IBM CP Optimizer
Commercial world
Mixed Integer Programming
CBC (solver)
MIP (wrapper)
Or-tools (wrapper)
Constraint programming
ECLiPSe
Choco
Gecode
Or-tools
Open-source world
Easy
I can declare model using existing solver.
Fairly difficult
I can solve problem with a sequence of easy problems.
Very Difficult
I must implement custom solver.
Business requirements are almost impossible to be collected upfront.
Performance is not satisfactory.
Solution quality is not satisfactory.
No solution found is not acceptable.
Validator
Solver
Optimization engines
Integration layer
Load balancer
Validator
Solver
Optimization engines
Start with business process and decisions
Start with small and iterate.
Use real data since the first day of the project.
Assume problems are infeasible or internally contradictor.
Deal with must vs nice to have requirements.
recommendation = optimization(prediction)
Saving up to 20% of cash management costs in Deutsche Bank
Challenge
• Factory throughput was too low
• Upgrade or not to increase throughput
Solution
• Integrated planning and scheduling optimization model
• Scenario generation to support investment decision
• Tailor made optimisation model
Effects
• Ability to support investment decision with numbers
Based on academic work by Roman Barták
Challenge
• Dynamic and unpredictable orders flow
• Complex tasks
Solution
• Automation by optimisation
• Tailor made optimisation model
Effects
• In progress - feasibility tests of the
working solution
Collect Pay Drive Deliver Drive
Collect Pay Drive Deliver Drive
Collect Pay Drive Deliver Drive
recommendation = optimization(prediction)
Contact info
● Private: wit.jakuczun@gmail.com
● Business:
○ wit.jakuczun@fourteen33.com
○ wit.jakuczun@wlogsolutions.com

More Related Content

PDF
Always Be Deploying. How to make R great for machine learning in (not only) E...
PDF
Driving your marketing automation with multi-armed bandits in real time
PDF
Continuous Delivery for Machine Learning
PPTX
Eric Ries Lean Startup Schematic View Of Agile Development And Customer Devel...
PPT
Colin Robb - SOA - Agile or Fragile?
PDF
Driving Innovation with Kanban at Jaguar Land Rover
PDF
The Disciplines of Continuous innovation
PDF
Fix-Price Projects And Agile – PyCon Sette
Always Be Deploying. How to make R great for machine learning in (not only) E...
Driving your marketing automation with multi-armed bandits in real time
Continuous Delivery for Machine Learning
Eric Ries Lean Startup Schematic View Of Agile Development And Customer Devel...
Colin Robb - SOA - Agile or Fragile?
Driving Innovation with Kanban at Jaguar Land Rover
The Disciplines of Continuous innovation
Fix-Price Projects And Agile – PyCon Sette

What's hot (20)

PPTX
Technical debt a Business Perspective
PDF
When we design together
PPTX
Agile and fixed budget projects
PDF
Why change code that works - On Technical Debt and Refactoring
PDF
Structured Authoring for Business-Critical Content
PDF
Experiencing Agility From Requirements to Planning
PDF
When Testers Feel Left Out in the Cold
PPTX
2009_06_08 The Lean Startup Tokyo edition
PDF
Technical debt strategy
PPTX
Thoughts on productivity in software development
PDF
Agile 103 - the three big questions
PDF
ML Playbook
PDF
Testing Transformation: The Art and Science for Success
PDF
May 2021 Embedded Vision Summit Opening Remarks (May 26)
PDF
Agile for Business Analysts
PPTX
Agile Development with Agile Contract
PDF
Way to Agile - USTH
PDF
Evidence Based Management - Measuring value to enable improvement and agility
PDF
Agile testing
PPT
Technical and Product Debt Management
Technical debt a Business Perspective
When we design together
Agile and fixed budget projects
Why change code that works - On Technical Debt and Refactoring
Structured Authoring for Business-Critical Content
Experiencing Agility From Requirements to Planning
When Testers Feel Left Out in the Cold
2009_06_08 The Lean Startup Tokyo edition
Technical debt strategy
Thoughts on productivity in software development
Agile 103 - the three big questions
ML Playbook
Testing Transformation: The Art and Science for Success
May 2021 Embedded Vision Summit Opening Remarks (May 26)
Agile for Business Analysts
Agile Development with Agile Contract
Way to Agile - USTH
Evidence Based Management - Measuring value to enable improvement and agility
Agile testing
Technical and Product Debt Management
Ad

Similar to recommendation = optimization(prediction) (20)

PPTX
Large scalecplex
PPTX
Solving Large Scale Optimization Problems using CPLEX Optimization Studio
PDF
CPLEX Optimization Studio, Modeling, Theory, Best Practices and Case Studies
PDF
Optimization Software Class Libraries 1st Edition Stefan Voß
PPTX
Synthesis of analytical methods data driven decision-making
PPTX
Addressing Uncertainty How to Model and Solve Energy Optimization Problems
PDF
Optimization Software Class Libraries 1st Edition Stefan Voß
PDF
Modeling at Scale: SigOpt at TWIMLcon 2019
PDF
Optimization: from mathematical tools to real applications
PDF
IBM - Decision Optimization
PDF
Constraint Programming - An Alternative Approach to Heuristics in Scheduling
PPTX
Machine Learning vs Decision Optimization comparison
PDF
Optimization Direct: Introduction and recent case studies
PDF
Progr dinamica de_vazut
PDF
Pragmatic Machine Learning @ ML Spain
PDF
Model-Driven Optimization: Generating Smart Mutation Operators for Multi-Obj...
PDF
Handbook of Metaheuristics International Series in Operations Research Manage...
PDF
Demystifying ML/AI
DOCX
Effective Software Effort Estimation Leveraging Machine Learning for Digital ...
PPT
or row.ppt .
Large scalecplex
Solving Large Scale Optimization Problems using CPLEX Optimization Studio
CPLEX Optimization Studio, Modeling, Theory, Best Practices and Case Studies
Optimization Software Class Libraries 1st Edition Stefan Voß
Synthesis of analytical methods data driven decision-making
Addressing Uncertainty How to Model and Solve Energy Optimization Problems
Optimization Software Class Libraries 1st Edition Stefan Voß
Modeling at Scale: SigOpt at TWIMLcon 2019
Optimization: from mathematical tools to real applications
IBM - Decision Optimization
Constraint Programming - An Alternative Approach to Heuristics in Scheduling
Machine Learning vs Decision Optimization comparison
Optimization Direct: Introduction and recent case studies
Progr dinamica de_vazut
Pragmatic Machine Learning @ ML Spain
Model-Driven Optimization: Generating Smart Mutation Operators for Multi-Obj...
Handbook of Metaheuristics International Series in Operations Research Manage...
Demystifying ML/AI
Effective Software Effort Estimation Leveraging Machine Learning for Digital ...
or row.ppt .
Ad

More from Wit Jakuczun (12)

PDF
Know your R usage workflow to handle reproducibility challenges
PDF
Large scale machine learning projects with r suite
PDF
Managing large (and small) R based solutions with R Suite
PDF
20170928 why r_r jako główna platforma do zaawansowanej analityki w enterprise
PDF
Wit jakuczun dss_conf_2017_jak_wdrazac_r_w_enterprise
PDF
Case Studies in advanced analytics with R
PPTX
Bringing the Power of LocalSolver to R: a Real-Life Case-Study
PDF
ANALYTICS WITHOUT LOSS OF GENERALITY
PDF
Showcase: on segmentation importance for marketing campaign in retail using R...
PDF
20150521 ser protecto_r_final
PDF
Rozwiązywanie problemów optymalizacyjnych (z przykładem w R)
PDF
R+H2O - idealny tandem do analityki predykcyjnej?
Know your R usage workflow to handle reproducibility challenges
Large scale machine learning projects with r suite
Managing large (and small) R based solutions with R Suite
20170928 why r_r jako główna platforma do zaawansowanej analityki w enterprise
Wit jakuczun dss_conf_2017_jak_wdrazac_r_w_enterprise
Case Studies in advanced analytics with R
Bringing the Power of LocalSolver to R: a Real-Life Case-Study
ANALYTICS WITHOUT LOSS OF GENERALITY
Showcase: on segmentation importance for marketing campaign in retail using R...
20150521 ser protecto_r_final
Rozwiązywanie problemów optymalizacyjnych (z przykładem w R)
R+H2O - idealny tandem do analityki predykcyjnej?

Recently uploaded (20)

PPTX
Business Ppt On Nestle.pptx huunnnhhgfvu
PPTX
ALIMENTARY AND BILIARY CONDITIONS 3-1.pptx
PPTX
MODULE 8 - DISASTER risk PREPAREDNESS.pptx
PPTX
IB Computer Science - Internal Assessment.pptx
PPTX
Microsoft-Fabric-Unifying-Analytics-for-the-Modern-Enterprise Solution.pptx
PDF
Fluorescence-microscope_Botany_detailed content
PPTX
Introduction to machine learning and Linear Models
PDF
22.Patil - Early prediction of Alzheimer’s disease using convolutional neural...
PDF
Lecture1 pattern recognition............
PDF
168300704-gasification-ppt.pdfhghhhsjsjhsuxush
PDF
Clinical guidelines as a resource for EBP(1).pdf
PPTX
Database Infoormation System (DBIS).pptx
PPT
Quality review (1)_presentation of this 21
PPTX
Introduction to Firewall Analytics - Interfirewall and Transfirewall.pptx
PDF
TRAFFIC-MANAGEMENT-AND-ACCIDENT-INVESTIGATION-WITH-DRIVING-PDF-FILE.pdf
PPT
ISS -ESG Data flows What is ESG and HowHow
PPTX
Introduction to Basics of Ethical Hacking and Penetration Testing -Unit No. 1...
PDF
Galatica Smart Energy Infrastructure Startup Pitch Deck
PPTX
Computer network topology notes for revision
PDF
Recruitment and Placement PPT.pdfbjfibjdfbjfobj
Business Ppt On Nestle.pptx huunnnhhgfvu
ALIMENTARY AND BILIARY CONDITIONS 3-1.pptx
MODULE 8 - DISASTER risk PREPAREDNESS.pptx
IB Computer Science - Internal Assessment.pptx
Microsoft-Fabric-Unifying-Analytics-for-the-Modern-Enterprise Solution.pptx
Fluorescence-microscope_Botany_detailed content
Introduction to machine learning and Linear Models
22.Patil - Early prediction of Alzheimer’s disease using convolutional neural...
Lecture1 pattern recognition............
168300704-gasification-ppt.pdfhghhhsjsjhsuxush
Clinical guidelines as a resource for EBP(1).pdf
Database Infoormation System (DBIS).pptx
Quality review (1)_presentation of this 21
Introduction to Firewall Analytics - Interfirewall and Transfirewall.pptx
TRAFFIC-MANAGEMENT-AND-ACCIDENT-INVESTIGATION-WITH-DRIVING-PDF-FILE.pdf
ISS -ESG Data flows What is ESG and HowHow
Introduction to Basics of Ethical Hacking and Penetration Testing -Unit No. 1...
Galatica Smart Energy Infrastructure Startup Pitch Deck
Computer network topology notes for revision
Recruitment and Placement PPT.pdfbjfibjdfbjfobj

recommendation = optimization(prediction)

  • 2. Once upon a time Wit met a Customer that needed demand forecasts to ...
  • 3. Customer: I need better demand forecasts. Me: I understand. Can I have a simple question? Customer: Yes Me: Imagine I created a demand forecasting model and provided you with desired 5M forecasts (numbers). What are you going to do with them? Customer: Well… I will take the forecasts and optimize my logistics decisions using the numbers. Me: I see. Why don’t we talk about the whole decision problem? Maybe the inefficiency is not in demand forecasts but in optimization part? Customer: Can you create a math model for such complex business problem with many constraints and exceptions? I thought it was impossible. Recommendations are calculated in “Sheet” and it is a bottleneck. Me: It is possible to build decision support system that uses mathematical optimization for your problem Customer: Great, let’s talk about the details.
  • 4. What has happened in the past? What is an optimal course of actions for the past? What can happen in the future? What is an optimal course of actions for the future? DataAnalyticsOptimization
  • 7. Forecasts/predictions are “just” a tool for better decisions. And better decisions are based on right recommendations. And right recommendations are result of optimizing business KPIs that are linked to business decisions. Optimization model deals helps to improve robustness of decision making process robust.
  • 8. Automation of the complex business process. Transition to central/global planning. Learning from best (optimal) decision.
  • 9. I was selling vehicle routing solution to a logistics company. I managed to persuade manager/owner to meet and talk with the team. After 1 hour presentation of the solution to the company I got one question How can I create an invoice in this solution? This is real story that happened to me. Similar story can be read in “Being wrong with Clarke & Wright” by Robert E.D.Woolsey
  • 11. There are three commercial highly efficient solvers for mixed integer problems. Solutions have been on the market for 25+ years…. and are still in development. Very resistant to parallelization and distributed computing techniques. Very sensitive to data. Tightly coupled with business. Only “auto” for mip.
  • 12. There is only one good library for mixed integer programming that is open-source.
  • 13. Graphics taken from SCIP solver webpage. Benchmarks from Hans Mittelmann
  • 14. Most common ML libs are open-source. Open-source is very efficient compared to commercial solutions. Easy to create distributed implementations Fairly insensitive to data. Less tightly coupled with business. Quite a few AutoML solutions that work.
  • 15. Problem is infeasible. Explanation is extremely difficult. Customers expect any-time feature. Debugging is hell :)
  • 17. Mixed-integer programming black-box hard to customize limited applicability (still wide!) Constraint programming white-box easy to customize not limited applicability Metaheuristics custom-box easy to customize not limited applicability Easy Difficult
  • 18. Mixed Integer Programming IBM CPLEX Gurobi Fico Local Solver Constraint programming Sicstus IBM CP Optimizer Commercial world Mixed Integer Programming CBC (solver) MIP (wrapper) Or-tools (wrapper) Constraint programming ECLiPSe Choco Gecode Or-tools Open-source world
  • 19. Easy I can declare model using existing solver. Fairly difficult I can solve problem with a sequence of easy problems. Very Difficult I must implement custom solver.
  • 20. Business requirements are almost impossible to be collected upfront. Performance is not satisfactory. Solution quality is not satisfactory. No solution found is not acceptable.
  • 21. Validator Solver Optimization engines Integration layer Load balancer Validator Solver Optimization engines
  • 22. Start with business process and decisions Start with small and iterate. Use real data since the first day of the project. Assume problems are infeasible or internally contradictor. Deal with must vs nice to have requirements.
  • 24. Saving up to 20% of cash management costs in Deutsche Bank
  • 25. Challenge • Factory throughput was too low • Upgrade or not to increase throughput Solution • Integrated planning and scheduling optimization model • Scenario generation to support investment decision • Tailor made optimisation model Effects • Ability to support investment decision with numbers Based on academic work by Roman Barták
  • 26. Challenge • Dynamic and unpredictable orders flow • Complex tasks Solution • Automation by optimisation • Tailor made optimisation model Effects • In progress - feasibility tests of the working solution Collect Pay Drive Deliver Drive Collect Pay Drive Deliver Drive Collect Pay Drive Deliver Drive
  • 28. Contact info ● Private: wit.jakuczun@gmail.com ● Business: ○ wit.jakuczun@fourteen33.com ○ wit.jakuczun@wlogsolutions.com