TIMES2JuMP project status report; Learnings on the feasibility of Migrating TIMES to Julia JuMP
0 likes13 views
TIMES2JuMP project status report; Learnings on the feasibility of Migrating TIMES to Julia JuMP Dr. James Glynn, ESMA, Dr Olexandr Balyk, Dr Siddharth Krishna (Open Energy Transition)
TIMES2JuMP project status report; Learnings on the feasibility of Migrating TIMES to Julia JuMP
- 1. TIMES2JuMP project status report Learnings on the feasibility of Migrating TIMES to Julia JuMP so far… Work in Progress Dr James Glynn (ESMA), Dr Olexandr Balyk, Dr Siddharth Krishna (Open Energy Transition) 2025.06.10 | ETSAP Workshop, NARA, JAPAN www.esma.ie
- 2. Outline 01 1. Motivation - Terms of Reference 2. TIMES workflow - barriers to entry 3. Code translation considerations 3.1. Fundamental differences 3.2. Data Structures & presolve 3.3. TIMES re-write rather than Translation 4. Julia JuMP Performance 4.1. On going benchmarking 5. Costs of Current workflow 6. Dialogue with ETSAP Community (Dev & Users), OpenMOD & JuMP 6.1. Future developers & TIMES users via Julia or Python
- 3. JuMP (Julia for Mathematical Programming) is an open-source algebraic modeling language (AML) that is deeply embedded within the Julia language. It allows users to express a wide array of optimization problems—including linear programming (LP), mixed-integer programming (MIP), quadratic programming (QP), second-order cone programming (SOCP), semidefinite programming (SDP), and nonlinear programming (NLP)—using a high-level algebraic syntax that is familiar to users of other AMLs 1. Open Source, Transparent & reproducible - Good for science & innovation 2. Reduce barrier to entry and attract new talent & vitality to the TIMES community 3. JuMP is been overhalled since the initial ETSAP JuMP study (2018 - Gothenburg). 4. Julia is engineered for high performance & as Models grow in scale this is important 4.1. In memory communication 4.2. Innovative future potential for TIMES - integrated ML for Demand estimation for example. 5. Re writing TIMES from GAMS to JuMP can have profound consequences, but comes with some considerable hurdles - we face some of these hurdles irrespective of this migration. 5.1. The choice is not solely a technical performance related question. 5.2. There are uncertainties within this strategic decision Motivation & Terms of Reference of TIMES2JuMP 02
- 4. 1. GAMS Commercial Licence with CPLEX $14,000 for 1 seat 2. Veda Advanced Commercial Licence $12,000 for 1 seat 3. TIMES code is succinct and elegant within GAMS. It’s current formulation nor data handling would not directly translate well to JuMP - a naive translation would lead to poor performance in JuMP. 3.1. Formulation needs reimagining - i.e. need an “Antti” and an “Oscar” to ensure performant TIMES code functionality in the transition - both world class in their TIMES, GAMS and JuMP fields 4. GAMS preprocessing and presolve model construction is sophisciated and world class TIMES workflow, barriers to entry, speed and costs 03 XL2TIMES
- 5. 1. JuMP requires quite different code logical construction approach to match GAMS presolve performance. 2. Review Paper: Dunning, Iain, Joey Huchette, and Miles Lubin. "JuMP: A modeling language for mathematical optimization." SIAM review 59, no. 2 (2017): 295-320. Code translation considerations 04
- 6. 1. Writing Efficient Julia Code is critical 2. Effective software engineering identified as a key risk. 2.1. I.e. access to professional JuMP programmers 3. Different data structure and preprocessing approch compared to GAMS required for Julia to performance effectively 4. Prototype developing lead by Olex 4.1. Collaboration with Oscar Dowson on expert structural review 4.2. Collaboration with Antti & Will (NZ) 4.3. Discussing engaging another (less busy) JuMP expert Julia JuMP Prototype Performance 05
- 7. 1. GAMS CPLEX costs agreement from 1999 - ETSAP is considerably benefiting from the age of this agreement. 1.1. CPLEX is typically sold via resellers and not directly from IBM. 2. Current commercial CPLEX studio & CPLEX Solver licence is €100k per seat - this was my first direct commercial quote. 2.1. TOO EXPENSIVE! - this started a journey with IBM 3. CPLEX is free of cost for academic users 4. GAMS is free of cost for academic users 5. ETSAP-TIMES does not need the full CPLEX Studio Suite - we need a solver bridge. (Thank you Evangelos for sparking this realisation) 5.1. Embedded Software agreement (ESA) €204k for 80 seats or €2,500 per user for CPLEX (2.5% the usual commercial cost) 5.1.1. ETSAP can distribute ESA licences within within the TIMES code or within the community as the ExCo wishes. 5.2. Took time to find common ground and understanding of use case with IBM (EMEA - Dublin, Paris & London offices). 6. Existing CPLEX Licences via GAMS would still be backwards compatible and available via Julia packages (GAMS.jl) Costs of Current workflow & quoted future costs 06
- 8. 1. Community Acceptance from TIMES Developers 1.1. Initial Unstructured Dialogue with TIMES Developers 1.1.1. Language Agnosticism and efficiency - avoid prolonged transition challenges 1.1.2. Parallel Code Development (not liked - never ending project potential risk) 1.1.3. Backward compatibility & comprehensive testing required for comfort and validation 1.1.4. Desire for improved code maintenance and version control - more collaboration desired 1.1.5. GAMSPy should be a useful stepping stone between programming languages. 1.1.6. Needs Urgency and Long Term Commitment (Longevity of JuMP & Python? 50 years?) 2. JuMP Developers 2.1. Oscar Dowson - Lead JuMP developer 2.1.1. Presolve Data Structures 2.1.2. Code logical construction needs to be very different to GAMS (Vector data structures rather than sets & filters) 3. OpenMOD, Online LinkedIn with Modelling Community moving towards JuMP in Next Gen Models 3.1. GENX (MIT - Princeton) 3.2. SIENNA (NREL US Gov) 3.3. SpineOpt.jj (EU Comission) 3.4. GENsysMOD (OSEmosys translation from GAMS to JuMP) Dialogue with ETSAP, JuMP Developers & OpenMOD 07
- 9. 1. Most Popular technologies among StackOverflow developer survey 1.1. https://survey.stackoverflow.co/2024/technology 1.2. Python 51% of survey 1.3. Julia 1.1% of survey respondents 2. Admired Vs Desired 2.1. Python - Admired 67.6% - Desired 41.9% 2.2. Julia - Admired 61% - Desired 2.1% 3. Salary / Cost 3.1. Julia Programmer Median Salary $70,000 per year (Decreasing) 3.2. Python Programmer Median Salary $67,000 per year (Decreasing) 4. Python is the #1 most popular programming language professionally (plenty of talent in the pool to draw from for a migration) 4.1. Julia is 33rd (0.42% of developers) 4.2. GAMS is 36th (0.3% of developers) Availability of Julia (JuMP) or Python (Linopy) developers? 08
- 10. 1. Complete Literature review 2. Complete Prototyping 3. Complete benchmarks 4. Complete Costs setup assessment given CPLEX ESA costs for Group ETSAP licence (as opposed to individual institutional licences). 5. Complete broader community round table panel discussions 6. Report Back OpenMOD Conversation: https://forum.openmod.org/t/feasibility-of-migr ating-times-to-julia-jump-from-gams/5326 LinkedIn Conversation: https://www.linkedin.com/posts/esma-ie_we-ha ve-been-tasked-within-the-etsap-community- activity-7336451659528605697-263x?utm_sou Next Steps 9
- 11. THANK YOU www.esma.ie Dr James Glynn 2025.06.10 | ETSAP Workshop, NARA, JAPAN
- 12. ESMA provides academically rigorous research consulting services on timescales that matter to decision makers, using best-in-class open energy systems models, leveraging the more than 40 years of institutional knowledge in the IEA-ETSAP TIMES open-source code to enable partnerships to co-create data-driven insights for optimal investment decisions, engineering operation, strategic long-term planning, policy uncertainty, and risk mitigation analytics in clean energy futures. ESMA understands Trust and Relationships. We incorporate community, city, national and global perspectives into our energy systems models, keeping the needs of the fair social good, community of practice, city, NGO, Government, finance and industry stakeholders in mind when considering our interconnectedness and dependencies. ESMA Ltd. is a data start-up that builds innovative best-practice computer math models (LP/NLP/MIP) based on systems thinking, engineering, science, and ecological economics to create data-driven decision intelligence and logically navigate the uncertainty space to plan, invest, build, operate, and live within a sustainable, secure future energy system. WHAT WE DO www.esma.ie
- 13. www.esma.ie