SlideShare a Scribd company logo
@mdaquin - ICCBR 2022 - 14/09/2022
A factorial study of neural
network learning from
differences for regression
Mathieu d’Aquin, Emmanuel Nauer, Jean Lieber
K Team, LORIA, France
k.loria.fr – @mdaquin
@mdaquin - ICCBR 2022 - 14/09/2022
@mdaquin - ICCBR 2022 - 14/09/2022
Example
2.6
0.3
1.1
0.4
1.5
@mdaquin - ICCBR 2022 - 14/09/2022
Example
2.6
0.3
1.1
0.4
1.5
@mdaquin - ICCBR 2022 - 14/09/2022
Example
2.6
0.3
1.1
0.4
1.5
@mdaquin - ICCBR 2022 - 14/09/2022
Example
2.6
0.3
1.1
0.4
1.5
@mdaquin - ICCBR 2022 - 14/09/2022
Example
2.6
0.3
1.1
0.4
1.5
1.7
@mdaquin - ICCBR 2022 - 14/09/2022
Example
2.6
0.3
1.1
0.4
1.5
1.7
real value: 2.7
@mdaquin - ICCBR 2022 - 14/09/2022
1.7
real value: 2.7
@mdaquin - ICCBR 2022 - 14/09/2022
Example
2.6
0.3
1.1
0.4
1.5
@mdaquin - ICCBR 2022 - 14/09/2022
Example
2.6
0.3
1.1
0.4
1.5
2.1
real value: 2.7
@mdaquin - ICCBR 2022 - 14/09/2022
A
2.6
0.3
1.1
0.4
1.5
B
2.8
0.1
1.2
0.1
1.5
A-B
-0.2
0.2
0.1
0.3
0.0
@mdaquin - ICCBR 2022 - 14/09/2022
A
2.6
0.3
1.1
0.4
1.5
B
2.8
0.1
1.2
0.1
1.5
A-B
-0.2
0.2
0.1
0.3
0.0
0.2
real value A: 2.7
real value B: 2.8
VA-VB: -0.1
@mdaquin - ICCBR 2022 - 14/09/2022
What happens when we do that?
How does it affect the performance of the network?
How does it affect the settings of hyperparameters?
@mdaquin - ICCBR 2022 - 14/09/2022
Case difference heuristic (CDH)
In CBR, corresponds to learning how to derive differences in
solutions from differences in problems of the case base.
Several approaches, including:
- Rule mining in the case base (adaptation knowledge
acquisition)
- Training a neural network on differences between problems
and solutions
@mdaquin - ICCBR 2022 - 14/09/2022
Case difference heuristic (CDH)
In CBR, corresponds to learning how to derive differences in
solutions from differences in problems of the case base.
Several approaches, including:
- Rule mining in the case base (adaptation knowledge
acquisition)
- Training a neural network on differences between problems
and solutions
@mdaquin - ICCBR 2022 - 14/09/2022
Case difference heuristic (CDH)
Some variants:
- Train the network with differences Pb1-Pb2 from the case
base, where Pb2 is a problem similar to Pb1. Predict
SolTarget-SolSource from PbTarget-PbSource
- Use multiple pairs Pb1-Pb2i
for training. Average the
predictions from multiple PbTarget-PbSourcei
- Train on (Pb1, Pb1-Pb2), and predict using (PbTarget,
PbTarget-PbSource)
@mdaquin - ICCBR 2022 - 14/09/2022
Some existing results
Liao and Liu, AIKE 2018: Feasibility and impact on the size of the
case base used (training data).
Leake et al, AAAI 2021: CDH with a neural network outperforms
KNN and CDH with rules.
Ye et al., ICCBR 2021: CDH in some cases can better generalise
(perform better on novel queries).
@mdaquin - ICCBR 2022 - 14/09/2022
@mdaquin - ICCBR 2022 - 14/09/2022
Does it achieve the same/better performance?
Does it require less data for training?
Does it converge faster or slower?
Should we use 1 case or multiple cases for training
and when applying the model?
@mdaquin - ICCBR 2022 - 14/09/2022
A factorial study
Training a neural network in the typical way and from
differences on multiple tasks/datasets varying several
factors to get answers to the previous questions.
@mdaquin - ICCBR 2022 - 14/09/2022
model, year, fuel type, transmission type,
fuel consumption, mileage, tax band
price sold.
frequency, angle, chord length, velocity,
displacement, thickness noise level
demographics, family situation, transport,
etc. grade obtained
date and time, duration, origin and
destination, airline, number of stops
price of seat
The datasets
Used cars:
Variants:
Toyota and Vauxhall
Airfoil:
Students:
Variants: Maths and
Portuguese
Flights:
Variants: Economy
and Business
@mdaquin - ICCBR 2022 - 14/09/2022
The factors varied
test_size: The dataset (before computing differences) is split into a
training set and a test set. The higher the test set size, the lower the amount
of data used for training.
epochs: The number of iterations over the dataset given to the neural
network for training.
ntr: For any pb1 in the training set (case base), the number of pbi used to
create the difference-based training set from (pb1-pbi, sol1-soli)
nte: The number of similar cases used in the case base (training set) from
which to predict the difference for a given target problem. Results are
averaged over all nte solutions obtained.
use context: Whether or not the original features of the problem are
included with the difference at training time and prediction time.
@mdaquin - ICCBR 2022 - 14/09/2022
The factors varied
@mdaquin - ICCBR 2022 - 14/09/2022
The factors varied
Almost 100K models trained both in a typical way, from
differences and from differences+context.
x2
x2
x2
19
19
19
19
20
40
12
10
5
5
5
5
5
10
10
10
@mdaquin - ICCBR 2022 - 14/09/2022
So, does it need less data?
R2
Flights economy
Base NN
Diff. no context
Diff. + context
@mdaquin - ICCBR 2022 - 14/09/2022
So, does it need less data?
Best results for students/maths
Best results for airfoil
@mdaquin - ICCBR 2022 - 14/09/2022
Does it converge faster or slower?
R2
Used cars/Toyota
@mdaquin - ICCBR 2022 - 14/09/2022
Does it converge faster or slower?
Best results for students/maths
Best results for airfoil
@mdaquin - ICCBR 2022 - 14/09/2022
Does it converge faster or slower?
Best results for flights/economy
Best results for flights/business
@mdaquin - ICCBR 2022 - 14/09/2022
Number of cases used
@mdaquin - ICCBR 2022 - 14/09/2022
Summary and the use of context
Minimal
settings to
reach within
0.2% of the
best result
@mdaquin - ICCBR 2022 - 14/09/2022
Conclusion
In all cases, obtained at least similar peak performance.
On average, requires more or less the same amount of data.
But easier to converge when learning from differences.
Adding context is not always beneficial, but is never significantly
detrimental.
The ideal number of cases used during training and during
retrieval depends on the specific scenario.
https://github.com/mdaquin/deltaML
@mdaquin - ICCBR 2022 - 14/09/2022
Limitations
The networks tested are similar to each other and relatively
simple.
Faster to converge does not mean less time required: Finding the
most similar cases to build the difference-based training set can
take a lot of time
→ needs a faster, more scalable KNN implementation
Not tested on classification tasks, or tasks where the differences
might rather be computed on the “representation layer”
(embeddings, CNN) than on the raw data.
@mdaquin - ICCBR 2022 - 14/09/2022
mathieu.daquin@loria.fr
@mdaquin
k.loria.fr
@mdaquin - ICCBR 2022 - 14/09/2022
Case difference heuristic (CDH)
Target problem
Target solution?
Source problem
Source solution
retrieval
adaptation
@mdaquin - ICCBR 2022 - 14/09/2022
Case difference heuristic (CDH)
Target problem
Target solution?
Source problem
Source solution
retrieval
adaptation

More Related Content

PDF
BLOCKCHAIN IMPLEMENTATION IN EDUCATIONAL SYSTEM
PDF
IRJET - A Research on Eloquent Salvation and Productive Outsourcing of Massiv...
PDF
AUTOMATED WASTE MANAGEMENT SYSTEM
PDF
Partial Object Detection in Inclined Weather Conditions
PDF
IRJET- Review of Tencent ML-Images Large-Scale Multi-Label Image Database
PDF
IRJET- Segmentation and Representation of Data Dependent Label Distribution L...
PDF
IRJET- An Data Sharing in Group Member with High Security using Symmetric Bal...
PDF
HOUSE PRICE ESTIMATION USING DATA SCIENCE AND ML
BLOCKCHAIN IMPLEMENTATION IN EDUCATIONAL SYSTEM
IRJET - A Research on Eloquent Salvation and Productive Outsourcing of Massiv...
AUTOMATED WASTE MANAGEMENT SYSTEM
Partial Object Detection in Inclined Weather Conditions
IRJET- Review of Tencent ML-Images Large-Scale Multi-Label Image Database
IRJET- Segmentation and Representation of Data Dependent Label Distribution L...
IRJET- An Data Sharing in Group Member with High Security using Symmetric Bal...
HOUSE PRICE ESTIMATION USING DATA SCIENCE AND ML

Similar to A factorial study of neural network learning from differences for regression (20)

PDF
Recuriter Recommendation System
PDF
Reinforcement Learning to Mimic Portfolio Behavior
PPTX
team12.project_ver_1_(1).pptx
PDF
IMAGE CAPTION GENERATOR USING DEEP LEARNING
PDF
Helmet Detection Using Yolo V3 Technique
PDF
IRJET- A Real Time Yolo Human Detection in Flood Affected Areas based on Vide...
PPTX
Capstone Project.pptx
PDF
Helmet Detection Based on Convolutional Neural Networks
PDF
SECURE FILE STORAGE IN THE CLOUD WITH HYBRID ENCRYPTION
PPT
pattern_evaluaiton_methods.ppt
PDF
IRJET - Conversion of Unsupervised Data to Supervised Data using Topic Mo...
PDF
IRJET- Generating 3D Models Using 3D Generative Adversarial Network
PDF
IRJET- Object Detection and Recognition using Single Shot Multi-Box Detector
PDF
Efficient Pseudo-Relevance Feedback Methods for Collaborative Filtering Recom...
PDF
Blockchain & Cloud-based, Digital Object Identifier System for better Environ...
PDF
IRJET - Coarse Grain Load Balance Algorithm for Detecting
PDF
A Literature Survey on Image Linguistic Visual Question Answering
PDF
ROAD POTHOLE DETECTION USING YOLOV4 DARKNET
PDF
CAR DAMAGE DETECTION USING DEEP LEARNING
PDF
STOCK MARKET PREDICTION USING NEURAL NETWORKS
Recuriter Recommendation System
Reinforcement Learning to Mimic Portfolio Behavior
team12.project_ver_1_(1).pptx
IMAGE CAPTION GENERATOR USING DEEP LEARNING
Helmet Detection Using Yolo V3 Technique
IRJET- A Real Time Yolo Human Detection in Flood Affected Areas based on Vide...
Capstone Project.pptx
Helmet Detection Based on Convolutional Neural Networks
SECURE FILE STORAGE IN THE CLOUD WITH HYBRID ENCRYPTION
pattern_evaluaiton_methods.ppt
IRJET - Conversion of Unsupervised Data to Supervised Data using Topic Mo...
IRJET- Generating 3D Models Using 3D Generative Adversarial Network
IRJET- Object Detection and Recognition using Single Shot Multi-Box Detector
Efficient Pseudo-Relevance Feedback Methods for Collaborative Filtering Recom...
Blockchain & Cloud-based, Digital Object Identifier System for better Environ...
IRJET - Coarse Grain Load Balance Algorithm for Detecting
A Literature Survey on Image Linguistic Visual Question Answering
ROAD POTHOLE DETECTION USING YOLOV4 DARKNET
CAR DAMAGE DETECTION USING DEEP LEARNING
STOCK MARKET PREDICTION USING NEURAL NETWORKS
Ad

More from Mathieu d'Aquin (20)

PDF
Recentrer l'intelligence artificielle sur les connaissances
PDF
Data and Knowledge as Commodities
PDF
Unsupervised learning approach for identifying sub-genres in music scores
PDF
Is knowledge engineering still relevant?
PDF
A data view of the data science process
PDF
Dealing with Open Domain Data
PDF
Web Analytics for Everyday Learning
PDF
Presentation a in ovive montpellier - 26%2 f06%2f2018 (1)
PDF
Learning Analytics: understand learning and support the learner
PDF
The AFEL Project
PDF
Assessing the Readability of Policy Documents: The Case of Terms of Use of On...
PDF
Data ethics
PDF
Data for Learning and Learning with Data
PDF
Towards an “Ethics in Design” methodology for AI research projects
PDF
AFEL: Towards Measuring Online Activities Contributions to Self-Directed Lear...
PDF
Profiling information sources and services for discovery
PDF
Analyse de données et de réseaux sociaux pour l’aide à l’apprentissage infor...
PDF
From Knowledge Bases to Knowledge Infrastructures for Intelligent Systems
PDF
Data analytics beyond data processing and how it affects Industry 4.0
PDF
Données ouvertes et traces numériques
Recentrer l'intelligence artificielle sur les connaissances
Data and Knowledge as Commodities
Unsupervised learning approach for identifying sub-genres in music scores
Is knowledge engineering still relevant?
A data view of the data science process
Dealing with Open Domain Data
Web Analytics for Everyday Learning
Presentation a in ovive montpellier - 26%2 f06%2f2018 (1)
Learning Analytics: understand learning and support the learner
The AFEL Project
Assessing the Readability of Policy Documents: The Case of Terms of Use of On...
Data ethics
Data for Learning and Learning with Data
Towards an “Ethics in Design” methodology for AI research projects
AFEL: Towards Measuring Online Activities Contributions to Self-Directed Lear...
Profiling information sources and services for discovery
Analyse de données et de réseaux sociaux pour l’aide à l’apprentissage infor...
From Knowledge Bases to Knowledge Infrastructures for Intelligent Systems
Data analytics beyond data processing and how it affects Industry 4.0
Données ouvertes et traces numériques
Ad

Recently uploaded (20)

PPTX
DISORDERS OF THE LIVER, GALLBLADDER AND PANCREASE (1).pptx
PDF
BF and FI - Blockchain, fintech and Financial Innovation Lesson 2.pdf
PDF
Fluorescence-microscope_Botany_detailed content
PPT
Quality review (1)_presentation of this 21
PPTX
1_Introduction to advance data techniques.pptx
PPTX
Moving the Public Sector (Government) to a Digital Adoption
PDF
Clinical guidelines as a resource for EBP(1).pdf
PDF
.pdf is not working space design for the following data for the following dat...
PPTX
Computer network topology notes for revision
PPTX
Introduction to Knowledge Engineering Part 1
PDF
Launch Your Data Science Career in Kochi – 2025
PPTX
IB Computer Science - Internal Assessment.pptx
PPTX
STUDY DESIGN details- Lt Col Maksud (21).pptx
PPT
Chapter 2 METAL FORMINGhhhhhhhjjjjmmmmmmmmm
PPTX
The THESIS FINAL-DEFENSE-PRESENTATION.pptx
PPTX
Business Acumen Training GuidePresentation.pptx
PPTX
Introduction to Basics of Ethical Hacking and Penetration Testing -Unit No. 1...
PPTX
oil_refinery_comprehensive_20250804084928 (1).pptx
PDF
Mega Projects Data Mega Projects Data
PPTX
ALIMENTARY AND BILIARY CONDITIONS 3-1.pptx
DISORDERS OF THE LIVER, GALLBLADDER AND PANCREASE (1).pptx
BF and FI - Blockchain, fintech and Financial Innovation Lesson 2.pdf
Fluorescence-microscope_Botany_detailed content
Quality review (1)_presentation of this 21
1_Introduction to advance data techniques.pptx
Moving the Public Sector (Government) to a Digital Adoption
Clinical guidelines as a resource for EBP(1).pdf
.pdf is not working space design for the following data for the following dat...
Computer network topology notes for revision
Introduction to Knowledge Engineering Part 1
Launch Your Data Science Career in Kochi – 2025
IB Computer Science - Internal Assessment.pptx
STUDY DESIGN details- Lt Col Maksud (21).pptx
Chapter 2 METAL FORMINGhhhhhhhjjjjmmmmmmmmm
The THESIS FINAL-DEFENSE-PRESENTATION.pptx
Business Acumen Training GuidePresentation.pptx
Introduction to Basics of Ethical Hacking and Penetration Testing -Unit No. 1...
oil_refinery_comprehensive_20250804084928 (1).pptx
Mega Projects Data Mega Projects Data
ALIMENTARY AND BILIARY CONDITIONS 3-1.pptx

A factorial study of neural network learning from differences for regression

  • 1. @mdaquin - ICCBR 2022 - 14/09/2022 A factorial study of neural network learning from differences for regression Mathieu d’Aquin, Emmanuel Nauer, Jean Lieber K Team, LORIA, France k.loria.fr – @mdaquin
  • 2. @mdaquin - ICCBR 2022 - 14/09/2022
  • 3. @mdaquin - ICCBR 2022 - 14/09/2022 Example 2.6 0.3 1.1 0.4 1.5
  • 4. @mdaquin - ICCBR 2022 - 14/09/2022 Example 2.6 0.3 1.1 0.4 1.5
  • 5. @mdaquin - ICCBR 2022 - 14/09/2022 Example 2.6 0.3 1.1 0.4 1.5
  • 6. @mdaquin - ICCBR 2022 - 14/09/2022 Example 2.6 0.3 1.1 0.4 1.5
  • 7. @mdaquin - ICCBR 2022 - 14/09/2022 Example 2.6 0.3 1.1 0.4 1.5 1.7
  • 8. @mdaquin - ICCBR 2022 - 14/09/2022 Example 2.6 0.3 1.1 0.4 1.5 1.7 real value: 2.7
  • 9. @mdaquin - ICCBR 2022 - 14/09/2022 1.7 real value: 2.7
  • 10. @mdaquin - ICCBR 2022 - 14/09/2022 Example 2.6 0.3 1.1 0.4 1.5
  • 11. @mdaquin - ICCBR 2022 - 14/09/2022 Example 2.6 0.3 1.1 0.4 1.5 2.1 real value: 2.7
  • 12. @mdaquin - ICCBR 2022 - 14/09/2022 A 2.6 0.3 1.1 0.4 1.5 B 2.8 0.1 1.2 0.1 1.5 A-B -0.2 0.2 0.1 0.3 0.0
  • 13. @mdaquin - ICCBR 2022 - 14/09/2022 A 2.6 0.3 1.1 0.4 1.5 B 2.8 0.1 1.2 0.1 1.5 A-B -0.2 0.2 0.1 0.3 0.0 0.2 real value A: 2.7 real value B: 2.8 VA-VB: -0.1
  • 14. @mdaquin - ICCBR 2022 - 14/09/2022 What happens when we do that? How does it affect the performance of the network? How does it affect the settings of hyperparameters?
  • 15. @mdaquin - ICCBR 2022 - 14/09/2022 Case difference heuristic (CDH) In CBR, corresponds to learning how to derive differences in solutions from differences in problems of the case base. Several approaches, including: - Rule mining in the case base (adaptation knowledge acquisition) - Training a neural network on differences between problems and solutions
  • 16. @mdaquin - ICCBR 2022 - 14/09/2022 Case difference heuristic (CDH) In CBR, corresponds to learning how to derive differences in solutions from differences in problems of the case base. Several approaches, including: - Rule mining in the case base (adaptation knowledge acquisition) - Training a neural network on differences between problems and solutions
  • 17. @mdaquin - ICCBR 2022 - 14/09/2022 Case difference heuristic (CDH) Some variants: - Train the network with differences Pb1-Pb2 from the case base, where Pb2 is a problem similar to Pb1. Predict SolTarget-SolSource from PbTarget-PbSource - Use multiple pairs Pb1-Pb2i for training. Average the predictions from multiple PbTarget-PbSourcei - Train on (Pb1, Pb1-Pb2), and predict using (PbTarget, PbTarget-PbSource)
  • 18. @mdaquin - ICCBR 2022 - 14/09/2022 Some existing results Liao and Liu, AIKE 2018: Feasibility and impact on the size of the case base used (training data). Leake et al, AAAI 2021: CDH with a neural network outperforms KNN and CDH with rules. Ye et al., ICCBR 2021: CDH in some cases can better generalise (perform better on novel queries).
  • 19. @mdaquin - ICCBR 2022 - 14/09/2022
  • 20. @mdaquin - ICCBR 2022 - 14/09/2022 Does it achieve the same/better performance? Does it require less data for training? Does it converge faster or slower? Should we use 1 case or multiple cases for training and when applying the model?
  • 21. @mdaquin - ICCBR 2022 - 14/09/2022 A factorial study Training a neural network in the typical way and from differences on multiple tasks/datasets varying several factors to get answers to the previous questions.
  • 22. @mdaquin - ICCBR 2022 - 14/09/2022 model, year, fuel type, transmission type, fuel consumption, mileage, tax band price sold. frequency, angle, chord length, velocity, displacement, thickness noise level demographics, family situation, transport, etc. grade obtained date and time, duration, origin and destination, airline, number of stops price of seat The datasets Used cars: Variants: Toyota and Vauxhall Airfoil: Students: Variants: Maths and Portuguese Flights: Variants: Economy and Business
  • 23. @mdaquin - ICCBR 2022 - 14/09/2022 The factors varied test_size: The dataset (before computing differences) is split into a training set and a test set. The higher the test set size, the lower the amount of data used for training. epochs: The number of iterations over the dataset given to the neural network for training. ntr: For any pb1 in the training set (case base), the number of pbi used to create the difference-based training set from (pb1-pbi, sol1-soli) nte: The number of similar cases used in the case base (training set) from which to predict the difference for a given target problem. Results are averaged over all nte solutions obtained. use context: Whether or not the original features of the problem are included with the difference at training time and prediction time.
  • 24. @mdaquin - ICCBR 2022 - 14/09/2022 The factors varied
  • 25. @mdaquin - ICCBR 2022 - 14/09/2022 The factors varied Almost 100K models trained both in a typical way, from differences and from differences+context. x2 x2 x2 19 19 19 19 20 40 12 10 5 5 5 5 5 10 10 10
  • 26. @mdaquin - ICCBR 2022 - 14/09/2022 So, does it need less data? R2 Flights economy Base NN Diff. no context Diff. + context
  • 27. @mdaquin - ICCBR 2022 - 14/09/2022 So, does it need less data? Best results for students/maths Best results for airfoil
  • 28. @mdaquin - ICCBR 2022 - 14/09/2022 Does it converge faster or slower? R2 Used cars/Toyota
  • 29. @mdaquin - ICCBR 2022 - 14/09/2022 Does it converge faster or slower? Best results for students/maths Best results for airfoil
  • 30. @mdaquin - ICCBR 2022 - 14/09/2022 Does it converge faster or slower? Best results for flights/economy Best results for flights/business
  • 31. @mdaquin - ICCBR 2022 - 14/09/2022 Number of cases used
  • 32. @mdaquin - ICCBR 2022 - 14/09/2022 Summary and the use of context Minimal settings to reach within 0.2% of the best result
  • 33. @mdaquin - ICCBR 2022 - 14/09/2022 Conclusion In all cases, obtained at least similar peak performance. On average, requires more or less the same amount of data. But easier to converge when learning from differences. Adding context is not always beneficial, but is never significantly detrimental. The ideal number of cases used during training and during retrieval depends on the specific scenario. https://github.com/mdaquin/deltaML
  • 34. @mdaquin - ICCBR 2022 - 14/09/2022 Limitations The networks tested are similar to each other and relatively simple. Faster to converge does not mean less time required: Finding the most similar cases to build the difference-based training set can take a lot of time → needs a faster, more scalable KNN implementation Not tested on classification tasks, or tasks where the differences might rather be computed on the “representation layer” (embeddings, CNN) than on the raw data.
  • 35. @mdaquin - ICCBR 2022 - 14/09/2022 mathieu.daquin@loria.fr @mdaquin k.loria.fr
  • 36. @mdaquin - ICCBR 2022 - 14/09/2022 Case difference heuristic (CDH) Target problem Target solution? Source problem Source solution retrieval adaptation
  • 37. @mdaquin - ICCBR 2022 - 14/09/2022 Case difference heuristic (CDH) Target problem Target solution? Source problem Source solution retrieval adaptation