DATA SCIENCE Lesson 5 Data Science Predictive Modeling and Modelling Methodology in Geographic Information System conference Jean-Antoine Moreau
3 likes2,155 views
This document is a lecture by Jean-Antoine Moreau on predictive modeling and geographic information systems. It discusses modeling methodologies including inductive and deductive approaches. It also describes the components of a geographic information system including databases, mapping, analysis and modeling systems. Predictive techniques like neural networks and their applications to business and geographic information systems are also covered.
DATA SCIENCE Lesson 5 Data Science Predictive Modeling and Modelling Methodology in Geographic Information System conference Jean-Antoine Moreau
- 1. Conference - Study : Jean-Antoine Moreau (Engineer - Lecturer) Predictive ModelingPredictive Modeling Modelling MethodologyModelling Methodology Geographic Information SystemGeographic Information System © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP.
- 2. Conference - Study : Jean-Antoine Moreau (Engineer - Lecturer) DATA SCIENCEDATA SCIENCE Lecture 5Lecture 5 © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP.
- 3. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 « Each problem that I solved became a rule, which served afterwards to solve other problems. » Rene Descartes
- 4. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 • Image processing; • Spatial analysis; • Cartography; • Database management system; • Statistical analysis; • Business application; Database (spatial) Geographic Information System Statistical Summary
- 5. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Database Structure of Spatial Character • Object-based spatial model; • Spatial objects; • Thematic objects.
- 6. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Spatial Object Physical Object Abstract Object
- 7. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Spatial Object Combinatorial Network Geometric Network Topological relation within 3D Objects
- 8. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 • The geographic data base is made: a geographical base, which includes spatial objects; a thematic base, which includes thematic descriptions (attributes) of these objects.
- 9. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Subsystems that make up the geographic information system:
- 10. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 The mapping system selects items from the basic geographic data, represents them cartographically on media (monitors, printers, tablets, plotters) in compatible formats.
- 11. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 The digital recording system • Converts analogic information into digital information.
- 12. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 The spatial analysis & modelling system • Extending the capabilities of traditional databases; • Consideration of the location of the observations; • Allows the spatial overlap of entity.
- 13. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 • The image processing system: – Remote image processing; – Includes statistical analysis procedures; – Allows the transformation of original image into a thematic content.
- 14. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 • The Statistical Analysis System: – Procedure of statistical analysis; – Thematic dimension; – Statistical procedures for analyzing the spatial dimension.
- 15. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Predictive Techniques • Digital representation techniques of spatial data: – approach in object mode, • vector structure; – approach raster (data) • structure or mesh.
- 16. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Raster storage
- 17. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Acquisition Creating layers Verification Integration Grouping Precision Creating for vector files Rasterization files Pearson correlation test Qualitative award Meaningful categories Predictive modeling Boolean supperposition Using the probability (location / time) Definition of the area Development of a predictive model
- 18. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Inductive approach location parameter model
- 19. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Deductive approach location hypothesis model
- 20. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Inductive modelisation • Inductive modeling consists in a Boolean superposition of all significant layers, with the logical AND operator.
- 21. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Methods • Neural networks; • The Temporal series; • Event models.
- 22. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Neural networks Modeling
- 23. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Neural networks Modeling
- 24. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Neural networks Modeling
- 25. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5
- 26. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 • The neural network model predicts from influences: – Input received information are factors of influence; – Dendrites, which transform information between neurons are chosen in mathematical functions; – The synthesis of information made is the phenomenon, to predict.
- 27. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Neural networks Function Modeling
- 28. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Neural networks Function with Data Base Modeling
- 29. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Neural networks Function with Data Base Modeling Business Information systemGeographic Information System
- 30. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Temporal series’ module is used for the prediction of fluctuating quantity. • Decomposition of a set of data by: – a trend; – cyclical items; – dependent components of past values of the series or trends of influencing factors; – random residues.
- 31. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 spatial representation
- 32. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 The fractal theory takes into account the continuity and discontinuity of developments.
- 33. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Chaos theory • System with high sensitivity to the initial conditions; – The initial conditions can be very different from forecasts.
- 34. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC A break … you can ask your questions « Change your opinions, keep to your principles; change your leaves, keep intact your roots » Victor Hugo
- 35. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Behavioral expectations • Decision to purchase of a customer, – according to its own characteristics; • Industrial component failure; • ....
- 36. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Behavioral expectations • Behavioral prediction; • Statistical theory; • Analytical models: – Analysis of variance; – Mathematical regressions; – Duration of models; • Artificial intelligence oriented real-time.
- 37. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Machine Learning • Optimization criteria; • Prediction indicator; • Search the data in relational form; • Automatic optimization; • Model “black box” depiction of the system’s model and illustrates the relationships between the inputs, states, and output.
- 38. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 systemic view
- 39. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Input Output Environment Black Box Model
- 40. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Black Box Model
- 41. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Predictive modeling - uncertainties and variations
- 42. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 • Concept of virtual measurement: – Measure calculated as opposed to added value, the real mesure; – Focus on the associated measurement uncertainty.
- 43. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 • Errors - Uncertainties: – Measurement error: the difference between the measured value of a quantity and a reference value; – Systematic error components of the measurement error, the variation is predictable. – Measurement uncertainty: Non-negative parameter characterizing the dispersion of the assigned values.
- 44. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Errors and Uncertainties in modeling • Formalization (mathematical) model: – Approximation, structural uncertainties. • Numerical implementation of the models: – Details of algorithms ; – Convergence thresholds; – Stochastic models. • Parametric uncertainties: – Accuracy of the parameters; – Model definition.
- 45. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Numerical uncertainties • Precision of calculations related to the finite; • Representation of numbers on computers; • Unit convertion; • Threshold effects; • Sensitivity to initial conditions; • Non-breeding calculations; • Stochastic method.
- 46. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Numerical uncertainties • Threshold effects; • Stop criterion of an algorithm; • Convergence threshold effect.
- 47. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Digital uncertainty • Non-repeatability; • The response order of CPU (Central Processing Unit) : – the outcome; – on the order of operations.
- 48. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 • Bayesian error estimation in density functional theory Error estimates for Density Functional Theory calculations. The approach which is based on ideas from Bayesian statistics involves creating an ensemble of exchange-correlation functionals by comparing with an experimental database.
- 49. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 The major uncertainty is the failure of a model to represent the experimental data.
- 50. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Predictive modeling Simplified process
- 51. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 • Spatial structure of the data – Geographical database, – The predictive models; – Referenced data.
- 52. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Information Modeling
- 53. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Define the Database Use of a database management system Creating a validated and secure repository
- 54. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Define the method of consultation Consultation tools
- 55. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 example: Method of decision trees. • Statistical method applicable to area discrimination and regression. • Statistical discrimination consists from a sample of data representation of considered phenomena, to build a model to predict the class of a new observation. • The class of an observation (a qualitative variable) is this the binary variable indicating the presence or absence (true or false proposition, boolean 1 or 0).
- 56. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 example: Method of decision trees.
- 57. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 example: Method of decision trees.
- 58. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 The generalization ability of the model is the quality of the model on new data.
- 59. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Construction of the decision tree • The first phase: – Récurssif binary partitioning algorithm; – The recursion stops when the subset is homogeneous. • The second phase: – Removal of the least significant statistically branches of the decision tree; – Breiman method; – Ranking according to the resistance to the reduction of the error criterion.
- 60. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 • The performance of decision trees should be assessed in their ability to predict: – presence; – no; – true; – false; – yes; – no; – ... geolocated
- 61. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC a break, to ask your questions. « There is nothing either good or bad but thinking makes it so. » William Shakespeare
- 62. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 • Correlation and regression: – The correlation is a statistical characteristic of the existence or absence of a connection in two samples X and Y, values taken on a group of subject. – The coefficient coeficient quantifies this relationship by a sign of the correlation (positive and negative), and by the strength of this correlation. – The degree of correlation is measured from 0 to 1. • 0: no correlation; • 1: perfect correlation.
- 63. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 • Calculating the variance between samples: • Calculation of the covariance: • The correlation index: Coefficient calculation
- 64. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 The correlation • The linear correlation was studied by: – a scattergram; – the linear correlation coefficient, which is a measure of the direction and intensity of the linear association.
- 65. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Strong linear relationship Positive or negative correlation almost perfect.
- 66. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Pearson test This is to test whether the data are consistent with a probability law defined a priori. We begin with the case of the multinomial distribution, for which the probability distribution is a finite number of values. In the general case, it is reduced to the multinomial distribution by splitting the data into classes.
- 67. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Spearman Test • Spearman's coefficient, like any correlation calculation, is appropriate for both continuous and discrete variables, including ordinal variables. • The Spearman correlation coefficient is defined as the Pearson correlation coefficient between the ranked variables.For a sample of size n, the n raw scores are converted to ranks , and ρ is computed from:
- 68. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Raster and Vector Structure • Raster : – Conceptual model = continuous surface; • Pixel; • Vector : – Conceptual model = discrete objects; • Points, ligne, polygone, bodies.
- 69. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 • Vector structure; • Raster structure; • Storage in the computer. • Vector data; • Storage of vector data; • Topology; • Generalization methods.
- 70. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 • Longitude; • Latitude; • Elevation; • Roads; • Building; • Plants; • Lakes; • … Spatial Vector in a Geographical Information system They have position in X, Y , Z
- 71. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 NON Spatial data in a GGeographical IInformation SSystem (GIEGIE) Data that describe the properties of the object. Examples: Tree layer > hight, perimeter, age; River layer > depth; House layer > owner, no, individual.
- 72. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Non-Spatial data in a GGeographical IInformation SSystem (GIEGIE) • Attribute forms : • Tables; • Text; • Documents; • Pictures; • Video; • Sound; • …
- 73. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 • Attribute data – Each object is represented as a row in the table; – The row in the table are linked to the geographical objects throughthe ID-number.
- 74. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 example Masts for Mobile telecommunication • Position for possible locations : Vector model points give a best precision; • Topography expressed as elevation in meters above sea level : – Raster model (continuous surface); • Building including their heights in meter: – Vector model, point or polygons; – The height presented in the attribute table.
- 75. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 • Simple exercice : About a molecule Using a real-time combinatorial model
- 76. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 • File extensions (examples): – The geometrical objects: • . shp – The link between the geometrical objects and the attribute data: : • . shx – The attribute data : • . dbf – The spatial reference (projected coordinate system) : • . prj
- 77. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 The file The contents of files • . shp - coordinate file ; • . shx - index list ; • . prj - Text string in a specified order ; • . dbf - rows and columns.
- 78. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Point • No dimension ; • No extension in space ; • Each point has a unique ID-value ; • One coordinate pair (3 coordinates, in 3D).
- 79. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Lines • 1 dimension ; • Each line segment = a start node + an end node; • Node in between = vertices (breakpoints); • Each line segment has an ID-number.
- 80. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Polygons – simple – 2 dimensional; – Lines that enclose themselves > surface is homogeneous; – Each polygon > an ID-number.
- 81. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Polygons – simple model Simple model = spaghetti model • Storage problem – Boundaries are stored twice; • Searches of the neighbor – Entire list of coordinates is searched until an identical pair is found.
- 82. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Polygone - simple model + a coordinate liste – List of coordinates; – List of the points; – Attribute table.
- 83. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 • How could objects be related to each other ?
- 84. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Topological line structure • Analysis of flows / transport; • Connectivity and direction.
- 85. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Topological Line Network Example • Required information about attributes (time, speed, …); • How they are connected; • Node: – start, end, intersections in separate table.
- 86. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Topological Line Wetwork The Graph
- 87. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Examples Models
- 88. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC A break … you can ask your questions « Start by doing what's necessary; then do what's possible; and suddenly you are doing the impossible. » Francis of Assisi
- 89. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Topological polygon structure • Shares lines stored once; • Different tables for describing different aspects of the topology.
- 90. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 File structure for a geometric object Two dimensions
- 91. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 File Structure for a geometric object three-dimensional
- 92. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Generalization • Set of data volume; • To synchronize data based on different scales (level of detail); • Define a good requirements; • To cut down on processing.
- 93. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Types of generalization • Simplification – reduce the number of coordinate pairs; • Smoothing – apply an equation to describe the shape; • Aggregation – aggregate several objects; • Merge – merge smaller objects to larger; • Collapse –change data type.
- 94. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Line generalization : polynominal equation Polynomial equation P(x) = B + B X + B X + …. + B X K0 1 n 2 2 n 0B , B , B … , B coefficients that decide the shape of the curve 21 k K = the degree of the polynomial
- 95. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Splines Cubic - polynominal P(x) = B + B X + B X + …. + B X K0 1 n3 2 2 n • 2 points, then a new cubic polynomial must be added; • Rules are added to keep a « smooth » passage between the polynomial.
- 96. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Vectors advantages • Small amount of data – storage efficiency; • Easy to update; • Easy to combine with attributed data; • Geometric precision; • Management of topological relation.
- 97. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Vectors disadvantages • Data continuity is not represented effectively; • Spatial analysis and filtering within polygon is impossible.
- 98. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Repetition raster / vector • Raster – Continuous varying surface.
- 99. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Examples of raster images • Photos; • Photogrammetry and Remote Sensing; • Scanned Images of Maps; • Modeling - terrain; • Land cover analysis for continuous surfaces.
- 100. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 0,0 Raster in co-ordinate space Position (i,j) Column: i Row: j n X axis Y axis
- 101. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Spatial Resolution • Spatial Resolution = minimum distance over which change is recorded; • Smallest units are the cells or pixels; • High resolution = small cell dimensions; • High resolution = lots of detail.
- 102. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 • Every cell has a positional coordinates; • These may be referenced in terms of cell positions (i,j) or in coordinate space (x,y); • Therefore we have (x,y,z) or (i,j,z).
- 103. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Representation of the reality Example • No cell can be empty, all cells must have a value; • Numbers represent an object; • The numbers present an ID-number in one complex attribute data (table).
- 104. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Raster representation example
- 105. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Raster representation example
- 106. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Raster representation example Vector data representation Raster data representation
- 107. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Raster representation example
- 108. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Raster Advantages / Disadvantages
- 109. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Raster advantages • Simple data structure; • Easy to generate; • Easy workflow; • Easy analysis.
- 110. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Raster disadvantages • Tends to generate huge files, depending on resolution; • Cell arrangement does not respect natural borders; • Low in precision.
- 111. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC a break … you can ask your questions « Continuous effort - not strength or intelligence – is the key to unlocking our potential. » Winston Churchill
- 112. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Storage
- 113. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Data in the computer • Binary system used for storing and computation; • Computer store data as arrays of switches;, that are either ‘on’ (1) or ‘off’ (0) called bits; • 8 bits yields one byte; • Several bytes strung together is call a word.
- 114. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Binary system = base 2 • base 10 = the decimal system; • base 2 = the binary system.
- 115. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Value of an array of 3 switches on the binary system
- 116. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 In the computer • Binary system • Array of 8 switches (0,1) = 1 byte; • 8 bits – 1 bits ;
- 117. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Larger number in the computer • 2 bytes =16 bits
- 118. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Larger and negative numbers • One bit has to be set aside to represent the sign • The range of numbers represented in decimal form : -32 768 to +32 768
- 119. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Decimal number
- 120. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Decimal number • Lowest decimal number that can be stored = 1.2 x 10^38 ; • Highest decimal number that can be stored => 3.4 x 10^38 ; • Precision is limited to 7 significant digits ; • Single precision real = 4 bytes <the decimal point can ‘float’ • Double precision real = 8 bytes.
- 121. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 INTEGER VALUE • Byte 1 byte; • Integer 2 bytes; • Double integer 4 bytes.
- 122. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 DECIMAL NUMBER – FLOAT • Single precision (7 significant number(s) 4 bytes) ; • Double precision (15 significant number(s) 8 bytes) .
- 123. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 • ASCII 1 byte / character ; • ANSI 1 byte / character.
- 124. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 • Grids can also store continuous values like elevation
- 125. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Grids Example • Shuttle radar topography mission (SRTM) • Digital Elevation Models (DEM)
- 126. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Grids Example
- 127. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Grids Example
- 128. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 • Mathematical operations can be applied to two raster and the result is in the output raster. • Functions include + , - , / , * , Log , Exp , Sin , Cos , Sqrt .
- 129. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Triangulated Irregular Network a 3 Data structure for representing surface rd
- 130. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Types of Data Representation Vector data representation Raster data representation Triangulated data representation
- 131. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Topology • Feature of the need to be connected using specific rules.
- 132. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Planar topology • Planar topology specifies the topological rules for features; • Topological properties include adjacency, connectivity, inside / outside, and direction.
- 133. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Planar topology
- 134. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Topology
- 135. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Topology
- 136. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Topological Relationships • Vector ans TIN (Triangulated Irregular Network) data can have topological structure; • Raster and image can not have a topology structure.
- 137. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Questions for each project • What data layers ? • Vector, raster, TIN, image ? • Topological structure : – Network connectivity ? – Planar topology ? • Attributes ? • Minimum required accuracy?
- 138. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC a break … you can ask your questions « A leader is best when people barely know he exists, when his work is done, his aim fulfilled, they will say: we did it ourselves. » Lao Tzu
- 139. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 SPATIAL OBJECTSSPATIAL OBJECTS ANDAND DATABASE MODELSDATABASE MODELS
- 140. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 • The objects in a spatial database are represented by real world entities with associated attributes; • The power of à GIS (geographical Information System) comes from its ability to look at data in their geographical context, and examine them between entities.
- 141. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Point data • The simplest type of spatial object; • Choice of entities which will be represented a point depends on the scale of the map study; • The coordinates of each point can be stored as two additional attributes; • Information on a set of points can be viewed as an extended attribute table; • Each point is independent of every other point, represented as a separate row in the database model.
- 142. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 LINE DATA
- 143. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 LINE DATA Network Entities • Infrastructures networks, • Transportation network, • Utility network.
- 144. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 LINE DATA Network characteristic • Network is a composed of : – Nodes – junction, – Links – chains in the data base model; – Diagramm; – Valency of a node is the number of links at the node.
- 145. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 • Example of link attributes • Direction of traffic; • Volume of traffic; • Length; • Number of lanes; • Time to travel along link; • Diameter of pipe; • Direction of pipe; • Direction of flow; • Voltage of electrical transmission line; • …
- 146. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 AREA DATA – Boundaries may be defined by natural phenomena; Boundary are defined by the phenomenon itself.
- 147. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 AREA Coverage • Entities are isolated areas; • Any place is within exactly one entity; • The database must be able to deal them correctly.
- 148. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Data structures for representative surface Digital model Result is a matrix of point
- 149. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Data structures for representative surface • Digital Elevation Model; • Based on sampling the elevation surface with a regular intervals; Result is a matrix of points.
- 150. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Using the technique of a spatial interpolation
- 151. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Topological Data Model Example 3D Urban Entity Node-Relation Structure
- 152. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Node – Relation Structure (NRS) • It represents : – Topological relationships; – Adjacency and connectivity relationship.
- 153. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Node – Relation Structure (NRS)
- 154. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Node – Relation Structure (NRS)
- 155. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Node – Relation Structure (NRS)
- 156. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Node – Relation Structure (NRS)
- 157. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC DATA SCIENCE – L5DATA SCIENCE – L5 Node – Relation Structure (NRS)
- 158. © Jean-Antoine Moreau copying and reproduction prohibited Managing my copyright ADAGP. Contact Mobile Website MOOC a break … you can ask your questions « Coming together is a beginning; keeping together is progress; working together is success. » Henry Ford