Database Conditioning Presentation ESRI PUG 2015
- 1. How did we get where we are? What are some of the issues in existing data ? • Obviously bad data (e.g. typos) • Subtle and more pernicious Why has it been so difficult to wield potential power of these databases given the advances of modern technical applications? Technical approach being presented has proven to work (or at least provide a good start) in addressing this problem* *Does not require new people, more IT people; just some training and organizational commitment Geoscience Databases The 800 pound gorilla in the room of the modern digital world
- 2. Database issues in ArcGIS – Geoscience specific My experience as a geoscientist with ArcGIS and databases • My job was to teach spatial analysis and map based play assessment • The reality of the job: resolving data issues / creating data sets from graphics and digitized hand drawn illustrations. Data issue remains one of the big stumbling blocks to ArcGIS usage in geoscience Perhaps you have heard these comments from a geologist: • I have found flaws in this database • I cannot use it with confidence • I’m starting over from scratch Reality is: World is mature with respect to exploration (lots of data out there) Applications are very capable and not limited in scope or scale Why are there so many persistent issues with data quality and data conditioning My perspective from a 30 year career in the Petroleum industry
- 3. Geoscience data has (at least) 5 degrees of freedom X Y (spatial coordinates) Z (value – e.g. depth of formation) Geologic time Changing geologic classification schemes (an evolving scientific discipline) Data sets that are captured reflect the concepts and time schemas that were in place the data was captured • Litho-stratigraphy • Stratigraphic “type sections” or locales • Biostratigraphy data • Palynology data • Chrono-stratigraphy from different schema around the world • Radiometric Dates – (needs reverse translation) Description of Data Problem Technical Issues: Data Complexity Note: This discussion is about the consistency of the data such that it can be analyzed as it currently exists. Interpreted tops, unconformity surfaces, etc. still are the primary venue of a geoscientist and a separate discussion
- 4. Organizational Issues Data issues are poorly appreciated at an executive level Industry has already poured lots of money into the problem Data problems are often viewed as principally an IT issue, not requiring a cross functional solution End users (geoscientists) often create their own local data (nut piles) and become married to their own databases and database structures Sloppy usage by geoscientists and mixing of chrono-stratigraphic and lithologic terms (ongoing issue) Confounding Issues (if you don’t know where you are going… all roads lead you there.) We have met the enemy and he is us !
- 5. • Provides fundamental structure to understand sedimentary stacking patterns and groupings of sediment packages • Geologic plays and play elements are organized by temporal stratigraphy • Historic calibration of field sizes are organized by age classification • These issues apply for both conventional and unconventional exploration Bottom line Lack of attention to how data gets classified has profound influence on what constitutes hydrocarbon potential or lack thereof and (by extension) exploration strategy Why the obsession with geologic time ?
- 6. Uniform and consistent time – stratigraphic formats for all the data Interpretation consistent with that common temporal framework How do we get there from here? First step is to get the existing databases into a consistent modern time framework • What does a typical “before” data set look like • How do you deal with databases that have inconsistent attribution • How can you systematically and thoroughly reclassify your data into a consist framework. Desired State of Databases (easy to describe…. challenging to do)
- 7. Some useful reference to geologic age classifications A Consistent Temporal Framework Lots of google help out there albeit dated But……… This reference is the 2004 standard. Standards are almost continually being modified
- 8. Geologic Stage to MYBP age refinement (1937 – 2004) Databases frequently lock in the age schemas at the time when they were captured. In many cases other classifications are also included: • Litho-stratigraphy • Type sections or locales • Local formation names • Ages having no formal definition (e.g. Middle Cretaceous or Middle Oligocene) Geologic Ages are a Moving Target Geologic Time changes through time International Commission on Stratigraphy
- 10. GTS2014 Stage Table – ArcGIS format Table shown below is based on GTS 2014 Classification Scheme Coarser time intervals provided for flexibility Can be joined to your data (once conditioned) to standardize it to the International Commission on Stratigraphy GTS2014* Translation Table of International Standard Geologic Ages
- 11. How do we get these databases under control and deal with all the legacy data chaos?
- 12. Process your data (Origin Data) using the frequency tool to produce a unique list of age names in a results table more commonly called a Frequency Table Build this using the highest resolution age intervals that are in the data table Extract a unique age list from your Data Origin DataFrequency Table
- 13. Add a column in your Frequency Table to contain correct ages that are standardized: The resultant table is called a Translation Table Note: This remapping step is the most important step to get right and will require geoscience experience Add a column in your Origin Table to receive the remapped attributes
- 14. Examples of issues in Database Tables Multiple spellings / Litho-stratigraphy Litho-stratigraphy Older or alternate age classification Inconsistent spelling / multiple entries for same interval Litho-stratigraphy / type section Biostratigraphy Litho-stratigraphy Litho-stratigraphy Inconsistent spelling / multiple entries for same interval Multiple spellings / Litho-stratigraphy Inconsistent spelling / multiple entries for same interval Trailing space
- 15. Join the Translation Table to the Origin Table To make permanent export the feature data set with the join in place
- 16. Interactive graphs based on statistics from reclassified age attributes Age intervals summarized in: • Total Organic Carbon • Hydrogen Index • Vitrinite Reflectance
- 17. Initial processing and standardization will take a little addition time but there are big benefits: 1. Future raw data sets will have a process in place to deal with them more efficiently 2. All your age specific analysis will become much less daunting to do: o Hydrocarbon discovered volumes by age o Geologic parameter by age o Play assessments 3. Data cleanup product is data that is vetted and able to be loaded into more formal databases (e.g. Oracle) • Use the frequency tool in ArcGIS to thoroughly look through your existing data • Commit to a chronostratigrapic standard (and age resolution) everyone wants to work in and carefully reconcile your data to it • Build and maintain the translation tables For your local data For vendor data For the age model standard Recommendations
- 18. The requisite sunset picture denoting THE END View from Tenerife (Canary Islands) Mt Teide at sunset
- 19. Resources http://www.chronos.org http://www.stratigraphy.org/upload/bak/geowhen/index.html http://www.stratigraphy.org/ICSchart/ChronostratChart2014-02.pdf Current International Commission on Stratigraphy: Time Stratigraphic Chart Current International Commission on Stratigraphy: Time Stratigraphic Chart