Leveraging FME to Overcome Challenges in Gas Pipeline Data Enhancement and Document Linking

Editor's Notes

• #17: Some of the challenges with linear feature reconciliation and alignment were: Pipes needed to be merged or split based on either critical values, or certain fittings such as elbows. Some pipe features would need to be created based on station (m) values. New and existing pipe features would need to be realigned to the centerline.
• #18: In this Sample Data Set: We merge where EVENTIDs are the same If splitting at a fitting, we need to identify where the EVENTIDs are different, split, and assign new EVENTIDs to the resulting pipe segments In GIS, PS were not split at certain fittings (in this example elbows). Since the PFL was constructed feature-by-feature, as we update GIS from the PFL we need to split the pipe at elbows now. This is where Advanced Attribute Value Handling comes into play: Reads through EventIDs Set condition to split pipe if current EventID is different than the previous or subsequent one in the Enable Adjacent Feature Attributes part of the Attribute Manager 1 pipe segment are now three distinct segments.
• #19: To get pipe segments realigned or placed on the centerline: A minimum and maximum station value was extracted for each segment Then the data is merged with a calibrated centerline feature class The min (BEGINSTATION) and max (ENDSTATION) values are then used to snip the centerline, creating the desired length and location of the pipe segments.
• #20: Since most of the PFL data will not need to be split or merged, we need to maintain the original data structure: The Aggregator manages all the necessary splits and merges. The list Range Extractor grabs the Min and Max station values And the List Exploder takes each element in the list and puts it back within it’s original row Resulting in all the original attributes being paired with the correct newly created EVENTID.
• #23: Each record will have 1 entry in the External Document Those records are related to the feature level through relational database tables
• #24: Online access to verifying documents:
• #25: Standard APDM Model Since we are doing this process outside of our GIS, we need to create unique identifiers for each table we insert records into.
• #26: Here is an overview of the workspace to populate the External Document table. Ed’s notes Mention APDM and how it relies on unique IDs to relate one table to another. Show how the database tables connect. Diagrams. Clarify, Clarify, Clarify. Explain how the sequential nature of the PFL necessitates splitting/merging
• #27: Since there is 1 record per ID, we merge the ID numbers with the External Document. If there is no match, then that number continues on in the data stream The ID number is used to extract metadata from the SQL database, and then the ID number is replaced in the hyperlink to create a unique link to that document.
• #28: The Schema Mapper maps all of the extracted values to their associated domain codes Each record is assigned a Unique ID and then inserted into the External Document Additionally, a list of all the newly created IDs is generated and output as query statement that can be copied and pasted into GIS for easy spot checking
• #30: Process and clean-up the EventID strings, filtering out non-EVENTIDs. Each Record in the Audit table needs to have the Attribute it is representing assigned as a domain code. Query by attribute, remove duplicate EVENTIDs, create and assign attribute for SDE, generate UUID for each record.
• #31: Check against existing records in Audit Table to prevent duplicates Insert record into Audit Table Merge on EntryID and populate 1 to many Audit Document table, establishing the relation between the External Document and the Audit Table.
• #32: Wanted to touch on the use of custom transformers. Each workspace as made into a transformer: Six separate workspaces were condensed into 6 transformers that were in turn combined into one workspace. The benefits included being able to:
• #33: The attributes and geometry of specific feature classes were updated and enhanced Critical values of features are related through associated relational database tables Verifying documents are viewable via web browser directly from GIS at the feature level