Improving substation reliability & availability
5 likes1,334 views
This document discusses improving substation reliability and availability. It describes electrical events that occurred after an expansion at a refinery, highlighting issues with microprocessor relays, control power supplies, and time synchronization. The importance of substation documentation, automation, data retrieval, monitoring, and critical auxiliaries is discussed. Reliability calculations are shown to not fully account for hidden failures and varying operating conditions. Overall recommendations are made to establish best practices for documentation, configuration, monitoring, and addressing reliability more holistically.
![Reliability, Numbers and Statistics
How applies to reliability?
RELIABILITY INDICES COMPARISON
WITH AND WITHOUT HIDDEN FAILURES CONTINGENCES
RELIABILITY
INDEX
NAME
HIDDEN
FAILURES
NOT
HIDDEN FAILURES
INCLUDED
CASE 1 CASE 2NAME
INCLUDED CASE 1 CASE 2
FAILURE PROB
1.1130E-03
1.1880E-03 1.6850E-03
INCREASE 107% 151%
FAILURE FREQ [ /YR]
0.4425
0.4586 0.6034
INCREASE 104% 136%
FAILURE DURATION
[HRS/YEAR] 22.03
22.68 24.68
INCREASE 103% 112%
Note: Hidden failure probabilities: Case 1 - 10e-3, Case 2 - 10e-2](https://image.slidesharecdn.com/pcic09an-40improvingsubstationreliabilityavailability09162009-150502224946-conversion-gate02/85/Improving-substation-reliability-availability-41-320.jpg)
Improving substation reliability & availability
- 1. Improving Substation ReliabilityImproving Substation ReliabilityImproving Substation ReliabilityImproving Substation Reliability and Availabilityand Availability 2009 PCIC2009 PCIC –– 40402009 PCIC2009 PCIC –– 4040 Authors:Authors:Authors:Authors: Robert M. SpiewakRobert M. Spiewak Jerry PittmanJerry Pittman Daniel WilsonDaniel Wilson Dashiell, LLC. Valero Energy Corporation Valero Port Arthur RefiningDashiell, LLC. Valero Energy Corporation Valero Port Arthur Refining Dominik PieniazekDominik Pieniazek Floyd WeisseFloyd Weisse VI E i i LLCVI E i i LLC D hi ll LLCD hi ll LLCVI Engineering, LLC.VI Engineering, LLC. Dashiell, LLC.Dashiell, LLC. 2009 Petroleum and Chemical Industry Committee Technical Conference2009 Petroleum and Chemical Industry Committee Technical Conference Anaheim, CaliforniaAnaheim, California September 14September 14--16, 200916, 2009
- 2. Today’s PresentationToday’s Presentation Improving Substation Reliability and AvailabilityImproving Substation Reliability and Availability Background Substation Documentation Automation, Data Retrieval, Historian andAutomation, Data Retrieval, Historian and Alarm Annunciation Monitoring and OperationMonitoring and Operation Critical Auxiliaries Reliability, Numbers and Statistics Recommendations Conclusions
- 3. Background
- 4. BackgroundBackground Project OverviewProject Overview Large expansion at an existing refineryLarge expansion at an existing refinery –– Electrical Reliability ImprovementElectrical Reliability Improvement –– Additional onAdditional on--site generationsite generationgg Multiple parties involved:Multiple parties involved: –– Two Owner/OperatorsTwo Owner/OperatorsTwo Owner/OperatorsTwo Owner/Operators –– Three engineering companiesThree engineering companies Two construction companiesTwo construction companies–– Two construction companiesTwo construction companies –– One third party testing companyOne third party testing company One UtilityOne Utility–– One UtilityOne Utility
- 5. BackgroundBackground Refinery Electrical SystemRefinery Electrical System
- 6. BackgroundBackground Electrical Events After Expansion Nov- 06 South Sub Ground on 13.8KV overhead line. Two new transformers installed with 400A resistor in existing 1000 A ground system. Both transformers ground relays set to trip on Pick Up. Sep- Refinery Balanced load to generation. Trip 80MW generator, refinery imports throughSep 07 Refinery Blackout, Post Hurricane Humberto Balanced load to generation. Trip 80MW generator, refinery imports through 230kV lines L1 & L2. Utility 230kV lines L554 & L591 in service. L554 faults on blown debris, no reclosing. Seconds later L591 faults on lightning strike, no reclosing. Refinery attempted to serve its load plus local community. Generation trips on overload and blacks out. N PP2 Li it d ti t PP2 I t d B il h t it hi fNov- 07 PP2 Limited generation at PP2, Import mode. Boiler house operators switching for maintenance to replace pole in tie line 205 A/B. Over current relay picked up and tripped remaining tie line 402A/B. Training issues as refinery eliminates tie lines and TGs. Mar- PP6 & PP7 GTG 1 out for maintenance. Loss of a transformer along with GTG1, causes 08 g , overload on second transformer. DC batteries and chargers for equipment are in PP5. Lost a single battery charger, batteries discharged, Microprocessor relays lost power, When power restored, microprocessor relay initiated breaker failure tripping first transformer, second transformer tripped on overloadoverload.
- 7. BackgroundBackground Electrical Events After Expansion (con’t) Jul-08 PP7 Installed temporary battery charger at PS5. Tripped overnight, repowering DC system relays previously causing trips had been disabled, a line differential microprocessor relay initiated an unexplained trip on repowering causing an outage.outage. Jul-08 230kV private L1 Test "blue Button" trips line under a scheduled utility/refinery communication test. Logic at both ends in microprocessor relays did not match causing trip. Mar- 08 SG Sub TR1 & TR2 Microprocessor relay. Set to trip on pick up instead of TOC; M SG S b TR2 t i d i ti d t h i t i t t tMar- 08 SG Sub TR2 tripped on energization due to harmonic restraint not set Apr-08 SG Sub Sync lights stay on all the time due to unproven wiring Jun- 08 69kV Ties Overcurrent set points in microprocessor relays set incorrectly
- 9. Substation DocumentationSubstation Documentation Requirements Last items considered on new projectsLast items considered on new projects Must specify your requirements beforeMust specify your requirements before awarding project, “Toolawarding project, “Tool--Kit”Kit” WhyWhy –– Downsized staffDownsized staff –– Document control issues with advancing electronicDocument control issues with advancing electronic formatsformats St & t i l d i ti f dSt & t i l d i ti f d–– Storage & retrieval during times of needStorage & retrieval during times of need –– Tracking recall or advisory bulletinsTracking recall or advisory bulletins –– FAT & SATFAT & SAT–– FAT & SATFAT & SAT
- 10. Substation DocumentationSubstation Documentation Special Provisions Microprocessor relays have replacedMicroprocessor relays have replaced h d i d l l i ith t l i dih d i d l l i ith t l i dihardwired relay logic without logic diagramshardwired relay logic without logic diagrams –– Complex relay schemes difficult to testComplex relay schemes difficult to test D t t ( l tti )D t t ( l tti )–– Data storage (relay settings)Data storage (relay settings) Electrical Operating ProceduresElectrical Operating Procedures FAT/SAT Test records is benchmark dataFAT/SAT Test records is benchmark data BOM and O&M manualsBOM and O&M manuals
- 11. Automation, Data Retrieval, St Hi t iStorage, Historian, andand Alarm AnnunciationAlarm Annunciation
- 12. Automation, Data Retrieval, …Alarm Annunciation What…? Utilizing Installed IEDs “Data Mining”, trending, equipment status estimation System Minimum Configurations Integration Best PracticesIntegration Best Practices Use data to “Increase productivity and d t”reduce cost”
- 13. Automation, Data Retrieval, …Alarm Annunciation Typical IEDs existing within substations Protective relays and meters O li t h d ltOn-line tap changers and voltage regulators Reclosers and digital fault recordersReclosers and digital fault recorders Bushing monitoring devices Circuit breaker monitoring devicesCircuit breaker monitoring devices Dissolved gas in oil monitors Partial discharge monitorsPartial discharge monitors CCTV cameras Phone event messengersPhone event messengers Substation access system
- 14. Automation, Data Retrieval, …Alarm Annunciation Example of monitoring class III transformer
- 15. Automation, Data Retrieval, …Alarm Annunciation Mining of Data… Data mining is the process of extracting patterns from data. As more data are available and gathered,g data mining is becoming an increasingly important tool to transform these data intop information. – It is commonly used in a wide range of profilingy g p g practices, such as marketing, surveillance, fraud detection and scientific discovery, – Time for industrial application.
- 16. Automation, Data Retrieval, …Alarm Annunciation “Data mining”… Can effectively help determine: – required maintenance intervals, – replace interval-maintenance with risk-based maintenance approach, – detect equipment deterioration at early stages, – or estimate other system properties.
- 17. Automation, Data Retrieval, …Alarm Annunciation “Data mining”… When such data is properly identified, the facility has adequate time to – acquire replacement parts, – schedule required maintenance, – importantly avoid catastrophic failure.
- 18. Automation, Data Retrieval, …Alarm Annunciation Configuration and best practices Some of the best practices: – Ensure system can integrate devices from various manufacturers – Ensure system supports multiple technologies / protocols – Utilize multifunction IEDs – Choose best IED for assigned function – Make system as flexible as possible
- 19. Automation, Data Retrieval, …Alarm Annunciation Configuration and best practices Integrated system shall provide: – Increased system functionality – Increased system reliability/availability (limit/eliminate outage time) – Increased system safety – Decrease a cost for automation – Decrease a cost for protection – Adequate tools at the fingertips of the engineer
- 21. Monitoring and Operation Remote monitoring For not-networked / not-connected apparatus obtaining data about the present state of the apparatus, IED, transformer or IED that monitors this transformer is: expensive, time consuming, could be risky. F “i t lli t” (IED t k d) d tFor “intelligent” (IEDs are networked), data is stored in a database and repository and t i il blremote access is available.
- 22. Monitoring and Operation First most manifested benefit In not-networked system, to obtain configuration files, SER, or DFR from IEDs, the engineer needed to travel to the site and obtain files from the devices In networked system, the engineer cany g access historical data from a historian, verify what is stored in the IED presently and howp y historical data compares with present data
- 23. Monitoring and Operation Examples of usage IEDs, CTs, VTs problems detection, System recovery, switching unexpected loads and disturbances, Analysis of changes required to the system for changing operating conditions, F l h i i f d h i– For example: changing consistency of gas and how it will influence starting and loading of the compressor etc., Automatic change of protection parameters, – For example: different seasons, mode of operations tetc., Isolation from utility for special conditions.
- 24. Monitoring and Operation Remote operation Remote control of: – circuit breakers, – switches, – lockout relays (LOR) and – other apparatus that at present requires anpp p q operator to control it locally, – CCTV cameras, – Access doors and gates.
- 25. Monitoring and Operation FOREMOST safety benefit Performing switching operation remotely eliminates arc-flash hazard exposure to the operator / electrician by removing him from the danger zone. – Note: safety procedures shall be established first before remote operation is allowed.
- 27. Critical Auxiliaries Almost always forgotten… Substation aspects considered as auxiliary in nature: – Control Power Supply – Time Synchronization – Enabling SER and DFR recorders
- 28. Critical Auxiliaries Control and auxiliary power supply Control Power Supply should be: – Maintained – Monitored – Redundant
- 29. Critical Auxiliaries Control and auxiliary power supply DC CONTROL POWER SUPPLY BLOCK DIAGRAM, NON- REDUNDANT WITH MONITORING AC SOURCE BATTERY CB TEST SOURCE DS1 DS2 MCBFCB DISTRIBUTIONDISTRIBUTION PANEL
- 30. Critical Auxiliaries Control and auxiliary power supply DC CONTROL POWER SUPPLY BLOCK DIAGRAM, AC CHARGING REDUNDANT, DC BUS NON-REDUNDANT WITH MONITORINGWITH MONITORING
- 31. Critical Auxiliaries Control and auxiliary power supply DC CONTROL POWER SUPPLY BLOCK DIAGRAM, 2 x 100% REDUNDANT WITH MONITORING
- 32. Critical Auxiliaries SER and DFR SER and DFR default settings enable only basic functionality at best case – Not utilized to collect operation and fault information – The table in paper / Attachment C lists analog channels and digital element recommended for monitoring and data-collection applications
- 33. Critical Auxiliaries Time synchronization Cannot depend on internal clock …circuit timing “drifts” with usage time, temperature, environmental instability and other factors Th ti t dd i ifi t i f tiThe time stamp adds significant information Not synchronized information creates havoc Time Synchronization could become a law Some utilities require Time SynchronizationSome utilities require Time Synchronization measurements at the Inter-Tie point
- 34. Reliability, Numbers, Statistics and Hidd F ilHidden Failures
- 35. Reliability, Numbers,…Hidden Failures What is reliability? Definition of reliabilityDefinition of reliability –– IEEE Std 100 “EE Standard Dictionary…”IEEE Std 100 “EE Standard Dictionary…” The ability to perform a required functionThe ability to perform a required function under statedunder stated ditiditi f t t d i d f tif t t d i d f ticonditionsconditions for a stated period of time…for a stated period of time… –– IEEE “Gold Book” addsIEEE “Gold Book” adds Th tit ti li bilit l ti it li bilitThe quantitative reliability evaluation… permit reliability indexes … to be computed from knowledge of thefrom knowledge of the reliability … of the constituent components of thereliability … of the constituent components of the systemsystem –– IEEE C37.100IEEE C37.100 Reliability: a measure of the degree of certainty that the relay, or relay system, will perform correctlywill perform correctly
- 36. Reliability, Numbers,…Hidden Failures What is reliability? What are we looking for when we evaluateWhat are we looking for when we evaluate equipment / system reliability performance?equipment / system reliability performance? –– How system should perform,How system should perform, –– Highest reliability vs. CAPEX cost,Highest reliability vs. CAPEX cost, –– Equipment replacement,Equipment replacement, –– Are we considering all parameters?Are we considering all parameters? –– Wide gap between design and operationWide gap between design and operationg p g pg p g p calculationscalculations
- 37. Reliability, Numbers,…Hidden Failures How calculated? Reliability modeling principles and data we are using do not consider: – Varying parameters environmental parameters – Random, unpredictable “chaotic” disturbances – Large set samples – Other major reliability influences Parameters used in reliability analysis areParameters used in reliability analysis are based on very small sampling set of data
- 38. Reliability, Numbers,…Hidden Failures Hidden Failure “Hidden failures”“Hidden failures” –– IN PROTECTION SYSTEM / UTILITYIN PROTECTION SYSTEM / UTILITYIN PROTECTION SYSTEM / UTILITYIN PROTECTION SYSTEM / UTILITY APPROACHAPPROACH “a permanent defect that will cause a relay system to“a permanent defect that will cause a relay system top y yp y y incorrectly and inappropriately removeincorrectly and inappropriately remove a circuit elementa circuit element as a direct consequence of another switching event”as a direct consequence of another switching event” IN PROCESS INDUSTY SYSTEMIN PROCESS INDUSTY SYSTEM–– IN PROCESS INDUSTY SYSTEMIN PROCESS INDUSTY SYSTEM “a permanent defect that will cause an EI&C system to“a permanent defect that will cause an EI&C system to incorrectly and inappropriately remove a circuit elementincorrectly and inappropriately remove a circuit elementincorrectly and inappropriately remove a circuit elementincorrectly and inappropriately remove a circuit element as a direct consequence of another usually abnormalas a direct consequence of another usually abnormal event in EI&C system and/or plant process”event in EI&C system and/or plant process”
- 39. Reliability, Numbers,…Hidden Failures Causes of Hidden Failure Examples Cause examples:Cause examples: – Circuit fault – Shortage of reactive power Random operation of DC power system– Random operation of DC power system – Inadequate testing / commissioning – Operation of E&IC system in “seldom used state”Operation of E&IC system in seldom used state
- 40. Reliability, Numbers,…Hidden Failures Hidden Failure “Host” Examples Hidden failure “host”Hidden failure “host” examples: – Control schemes from previous projects – Not performing FAT or SAT to minimize cost Removing monitoring equipment– Removing monitoring equipment – Selecting vendor/contractor based on capital cost – Applying firmware without testing – Not updating firmware
- 41. Reliability, Numbers and Statistics How applies to reliability? RELIABILITY INDICES COMPARISON WITH AND WITHOUT HIDDEN FAILURES CONTINGENCES RELIABILITY INDEX NAME HIDDEN FAILURES NOT HIDDEN FAILURES INCLUDED CASE 1 CASE 2NAME INCLUDED CASE 1 CASE 2 FAILURE PROB 1.1130E-03 1.1880E-03 1.6850E-03 INCREASE 107% 151% FAILURE FREQ [ /YR] 0.4425 0.4586 0.6034 INCREASE 104% 136% FAILURE DURATION [HRS/YEAR] 22.03 22.68 24.68 INCREASE 103% 112% Note: Hidden failure probabilities: Case 1 - 10e-3, Case 2 - 10e-2
- 42. Reliability, Numbers and Statistics Reliability Data in “Gold Book” is a “generic” Small sample set Sample set collected in minimal certainSample set collected in minimal certain conditions Calculated indicesCalculated indices do not represent realityCalculated indicesCalculated indices do not represent reality… Paper is focusing on improving VAILABILITY f th EI&C t t “ li bilit i di ”of the EI&C system, not “reliability indices”
- 43. Summary
- 44. Summary Formulate practices, standards, and data documentation applicable to your organization Include in RFQ requirements special needs that are not included in industry guides,y g specification, practices… Evaluate how the project should beEvaluate how the project should be executed Assure contractual “cooperation” with utilityAssure contractual cooperation with utility for “post mortem” analysis
- 45. Summary Include provisions in POs allowing for third party to provide testing services without altering contractual responsibilities and obligations Incorporate procedures for updating andp p p g storing advisory bulletins and recall information for all apparatus and IEDspp Apply automation for communication with IEDs and file managementIEDs and file management
- 46. Summary Implement procedures and software for analysis of the collected data Apply reliable engineering principles for ACpp y g g p p and/or DC control power sources design Test periodically power system alarms andTest periodically power system alarms and monitoring Start collecting power system data applyStart collecting power system data apply time synchronization
- 47. Conclusions
- 48. Conclusions Reliability and availability of the EI&C system in process plants: – Difficult to estimate – Not considers “project” factors – Sensitive to input parameters and environmental conditions – Based on very small sample set of data
- 49. Conclusions The reliability and availability of EI&C system can be improved by focusing on time-tested: – Good engineering practices, – Appropriate and correct documentation, – Apparatus/equipment testing, commissioning and monitoring, – Applications of available technologies to performed monitoring tasks with minimum human supervision, – Perform tasks routinely to eliminate “hidden failures” in the system.
- 50. Conclusions Data collected during monitoring of the EI&C system and equipment could – “Foresee” approaching equipment failure, trigger an alarm i.e. safe time and money! – Aid at verifying, adjusting, calibrating power system models for power studies Use data to your advantage!!!Use data to your advantage!!!
- 51. Do not forget about Test Switches… TEST SWITCH IS YOUR FRIEND !!!TEST SWITCH IS YOUR FRIEND !!!
- 52. Questions?
- 53. Thank you