Jom first guess upgrade in min temp tool (jan 2015)
Download as DOCX, PDF0 likes204 views
The document discusses the upgrade of the 45th Weather Squadron's minimum temperature prediction tool, which was necessary due to a shift in operational requirements after the end of the space shuttle program, resulting in a colder temperature advisory threshold. A new linear regression equation for predicting minimum temperatures was developed, showing significant improvements in accuracy with a 59% reduction in RMSE and an 84% reduction in bias compared to the previous algorithm. The new 'first guess' equation uses 1000-850 mb thickness to enhance forecasting accuracy, despite the need for additional correction factors to refine results further.
Jom first guess upgrade in min temp tool (jan 2015)
- 1. 1 Optimization of the 45th Weather Squadron’s ‘First Guess’1 Minimum Temperature Prediction Equation2 3 JAMES S. BROWNLEE4 Florida Institute of Technology, Melbourne, Florida5 6 WILLIAM P. ROEDER7 45th Weather Squadron, Patrick AFB, Florida8 9 ABSTRACT10 An upgrade was made to the 45th Weather Squadron’s (45 WS) Minimum Temperature tool.11 This update was desired since the initial 45 WS minimum temperature tool contained several12 elements that had been tuned subjectively. More importantly, there was a change in 45 WS13 operational requirements for minimum temperatures advisories to significantly colder14 temperatures. The previous warmest low temperature advisory was ≤ 60F. After the end of the15 Space Shuttle Program in 2011,the warmest 45 WS temperature advisory became ≤ 35 F. Since16 the post-Space Shuttle temperature advisories represented a significantly colder regime, a re-17 optimized algorithm was desired. The 45 WS minimum temperature tool consists of a ‘first18 guess’based on the 1000-850 mb thickness and correction factors for various local19 meteorological effects. In this project, the ‘first guess’equation was re-optimized and represents20 a substantial improvement over the previous equation. This re-optimized ‘first guess’ equation is21 the first and most important step for upgrading the entire low temperature tool.22 23 1. Introduction24 25 The 45th Weather Squadron (45 WS) provides weather support for the Cape CanaveralAir Force26 Station (CCAFS), NASA’s Kennedy Space Center (KSC),and Patrick Air Force Base (PAFB) (Roeder et27 al. 2005). Most of the support provided by the 45 WS is for operations at KSC and CCAFS that includes28 space launches, preparation for space launches, personnel safety,and resource protection. One of the29 many support functions of the 45 WS are the low temperature advisories, which are listed in Table 1. The30 minimum temperature advisories are the most frequently issued warning, watch,or advisory product31 issued by the 45 WS during the winter months (Roeder et al. 2005). These minimum temperature32 advisories are critical because if the temperature gets too low, icing damage can occur to refrigerated lines33 exposed to the outdoors at various facilities (Roeder et al. 2005).34 The minimum temperature tool used by the 45 WS needed to be updated because the prior tool was35 developed to include Space Shuttle operations. During the Space Shuttle Program, the 45 WS was36
- 2. responsible for temperature advisories of ≤ 60F. When the program ended in 2011, the 45 WS low37 temperature advisories changed. The warmest low temperature advisory became ≤ 35F. As a result of38 this colder temperature regime, an update to the minimum temperature algorithm was needed.39 The current minimum temperature forecast tool in use by the 45 WS uses the 1000 mb to 850 mb40 thickness to make a ‘first guess’ minimum temperature forecast. This minimum temperature is predicted41 through the use of a linear regression equation (Roeder et al. 2005). This method of using thickness42 values for predicting both minimum and maximum temperatures has been utilized at many different43 forecasting centers (Struthwolf 1995; Massie and Rose 1997; Rose 2000), and many of these forecasting44 techniques utilize linear regression equations (Massie and Rose 1997; Rose 2000). In a similar manner to45 Rose (2000), the forecasted temperature at the 45 WS is further modified by severalcorrection factors to46 incorporate local effects. These localeffects are wind speed, cloud cover, wind direction, nocturnal47 inversion, dew point, boundary layer humidity, and mid-level humidity. After the correction factors are48 applied, the final expected minimum temperature is provided as guidance to the forecaster. This49 minimum temperature algorithm is shown in Fig. 1. This ‘first guess’ temperature prediction is the most50 critical part of the forecast,if this number is significantly in error, then the entire temperature forecast is51 wrong. A new re-optimized linear ‘first guess’ temperature prediction equation was produced by this52 research project.53 54 2. Data and methods55 56 The previous ‘first guess’ equation is a linear regression equation that uses the 1000-850 mb thickness57 to predict the ‘first guess’ minimum temperature. A linear equation has the following form:58 bmxy 159
- 3. The slope ‘m’ and intercept ‘b’ were previously optimized by linear regression by the 14th Weather60 Squadron (14 WS), the Air Force climatology center,using radiosonde and temperature data from the 4561 WS tower network with temperatures ≤ 60F. The previous operational ‘first guess’ linear regression62 equation is shown below:63 32 5 9 *15.273592.15*1979.0 8501000 mbThicknessFMinTemp 264 This previous ‘first guess’ equation was created by the 14 WS in 2004 at the request of the 45 WS. The 4565 WS then further refined this linear equation by ‘regression through the origin’, adjusting temperatures to66 Kelvin. The optimization of the new ‘first guess’ began with data provided by the 14 WS. These data67 included the following for all days where 45F was observed at any of the 45 WS weather towers,or68 surface observations at the KSC Shuttle Landing Facility (KTTS) or the CCAFS Skid Strip (KXMR); the69 1000-850 mb thickness nearest in time to the lowest temperature, and all surface observations at KTTS70 from 2-hr after sunset before the lowest temperature to 1-hr after sunrise after the lowest temperature. The71 data for these “cold events” (≤ 45F) were for Jan 1986-Apr 2014. The new ‘first guess’ was optimized72 using the data from 1986-2009, while 2010-2014 data were used for independent verification. The sample73 size for each of these partitions is listed in Table 2. Even though the warmest threshold for the 45 WS74 advisories is ≤ 35F, the threshold of ≤ 45F for cold events was chosen, based on the frequency of75 occurrence for CCAFS/KSC,to ensure a large enough sample size for the optimization. In addition, this76 ensures that most of the events are for cold front passages,which are the primary mechanism for the77 colder events at CCAFS/KSC. This also allows a margin for the forecaster’s guidance as the temperatures78 begin to approach the warmest advisory threshold.79 The new linear ‘first guess’ equation was optimized using two different methods. The first method80 involved using the ‘Solver Tool’ in EXCEL. The ‘Solver Tool’ in the EXCEL spreadsheet optimized the81 slope and intercept of the previous ‘first guess’ equation by minimizing Root Mean Square Error (RMSE)82
- 4. of the previous equation over a certain number of iterations. After the optimization was complete using83 the ‘Solver Tool’, the previous ‘first guess’ equation became the following new equation:84 32 5 9 *15.27315.228*0371.0 8501000 mbThicknessFMinTemp 385 The second version of the new ‘first guess’ equation was created using the ‘Trend Line’ linear regression86 function in EXCEL. This second equation is shown below:87 4.92*0091.0 8501000 mbThicknessFMinTemp 488 Even though they appear quite different, Equation 3 is virtually identical to Equation 4. Both of these89 equations calculate the ‘first guess’ temperature in Fahrenheit. However,the ‘Solver Tool’ equation was90 an adaptation of the previous ‘first guess’ equation that solves for the temperature in Kelvin and then91 converts it to Fahrenheit. The ‘Trend Line’ equation solves for the low temperature in Fahrenheit directly.92 Unlike Equation 3, the ‘Trend Line’ equation is an analytical solution. As expected, the 'Solver Tool'93 solution converged to the solution from the least squares linear regression as provided by the EXCEL94 'Trend Line' function. Indeed the least squares 'Tread Line' linear regression solution and the 'Solver95 Tool' solution both have the same correlation coefficient (r2 = 0.2459), and the average error between the96 two solutions is only 0.11F over the 1986-2014 data set. A t-test shows they are the same solution at the97 99.99992% significance level. Presumably, if more iterations of the 'Solver Tool' solution had been98 conducted, its solution would have become even closer to the least squares linear regression solution.99 After the optimization of the linear equation was finished, the bias and RMSE were calculated for the new100 ‘first guess’ equation. Since the linear regression in Equation 4 is statistically optimized, it is the preferred101 solution, even though Equation 3 is very similar.102 In the data there were six days when the predicted temperatures were exceptionally high. This was103 due to the large thickness values reported on each of those days. These large thickness values resulted in104 unrealistically high predicted temperatures,and as a result, the errors between the observed temperatures105 and predicted temperatures for these six events were very high; these six data points were considered106
- 5. erroneous outliers and removed from the data set. By removing these outlier points, a more realistic107 RMSE and bias could be achieved.108 Alternate regressions were also considered. The previous 45 WS minimum temperature tool found a109 slight performance improvement using a ‘regression through the origin’ with the 1000-850 mb thickness110 and the minimum temperatures in Kelvin. ‘Regression through the origin’ is justified a priori since the111 hypsometric equation would predict zero thickness at zero absolute temperature. With the new data set in112 this study, the ‘regression through the origin’ was also slightly better than the normal linear regression.113 However,the improvement was not statistically significant and so was not selected for operational use.114 In the original upgrade to the 45 WS minimum temperature tool in 2004, the ‘first guess’ based on the115 1000-850 mb thickness performed much better than the 1000-500 mb thickness ‘first guess’, which was116 replaced at that time. This made good meteorological sense since the cold events are mostly due to arctic117 outbreaks, which are much shallower than 500 mb. In this project, the possibility that the arctic layer is118 so shallow that its top is closer to 925 mb than 850 mb was also considered. Others have found the 925119 mb thickness to be useful in predicting low temperatures (Rose 2000). However,a ‘first guess’ based on120 the 1000-925 mb thickness did not perform quite as well as the 1000-850 mb thickness, even after three121 outliers were eliminated. Therefore,a 1000-925 mb ‘first guess’ was not selected. The possibility that122 the 1000-925 mb thickness might work better than the 1000-850 mb thickness for colder events was also123 considered. A 1000-925 mb ‘first guess’ for minimum temperatures ≤ 36F was found to perform slightly124 worse than the 1000-850 mb ‘first guess’. Thus this potential two-tiered ‘fist guess’ was not selected,125 where the 1000-925 mb thickness would be used at the lower temperatures and the 1000-850 mb126 thickness would be used at the warmer temperatures below 45F. Likewise, the 1000-925 mb thickness127 was considered for minimum temperatures from ≤ 45F to > 36F performed slightly worse than the128 1000-850 mb thickness. Therefore,the final result is to use the 1000-850 mb thickness ‘first guess’129 discussed previously.130
- 6. The same temperature stratification used in the 1000-925 mb regressions was also applied to the131 1000-850 mb regression. However,neither the ≤ 36F nor the ≤ 45F to > 36F regressions using the132 1000-850 mb thicknesses were statistically significantly better than the non-stratified 1000-850 mb133 regression. The plot of the colder temperature stratification suggested a 1000-850 mb regression through134 the origin with temperatures in Kelvin might be advantageous. However,this regression was not135 statistically superior to the overall 1000-850 mb thickness regression. As a result, none of these alternate136 1000-850 mb regressions were selected. Despite the severalalternate regressions considered, none137 showed a statistically significantly benefit over the 1000-850 mb thickness regression. Therefore,the final138 result is to use the 1000-850 mb thickness ‘first guess’ discussed previously and shown in Equation 4.139 140 141 3. Analysisand discussion142 143 a. Comparison of the Accuracy of the New Linear ‘First Guess’ Equation and the Previous Equation144 145 Table 3 compares the RMSE and bias for the previous and new ‘first guess’ equations for the146 development (1986-2009) and verification (2010-2014) with the six outlier 1000-850 mb thicknesses147 excluded. Table 4 compares the bias for the previous and new ‘first guess’ equations for the same time148 periods. The new ‘first guess’ equation has a RMSE of 4.83F on independent data,compared to the149 RMSE of 11.74F in the original equation, an 59% improvement. The new ‘first guess’ has a bias of150 1.31F on independent data, compared to the bias of 8.22F in the original equation, an 84%151 improvement. The bias indicates that the new ‘first guess’ still tends to over-forecast slightly. The RMSE152 is the typical expected magnitude of error for individual forecasts,regardless of polarity, i.e. ±5F. The153 bias is the average error over many forecasts,where the individual ± errors tend to cancelout each other.154
- 7. Individual errors of ~5F may not appear to be good performance, but recall that this is just for the ‘first155 guess’; the correction factors will further reduce the error for the entire tool.156 Figure 2 shows the linear correlation between the observed minimum temperatures and the observed157 1000-850 mb thickness values. From Fig. 2, it is evident that the linear relationship between these two158 parameters is rather weak. According to the correlation coefficient in Fig. 2, the linear regression line,159 which is Equation 4, only explains 25% (r2 = 0.2459) of the variance. However,the method is still useful160 as a ‘first guess’ given the previous discussion on the ‘first guess’s’ improved performance at predicting161 low temperatures. However,it also shows the need for the correction factors to refine the forecast,and the162 eventual goal of this project is to optimize all of the correction factors which are listed in Fig. 1.163 Figure 3 compares the temperature predictions made by the previous ‘first guess’ equation and the164 recorded low temperatures that occurred during each cold event from 1986-2014. Figure 4 compares the165 temperature predictions made by the new linear equation, and the observed low temperatures that166 occurred on each cold event day during the same time period. These two figures clearly show that the new167 equation’s temperature predictions are much more accurate than the previous equation’s predictions168 Figures 5 and 6 compare the low temperature prediction accuracy of the previous and new linear169 equations for all cold days which occurred during the independent verification period (2010 to 2014). It is170 interesting to see how well the new equation can handle predicting temperatures that occur during171 extreme cold air outbreaks, and a series of such outbreaks occurred at CCAFS/KSC during the first few172 months of 2010. During that year and for the rest of the selected time period, the previous linear equation173 had considerable difficulty in predicting the minimum temperatures for each day. On almost every cold174 event day, the previous equation predicted temperatures which were higher than the observed minimum175 temperatures. From both of these figures, it is quite clear that the new equation made more accurate176 temperature predictions. It should be noted that in Figs. 4 and 6, there are some events when the new177 equation slightly under predicted the observed low temperatures. Overall, though, Figs. 4 and 6 show that178 in most cases, the new equation made fairly accurate temperature predictions.179
- 8. As a further test of the new equation’s performance a z-test was performed which showed that the180 bias of the new ‘first guess’ was not statistically significantly different than zero at the 12.75%181 significance level, i.e., the new technique appears to be unbiased. However,the RMSE is statistically182 significantly different than zero at the 1.03 x 10-200 % significant level, thus the ‘first guess’ is not a perfect183 predictor of the minimum temperatures. This latter result reinforces the need for the correction factors in184 the Minimum Temperature Tool to incorporate local effects and refine the final prediction. Overall185 though, it is quite evident that the new equation does a much better job than the previous equation at186 predicting low temperatures during cold events at CCAFS/KSC.187 188 b. Reasonsforthenew equationsincreased accuracy189 190 The new ‘first guess’ equation is much better at predicting cold events at CCAFS/KSC than the191 previous operational equation. One significant reason for this is that the old equation was optimized for192 days when the low temperature was ≤ 60F. The data used to construct the previous linear ‘first guess’193 equation contained temperatures as high as 60F. Climatologically, there are many more days with194 minimum temperatures in the 60-45F range than ≤ 45F, so the previous ‘first guess’ equation may have195 been overly tuned to the warmer range of the previous low temperature advisories. Since the previous196 linear regression equation was fitted for a data set which included low temperatures that high, the197 equation is not as useful in predicting much colder temperatures; the old equation has a warm bias. This198 warm bias is responsible for most of the larger RMSE and bias values that occurred when using the old199 operational equation. As a result of this warm bias, a new ‘first guess’ equation was needed; an equation200 constructed using colder temperatures. Since this new equation has been tuned with much colder201 temperatures the equation makes temperature forecasts that better match the 45 WS’s new temperature202 advisory regime of ≤ 35F.203
- 9. 204 c. Other work205 206 As mentioned earlier, the 45 WS minimum temperature tool consists of a ‘first guess’ based on the207 1000-850 mb thickness and seven correction factors (Roeder et al. 2005). These correction factors208 consider wind speed, clouds, nocturnal inversion, dew point, on-shore/off-shore flow, low altitude209 humidity, and mid-altitude humidity. Most of these correction factors were tuned subjectively and could210 be improved by objective optimization. The wind speed correction factor was briefly examined in the211 current project reported in this paper. This correction factor appeared to be working well and no further212 work was done to optimize this correction factor to concentrate resources on optimizing the ‘first guess’,213 which had much more room for improvement and had more impact of the performance of the minimum214 temperature tool.215 The 45 WS is currently working with a student at the Florida Institute of Technology to optimize the216 cloud correction factor. The ‘first guess’ equation might show even better performance if based on the217 1000-925 mb thickness since the new colder advisories are mostly due to arctic air mass outbreaks that218 are relatively shallow. The remaining correction factors in the 45 WS minimum temperature tool should219 be objectively optimized in the future.220 221 4. Conclusions222 223 In this project, the optimization of the linear ‘first guess’ equation was performed. From the analysis,224 it was shown that the new optimized linear ‘first guess’ equation is superior to the old operational225 equation. The results showed that during all recorded major cold events that occurred in East Central226 Florida from 1986 to 2014, the new linear equation made more accurate low temperature predictions than227
- 10. the old equation. In addition to that, the new equation made low temperature forecasts that are in line with228 the new low temperature advisories. This increased accuracy is reflected in the observed reduction of both229 the RMSE and bias values. Much of the larger RMSE and bias that occurred with the old operational230 equation was due to the warm bias of that particular equation, and thus that equation is not useful with the231 new low temperature advisory criteria. In closing, it is recommended that the new linear ‘first guess’232 equation be used in place of the previous linear ‘first guess’ equation. Another option would be to use the233 new linear equation during very strong cold air outbreaks and use the previous equation during less severe234 cold air outbreaks. Either way, the results of this analysis show that the new linear ‘first guess’ equation is235 a major and much needed first step in updating the 45 WS’s low temperature tool.236 Acknowledgments. The 14th Weather Squadron, the U.S. Air Force climate center, provided the data237 CCAFS/KSC weather data used in this study.238 239 240 REFERENCES241 Massie, D. R. and M. A. Rose, 1997: Predicting Daily Maximum Temperatures Using Linear Regression and Eta242 Geopotential Thickness Forecasts. Wea. Forecasting, 12, 799–807.243 Roeder, W. P., McAleenan, M., Taylor, T. N., and T. L. Longmire, 2005: Applied Climatology In The Upgraded244 Minimum Temperature Prediction Tool For The Cape Canaveral Air Force Station and Kennedy Space Center, 15th245 Conference on Applied Climatology, 20-23 Jun 2005, 7 pp.246 Rose, M., 2000: Using 1000-925 mb Thicknesses in Forecasting Minimum Temperatures at Nashville, Tennessee.247 Technical Attachment SR/SSD 2000-25.248 Struthwolf, M. E. 1995: Forecasting Maximum Temperatures through Use of an Adjusted 850- to 700-mb249 Thickness Technique. Wea. Forecasting,10, 160–171.250 TABLES AND FIGURES251
- 11. Table 1. Cold temperature advisories provided by 45 WS.252 Temperature Threshold Duration Desired Lead-time ≤ 35F any occurrence 4 hr ≤ 32F ≥ 4 hr 16 hr ≤ 28F any occurrence (if wind > 10 kt) 16 hr 253 254 255 Table 2. Partitioning of the cold weather events (≤ 45F) at CCAFS/KSC in optimizing the 45 WS256 minimum temperature tool (6 outliers removed).257 Time Period Description Number of Events Percent of Events Jan 1986-Apr 2014 All Data 595 100% Jan 1986-Dec 2009 Development Data 476 80% Jan 2010-Apr 2014 Independent Verification 119 20% 258 Table 3. RMSE for the previous and new ‘first guess’ equations for all cold events (≤ 45F) at259 CCAFS/KSC. The bias values were calculated using Equation 4. The six outlier 1000-850 mb thicknesses260 were excluded in these calculations.261 262 Time Period RMSE of previous ‘First Guess’ Equation(F) RMSEof new ‘First Guess’ Equation (F) Jan 1986–Dec 2009 8.87 3.47 Jan 2010-Apr 2014 11.74 4.83 263 264 265 Table 4. Bias for the previous and new ‘first guess’ equations for all cold events (≤ 45F) at266 CCAFS/KSC. The bias values were calculated using Equation 4. The six outlier 1000-850 mb thicknesses267 were excluded in these calculations.268 269 Time Period Bias of previous ‘First Guess’ Equation(F) Bias of new ‘First Guess’ Equation (F) Jan 1986-Dec 2009 6.35 0.03 Jan 2010-Apr 2014 8.22 1.31 270 271 272 273
- 12. Figure 1. Schematic of the minimum temperature algorithm used by the 45 WS to make low temperature274 forecasts.275 276 277 Figure 2. Linear regression of the observed minimum temperatures and the observed thickness values.278 279 280 281
- 13. 282 Figure 3. Days on which the observed low temperature reached 45F or less from 1986 to 2014 (black283 line) along with the low temperature predicted for each day by the old operational ‘first guess’ linear284 equation (red line). The six outliers were removed here.285 286 287 288
- 14. 289 Figure 4.Days on which the observed low temperature reached 45F or less from 1986 to 2014 (black290 line) along with the low temperature predicted for each day by the new ‘first guess’ linear equation (red291 line). The six outliers were removed here.292 293
- 15. 294 Figure 5. Days on which the observed low temperature reached 45F or less from 2000 to 2014 (black295 line) along with the low temperature predicted for each day by the old operational ‘first guess’ linear296 equation (red line).297 298
- 16. 299 Figure 6.Days on which the observed low temperature reached 45F or less from 2000 to 2014 (black300 line), and the low temperature predicted for each day by the new ‘first guess’ linear equation (red line).301 302 303