M.S. Capstone Seminar

Editor's Notes

• #2: Good morning. Thank you for coming to my capstone seminar. My name is Satoshi Hirabayashi. I will talk about my research, examining the impact of raster datasets on flood and low streamflow regional regression models using custom gis applications
• #3: Here is outline of today’s talk.
• #4: First is abstract.
• #5: Here is my research title. This is quite long. So, I will break it down a little bit for you to better understand. First, I will use a lot of raster datasets in this study. Then, because there are so many data, GIS operations will be customized to easily process these datasets. Then, outputs from the GIS application will be used as inputs to the hydrologic modeling. Flood and low streamflow will be modeled. And finally, the impact of raster datasets on those models will be examined.
• #6: Introduction.
• #7: Of interest is to estimate or predict frequency and magnitude of extreme hydrologic events, flood and low streamflow. There are common statistics to estimate flood and low streamflow. The common flood statistic is 100 year flood, Q100, represents the annual maximum streamflow that is exceeded on average once every 100 years. The common low streamflow statistic is 7-day, 10-year low streamflow, Q7,10, which represents the average annual 7-day minimum streamflow that is exceeded on average in 9 out of every 10 years . Q100, annual maximum flow that occurs once every 100 years on average. Q7,10, annual 7-day minimum flow that occurs once every 10 years on average. There are two situations in the rivers, gauged or ungauged. At the gauged river sites, the streamflows are gauged and recorded. In this case, typically flood and low streamflow statistics can be estimated with a frequency analysis. On the other hand, at the ungauged river sites, streamflow records are not available. In this case, one common way to estimate flood and low streamflow is a regional regression model. This is my interest here, to predict flood and low streamflow at ungauged sites with the regional regression models.
• #8: Regional regression models is an equation between Q and Xs. Q is a response variable, in this study, Q100, or Q7,10. Xs are explanatory variables, watershed characteristics. At ungauged sites, watershed characteristics can be measured. But stream data is not available here, so of course, so you can’t calculate Q. So you have Xs here but you don’t have Q here. That means, you can’t determine the relationship here. What you can do is to choose a gauged site in a same region as the ungauged river sites. At gauged sites, watershed characteristics can be measured. And Q can be calculated from streamflow records. So you have both Q and Xs here, so using these values, you can determine the relationship between Q and Xs. Once you develop a regression model, you can apply the developed model to predict the low streamflow statistics at ungauged sites because they are At the same region.
• #9: Regional regression model is a mathematical equation. Here Q is flood or low streamflow statistics and Xs are watershed characteristics. Alpha, beta, gamma are model parameters to be estimated. By taking logarithm on both side, a log-linear model is obtained. There is one response variable and two or more explanatory variables. So, this relationship is analyzed with the multiple linear regression technique. I used ordinary least squares regression procedure to estimate model parameters, alpha, beta, gamma.
• #10: Now, I will introduce some past studies on flood and low streamflow regional regression models. Traditionally, the regional regression models to estimate flood work relatively good with the standard error of the models is 30-60 %. But, low streamflow regional regression models performed poorly, the standard error is more than 100 %. Some reasons for this poor performance have been pointed out. And here are two reasons. Traditionally, the watershed characteristics were manually derived from the paper map. So the data quality was probably insufficient. Another potential reason is that important watershed characteristics were not used in the model development. This seems to be due to the insufficient data quantity.
• #11: My advisor, Dr. Chuck Kroll took countermeasures for data quality and quantity. Two primary data involved in the regional regression model development are streamflow records and watershed characteristics. For streamflow records, they employed data from USGS’s Hydro-Climatic Data Network (HCDN), which meet certain measurement accuracy criteria. So, the quality is good. Also, they employed more than 1500 sites with averagely 44 years of record length. For the watershed characteristics, they used many raster data, 1465 data, and they derived watershed characteristics digitally. As a result, they councluded that the low streamflow regional regression models were improved, throughout the United States.
• #12: I will introduce Kroll et al’s processes. There are two major processes, watershed boundary delineation and watershed characteristics derivation. From DEM, and gauging site location, the watershed boundary of interest area can be delineated. This delineated watershed boundary is then overlaid on the raster data, such as climate data, and the spatial average is calculated. This is going to be a watershed characteristic. They didn’t use a GIS package, because there were so many data, and would take so long due to the computer power. So they developed programs in FORTRAN.
• #13: Kroll et al’s study motivated this study. They used 1 DEM with approximately 1km horizontal resolution. But now DEMs with finer resolution are available, using finer resolution DEM can enhance the quality of watershed boundary. They used two different climate datasets, one is 49 km resolution, the other is 4km, and the number of each dataset was fourteen hundred and thirteen. The soil data is 1km resolution and 12 data. If I use a finer resolution, quality of watershed characteristics can be improved. And if I use more data, quantity of watershed characteristics can be increased.
• #14: Recently, people use a GIS package to process geospatial data. GIS is very useful to derive input parameters to hydrologic models, But, what if there is a large amount of data?? In this study almost fifteen hundred data need to be processed. Can you do that manually? I don’t wanna do that, because, it is tedious, time-consuming, easily produce human errors, ….
• #15: Here is outline of today’s talk.
• #16: To develop a GIS system to efficiently and effectively derive watershed characteristics. As I said, I will use DEMs with finer horizontal resolution to delineate watershed boundaries, and examine the impact of the horizontal resolution on flood and low streamflow regional regression models. Also, I will use raster data with finer resolution, and new raster data to derive watershed characteristics. Then I will examine the impact of these new raster datasets on flood and low streamflow regional regression models. Of these watershed characteristics, I will examine the most important ones to include in flood and low streamflow regional regression models.
• #17: To develop a GIS system to efficiently and effectively derive watershed characteristics. As I said, I will use DEMs with finer horizontal resolution to delineate watershed boundaries, and examine the impact of the horizontal resolution on flood and low streamflow regional regression models. Also, I will use raster data with finer resolution, and new raster data to derive watershed characteristics. Then I will examine the impact of these new raster datasets on flood and low streamflow regional regression models. Of these watershed characteristics, I will examine the most important ones to include in flood and low streamflow regional regression models.
• #18: To develop a GIS system to efficiently and effectively derive watershed characteristics. As I said, I will use DEMs with finer horizontal resolution to delineate watershed boundaries, and examine the impact of the horizontal resolution on flood and low streamflow regional regression models. Also, I will use raster data with finer resolution, and new raster data to derive watershed characteristics. Then I will examine the impact of these new raster datasets on flood and low streamflow regional regression models. Of these watershed characteristics, I will examine the most important ones to include in flood and low streamflow regional regression models.
• #19: To develop a GIS system to efficiently and effectively derive watershed characteristics. As I said, I will use DEMs with finer horizontal resolution to delineate watershed boundaries, and examine the impact of the horizontal resolution on flood and low streamflow regional regression models. Also, I will use raster data with finer resolution, and new raster data to derive watershed characteristics. Then I will examine the impact of these new raster datasets on flood and low streamflow regional regression models. Of these watershed characteristics, I will examine the most important ones to include in flood and low streamflow regional regression models.
• #20: Here is outline of today’s talk.
• #21: This map shows the standard error of the regional regression models developed by Kroll et al. They generally improved the low streamflow model performance across the country, but, these areas are not very good. Standard error was more than hundred. The models in these regions may be improved with DEMs with finer resolution and more explanatory variables.
• #22: Flood regional regression models across the country was developed by the USGS. The performance of these flood models are relatively good. This map shows the number of explanatory variables in the models. In this region, only one variable is used in the models. For instance, in Tennessee, only drainage area is used in the model. Based on these results, the study region of this study was decided. Models in these regions may be improved with more explanatory variables.
• #23: Here, Tennessee, Kentucky, and North Carolina region, 36 USGS gauged sites are employed in this study. They are unregulated streams, which means there is no addition or withdrawal of water. They are not affected by regulation, such as reservoir releases. Streamflow data for these sites are recorded in the USGS national water information system (NWIS). They have more than 10 years of streamflow records.
• #24: This list presents the employed data. General watershed characteristics for the study region were obtained from the USGS. DEMs I used are Hydro1k DEM, 1-degree DEM and National elevation dataset (NED) DEM, horizontal resolution of these DEMs are 1 km, 85m, and 30 m, respectively. Soil data is USDA’s STATSGO. Climate is PRISM. Soil and climate data are same as Kroll et al’s, but the resolution of climate is much finer. They used 49 km, but this study used 4 km. I employed hydrology and hydrogeology data from USGS kansas water science center. As an initial investigation into methods to quantify hydrologic information from remotely sensed imageries that may aid in the prediction of extreme hydrologic events. Here I used raw data obtained by the MODIS (Moderate Resolution Imaging Spectroradiometer) sensor onboard NASA's Terra satellite.
• #25: This figure shows the overall GIS processes in this study. There are three main tasks, watershed boundary delineation, preprocess, and spatial statistics calculation. Watershed boundaries are delineated with Arc Hydro tools.
• #26: Basic idea here is that the water flows from points with higher elevation the lower elevation. So, using the elevation information in DEM and outlet point of the stream where the gauging site are located, watershed boundary for each gauging site can be delineated. The first step is fill sinks in DEM, if a cell is surrounded by higher elevation cells, the water is trapped in that cell and cannot flow. The fill sinks function modifies the elevation value to eliminate these problems. The second step is determining the flow direction. From the flow direction grid, watershed boundary is delineated. So I got three watershed boundary sets, which are denoted as WS1k, WS85, and WS30, representing the horizontal resolution of DEM used.
• #27: STATSGO toolset create raster datasets from original polygon data and attribute tables.
• #28: Developed STATSGO toolset take original STATSGO data as inputs and output the raster data to be used for the watershed characteristics derivation. Original STATSGO data I used is 1 polygon data and 2 attribute tables, soil component table and soil layer table, for each state. These data are inputted to the developed STATSGO toolset. Soil characteristics were weighted according to component percentage and layer thickness, then weighted values were joined to polygons, and a raster data representing a weighted soil characteristics. These processes were performed for each state, and then created raster for each state were merged into one raster data. Raster data representing max and minimum of these soil characteristics were created.
• #29: Raster toolset performs preprocesses for hydrologic, hydrogeologic, MODIS, PRISM raster datasets.
• #30: Developed raster toolset take original PRISM data as inputs and output the raster data to be used for the watershed characteristics derivation. Original PRISM data is a raster data in ArcInfo ASCII GRID format. To be used in watershed characteristics derivation, these raster data need to be converted to ESRI GRID format. Since we employed more than fourteen hundred data, it is very tedious to process all of these data manually. So I developed batch algorithm in the raster toolset so that there is only a few user interaction for the process. So physical paths to raster data are listed in the raster list file, and the batch process reads this file, retrieve raster data, then convert them from ASCII GRID format to the ESRI GRID format. Also, projection of these raster data need to be converted to the same projection as the watershed boundaries. Again, this process can be performed in a batch process, and minimum of these soil characteristics were created. Original PRISM data covers conterminous United States. But that is too large for this study, it would take longer if you used this data for watershed characteristics derivation, so I decided to develop a clipping function in the raster toolset. This is again performed in a batch process. As a result of those batch processes, clipped PRISM raster datasets were prepared. And they include …
• #31: Hydrology and hydrogeology data are provided in the ESRI GRID format. So There is no need to convert the format. But still projection conversion and clipping are necessary, they can be done the batch process in raster toolset. As a result, four raster datasets, that represents Base flow index, groundwater recharge, infiltration excess overland flow, saturation excess overland flow.
• #32: MODIS data are originally in hierarchical data format (HDF) format. These data are first processed in ENVI and exported to raster data in ESRI GRID format. Then, using the batch process I have talked, the projection conversion and clipping were performed. Prepared raster data are calibrated radiance on march 6, 2000, that is in the typical wet season in the study region And Sep. 6, Oct, 3, and so on in the typical dry season. My expectation is this data in wet season has some impact on flood models, and these dry season data have some impact on low streamflow models.
• #33: Table toolset creates a table where watershed characteristics will be stored.
• #34: Developed table toolset take gauging site and table list file as inputs and create initial watershed characteristic tables where the watershed characteristics for each watershed are stored.
• #35: Now all of these raster data and a table are ready. Then, paths to raster datasets are listed in a raster list file. Spatial statistics toolset computes the spatial statistics and save them in the table.
• #36: Paths to raster datasets and type of statistics to be computed are listed in a raster list file. You can specify, mean, maximum, minimum, standard deviation, and so on. Also this toolset takes watershed boundary data that will be overlaid on raster data. Then, spatial statistics are computed in the batch process and calculated statistics are stored in the table.
• #37: Now I summarize the GIS processes. These watershed boundaries were delineated from DEMs with 1km, 85m, and 30m horizontal resolution. These raster data were pre-processed with the developed toolsets. And then, the watershed boundaries were overlaid on the raster data and spatial statistics were computed and save in the watershed characteristic tables for each watershed boundary set.
• #38: In addition to the watershed characteristics explained so far, I added 4 topography variables computed from DEMs. And also, added 1 hydrology variable, surface flow index (SFI), which is computed as 100 - base flow index (BFI). Base flow is the component of streamflow that is contributed from ground water discharge. BFI is average percentage of base flow in annual streamflow. And SFI is average percentage of surface flow in annual streamflow. Also, I summarized 40 year monthly record of precipitation, max and minimum temperatures. They are 40-year record from Jan to Dec, totally 480 variables. They are divided into four seasonal windows and averaged. Then, they are fitted to a certain distribution and these percentiles were computed, resulting 36 variables. So, 480 variables of monthly precipitation were summarized to 36 variables. Max, and min temp are also summarized.
• #39: The watershed char. Table is a big table, and we categorized variables in the table into 4 classes.
• #40: Then, from A, B, C, D category, I made five combination of explanatory variables. So, performance of the model developed with only USGS traditional watershed characteristics were this. Then, add topography, soil, and climate data and build the models again, they, these models may be better than the model with only A. and so on.
• #41: For each watershed boundary set, potential explanatory variable combinations were used to develop the models. Criteria for the model construction are, # of explanatory variables, I developed model with 2, 3, and 4 explanatory variables. To determine which explanatory variables enter the model, I employed the stepwise regression procedure with 5 % of significance level to enter and stay in the models. Model parameters should be significant at 5 % significance level.
• #42: Here is outline of today’s talk.
• #43: This chart presents the standard error for all the flood regional regression models. Model with WS1k and A variable combination, A+B variable combination, and so on. Within a single watershed boundary set, models with the ALL variable combination were the best. So I use these results to compare DEM effects. Then, I will use WS1k result to compare the explanatory variable effects.
• #44: Same
• #45: Flood regional regression models developed with the ALL potential explanatory variable combination. They are almost same.
• #46: Low streamflow regional regression models developed with the ALL potential explanatory variable combination. They are almost same.
• #47: Why it happens? Now I will show you how the delineated watershed boundary are actually differ. This is WS1k polygon.
• #48: If I overlay WS85 polygon.
• #49: Then, I overlay WS30.
• #50: Now, let’s take a look at the variable combination impact on the flood model performance. Models developed with only A is the worst. Also, A+D combination produces the worst performance. A+D, is a traditional watershed characteristics and MODIS data, so MODIS data has nothing to do with flood models. Adding B, C, and B,C,D improved the models. Now let’s take a closer look at the explanatory variables in the models. Here are the model equations. For A, A+D, A+C, only two explanatory variables entered the model, satisfying the criteria. For A+B, up to four variables entered. And, note that the variables entered the model in this order. So, 2-variable model contains DA and DTMIN75, 3-variable model contain DA, DTMIN75, and FOREST, and so on. Which category are these variable belong to? A, B, C or D? Drainage area from A entered all models. Only with the SFI, surface flow index, the model is improved a lot.
• #51: This is for the low streamflow models. It shows the same trends as the flood except for A+D combination. A is the worst, other combinations are better. For flood models, A and A+D were exactly same, but here A+D combination produced the better models. Now let’s look at the model equations and category of the explanatory variables. Drainage area from A entered all models.
• #52: This is a typical hydrograph of a storm event. Sources of these streamflow are roughly divided into two flows, base flow, (or ground water discharge) and surface flow. Base flow index is a long-term average percentage of base flow in annual streamflow. Usually streamflow records are needed to compute BFI. But the new approach was taken here, the BFI for more than eight thousand gauged sites were computed. And then, they were interpolated for the conterminous US.
• #53: Here is outline of today’s talk.
• #61: Both Q100 and Q7,10 can be computed from the streamflow records using a log-Pearson type 3 distribution. First, the annual maximum flow for each year is calculated, then natural log is taken, and the histogram is made. And it is assumed that this distribution fits a log-Pearson type 3 distribution. Q100 is this point that represents the 99 th percentile of the distribution. Calculation of Q7,10 is similar. First,the annual minimum flow for 7 consecutive day for each year is calculated, then natural log is take, and plotted in the hitogram. And it is assumed that this distribution fits a log-Pearson type 3 distribution. Q7,10 is this point and represents 10 th percentile of this distribution.
• #62: In addition to the watershed characteristics explained so far, I added 4 topography variables computed from DEMs. And also, added 1 hydrology variable, surface flow index (SFI), which is computed as 100 - base flow index (BFI). Base flow is the component of streamflow that is contributed from ground water discharge. BFI is average percentage of base flow in annual streamflow. And SFI is average percentage of surface flow in annual streamflow. Also, I summarized 40 year monthly record of precipitation, max and minimum temperatures. They are 40-year record from Jan to Dec, totally 480 variables. They are divided into four seasonal windows and averaged. Then, they are fitted to a certain distribution and these percentiles were computed, resulting 36 variables. So, 480 variables of monthly precipitation were summarized to 36 variables. Max, and min temp are also summarized.
• #63: Now let’s take a look at how the DEM horizontal resolution affects the derived watershed characteristics. Here is an example. These polygon data are watershed boundaries included in WS1k, WS85, and WS30. For an example, if they are overlaid on the 30-year average January precipitation and grids within the polygon are averaged. Differences are very small.
• #64: Of interest is how the derived watershed characteristics changed based on both the horizontal resolution of the DEM and raster datasets. This chart shows the average percent difference of derived watershed characteristics between each of the three sets of watershed boundaries (WS1k, WS85, and WS30). This indicates that, when you use MODIS data, which is the finest resolution, average difference between WS1k and WS30 is the largest, but is just 3.1 %. Other cases produce the smaller difference. Therefore, there are only small differences in watershed characteristics, and that leads to the small differences in the performance of developed models.
• #65: Now, let’s take a look at the variable combination impact on the flood model performance. Models developed with only A is the worst. Also, A+D combination produces the worst performance. A+D, is a traditional watershed characteristics and MODIS data, so MODIS data has nothing to do with flood models. Adding B, C, and B,C,D improved the models. Now let’s take a closer look at the explanatory variables in the models. Here are the model equations. For A, A+D, A+C, only two explanatory variables entered the model, satisfying the criteria. For A+B, up to four variables entered. And, note that the variables entered the model in this order. So, 2-variable model contains DA and DTMIN75, 3-variable model contain DA, DTMIN75, and FOREST, and so on. Which category are these variable belong to? A, B, C or D? Drainage area from A entered all models. Only with the SFI, surface flow index, the model is improved a lot.
• #66: This is for the low streamflow models. It shows the same trends as the flood except for A+D combination. A is the worst, other combinations are better. For flood models, A and A+D were exactly same, but here A+D combination produced the better models. Now let’s look at the model equations and category of the explanatory variables. Drainage area from A entered all models.