Government 1008: Introduction to Geographic Information Systems Objectives: Creating geodatabases Joining attribute tables Attribute and location based queries Spatial joins Creating spatial and attribute data Estimated Time: 4 5 hours LAB EXERCISE 4: Got Database? 1.0 Problem: Using database techniques to query and reason with your data In this lab exercise you will investigate the relationships between crime, density, and housing value and speculate about how spills or crashes may play a role in describing them. You will learn to combine multiple data tables and layers from various sources in making queries and generating statistics. Recall what you found in previous lab exercises about Environmental justice locations in Massachusetts and use that knowledge in generating your spatial questions. Data used in this part of the lab exercise Data in the massgis folder 1.0 Geodatabases To manage your own spatial database, you can create a geodatabase 1. A geodatabase is a collection of geographic datasets of various formats. In ArcGIS a geodatabase is a collection of geographic datasets of various types held in a common file system folder, a Microsoft Access database, or a multiuser relational database (such as Oracle, Microsoft SQL Server, or IBM DB2). The geodatabase can contain three primary dataset types: Feature classes Raster datasets Tables A file geodatabase (*.gdb) offers all the functionality of a geodatabase, but it is stored in a file structure, whereas, with a personal geodatabase (*.mdb) your data will be stored in a Microsoft Access readable database (which you can access from outside of ArcGIS in Microsoft Access for non spatial queries). 1. Start up ArcCatalog. 2. Select from the main menu in ArcCatalog: File, then New, then File Geodatabase (not Personal Geodatabase) or right click in the TOC to select New and then File Geodatabase. Rename this new geodatabase to be lab4. Import spatial data files from the massgis folder to this geodatabase as follows: 1 http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#/what_is_a_geodatabase/003n00000001000000/
3. Right click against the geodatabase lab4.gdb in your folder to get a menu as shown in the screen on the right where you should select Import and Feature Class (multiple). 4. You will get a new window. Import the shapefiles: census2000blockgroups_poly (a shapefile showing census block groups in Massachusetts) and TownsMA from your lab2 (or census2000towns_poly in massgis all the towns in Massachusetts). 5. Right click against the geodatabase lab4.gdb in your folder to get a menu where you should select Import and then Table (Multiple) to get a new window. Import the tables MassIncomeHousingValue_Census1990blockgroups.xls (housing values from Census 1990), housing_value.dbf (housing values from Census 2000) from the massgis folder, ucr_90_s_v.dbf and ucr_00_s_v.dbf from the folder spoliver_sf_crime_towns in massgis. 6. You can rename the tables and shapefiles within the geodatabase in ArcCatalog The census2000blockgroups_poly shapefile is documented at http://www.mass.gov/mgis/cen2000_blockgroups.htm and http://www.mass.gov/mgis/bg_fields.htm explains the variables in the file housing_value.dbf. All the crime data for MA are in the folder spoliver_sf_crime_towns in massgis and the two tables you imported show crime statistics by town for 1990 and 2000. More on the spatial data files can be found at: http://www.mass.gov/mgis/crime_statistics.htm and on the database at http://maps.massgis.state.ma.us/spoliver/. 7. Open ArcMap and add (or drag from ArcCatalog) the shapefile for Towns and all the tables in your lab4 geodatabase. 1.1 Adding Fields to Tables In order to add a field to a table, three conditions must be met: You must have write access to the data You should not be currently editing the data in ArcMap or ArcCatalog No other users or applications should be accessing the data. If any of these conditions are not met, the Add Field option will be grayed out or it will not work. To be able to edit a spatial data file, therefore, you must copy it to your own n: folder. 2
You are going to edit the attribute table of the Towns shapefile to include population density variables: 8. Open the attribute table for the TownsMA or census2000towns_poly shapefile (right click in table of contents TOC and then select Open Attribute Table). 9. In the table, click on the Table Options icon at the top right corner of the table window. Select Add Fields from the menu that drops down. 10. In the Add field dialog, enter DENSITY90 into the Name space, Double into the Type space, set the Alias and Default. Click OK. You can see that the new field appears in the last column (on the extreme right) of the table. 11. Also add fields called DENSITY00 and DENSITYCH. FIELD TYPES Text: textual data, letters and/or numbers Short Integer: integer numbers values between +/-32,767 Long Integer: integer numbers values between +/-2.16 billion Float: real numbers (positive numbers) to a decimal precision of 7 places (7 significant digits) (this field type is also referred to as Single) Double: real numbers (positive numbers) to a decimal precision of 15 places (15 significant digits) Date: stores dates (and or times); in geodatabases, this field can store the date and the time; in shapefiles and coverages it can only store the date portion and date fields will display differently in different applications BLOB: stores references to Binary Large OBjects like video, audio, word files, or other binary data PRECISION: For numeric data, precision is the total number of digits allowed (Left and Right of decimal point) SCALE: Number of allowable digits to the Right of the decimal point and only for Float and Double field types You should have three new fields, DENSITY00, DENSITY90 and DENSITYCH that have no data as yet within the columns. Note: To see variable characteristics for the other variables in the Towns attribute table, right click on shapefile in the TOC. Then select Properties and click the Fields tab. 1.2 Calculating new field values Now fill the new fields with data: 12. In ArcMap, go to main menu Customize and then Toolbars and check Editor followed by the Close button. 13. On the editor toolbar, select the Editor pull down menu and click on Start Editing. 14. During the editing of a table it is common to select a single or a number of records by mistake. They instantly become highlighted. To clear this selection, click on the Table Options menu and select Clear Selection. 15. Left click on the DENSITY90 field (column) heading in the table to highlight it. 16. Then right click on column heading. Scroll down the menu that appears and select Field Calculator to get a new window. 3
17. In the Field Calculator window enter the formula for calculating density: double click [POP1990] on the Fields, click or type / and then double click [SQ_MI] on the Fields. You can also type all the calculations you need but you need to be careful to type exactly the names of the fields including brackets. 18. Click on OK. The values for DENSITY90 (in population per sq mile) will now appear on the table. 19. Repeat the two previous steps for DENSITY00 (change the population variable to POP2000) and for DENSITYCH which is the difference between DENSITY00 and DENSITY90). 20. Once you have created the three new variables save your work by clicking on Editor and then Save Edits. 21. You can also perform statistical operations on the field. Highlight the column DENSITY90 in the table, right click on it and select Statistics. This brings up the statistics window that provides basic statistics for the field. 1.3 Attribute Queries We can now query the new fields that have been created. We want to select all towns that have DENSITYCH less than 0 or in other words all towns where density decreased between 1990 and 2000. You can perform attribute queries through the geographic data view or through the attribute table. As the table and the map window are dynamically linked, your selection results should be visible in both places. 22. To access the attribute query dialog, click on the Select By Attributes icon at the top of the Table or go to the main menu Selection, and click Select By Attributes. 4
23. A Select by Attributes window will appear this is a RQBE (relational query by example) where you can construct a query without needing to know the underlying Structured Query Language (SQL). However, some SQL is also visible as in SELECT * FROM census2000towns_poly WHERE. 24. Under fields, scroll down to DENSITYCH and double click on it. Then click on < and then enter 0. Click Apply and then close the Select by Attributes window. 25. The towns with a fall in density are highlighted in both the map as well as the table. Note: After the query, click on Clear Selection icon in the Table icons or from the Table Options pull down menu or from the Selection main menu item in the geographic data view if you want to get rid of the selection and make a new selection query. 1.4 Sorting Data 26. Click on DENSITY00 field in the table to highlight it and then click on the right mouse button. From the accompanying menu, you can select Sort Ascending or Sort Descending. Before you exit you must save your edits. 27. To do this, click Stop editing on the Editor toolbar. When you click stop editing, you will be asked if you want to save edits (if you have not already saved them). Click yes and your edits will be saved. Question 1 (20 points) Answer these questions before you move on to the next section Question 1a (10 points): Which towns had the greatest fall in density (look at the top 10)? What would you predict about housing values in towns where density fell? 5
Question 1b (10 points): Which were the densest towns in 2000 (again look at the top 10)? Did density rise in those towns between 1990 and 2000? What do you predict about crime trends in the communities with a steep fall (or steep rise) in density? 1.5 Joining tables If you open the attribute file associated with the Towns shapefile you will notice that it does not have any crime related variables. However, we know that the crime tables and Towns have a common key: TOWNS_ID. The Towns attribute table shares a field called TOWN_ID in common with the crime tables for 1990 and 2000. 28. Right click in the TOC against the table ucr_00_s_v and then select Joins and Relates and then Join to get a new window called Join Data. Complete the details of the join as shown. What do you want to join to this layer? should be Join attributes from a table. The key is TOWN_ID. Check against Keep only matching records. Click OK. If it asks to index your data click on OK. 29. Then, right click on the shapefile TownsMA or census2000towns_poly in the TOC. Select Joins and Relates and then Join to get a new window called Join Data. Check against Keep only Matching Records. See screenshots below for other details. Now the attributes from both the crime tables will be visible in the Towns attribute table when you scroll to the right. 30. Query this table using Select by Attributes (main menu Selection and then Select by Attribute). For example, I queried all locations where murder rates (and total robberies) in 2000 were more than murder rates (and total robberies) in 1990. Note: Joins are stored in the ArcMap document itself but are not permanent unless you save this data as a new shapefile. Note: If you plan on moving the data at some point, you should 6
save your ArcMap documents with relative paths (main menu File and then Map Document Properties and check against Store relative path names to data sources). If data is moved, you can repair tables and layers after opening the document, but unless the target table and the join tables are in the same workspace, the joins will not be repaired. If you save your document with relative paths, tables and layers will be restored automatically with joins as long as the document has been moved relative to the where the data has been moved. 31. Add the census2000blockgroups_poly shapefile in the Lab4.gdb to your map document and join the housing value by census block groups data table to it. How could you join them? Hint: census2000blockgroups_poly and housing_value.dbf share a common key called LOGRECNO; census2000blockgroups_poly and HousingvalueIncomedata_ share a common key BG_ID. These were all one to one joins. To do one to many or many to many joins you need to use a similar tool called Relate or aggregate the data in some way. Remember to save your lab4.mxd document. Question 2 (40 points) Question 2a (10 points): Create a map for one crime related variable from the crime data that you joined to Towns in MA. (This does not have to be a layout that includes north arrow, scale etc. Just attach a screenshot of the map and legend). Briefly describe what it suggests about that crime. Question 2b (10 points) Describe briefly one crime related query that shows change in that crime variable between 1990 and 2000. Attach screenshots. Question 2c (20 points) Create a map showing housing values in Massachusetts in 1990 versus 2000. Use this map to describe in a few paragraphs whether it confirms your hypotheses about the relationships between housing value and density (in question 1a). (This does not have to be a layout that includes north arrow, scale etc. Just attach a screen showing a screenshot of the map and a legend). 1.6 Spatial Queries 32. Add the final version of spills or crashes dataset that you created in the previous lab exercise to your ArcMap document. 33. Go to the main menu Selection then Select by Location. In the Select by Location window try out spatial queries that could assess possible relationships between housing values, and crash (or spill) locations. 7
34. Add other layers from massgis that may help you explain any patterns that you noticed: for example Environmental_Justice_Populations, etc. The screen on the right shows how to set up a query to find all Environmental Justice designated block groups that are within 0.5 mile of hazardous material or oil spills. 35. You can switch the selection in the table to conduct statistics for different subsets. For example Calculate statistics for mean percentage minorty in EJ block groups 0.5 miles from spills versus those block groups which are over 0.5 miles from it. Question 3 (20 points) Do this after you do the next section Write a brief description of the results of your spatial quer(ies). You could test theories and reason about a few of these theories regarding the relationships between population density, income, housing values, and spills (and/ or crashes). Include the screenshot of at least 1 spatial query, and 2 graphs (using the Statistics tool). If you think it is relevant to your question, incorporate the crime trends that you mapped in the previous question. You could concentrate on one location (like the city of Springfield, Boston or Cambridge) or look at overall trends within the state or a region in the state. Refer to your map in Question 4 for this description. I will grade this based on your coherence in stating your hypotheses about relationships (or questions about trends) and the appropriateness of the visual material you attach to answer the questions you set out to not on the quantity of visuals or material that you attach. Examples of visuals you could attach: Statistics on median housing value in EJ block groups within 0.5 km of hazardous material spills versus median housing value in EJ block groups more than 0.5 km of hazardous material spills 1.7 Spatial Joins in ArcGIS To find out the number of hazardous materials/ oil spills or crashes in each of the census block groups you would create a spatial join between the two shapefiles: spills/crashes and census block groups. Recall (from Lab Ex 2) that you can only do these overlays if they both are georeferenced to the same coordinate system, projection, etc. 36. Right click against census block groups in the TOC, then select Joins and Relates and then Joins to get a Join Data window. In this window, change What do you want to join to this layer? to be Join data from another layer based on spatial location. Accept the defaults and save your output to censusbg_spillscount or censusbg_crashescount your folder. Click OK. 8
37. When you open the attribute file for the new shapefile (that is automatically added to your ArcMap display) you will find that it has a new variable Count_ that shows the number of spills (or crashes) in each census block group. Where do you find many spills/ crashes? (Do an attribute query to find such block groups). Notice that there are data files associated with census block groups that have many other variables that may be of interest to you (which you could also join). These files are documented at http://www.mass.gov/mgis/cen2000_blockgroups.htm 1.8 Creating new attribute tables 38. First, add the Spatial Analyst extension added. In the main menu click Tools and then Extensions and then make sure Spatial Analyst is checked. 39. Add one of the following rasters from massgis depending on the type of analysis you would like to reason about: income99, density00 or lichildden if you choose to skip 1.8.1. If you add income you are interested in asking about the differences in income between various census block groups with different numbers of spills/ crashes. Therefore your spatial question is Are there differences in income between block groups which have no spills versus ten or more spills? 1.8.1 Creating a raster data set from a vector data set (Optional) 40. Click on the Conversion Tools in ArcToolbox and then To Raster and then Feature to Raster to get a new window. 41. Create a new raster that shows median housing value by 500x500m cells that were converted from the block groups data (see screen that follows). 1.8.2 Creating an attribute table 42. Click on the Spatial Analyst tools in ArcToolbox, then select Zonal followed by Zonal statistics as Table to get a new window as shown. 9
43. Let spills or crashes be your zone data set and the field you are interested is the number of spills or crashes (Count_). In other words you want to find the average housing value (this is an average of median housing value not a great variable statistically speaking) for each spill/ crash category (the number of spills or crashes varying from 0 to the maximum number of crashes or spills). Click OK to get a new table. 44. You can open this table in Excel and make a graph. Or click on the Table options icon and then in the drop down menu select Create Graph to get a new window where you want a Vertical bar graph with the X field is COUNT_, X label field is COUNT_ and the Value field is the (housing value) mean. The graph appears to indicate a falling trend in mean of median housing value in owner occupied units as the numbers of spills in the census block group increases. Save the final graph that you create. Question 4 (20 points) Create a layout with the graph you created in Section 1.8, and a map of any variable that is relevant to your discussion in Question 3. (Include legend, title, north arrow, source, scale in this map). 10