ArcMap Text Points and Census Tables Lab

Due in two weeks

Objective: The objective of this lab is to keep you going with ArcMap. You will: symbolize text files with point coordinates as maps; convert them to shapefiles; join tabular data in text files to shape files; and use selection, sorting, summarizing, and spatial and attribute joining to answer a set of questions. You will also learn a bit about the US Census Bureau's Census2000.

Data Resources

These data used to be in two directories, \\Odin\Data\Geog488 and \\Odin\Data\Census2000. I've tried to get all you need for the lab in the first of those, but, if you need to, look in the other. You probably already have the Data directory "mapped" as a drive.

  • oahucbpts.shp, etc. - the shapefile of Oahu Census Block (centroid) points that you used in the previous lab.
  • hosp.txt - text file with latitude, longitude, feature class (hosptial), and name for a set of points representing hospitals on Oahu.
  • schools.txt - text file with latitude, longitude, class (school), and name for a set of points representing schools on Oahu
  • in \\Odin\Data\Census2000 ...

    Part One: Text files of point data to maps

    Idea: It is real handy to be able to symbolize a text file containing coordinate points and data on a map. That is what this part is about.

    (Aside: Write-hint: Notice that text files are ASCII BCD files that you can 'type' and edit with editors like 'vi', 'notepad' or 'edit'. Notice the format: quoted strings as column headings, a ".txt" extension. Try that to see how close two points can be in ArcMap by typing a file of points that get very, very, very close to each other and then seeing what "ArghGIS" does with them.) it.)

    Pre-set-up: First, examine the three text files (bglinks.txt, schools.txt, hosp.txt) with a text editor or the DOS "more" or "type" commands. See what's in them and how they are layed-out?

    Set-up: Start with an empty ArcMap project. Add the three text files (schools.txt, hosp.txt, and bglinks.txt) to the project (add data button).

    Display each as a map:

    Convert each to a shapefile in your (U:/) directory:

    Part One Questions:

    1. How many census blockgroup centroids are in the oahucbpts data?
    2. How many hospital points are in the data?
    3. How many school points are in the data file?
    4. How well do these data match your understanding of those phenomenon on Oahu? (What's wrong? Are things in the right places? Are they complete? How would/do you know?)
    5. What proportion of Oahu's blockgroup centroids have elevations above 10 meters?
    6. What proportion of Oahu's population is in blockgroups whose centroids have elevations above 10 meters?
    7. What percentage of Oahu's population lives within 5 km of a hospital?
    8. What percentage of Oahu's population lives within 5 km of a school?
    9. For the following consider a degree of arc to be approximately 110 kilometers.
    10. What is the mean distance of a Oahu census blockgroup centroid to a hospital? (spatial join, new field, calculate)
    11. What is the mean distance of a Oahu census blockgroup centroid to a school?
    12. What is the mean distance of a person on Oahu to a hospital?
    13. What is the mean distance of a person on Oahu to a school?
    14. What big assumption are you making to answer those last two?

    Part Two: Joining data to shapefiles

    Idea: Joining tables to shapefiles lets one access tabular data when needed without having to have it stored "in" the shapefile. I.e., one shapefile can be used with many different tables without having to become a huge single shapefile nor multiple shapefiles. Joining depends on two tables each having columns that contain matching "keys", so that records whose keys match can be joined together. The columns can have different names. It is the data in the columns that are used to make the join.

    In this case, you'll join three files: First, an intermediate text file (which contains two key fields) will be joined to a shapefile. Then, second, a table of census data will be joined to that. ( Yes. This is overkill, and maybe a little inefficient, but it gives you more practice with the joining process .) As the figure below is supposed to illustrate, the shapefile has a FIPS code field but no LOGRECNO field, and the hiXXuf3.txt files have LOGRECNO fields but no FIPS code. These are, therefore, hard to join. The bglinks.txt file has both FIPS and LOGRECNO fields and can be used, in the middle, to join them.
    bg15_d00.shp bglinks.txt hiXXuf3.txt

    Before set-up: Read the AAA_README file. Skim over the sf3.pdf description of the census data, especially Chapter 5 and Chapter 7. In Chapter 7, the Table (Matrix) Section is especially useful for locating particular data. It points to the segment file and gives the reference variable name for particular data. The segment (e.g., 01) is used in the name of the data file (e.g., hi01uf3.txt). So, the data for segment 08 are in the file named hi08uf3.txt. Within a segment file, the data dictionary reference name is used as the column heading (e.g., P089009). The data dictionary ties this number to its meaning.

    Set-up: With a blank ArcMap project, add the bg15_d00.shp file to the display. Join the bglinks.txt file to it. Join which ever segment file(s) you need.

    The steps in a tabular join are:

    The example above would add the data in bglinks.txt (including the LOGRECNO) to the records in the bg15_d00.shp shapefile. To join the data in a segment file to these enhanced shapefile records, do it again but now use the appropriate hiXXuf3.txt file and join on the LOGRECNO fields.

    N.B. There is an oddity in ArcGIS's (v9.2 and 9.3) handling of these text files. It seems that the software can usually display the text tables once they are read, but if there are 255 fields of data it can not display the table. I have not tried more than 255 fields but suspect that that would fail also. However, if you do not display the table but just go ahead and join it to a shape file, the 255 field data tables will work. Three of the seventy-six segment files have 255 fields. They are: "hi06uf3.txt", "hi59uf3.txt", and "hi61uf3.txt", but now you know how to use them too.

    Part Two Questions:

    1. Which Census data segment and variables contain the total population counts for each census blockgroup?
    2. Are there any blockgroups in which that count does not match the number given in the bglinks.txt POP100 field?
    3. What data segment and variables contain the number of females and males in a census blockgroup?
    4. Which Oahu Census block groups have the most disparate ratios of males to females? Is this a pattern that you can explain?
    5. What data segment and variables contain the number of people below 18 years old, 18 to 25, 25 to 65, 65 and older?
    6. Describe the distribution of 18 to 25 year olds across the state. Where are they concentrated? Missing?
    7. Using these polygon data, what is the average distance from an Oahu census blockgroup to a hospital? Why wouldn't this number be the same as for centriods in Part One?
    8. What data segment and variable will let you get at the median household income in the blockgroups?
    9. Describe the state-wide pattern of median household income.

    Hints and work-arounds

    Help - is on the main menu and in the "Getting to Know ArcGIS" book.

    Starting ArcMap - START:Programs:ArcGIS:ArcMap

    Adding a shapefile to the map - Add layer; connect to folder; select files

    Arrange layers - drag and drop in legend

    Hide (or Show) layers - check box in the legend

    Tool bar tools - zoom in, out, previous, next, full; pan; identify; measure; select feature

    Getting the table for a layer - left click name in legend, then choose "view table"

    Symbolizing data - double left click the layer's name in the legend / table of contents. Under the symbology tab, you can symbolize a layer as one class, several classes (and by different strategies), or as unique values (individual symbols) using graduated colors and or sizes.

    Selection - by location (for locating things that are in some spatial relation to other things, i.e. close-to, inside, intersecting, etc)

    Selection - by attributes (use SQL-like query to select records from a table.

    Statistics - calculate statistics (mean, minimum, maximum, total, standard deviation etc for (selected rows) of a column of data in a table) (when viewing a table, right-click on a column heading and select 'statistics')

    Summarize - create a new table summarizing (mean, minimum, maximum, total, standard deviation etc) the values associated with unique values in a column in a table. When viewing a table, right-click on a column heading and select 'summarize'. Be sure that you are trying to write the table in a directory to which you have write permission.

    Spatial Join - click the layer name, select joins, in dialog select join based on location of features (i.e., spatial join) - this will add a distance field to the joined table.

    Table Join - uses matching of key fields to join records - appends attributes of the non-spatial table to the layer's attribute table.

    Table relate - use when you need a many:many relation; does not append to table records but does allow selection by table values to propagate.

    Adding text data (X,Y, data) to a map. Click [Add Data] button. In the dialog navigate to text file, click add. In Table of Contents (TOC), right click the text file name, select open, right click on Display X.Y Data, select fields for X and Y.

    Joined data showing as '0', 'null' or blank fields? ArcMap v 9.0 (like AV3.2) seemed to have trouble with multiple joins. One solution is to make one join, then Data -> Export that as a new shapefile, bring it in to the map and make the second join, and save it as another new shapefile. I know, it seems like the software should be able to not have things as new shapefiles, but... this seems to be a work-around.

    Trying to do something with an XY Event layer? ArcMap won't let you do it? ArcMap almost lets the XY data act like a shapefile, but not quite. Use the Data -> Export trick here as well, to make the XY Events into a real shapefile, and then use that instead.

    In tables, the "freeze fields" will keep frozen columns visible at the left side of the table while you pan across other columns.

    When you create a floating point number field in a table, ArcMap lets you specify its 'precision' and 'scale'. Accepting the default '0's works fine and gives you (apparently) the native floating point binary representation for the computer.

    See the "N.B" note above about text tables with 255 (or more?) data fields or columns. Don't display the table, just join it blind. I think you don't even have to add it to the project first.