In-Class Exercise 9 [GIS for Historical Studies]

In this exercise we will georeference a Sanborn map and  then link it to historical data and then produce a map showing faunal data overlaying a historical map for the block. We'll download Sanborn maps for a neighborhood in Oakland and then we'll link it to faunal data from an archaeological consulting project that occurred there in the late 1990s. 

Part 1. Mapping Faunal Data

Part 2. Historical Map Sources as a WMTS

Part 3. Mapping Census Data

The Sonoma State team that ran the project (Adrian and Mary Praetzellis) have put a lot of data online

Putting the "There" There: Historical Archaeologies of West Oakland Links to an external site.

https://www.sonoma.edu/asc/cypress/finalreport/index.htm Links to an external site.

However the detailed report on Block 1 and associated Faunal data in an Excel document was kindly provided by Dr. Adrian Praetzellis for this course.

 

Part I. Mapping Faunal Data

• Open the PDF located in the class Box under “Oakland-Mandela/Block1/B1_report.pdf”

Look in the Appendix “Feature association by Block” at the end of the PDF. It’s B-1 or page 331 of 365 in the PDF. This table allows us to link the references in their Excel document.

Faunal Analysis results table

•  Open in Excel and view the data.
• You can see the data are not structured appropriately for a summary map. These are set up to analyze the relationship between meat cuts and cost and other data about the property such as socio-economic and ethnic affiliation.
• restructure data in Excel so that columns indicate mappable fields such as Beef_Low Beef Medium, Beef_High. 
• Each row should correspond with an archaeological feature. The number should be provided in an Integer only field (for further joins). In this example most of these were excavated privies.

 

Sanborn Maps

*Update* The Library of Congress now hosts many Sanborn Maps from prior to the 1920s in full color. The UMI subscription described below is no longer is needed.

Original: We have a campus license for access to the Sanborn Maps fire insurance maps that date from 1866-1970. The scans online are black and white but the color data is mostly about construction material so we can do a lot with just the black and white.

This exercise focuses on a neighborhood in West Oakland where freeway replacement after the 1989 Loma Prieta earthquake involved mitigating sites in a late 19^th century neighborhood.

• From a computer with our library proxy access visit the Sanborn maps:

http://sanborn.u Links to an external site.mi.com/ Links to an external site. and select 1889 maps for Oakland, CA.

Note that there are 25 map sheets cut into tiles so finding the correct sheet(s) is a bit of work, but you will notice that the third sheet, 0B, is an index. Search the index for the range of addresses you are looking for.

We can use an example address like 671 Sixth Street. Near the center you’ll note Sixth Street (the * indicates only half of the street is shown). It looks like 671 6^th Street (an odd numbered address) is on *18 sheet.

• Close this view
• Use the “Jump to Sheet” menu on the thumbnail page to jump to 18_b. Look at the street names: Fifth, Sixth. We’re on the right map sheet tile! We're focusing on the block labeled "50"
• Click “Download Map” on the top bar. A PDF map will open in a new tab. Hover in the lower-right corner and click on the Disk icon that appears. Save the map as “18b_sheet.pdf”
• Open 18b_sheet.pdf in Acrobat.
• File > Save As Other > Image … > PNG.
• In Windows right-click and Preview the PNG.
• Open the B1_Plan.png map in the Mandela.Zip file you downloaded earlier and Preview that file.

What if you don't have Acrobat on your computer?

You can get the block 50 looking as good as possible and then do a screen shot. In Windows you can go to the Start Menu and search for Snipping Tool. Start typing S...n...i...p. Click the icon.

Do a screenshot and save it as PNG.

Note the street names on the four sides. The block is bounded by 5^th and 6^th and Grove and Castro streets.

Georeferencing Sanborn Maps in ArcGIS

Sanborn maps don’t have an reference indicated but we can use street corners.

• Open a Blank map in Arcmap 10
• Add Data from ArcGIS. Add Street Maps. Zoom into Oakland near Jack London Square…
• There’s a problem: are a lot of modern freeways!
• Search for USA Historical Topo Maps and Add (don’t “Open”) so it gets added to the same project.

When is this map from? Click the Identify tool choose “Visible Layers” and click map. Note the Date on Map field. Also the TIF file has the date in the filename.

Georeferencing HINT:

Neither Castro nor Grove appear on the 1959 map. Castro is unlabeled (though it does appear on the Bing hybrid layer under I-980) and Grove was renamed Martin Luther King Jr Way in Oakland and Berkeley. So start with the corner of 6^th and Grove on the B1_Plan map and proceed around with Castro the street to the west of Grove.

This is the block labeled 50

*Update* the Library of Congress version of the Sanborn sheet 18 showing Block 50 can be downloaded for free Links to an external site..

Can we Georeference the Sanborn map using the 4 corners of the block?

What are the problems with this approach? What is the entire spatial reference dependent upon?

How can we improve the referencing?

Other method

• Visit this website

http://codeforoakland.org/data-sets/ Links to an external site.

which aggregates data appearent for the city of Oakland (found with search “oakland parcel shapefile”

• Scroll down to “Parcels” Tax parcel footprints

Note that there are Shapefiles in Geographic and State Plane III. Quiz: what is Geographic?

• Add the “Parcel_SPIII_clip.shp” file from the Mandela directory.

Let’s georeference to the Parcel boundary. This is typically very accurate data – why?

Property boundary disputes and … taxes.

Note the gap in the Parcels data. I took a block from elsewhere and copy/pasted it here and nudged it around to align. Let’s georeference to this.

historical Aerial images are available going back over 50 years in much of the US. Let's make a quick visit to see what's available at

HistoricalAerials.com Links to an external site.

Search on 671 Sixth Street, Oakland, CA

Determine where the block is and click to zoom in on image. Right click and "Save as..." to save the screen shot (pan around to keep watermark off of the block we're interested in).

 

It can be hard to Save As... the exact right view.  Your best result may come from getting it to look good (large but not pixelated) and then again use Snipping Tool to save a screenshot.

This time use JPG because it has shades of gray rhater than indexed colors.

Save and georeference this and other images using landmarks present in other layers. For higher quality referencing a field visit may be required: if some features are present today collect differential GPS points from those locations and georeference the imagery using those points.

Block Plan Map

The Sonoma State Report includes map that shows excavated features and other reference information. What can we use to georeference this map?

once the map is georeferenced we can trace off features and link them to the data table (for this example -- if GPS or higher precision coordinates are available for archaeological features use those instead).

Join and symbolize

Download CSV file. Join to Privies layer

View annotation table

How best to symbolize?

One possibility is Pies for each meat type with the cut shown as a section of the pie. In this example the pie is scaled to the weight of the bones from that cut in each context.

Quantities… how about stacked bars with one color for each meat type (e.g., pink for beef, brown for mutton, green for pork).

Here's the beginning of Stacked Bar representation (beef only).

Would it make sense to use a raster data model? What are the pros and cons?

 

Save a PDF of the representation you come up with and upload

Your map should have a scale bar, a north arrow, and a meaningful title.

 

Part II. Historical Map Sources as a WMTS

We discussed in passing the Georeferencer tool that's appearing on historical cartography websites. The David Rumsey map collector site has added one and we can use it to georeference Oakland map tiles

• Proceed to http://www.davidrumsey.com/ Links to an external site.
• Search on: Thompson & West Oakland 1878
• On the left side choose the filter "Atlas Map" and Where: Oakland CA.
• The "Old Oakland" neighborhood we're working in is on Vicinity 5.

Let's spend 58 seconds watching this clip

 

Once this step is complete we'll be able to bring in the map as a Open Geospatial Consortium Web Map Tile Service (OGC WMTS) right into Arcmap or Google Earth.

What's neat about this approach is that we get the historical map resource and the library or organization like Rumsey gets their maps georeferenced

We both get something we want!

 

I think this is an important development for archival sources and we'll probably see lots of these kinds of maps in the future. Let's each do a page from the 1878 atlas and georeference it to the Streets layer for context.

Once Georeferencing is complete click on "This Map" tab and look at the products offered: Worldfile, KMZ, and Embed OGC WTMS. Click the Embed link.

There are instructions once you sign in. 

The study area is on this map Old Oakland Vici Links to an external site.nity 5 map Links to an external site.

Click orange button "View in Georeferencer", then "This Map" tab and choose OGC WMTS.

As described in the Georeferencer Help you then

• open the ArcCatalog under "GIS Servers"
• Click "Add WMTS Server".
• Paste the link in the top (no Login / Password)
• Choose Affine drag it into your map!

B. Georeferencer exercise

Let's spend a little time georeferencing some of the 1878 map sheets.

Please coordinate so we don't all do the same Atlas book page of Oakland and Berkeley (change your city filter).

 

Part III. Census Data about these addresses

We're going to link additional historical data to this map. 

But first, a data presentation aside: Please visit this link http://sel-columbia.github.io/nycenergy Links to an external site. Note the presentation style. It's hosted out of Github and uses Mapbox technology which is an open source mapping system. Let's discuss web mapping options for these kinds of data.

 

We're going to integrate Census bureau information from individuals because data older than 72 years is available at the individual level as confidentiality concerns don’t apply to the deceased.

Returning to our address of 671 Sixth Street, the report tells us that Theodore Belding and his family lived there in 1880.  The census sheet gives us additional information, including his neighbors.  Looking at the sheet, we can see that his neighbor at 677 Sixth Street was William Ilse from Germany. Privy 993 was excavated from Mr. Ilse's residence.

 Hint: You can access census records free with library access: http://www.ancestrylibrary.com/ Links to an external site.

Under databases, search ancestry, you will be prompted to log in, but from there, you can search public records. 

These census data can be connected with other tabular data and used for labeling and analysis. 

 

Creating data from Census and Digitizing Sanborn Houselots

This example has a further challenge: The parcels don't exist in the modern Parcel shapefile and had to be constructed from scratch. In lieu of a higher accuracy historical parcel map they can be traced off of the georeferenced Sanborn map.

Using the information we have gathered from the Census, (I have only included those from the 1880 census as they would have been contemporaneous neighbors) we can create an excel table with additional information, such as name, address, occupation, ethnicity etc.  I have created the table for you.  Hint: excel tables must be older than 2003, in .xls format, not .xlsx or else ArcMap will give you a database error message when attempting to import it.  You can find that table in the shared Class box --> Oakland-Mandela --> Block 1 --> Blockcensus1880.  We can join this table to the others so that personal census information is now included, which fields will we join? 

We will create Polygons using the Editor tool and trace lots from the Sanborn maps. Lets begin by creating a shapefile in ArcCatalog, using a polygon feature.  

 Bring your new shapefile into ArcMap and begin an editing session:

Recall that we can now include attribute information to this digitized Sanborn houselot:

Further considerations:

• A spatial join can be used to link privy locations, house footprints and other data inside the parcel to the parcel boundaries.
• Digitizing the house footprints from Sanborn maps can yield additional analyses such as linking square footage of home with other data from the census and other sources. Higher density and multistory appear relatively compact in the footprint. Some Sanborn maps indicate the number of stories per structure which can be factored into the Sq Footage calculations.

 

Additional Resources

Aggregate census data at the National Historical GIS site

https://www.nhgis.org/ Links to an external site.

• Click “Select Data and GIS Files” on bottom of left nav bar