Programming

Working with Python Cursors to Edit Tabular Data

Problem: We have been tasked with writing a Python script to automate several spatial operations within ArcMap, including changes to the attribute tables of spatial features.

Analysis Procedure: Because we are working with ESRI’s suite of tools, this script required the use of arcpy cursors to allow access and editing capability for the specified data. The arcpy cursors were created in this project through arcpy function calls on specified tables. There are three main types of cursors- search, update, and insert. The search cursor is best suited for reading attribute values in a row while the update cursor makes changes to existing rows or deletes the existing rows entirely. Finally the insert cursor adds new rows to a specified table.

Obviously the ability to read, write, and edit tabular data is vital to automate spatial processes and the arcpy cursors can be used to create python dictionaries, lists, and other data objects within the script. This project required two separate scripts featuring combinations of all three arcpy cursors to alter various types of data. The scripts are displayed in the slideshow below.

Results:

  • Python Script that uses an Update Cursor to delete rows in a given table based on user-defined values.
    Python Script that uses an Update Cursor to delete rows in a given table based on user-defined values.
  • Python Script that generates new line features connecting polygon centroids using Search Cursor

Reflection: Working with the three arcpy cursors to complete this project allowed me to become more familiar and comfortable using GIS-specific scripting techniques, particularly the ESRI arcpy package. While it is clear Python is a flexible scripting language and well suited to many applications, the GIS specific functionality of the arcpy package ensures that there are no issues when reading, editing, and creating tabular attribute data that may be stored in an Enterprise Geodatabase or other type of Enterprise system.

Automating Spatio-Temporal Crime Mapping Analysis

Problem: Monitoring spatial patterns in violent crime, though important, may occur only once a year due data manipulation challenges. Creating crime maps requires preprocessing steps that are time consuming to perform by hand.  Batch geoprocessing to automate intake of CSV files containing crime data (or access the online repositories directly for the data), as well as geocoding data and hot-spot analyses of this data, will improve our understanding of violent crime spatial patterns.  Automating this process can enable more frequent monitoring of violent crime incidents–on a monthly, weekly, or even daily basis.  I have been tasked with creating a tool for this purpose and applying it to violent crime data from the Town of Chapel Hill, North Carolina.

Analysis Procedure: To address the batch processing aspect of the crime pattern analysis procedure, I wrote a series of python scripts featuring a FOR-loop that: for all the types of violent crime in the CSV file of crime data, sets each type as a dictionary key and use those keys to update the ‘master crime table’, ensuring each record in the master table is is categorized by the crime type. Once the master crime table is written, the python script processes address components and Geocodes each crime record to produce a point feature shapefile representing locations of each crime.

Next a defined function, “selectbyCrimeType” is called which allows the user to filter geocoded crime records by crime type. This function relies on a wildcard string supplied by the user and provides a way for the user to configure the tool to visualize specific kinds of violent crime. Next the script calls a second defined function, “vcptsHotSpot” for the crime point shapefile. This custom function collects crime events, generates a count of events at each geographic location, and uses the count field to generate a Spatial Weights Matrix. This matrix acts as the input for the subsequent Hotspot analysis which is executed by the script before finally generating two HTML reports featuring hotspot analyses- one regular hotspot analysis and one optimized hot spot analysis.

  • Graphical User Interface
  • Crime Mapping Script Parameters
  • Crime Mapping Psuedocode
  • Defined Functions used in Python Script

Results:

  • HTML Report Crime Hotspot Analysis, generated by custom Python script

Reflection: While there were some stumbling blocks in completing this project, I was able to successfully automate a lengthy and complex process that can be used to help Police and governments address violent crime. Prior to this project I had limited experience using Python scripting but I was consistently impressed with how flexible and powerful scripted tools can be. I plan to continue exploring python scripting as I move through my career and hope to build more complex customized tools.