ArcMap Woes Part 6: Random failure of gp tools on grouped layers

I had a series of errors recently while I was trying to run a few geoprocessing tools manually on layers within a group layer. The error message kept saying it couldn’t open the layer, even though the layer was clearly in the map. Even restarting ArcMap did not help.

I eventually discovered that geoprocessing tools may randomly fail on layers which are part of a group layer. I say randomly because sometimes the tools would run as expected, other times not. I’ve only experienced this anomaly when using the ArcMap interface – running the same sequence of tools on all the layers in a group layer in a for loop in Python does not give this error.


ArcMap Woes Part 4: Permissions errors

Sometimes when I’m in ArcMap, I can’t start an edit session. Normally, I restart ArcMap and everything is fine.

Recently, for about a week, I could not edit inside any file gdb, or create new feature classes inside of an existing one or a new one. The gdb was locking itself when I was in an mxd that was accessing the feature classes inside it.

No other application was accessing it at the time i.e. ArcCatalog was closed. I would reboot the VM, open a blank map document, drag in the feature class I wanted, then try to edit that feature class, or create a new feature class in that gdb. No such luck.

I mentioned that this happened for about a week. After that, everything was fine again. I’m not sure why. Windows Updates have broken file gdbs before, but I don’t think it was that.

I think that IT changed some group policies and that affected my host where the data is stored, which in turn affected the VM. I know that sounds crazy, but I’m used to accepting crazy by now when it comes to Arc. It’s still not enough to make me go to QGIS though.

View list of ArcGIS Server Data Store Items

Recently I had a task where, for >10 ‘clients’, I had to:

  1. Create a unique database user
  2. Grant privileges to certain datasets in a SDE database
  3. Save an mxd showing only what was relevant to that client
  4. Publish a feature service for consumption in ArcGIS Online

While I will leave the specifics of that delightful script for another post, during debugging I came up against an error where the script would fail if the database connection had already been registered with ArcGIS Server.

In lines 15 and 16, I get the location of the ArcCatalog connections folder for the current user. This is dependent on knowing which version of Arc is installed. I have found a slightly better way (written quite verbosely for clarity):

My dev environment

A while back, I made a big commotion about migrating my dev environment from Eclipse with PyDev to Visual Studio 2012 with PVTS. That did not last as long as I intended.

While there is nothing wrong with the setup, my scripts simply aren’t big enough or interwoven enough to warrant a gigantic IDE. Also, my complete failure in getting Git to play nice with it has led me to fully switch over to Notepad++.

I initially installed it to be able to quickly edit ArcGIS for Windows Mobile .amp files. I also used it when I was dabbling with HTML5. I then started noticing how convenient it was to open up my Python scripts in there, and how I could open a new file, scribble some pseudocode before leaving work, and having it persist between sessions.

I was hooked. I set it up with two views so I could view files side by side, played with a few themes before settling on Navajo, and  I was off.

Three problems though: I could not, for the life of me, run Python scripts from the editor. It was incredibly frustrating to have to do my edits, then open the file in Idle to run. Secondly, no arcpy intellisense (sad). Lastly, it does not recognise the .pyt extension of ArcGIS Python toolboxes.

Which is why I am now creating all my Python toolboxes and other scripts in PyScripter. I tweaked it to recognise .pyt files using Joel McCune’s technique, so now I can get some arcpy intellisense in my .pyt files now. I still miss my Notepad++ though, so I still find myself writing one-off scripts there, while leaving my bigger projects in PyScripter.

ArcGIS Diagrammer: Sad to see you go

I would often use ArcGIS Diagrammer for bigger projects, ones which would require documentation. I’ve never actually included any of my diagrams along with the project documentation, mostly because I was never asked for it.

Once I had worked with it long enough, I started to notice that it could be quite cumbersome and unintuitive at times. If something did not work as expected, I would first carry out the action manually in a file gdb, export the xml and look at it in Diagrammer to see the workflow.

Eventually, I just ended up going back to creating my templates in a template gdb. If I ever need a snapshot of my gdb in a Visio style, I will definitely open it up again, but it just doesn’t fit into my daily workflow.

What I am a bit irked about is the fact that ESRI never bothered to come up with a more official version of this product.

Coming up with a GIS Dataset naming convention

I have tried for several years to come up with/stick to a decent naming convention for GIS data. A colleague of mine came up with a 3 letter prefix scheme followed by an underscore which was pretty nifty:

  • ftr – Feature class
  • tbl – Table
  • svw – Spatial view
  • cvd – Coded value domain
  • vew – Table view
  • loc – Address locator
  • gcs – Geocoding service
  • svc – Service
  • ras – Raster
  • rvd – Range domain

This was done because when looking at the tables via SQL Server Management Studio, one cannot determine which tables are actually GIS tables/feature class attribute tables, and which are native SQL tables at a glance. This solved that problem, and conveniently grouped the different types of data at the same time.

However, a convention only works when everyone sticks to it. The use of plural vs singular is another bone of contention. I am of of the opinion that all feature classes should be named in the singular. This approach has stuck with me since mapwork classes in primary school (ah the 90s). Legends on maps always indicated “River”, “Road”, “Farm”, never “Rivers”, “Roads”, or “Farms”.

This is why when someone gives me shapefiles or whatever that have plural names, I immediately change it to singular. I WILL NOT TOLERATE THIS LUNACY. These naming issues fall under the greater problem of ever expanding formats for GIS data, along with the issue of “How on earth are we going to store all this stuff?!”

And no, saying “in the cloud” is not a solution. Someone still has to manage that data, and structure it in a way that makes sense and somehow anticipate further expansion of the system. That, however, is a story for another day.

ArcMap Woes Part 2: Assigning keyboard shortcuts

I came across this gem the other day. I was in the midst of a massive digitising session, ArcMap was being “slow” and my frustration levels were at a monthly high. I decided to spend a few minutes assigning keyboard shortcuts for the editing mode tools Edit Vertices and Continue Editing. What happened next just pushed my frustration levels higher – the shortcuts would not “assign”, no matter how many times I tried it, or what manner or key combination I used.

That GeoNet thread confirmed that there is a bug of some sort which prevents it from working properly. So I gave up, opened a blank mxd and carried on in there.