Editing assignment types in Workforce for ArcGIS

A Workforce for ArcGIS project is based on a predefined schema. This schema states that assignment types are populated into an associated coded value domain on the Assignments feature layer when created.

Sometimes assignment types need to be modified, perhaps because of a typo. On the Workforce front end, it is impossible to edit an assignment type. It can only be deleted (if the type is not being used by an assignment).

One can work around this by opening the Workforce project in ArcGIS Online, open the Assignment layer properties and directly edit the coded value domain.

How to convince someone to move from paper-based forms to electronic surveys

I spent March of 2018 working closely with a bridge engineer. For years, they had been using four paper questionnaires to capture inventory and inspection data of structures such as bridges, culverts, gantries etc. They would arrive at a structure, select the appropriate form, measure and write everything down, take a load of photos, draw a sketch or two and move on to the next one.

When they got back to the office, they would get a few students to manually input the captured data into an electronic form, which would store it in a database on their local machine. The students would also need to manually link photos to the correct structure. As with most projects, time would always run out, so the engineers would also need to help out with this electronic transfer process.

Once the screaming in my head subsided, I asked him, “Why? Why is it being done this way? It’s 2018. The process you are describing should have been phasing out at least 5 years ago already.” He didn’t really have a straight answer for me, beyond “this is the way it’s always been done”.

Over the next few weeks, I showed him that there is a better way. I used the first week to convert the largest of the four forms using Survey123. The form was fairly complex – with no access to a table layout or even a grid theme, I had to make a number of design decisions which wouldn’t impact the user experience too much while still retaining (and even enhancing) the functionality available in the paper form.

After my initial stab at it, we spent 2 more weeks going back and forth, fine-tuning choice lists, removing unnecessary questions, changing section groups, enforcing relevant fields, choosing repeats and optimising calculations. We ended up with an xls of 250 rows of questions and 35 choice lists. I was fortunate to have access to one of the databases they’d used on a previous project, so I was able to extract the choices from the lookup tables using SQL and Python.

I spent the last week of March replicating the other three forms. They were similar enough that I could copy and paste much of what I had implemented on the first form, but different enough that I couldn’t keep everything in one form. Once that was completed, I published all the surveys and gave it to him to thoroughly test (I believe my exact words were “Try to break it”).

By the first week of April, I had fixed most of the bugs and we were ready to train the students on it in the field. I set up a web app allowing him to view the surveys as they were submitted. He could immediately send a message to the students in the WhatsApp group I set up on their tablets if they were measuring components incorrectly, or not describing items properly. He was fully converted.

In my next post, I’ll detail how I took everything down and rebuilt it from its ashes by adding Workforce for ArcGIS and Operations Dashboard to create a more efficient system.

The grid theme is finally available in Survey123

I got a lovely surprise when I opened my emails on Friday – ESRI has finally gotten around to implementing a beta version of the grid theme in Survey123.

I know I’ve been quiet on the blog for a while, but that’s because I’ve been keeping my rants about GIS offline (mostly). In particular, my utter frustration at not being able to replicate a very critical paper-based form digitally because of the lack of a grid theme.

I even went as far as chucking Survey123 completely, setting up ODK Collect from scratch and trying out the grid theme there. I was not happy with the results at all, so I resigned myself to a compromise: replicating the forms in Excel and extracting the data automatically from the spreadsheets as the team uploads them to OneDrive in the field. Terrible, I know, but still better than paper.

Since ESRI is making this tentatively worded claim in the beta forum:

It’s now possible to design a digital smart form that closely resembles its paper form predecessor!

I will be putting it to the test today. It’s still not the holy grail though, which would be a proper table theme, but it at least takes us a step closer.

A rant about attachments

I remember when attachments were first introduced in ArcGIS Desktop (10.2 I think? whoops it was ArcGIS 10). It was a very useful feature, and more functionality was added over the years.

It also made mobile data capture even easier. The fieldworkers would go out, do their assessments, and attach multiple photos to their points. However, attachments with Collector has caused me so much frustration. Specifically, syncing with attachments.

The nature of the work we do (and the economic environment we are in) means that by default, I take the maps offline so that the fieldworkers can carry out their assessments, and then sync back to AGOL when they are on lunch break (or whenever they can pick up WiFi). I discovered a few years ago that once one hits a certain threshold (like 20 attachments in the map), there are going to be problems syncing.

It will just outright fail, or take very long and may need to be attempted a number of times. Why is this? I don’t know. Over the years, I’ve encountered this issue on all types of devices – the latest iPhones, low-end Android tablets, high-end Android tablets, mid-range Android phones…

What it seems like to me is that Collector “expects” a certain connection speed, and when it doesn’t get it, it times out and rolls back the sync. Fair enough – I’ve found multiple delta tables on devices I’ve needed to recover the databases from due to failed sync attempts. On a current project, they are using rugged devices which have really awful network chips (as in, I need to stand about 1 or 2m away from the access point so that I can take the maps offline). Naturally, at the end of the first day, each device had dozens of features with multiple attachments each, which refused to sync.

They have been out in the field for 2 weeks. Everyday, I have to manually retrieve the databases from the device, recover them, and push them out into appropriate geodatabases once I’ve determined what’s inside them.

So clear

I can deal with all of that, because Python is a tool that I maaay have mentioned here before. What I cannot deal with is the fact that attachments are still lost during geoprocessing. The fact that it was added as an environment setting in ArcGIS 10.5 and has been available in ArcGIS Pro for a while is of little comfort to me as I currently have access to neither.

Fine. I store the GlobalIDs in another field, merge the features together into their correct feature classes, enable attachments and insert the records from the corresponding attachment tables. Of course, I forget that the relationship class is now messed up, as it’s linking through the (now incorrect) GlobalID fields instead of the fields I stored the original IDs in.

After staring at the screen cross-eyed, I then realise that I only need to provide the attachments as jpgs in a folder, which I can extract from the tables using the original IDs and write into subfolders based on the feature type. I don’t actually need to link them back together since the technician does not need to view the photos to complete the work in ArcMap. /endrant

Testing out the new Collector app on Windows 10

I was very happy to see a post from ESRI appear in my feed this week about the new beta for Collector on Windows 10. It has irritated me that the only app available for Windows Phone was the ArcGIS app, which was pretty much a Collector app designed according to Modern principles anyway.

I am very pleased with this development, and am even more pleased that the app did not crash after I loaded it up on Astro. It only crashed while my colleague was attempting to navigate out of the Measure screen, but that could also just be my tablet acting up.

I will continue testing the app, hoping that related tables will appear soon, and that ESRI will turn into a universal app.