I got a lovely surprise when I opened my emails on Friday – ESRI has finally gotten around to implementing a beta version of the grid theme in Survey123.
I know I’ve been quiet on the blog for a while, but that’s because I’ve been keeping my rants about GIS offline (mostly). In particular, my utter frustration at not being able to replicate a very critical paper-based form digitally because of the lack of a grid theme.
I even went as far as chucking Survey123 completely, setting up ODK Collect from scratch and trying out the grid theme there. I was not happy with the results at all, so I resigned myself to a compromise: replicating the forms in Excel and extracting the data automatically from the spreadsheets as the team uploads them to OneDrive in the field. Terrible, I know, but still better than paper.
Since ESRI is making this tentatively worded claim in the beta forum:
It’s now possible to design a digital smart form that closely resembles its paper form predecessor!
I will be putting it to the test today. It’s still not the holy grail though, which would be a proper table theme, but it at least takes us a step closer.
I’ve been quite busy over the last few months. I’ve fully switched over to ArcGIS Pro, and I’ve been spending most of my time developing a fieldwork workflow using Workforce, Survey123, Collector and ArcGIS Online. I also (finally) forced myself to dabble with the ArcGIS Python API, which has fallen neatly into place alongside my normal fiddling with ArcPy.
I thought I’d write this post quickly just to document something I’ve noticed when publishing a feature layer symbolised using unique values from a coded value domain to ArcGIS Online. Despite the descriptions displaying correctly in Pro, if one edits the symbols and then does not delete all the feature templates and recreate them, editing them in an online web map will result in the codes being shown on the edit dialog, and not the descriptions.
The way around this is to, well, delete the feature templates and recreate them before publishing. I’ve had this happen a few times, where everything looks great online, and then someone tries to edit it in Collector using the codes.
According to my notes, I first used the Make Query Table tool in my first week at Aurecon, back in March 2012. It was the first of many, many times, because often when receiving spatial data in a non-spatial format from a non-GIS user, the first thing that gets thrown out is any trace of the original spatial component.
At some point, I realised the tool’s expression parameter was a bit wonky. As I have come up against this problem every few months since (forgetting it happens each time because I only thought to write down a note about it now), I have decided to immortalise it in a gist below.
I’ve been thinking quite a bit lately about how to store spatial data. It’s something I’ve covered here and my attitude towards this topic has evolved over the years.
The organisations I’ve worked in have mountains of spatial data accumulated over the years. The data is stored in shapefiles, geodatabases, normal databases, spreadsheets, documents, reports, photos…Why is it this way? It doesn’t have to be this way. It shouldn’t be this way!
In the course of my research for a topic for my project for next year, I’ve honed in on the methods for implementing an enterprise geoportal within an existing spatial data infrastructure. However, I feel like my focus is shifting to the data that the geoportal is trying to expose to a larger audience.
The concepts of a spatial data warehouse and a spatially enabled operational data store have been intriguing me. A regular GIS task involves comparing spatial data across a time period, analysing trends and presenting the results in a map or report. Why aren’t we storing this historical data in a SDW that’s optimised for reporting?
Non-spatial data can come from a variety of sources as well – spreadsheets, other databases etc. Another common GIS task is to spatially enable these datasets. Why are we not storing the outputs in a spatial enabled operational data store in an open format like GML?
I think it’s because to plan and implement a SDW/S-ODS takes time (and money). With a normal EDW, the organisation will not need much convincing to see the benefit of implementing one. “Spatial” is still seen as an “add-on”, or a “nice-to-have”.
In my ongoing quest to do absolutely everything through Python, I’ve been looking a lot lately at manipulating databases. I’ve been using arcpy to access GIS databases for years, and last year I finally got around to using pyodbc (and pypyodbc) for accessing SQL Server databases.
Now that I’m in an Oracle environment, Oracle has provided the cx_Oracle library to directly connect to databases. I have yet to test that though. What I’m interested in at the moment is creating and accessing databases for personal use.
I considered MongoDB for a while, but I don’t think I want to go NoSQL yet. This is why I have been experimenting with SQLite (through the sqlite3 library), as it is included in the Python install, and has the delightful SpatiaLite extension. The slogan goes against my one of my mottos (Spatial is Special) while supporting my other motto (Everything is Spatial).
For the first time, the Honours class was not given a list of topics to choose from. Instead, we were instructed to choose any computing centric topic, and post a discussion about it where it could be ripped apart. Obviously, the whole point of me doing this course is to focus on the computery side of GIS, so I settled on spatial data infrastructure (SDI).
There is a lot of literature available on this topic, but most of it focusses on the implementation of SDIs within the public sector. Rightly so – implementing it correctly is expensive, and once the government has bought into the idea and made their data available, private companies should follow suit.
Even then, the whole SDI topic can be very broad, as it is made up of several components. In my current role, while I would have input into all of the components, I wouldn’t necessarily have the mandate for those components. I then narrowed it down to the component that I would be responsible for: the geoportal within the SDI. I’ll be focussing on methods and challenges involved in the implementation of a geoportal as a component of a SDI within the financial constraints of a large enterprise environment.
Today I finalised my changeover to Visual Studio. I know I’ve been down this road before, but this time it’s different (probably).
This time around, I’m using Visual Studio 2015 Professional with Python Tools For Visual Studio. My shiny MSDN licence gives me access to it, along with Visual Studio Online. I mention that because after thoroughly messing up my Bitbucket account, I’ve now decided to play nice with the MS way of doing this, and using VSO for my source control.
That is not what this post is about though. Things were looking a bit down when I tried to
import arcpy and VS could not resolve the import. After immediately googling the issue, I decided to look back at the editor and saw that the squiggly line was gone. Turns out VS needed a few minutes to decide that it liked ArcPy after all.
Like Joel McCune’s PyScripter workaround, I figured VS would need something as well to recognise Python toolboxes. After some clicking around, the solution was trivial:
Tools > Options > Text Editor > File Extension. Type
pyt into the text box and choose
Python Editor from the dropdown list.
The settings take a few minutes to take effect, but it’s evident it’s working once the pyt file changes to the Python colours. Now I just have to figure out how to structure all my files within a solution…