Enable Intellisense in pyt files in PTVS

Today I finalised my changeover to Visual Studio. I know I’ve been down this road before, but this time it’s different (probably).

This time around, I’m using Visual Studio 2015 Professional with Python Tools For Visual Studio. My shiny MSDN licence gives me access to it, along with Visual Studio Online. I mention that because after thoroughly messing up my Bitbucket account, I’ve now decided to play nice with the MS way of doing this, and using VSO for my source control.

That is not what this post is about though. Things were looking a bit down when I tried to import arcpy and VS could not resolve the import. After immediately googling the issue, I decided to look back at the editor and saw that the squiggly line was gone. Turns out VS needed a few minutes to decide that it liked ArcPy after all.

Like Joel McCune’s PyScripter workaround, I figured VS would need something as well to recognise Python toolboxes. After some clicking around, the solution was trivial: Tools > Options > Text Editor > File Extension. Type pyt into the text box and choose Python Editor from the dropdown list.

The settings take a few minutes to take effect, but it’s evident it’s working once the pyt file changes to the Python colours. Now I just have to figure out how to structure all my files within a solution…

My dev environment

A while back, I made a big commotion about migrating my dev environment from Eclipse with PyDev to Visual Studio 2012 with PVTS. That did not last as long as I intended.

While there is nothing wrong with the setup, my scripts simply aren’t big enough or interwoven enough to warrant a gigantic IDE. Also, my complete failure in getting Git to play nice with it has led me to fully switch over to Notepad++.

I initially installed it to be able to quickly edit ArcGIS for Windows Mobile .amp files. I also used it when I was dabbling with HTML5. I then started noticing how convenient it was to open up my Python scripts in there, and how I could open a new file, scribble some pseudocode before leaving work, and having it persist between sessions.

I was hooked. I set it up with two views so I could view files side by side, played with a few themes before settling on Navajo, and  I was off.

Three problems though: I could not, for the life of me, run Python scripts from the editor. It was incredibly frustrating to have to do my edits, then open the file in Idle to run. Secondly, no arcpy intellisense (sad). Lastly, it does not recognise the .pyt extension of ArcGIS Python toolboxes.

Which is why I am now creating all my Python toolboxes and other scripts in PyScripter. I tweaked it to recognise .pyt files using Joel McCune’s technique, so now I can get some arcpy intellisense in my .pyt files now. I still miss my Notepad++ though, so I still find myself writing one-off scripts there, while leaving my bigger projects in PyScripter.

GitHub: Deciding what to do with it

I decided a while ago that it would be a good idea to share more of my code with others, besides copying and pasting it into this blog. I have linked to a few scripts stored on my OneDrive, especially since I don’t know what to do with all that space, but it doesn’t feel right to keep it there either.

This brought me back to an idea that has been swirling around in my mind over the last few years: GitHub. Since I code alone, I never saw the need for source control and versioning. My workspace is stored on my OneDrive, so my code is always in sync.

I know that this approach is incorrect. I know that if the team were to expand, or if I switched to programming full-time instead of the mish-mash of stuff I do now, I would need to know how to use it.

I logged back into my GitHub account, installed the Windows app, and then stared at it for a bit. How am I supposed to convert my workspace structure into here? After some Googling, reddit gave me the answer.

I’m going to replicate my project folder structure using repositories, and use the local clone of that as my Visual Studio workspace. I will reproduce the scripts in my General folder as Gists, as they are general purpose snippets that don’t need to be grouped together anyway.

GIS, geocomputation, geospatial, geoinformatics, geo-what?

Over the last few months, I’ve been thinking a lot about what I actually do, or what I’m “called”.

I’ve struggled with this since university. In 1st year, whenever introductions were made and I said I was studying Applied Geoinformatics, I would get this response:

When I tried to explain it, I’d always make it worse somehow, so then I’d get this face:

One time, when I told someone that, she asked what it was called in Afrikaans. Saying “Toegepaste Geoinformatika” did not help matters at all. It also did not help that 11 of us started; we were down to 6 after 2 weeks of Computer Science, 5 by the end of the year, and 2 in third year.

The GIS field has changed a lot over the last few years, and rapidly as well. I started out as a GIS intern, then a GIS Operator, when all I was doing was data capture and ad hoc map production. I then started identifying as a GIS Technician once I started doing analysis and more advanced cartography.

After a few months of that, and once the programming snake bit me (Python joke! Or not?), I started identifying as a GIS Developer. Every desktop GIS task I was assigned, I found a way to work some programming into it. I barely use the desktop interface anymore, everything I do is now done from Visual Studio, or at least from ArcCatalog where I don’t have to deal with the drama in ArcMap.

This issue is quite big, especially in a field that is rapidly evolving. Caitlin at GIS Lounge mentioned it:

Q. What do you do?

A. I work in GIS

Q. Um, GIS? What’s that?

A. It stands for Geographic Information Systems.

Q. Um, what’s that?

A. Oh, it’s sort of like Google Maps.

She had a survey form at the bottom of the article, so you can fill in what you call yourself. I was going to fill it in, when I realised I actually can’t sum up what I do in one word. James Fee mentioned this in his first newsletter late last year:

“What do you do?” Remember this question? I used to get it all the time and it was so hard to explain. I’d go into maps, databases and then the Internet. People sort of nod and seem to agree they understand just so you’ll stop talking about intersecting polygons and buffering the result. Then when Google Earth exploded on the scene, I’d used to just always say, “You know, like Google Earth…” and the other person would get all excited and say they looked up their hometown and saw their elementary school and how awesome it was that Google could find it.

I came across Geocomputation recently, which defines itself as “The Art and Science of Solving Complex Spatial Problems with Computers”. I then thought about identifying as a Geocomputation Specialist, but that didn’t seem right either.

Finally, I turned to what probably should have been my first port of call: Wikipedia.

Geoinformatics is the science and the technology which develops and uses information science infrastructure to address the problems of geography, geosciences and related branches of engineering…combines geospatial analysis and modeling, development of geospatial databases, information systems design, human-computer interaction and both wired and wireless networking technologies. Geoinformatics uses geocomputation and geovisualization for analyzing geoinformation.

Geoinformatics Specialist!

Generate chainage along a line feature

This is a request which comes up quite often. Normally, the request is to create points along a pipeline at every 1000m (or some other integer). Sometimes it will be for chainage along a road, but it’s always a line feature which defines a route of some kind. The first time I did this, I used ET GeoWizards, which was an excellent starting point. However, with access to an ArcInfo Advanced licence, I decided to try it using only geoprocessing tools found in ArcToolbox. This did not go well. I then turned to something which I should have turned to at the beginning – Python.

	#
	# @date 27/11/2014
	# @author Cindy Williams
	#
	# Generates chainage at a set distance along line features
	# and saves it to an existing, empty point feature class.
	# The point feature class should have a predefined schema
	# in case other attributes need to be transferred.
	#
	# For use as a standalone script.
	#
	import arcpy
	arcpy.env.workspace = r"C:\Some\Arb\Folder\work.gdb"
	pts = [] # Empty list for chainage points
	route = arcpy.management.MakeFeatureLayer("ftr_route") # Feature class containing route features
	chain = arcpy.management.MakeFeatureLayer("ftr_chain") # Empty points feature class
	i = 1000 # Set the chainage amount
	# Remove previous attempts at generating chainage
	arcpy.management.TruncateTable(chain)
	print("Chainage feature class emptied.")
	# Loop over all the features in the route feature class
	with arcpy.da.SearchCursor(route,["SHAPE@", "Route_Name"]) as rows:
	for row in rows:
	print("\tGenerating chainage along " + row[1])
	leng = row[0].length # Get the length of the current feature
	pts.append((0,(row[0].positionAlongLine(0)), row[1])) # Get the start point
	n = i # Start the count for current feature
	while i < leng:
	# Add a point at every i metres
	pts.append((i,(row[0].positionAlongLine(i)), row[1]))
	n += i
	pts.append((leng,(row[0].positionAlongLine(leng)), row[1])) # Get the end point
	# Open a cursor on the points layer
	cursor = arcpy.da.InsertCursor(chain,("LEN","SHAPE@XY", "Route_Name"))
	# Insert list of points into chainage feature class
	for row in pts:
	cursor.insertRow(row)
	del pts, cursor
	print("Script completed.")

view raw

generateChainageAlongLineFeatures.py

hosted with ❤ by GitHub

Visual Studio awesomeness

I should have switched to VS sooner. I like having multiple scripts docked next to each other – I know Eclipse can also do this, but it seems easier to dock the windows in VS. It easily uses the arcpy Intellisense, and the default code formatting is very nice.

One complaint though, is having to set a startup file per project if I want to run it using the Start button on the standard toolbar. Since all my scripts are standalone, I’ve been right clicking in the script, and selecting Start without Debugging to actually run the script. Still, it’s no worse than right clicking > Python Run in Eclipse.

Overall, the transition was painless, and I’m conforming to the standards we have set within our unit. Not a moment too soon, since Microsoft just did something totally out of character.

Migrating from Eclipse to Visual Studio

Microsoft has released the final version of Python Tools for Visual Studio. I’ve mentioned my love-hate relationship with Eclipse before, as in, I used to hate it, but I love it now. PyDev has only ever been good to me, and I’m grateful that as I was teaching myself Python (and by extension, how to love programming) that it was available as a part of that process. For that reason, I will keep Eclipse around.

However, my unit is fully tied in with Microsoft, and I am the only member of the team currently not using Visual Studio. I have Premium 2012 installed, from when I was fiddling around in C#, and I kept it with the intention of using it as my IDE for JavaScript (which I’m still trying to phase into), but now I’m finally ready to make the switch. I thought I would document my process here in case anyone else wants to migrate from Eclipse to Visual Studio.

My files in Eclipse are organised into PyDev Projects, similar to the way my OneNote is structured. Each unit has a PyDev Project, each Project has separate packages for a company project, and each package has various modules which are all standalone scripts. I don’t link my scripts at all, so this method of organising is purely to keep everything together in logical fashion.

I will need to replicate this structure inside Visual Studio. After a false start (creating a solution file for one PyDev Project, then trying to add another PyDev Project to it), it appears that the best method would be to relate the structures as follows:

• Solution = Eclipse Workspace
• Python Application = PyDev Project

All I ended up doing was make a copy of my folder structure into the folder I am using as my VS workspace. I then created a new Project from Existing Python code and set the source folder to the workspace folder.