The search for the ultimate productivity system continues

I spend far too much time thinking about systems. I have a system for everything – my email accounts, my household, my work, my studies and for how all those systems interact. Sure, everyone has a system for those, you might say. For me though, it is my preferred form of procrastination, in the sense that I have a tangible result (a new system!) at the end of it, as opposed to just whiling away the time on YouTube.

Let’s take a look at the evolution of my productivity system. I started out with OneNote in 2011, integrating it with Outlook Tasks in 2012. That plodded along for a bit, until I realised that tracking tasks in Outlook is ridiculous and I needed something web-based. I fiddled around a bit with some to-do list managers and settled on Wunderlist. I kept trying to link it with OneNote, but that turned out to be a useless fight.

I then started messing around with Trello, essentially duplicating what I was doing in Wunderlist. I waffled between the two until Microsoft bought Wunderlist, and now created To-Do, which will eventually replace Wunderlist but has very few features at the time of writing. There’s also Planner, which is the Office 365 version of a combination of Trello and Jira I guess? I don’t know.

I feel like I’m in limbo. At work, I’m currently using Planner with the team to track project tasks, linked to the relevant Notebooks, Groups and SharePoint libraries (accessible through the OneDrive mobile apps). I did link Wunderlist to Planner, but that just seemed weird.

I’m still using Wunderlist for daily task reminders, bill reminders, shopping list and food prep. I’m using Trello to track the food inventory of my household. I’m (kind of) using the normal Outlook calendar to track appointments. In other words, my system is a mess and I need to throw everything out and start over!

Enable Intellisense in pyt files in PTVS

Today I finalised my changeover to Visual Studio. I know I’ve been down this road before, but this time it’s different (probably).

This time around, I’m using Visual Studio 2015 Professional with Python Tools For Visual Studio. My shiny MSDN licence gives me access to it, along with Visual Studio Online. I mention that because after thoroughly messing up my Bitbucket account, I’ve now decided to play nice with the MS way of doing this, and using VSO for my source control.

That is not what this post is about though. Things were looking a bit down when I tried to import arcpy and VS could not resolve the import. After immediately googling the issue, I decided to look back at the editor and saw that the squiggly line was gone. Turns out VS needed a few minutes to decide that it liked ArcPy after all.

Like Joel McCune’s PyScripter workaround, I figured VS would need something as well to recognise Python toolboxes. After some clicking around, the solution was trivial: Tools > Options > Text Editor > File Extension. Type pyt into the text box and choose Python Editor from the dropdown list.

The settings take a few minutes to take effect, but it’s evident it’s working once the pyt file changes to the Python colours. Now I just have to figure out how to structure all my files within a solution…

CLD202x SharePoint Basics for IT Professionals: Yes please

I was planning on posting this a few weeks ago when I signed up for this course, but I’ve only gotten around to it now. I am very pleased with the course offering by Microsoft on edX, and this course is no exception.

Over 2 years ago now, when our GIS team split into Enterprise and Desktop, I was introduced to SharePoint. I had left the City just before they made switch to using SharePoint libraries as storage space instead of My Documents, so I had only dabbled with it briefly (and from an end user perspective).

During the last 2 years, I have had to interact with it in multiple ways – setting up subsites, managing permissions, creating content (such as embedding YouTube playlists, storing geometry in lists, configuring spatial viewers), and generally whatever else came up.

I never received any formal training in it, and simply clicking around in SharePoint while trying to figure it out, as I am inclined to do with other technologies, does not really work. There’s a reason this site exists. This is why I am pleased that Microsoft thought to offer this MOOC – it’s clearly needed.

The notes are very detailed, and explained very nicely in text form. This makes it very easy for me to copy into OneNote, and highlight as needed. The concepts are explained in a way that I would never have gotten from our seasoned SharePoint dev. Not that he doesn’t want to explain – he has shown me so many things on SharePoint – but it’s often the little things that one does automatically that trips up newbies.

For example, one of our clients had recently started using ArcGIS full time after only using it for very basic data capture before. He asked me why ArcMap was complaining about the projection of two feature classes being different when they were both in the same projection. I pointed out that while they were both in LO19, the one projection’s name was “LO19”, while the other one was “Transverse_Mercator_19”. It shouldn’t make a difference, but it does. The central meridian might differ by 0.00001 or something.

Another example is when one runs the Append tool in an ArcMap session, with the target table opened in the map. The tool will complete successfully, but the newly added rows won’t appear in the table even if you click away to a different table. You have to close the table, then open it again to force a refresh. Most of my ArcMap woes are solved this way – closing off whatever I’m doing and trying it again, or forcibly ending the programme. This is something I know after dealing with ArcMap for over 8 years now. It’s the GIS equivalent of

ArcGIS has a seemingly endless amount of this type of issue, and if I were teaching a total n00b how to use it, I would never even think to mention these things, because dealing with them is a reflex now. From what I can tell, SharePoint is similar, except that it probably has even more of it. Nonetheless, I’m just happy that someone has taken the time to lay out the basics in this format.

Testing out the new Collector app on Windows 10

I was very happy to see a post from ESRI appear in my feed this week about the new beta for Collector on Windows 10. It has irritated me that the only app available for Windows Phone was the ArcGIS app, which was pretty much a Collector app designed according to Modern principles anyway.

I am very pleased with this development, and am even more pleased that the app did not crash after I loaded it up on Astro. It only crashed while my colleague was attempting to navigate out of the Measure screen, but that could also just be my tablet acting up.

I will continue testing the app, hoping that related tables will appear soon, and that ESRI will turn into a universal app.

The latest Bing Maps Preview is a step in the right direction

In December, when I was moaning about the lack of a spatial tool to fit my needs, I had a good long whine about Bing Maps. I not only want to use Bing Maps, I want to like to use it.

I use it on most days when I have ArcMap open; I use it (sometimes) on my Nokia Lumia 1020 for navigation. This is why I am quite excited by the latest preview. My biggest gripe with it previously was that I could not view my saved favourites in the web app, only on my Windows devices, even if I was signed in with my Microsoft account (???).

I am extremely pleased to see that this has since been rectified. There are a bunch of other cool tweaks which have been added as well, but the “saved places in webapp now appears in Windows apps” aspect is enough to get me to switch from Here navigation to Maps navigation on my phone. Read more about it here on the Bing Maps blog.

DAT201x: Final recap

I really enjoyed this MOOC. I even considered upgrading to a verified certificate, but that was before I realised that after conversion, it would cost me ~ R1100. I might as well take the actual exam (~R747) and use my UNISA school discount to pay ~R422.

Overall, I think it was a very comprehensive basic -> intermediate course on using SQL with SQL Server 2012. It was great for me to able to apply some of the theory I’ve learnt over the years, and to understand more about the technology which underpins our SDE.

I hope that Microsoft will be offering more courses in this format in the future, as I find it a much better environment for learning than MVA. I also like how they’ve tied this course into convincing people to write the official exams. I’m definitely considering it seriously (after studying. A lot).

Now that my schoolwork is done for the moment, I have about two months to get a couple of MOOCs in. Also, I should look at ESRI Certification again, as it is a stormcloud hanging over my head, especially now that my PLATO application is complete.

Using INDEX & MATCH instead of VLOOKUP

I’ve been using Excel since Office 97. I was about 8 years old, and had no idea what I was doing. By the time I had progressed to VLOOKUPs and PivotTables, I was a bit older (and wiser, I’d like to think).

It was to my chagrin that I discovered about 2 years ago, that instead of creating a clusterbomb of VLOOKUP/CHOOSE (sometimes with some nested IFs sprinkled in for good measure), I could have been doing INDEX/MATCH all along.

=INDEX(x, MATCH(y, z, 0))

x = Range to return a value from
y = Value to look up
z = Range to look up the value in

I’ve made a small spreadsheet to demonstrate. Please ignore the fact that copying the formulas from Excel Online has changed them to the American region structure, hence the semi-colons as delimiters instead of commas.

Cell E2 does the standard VLOOKUP

=VLOOKUP(D2;$A$2:$B$9;2;FALSE)

while F2 uses INDEX/MATCH to achieve the same thing

=INDEX($B$2:$B$9; MATCH(D2;$A$2:$A$9;0))

That looks a bit more complex than the VLOOKUP. The real power is demonstrated in the next example. To achieve E3, VLOOKUP does the following

=VLOOKUP(D3; CHOOSE({2\1};$A$2:$A$9;$B$2:$B$9);2;FALSE)

while INDEX/MATCH follows the same structure as before in F3

=INDEX($A$2:$A$9; MATCH(D3;$B$2:$B$9;0))

This is why I am now more inclined to use INDEX/MATCH. The ability to do a lookup in any column, and return a value from any column, is great. This really helps for those times when I’m not just rather loading the data from the spreadsheets into Python dictionaries and pulling it apart there.

DAT201x Querying with Transact-SQL: A timely MOOC

I mentioned a while ago that while I was chuffed that Microsoft has started offering MOOCs on edX, I was not at all pleased about the peer assessment requirement in DEV203x.

Fortunately, another MOOC caught my eye: DAT201x Querying with Transact-SQL. The timing could not be better – I’m currently preparing for my first university exam in 5 years, so I’m trying to get back into that frame of mind. The exam is focussing more on the theory of database administration, best practice etc (so pure study work – my favourite!), but one of the assignments included a question where the answer was to write a query in T-SQL.

While I did that in undergrad, things have gotten understandably hazy over the years. Coincidentally, I had to modify some queries at work on one our SQL Server databases recently, so after not doing anything pure SQL related in 5 years (besides the SQL stuff in ArcGIS), in the same month I get hands-on experience at work, plus this MOOC to refresh my memory. Just in time for the exam in 3 weeks. Yay!

Also, no peer assessments. Just pure learning.

DEV203x Introduction to Bootstrap: Uh-oh

I was very excited to see that Microsoft is now offering MOOCs through edX. I immediately signed up for Introduction to Bootstrap and Transact-SQL. Bootstrap opened up on 1 April, so I spent some time getting the videos, downloading the notes and preparing my OneNote section for it.

I idly clicked over to the first lab, which looked promising – a detailed assignment, clearly designed to test the knowledge gained from the videos and notes, with clear submission rules. I then scrolled to the bottom and saw this:

I’ve mentioned in several previous posts that peer assessments in a MOOC is a dealbreaker for me. I’ve changed my opinion somewhat on the use of video lectures (just barely though), but making peer assessments a requirement is a bit much for me.

While my need to finish what I’ve started is strong, I have many other things to do as well, so for this one I may be content to just grab the content for self-paced learning/reference at a later stage.

An example of inefficiency

This incident happened to me about 18 months ago, but I still can’t quite believe it. Over the years, from dabbling with computers for fun, to studying them, to working with them full-time, I have had to find solutions to many problems. Often, these solutions would involve workarounds of some kind, or using a group of various tools to solve the problem.

On this particular day, an engineer from another unit asked me for a basic locality map – just a point overlaid on imagery showing the study area, with three labels. Very simple. I was about to ask him to send me a kmz file of the area when he said he would email me the map that he had created. Hmm.

I opened the JPG while he came to my desk to explain his workflow. See, he had snipped a piece of the imagery from Google Earth and pasted it into a Word doc. Next, he added labels using text boxes, then made a screenshot of that and pasted it into Paint so he could save it as a JPG. Also, he wouldn’t really be asking me to do it because he had already spent a few hours doing that, but because it was going into a report, it needed to have the correct imagery in the background. That should take me five minutes to do right?

So to recap: an engineer, who gets paid at least double what I do, spends a few hours hacking together a map using a barebones GIS viewer, a word processor and a barebones graphics programme. Out of necessity, he asks me to redo the “work” has done, and prescribes a time frame for how long he thinks my work should take.

The same task would take me about 15 minutes to do correctly and neatly, adhering to cartographic principles and corporate policy, using the professional GIS software that I have as a result of, you know, being a GIS Professional.

Needless to say, when I sent the map to him half an hour later, I did not receive an acknowledgement email (and definitely no thank you email), nor did I receive a cost code for the time I had spent fixing his “map”.