FME Workshop

4 December 2017

FME diagram3

Blog written by Daniel Nutsford, Senior Consultant at Interpret Geospatial Solutions

On Wednesday 29 November, Interpret held a workshop for our clients in Christchurch, to share our experience in FME and help others to better understand the capabilities of FME.

Attendees ranged from FME experts, to those who had never used the software before. I sat somewhere in the middle of this group - while I have used FME a handful of times and witnessed FME magic in various work places over the years, I still count myself as a bit of an FME rookie. It might not be surprising then that I was quietly impressed with some of the live demos my colleagues presented. Several FME workflows were presented over the afternoon, some covering basic data translation and others demonstrating much more complicated and automated workflows. Below is a quick summary of some my personal highlights.

FME and Webhooks – Todd Davis 

Interpret is a member of the Statistics NZ FME Vendor Panel, providing support to help create workspaces, custom transformers and get the most efficiencies out of FME processes for significant projects within the organisation. Todd, a certified FME professional, has been working closely with Statistics NZ along with various other organisations.  During the workshop, Todd shared some of his FME knowledge on these projects and demonstrated how easy it is to work with web services using FME Server. With a simple FME workbench, webhooks and a twilio account, he triggered an automated phone call to one of the workshop participants. 

FME can act as a wrapper between websites that can’t natively work together. It pulls information in one from end, processes it accordingly and spits it back out again. Todd’s workbench (which was published to FME Server) connects to an Uptime Robot Webhook and monitors the realtime status of websites. As soon as a website goes down, FME receives a notification from Uptime Robot, reformats the response and creates a customised call to twilio - a webservice that allows for the creation of automated phone calls and text messages. If you’re managing critical websites or services a customised phone call or message to the right person is much more useful than a generic email or alert.

This is just one example of how FME can be used to integrate webservices. Via other methods this task is complex and would require significant programming, but with FME Server and the right skill level a full prototype was up within 30 minutes.

PDF to GIS Translation – Alex Oulton

Alex, also a certified FME professional, demonstrated how FME can automate complicated and time-consuming tasks by chaining together multiple workbenches. Alex was recently tasked with extracting traffic signal loops from technical PDF drawings. Traditionally the extraction is done one by one due to the complexity of the PDFs and is a time-consuming process. Here’s how Alex automated the workflow with FME. This demo got very technical, very quickly!

As a precursor to FME, the PDFs were first passed through a conversion process to extract CAD vectorised linework and text strings. Based on the success of this process, OCR (Optical Character Recognition) and raster to vector tracing software was additionally used if required. The CAD drawings were then consumed by FME. As the CAD file moves through the workflow it is processes by three separate workbenches.

1)      Pre-validation – Workbench checks each CAD file for valid line and text content. Files that fail checks are flagged for resubmission (requires human help)

2)      Traffic Signal Loop Identification – Cutting a long story short, FME automatically detects the traffic loops within the CAD file using shape detection (loops are visualised as polygonal areas). It then scans the traffic loop for any valid text – the traffic loop value? To orientate the entire intersection plan, another process detects the north arrow from the plan and, combined with an azimuth calculator, generates the angle required for True North correction.

3)      Postprocessing and publishing - A final process validates the output, applies a spatial projection, and modifies the traffic loop location to appear correct when overlaid on the road network. The results of the process can then be viewed on an interactive online map. 

While this demonstration was highly complex and required an FME specialist to develop, it highlights that complicated workflows with very specific applications can be broken down into several workbenches and processed accordingly.

FME and 3D – Hamish Kingsbury

Hamish has been working with the NCTIR (North Canterbury Transport Infrastructure Recovery) team on secondment, creating and maintaining FME Desktop and Server workflows. This work has improved processes, timeliness and capability for reopening State Highway 1 following the 2016 Kaikoura Earthquake.

The work Hamish has been doing at NCTIR uses high res imagery and lidar (<0.5m), which conveys the scale of the slips in an easy to understand format and allows the user to create their own web scenes on demand. It uses an opensource WebGL JavaScript library called three.js, which requires the imagery as a jpeg and the heights as a csv.  FME was used to crop and convert the lidar and imagery to the correct locations and formats. The web scenes are viewable on phones and tablets as well as desktops and laptop computers.

NCTIR Dec17

FME, APIs and ArcGIS Online – Daniel Nutsford 

While I am pretty new to FME, I have plenty of experience with similar workflow builders such as Esri’s ModelBuilder and I was curious to see how FME compares. I decided to do some work with Auckland Transport’s realtime GTFS feed. After a few hours of playing around I was pleasantly surprised at how easy it was to publish live bus locations to a Web Map hosted on ArcGIS Online.

Connecting to webservices like APIs is FME 101. The workbench calls the Auckland Transport realtime API, and parses the JSON response. After some basic attribute manipulation and a vertex creator to plot the lat/lon values, FME connects to ArcGIS Online and overwrites an existing hosted feature service with the new bus locations. FME and Esri get along like a house on fire, and connecting to, updating and publishing services is very easy (I could also have published to GIS Server if I wanted). While my workbench was far from pretty and most likely offended many of the FME experts in the room, it was incredibly simple to create and required absolutely no programming. A big thumbs up from me!

That’s a very brief recap of just some of the demos presented at the Christchurch FME workshop. Interpret have another two workshops scheduled for Auckland and Wellington in February 2018. If you are interested in attending either of these workshops, please email melanie.gurden@abley.com with your details so we can send you an invite.