This article is specific to FME Server 2017 and 2018. For FME Server 2019+, please refer to Run a Workspace When Data Arrives in a Directory.

Introduction

The Directory Watch publisher allows you to have FME Server watch a directory, and when a new file has been moved into the directory, a notification message containing the file path will be sent to a topic. From here a workspace can use the file in a translation.

In this example, we will be creating two directories in the Temp Shared Resource folder called "input" and "output". We will set-up FME Server to watch the "input" directory for new files, and when a new file is uploaded, a parent workspace will run. The parent workspace will pass the file to a child workspace which will convert it to a different format. The output from this translation will then be copied to the "output" directory, and will (optionally) send an email with a download link to a list of people.

Downloads

directorywatchpublisher2017.zip

Part 1: Watching a directory

1. Create the directory to watch

Open the FME Server Web Interface and log in. On the side menu bar click on Resources, then double-click on the Temp resource to open it. Click on the New Folder button and call the new folder convert_tool. Within the convert_tool folder and create two new folders named input and output

convert_tool directory in FME Server with input and output folders

2. Create the Directory Watch publication

From the side menu bar, go to Notifications and select the Publications tab. Click New to create a new Publication with the name Convert_Tool (title case). Select Directory Watch as the Protocol and then for Topics To Publish To create a new topic called convert_tool (all lowercase)

For the Directory to Watch, click on the [...], then select Temp > convert_tool > Input

For Filter, only monitor CREATE operations and change the Poll Interval to one minute. Click OK to create the publication.

Publication Convert Tool parameters in FME Server

3. Verify the directory watch publication

The Topic Monitoring page allows you to view the messages being published to a topic. Please note that you will need a browser which supports WebSockets in order to make use of this functionality. Check if your browser supports WebSockets.

Back in the Notifications, click on the Topic Monitoring tab. Click the Topics to Monitor drop-down menu, and select the convert_tool topic, once you’ve selected the topic, it will automatically start monitoring as long as this page is still open.

Using a different web browser tab, go to the Resources page, and upload two files to the input directory you created earlier (Temp > convert_tool > input). After a minute, on the topic monitoring page, you will see two messages, one for each file uploaded

Topic Monitoring convert_tool in FME Server

Part 2: Working with the incoming files

A common pattern when working with notifications that provide attachments (e.g. email, directory watch) is to use a FeatureReader transformer to read the data and then do something with it. Another pattern that fits well with the directory watch publication is to use a FMEServerJobSubmitter transformer to call a child workspace to convert the data; and when that workspace completes the parent workspace can copy the data to a directory, email a link to the data or more.

For this example, we will make use of the workspaces that are included within the zip file in the downloads section; namely, 'easyTranslator2.fmw', and 'convert.fmw'. The 'easyTranslator2.fmw' workspace, (the child workspace), will be used to convert DWG files into SHAPE format and then create a zip for the newly created files. The 'convert.fmw' workspace, (the parent workspace), will then copy the zip file to a FME Server Resource directory for the user to access using a file copy writer.

Note: easyTranslator2.fmw differs from the original easyTranslator.fmw, which is provided with FME Server in the Samples repository.

1. Publish the child workspace

Open the easyTranslator2.fmw workspace and publish the workspace to FME Server in the Samples repository as a Job Submitter Service.

2. Publish the parent workspace

Open the convert.fmw workspace, and then open the properties of the FMEServerJobSubmitter transformer. Select your connection to FMEServer or Embed Connection Parameters. For Repository select Samples, and for Workspace select easyTranslator2.fmw.

Update the Job Parameters to match the image below:

FMEServerJobSubmitter parameters in convert.fmw in FME Workbench

Click OK to save the parameters.

Publish the workspace to FME Server, enter the Connection Parameters, click Next. Publish the Workspace to the Samples Repository, click Next. For Upload Connections, check your FME Server connection and click next. At the Register Services step, unselect Job Submitter and select the Notification Service. Configure the Notification service by clicking the "Edit..." button next to it:

• Set the Subscribed To Topic to convert_tool
• Set the Parameter to Get Topic Message to Source Text File(s)

Publish convert.fmw to FME Server using the Notification Service and Convert_Tool Topic in FME Workbench

You may or may not see a warning box appear - New notification topics will be created - click yes to accept and continue

3. Use the convert tool

Back in FME Server go to the Resources page, and navigate to Temp > convert_tool > input. Upload one of the provided DWG files to the input directory. Wait for a minute or so and then confirm that the file is now in the Temp > convert_tool > output folder. You should see a zip file - download it and you will see a shapefile inside.

Note: If you don't see the zip file, check to see if the job was run and what the status was. Go to Jobs and click on the most recent convert.fmw job to read the job log.

Converted file in Output after uploading a file to Input in FME Server

Part 3: Sending a link to download the result (Optional)

You can go a step further by offering a link to the zip file in the FME Server Resource directory and sending that link via email.

1. Create the topic

Create a new topic by going to the Topics tab on the Notifications page. Name the topic email_results

2. Create the email subscription

From the Subscriptions tab, click new to create a new email subscription.

• Name the subscription email_results.
• Select the email_results topic for Topics Subscribed To
• Select the Email protocol
  • Fill in required SMTP information
  • Set Email Format to HTML
  • Click OK

Email subscription in FME Server, set up for use with gmail accounts.

Please see the Sending Email Tutorial for more information about setting up an email subscription.

Also, if you use a gmail account, you will likely need to create an app-specific password for FME Server or allow less secure apps to access your Gmail account as outline in this article.

3. Get an access token

In order to configure a workspace to send an email, you must first generate a valid FME Server token. In the top right-hand corner of FME Server click the drop down menu attached to your user name and select Manage Token. Copy the string of numbers and letters listed Current Token.

FME Server token from Manage Token menu

4. Re-publish the parent workspace

Back in FME Workbench in the convert.fmw workspace. Enable the objects in the Emailer Bookmark by right clicking on the bookmark and selecting Enable All Objects in Bookmark.

In the navigator panel, under User Parameters > Published Parameters, click on the HostName and enter the hostname for your FME Server, and then click on Token to enter your FME Server Token.

Then click on Email and enter the email you wish to receive notification updates to, every time a file is uploaded into the input folder.

Published Parameters to edit in convert.fmw in FME Workbench

Publish to FME Server, at the Register Services screen, click "Edit..." next to Notification service. Expand Notify on Job Completion, set Post to Topics on Success: to email_results and set Post Data From: to Destination Text File. Finish publishing

Publish convert.fmw to FME Server using the Notification Service and email_results Topic in FME Workbench

5. Use the convert tool

Open the FME Server Web Interface > Resources > Temp > convert_tool > input. Upload one of the provided DWG files input directory. You should receive an email with a link to download the zip file. It takes a couple minutes for the email to be sent, if you haven’t received an email after 5 minutes, double check the job logs as well as the email parameters.

Successful conversion emailed results

Conclusion

Using Directory Watch to convert files can be an efficient way to change file formats to a useful format for your organization if you regularly get different formats from outside sources. This way all the files will be converted in the same way and the original and new files will be contained in one spot. For receiving email updates, this can be useful to monitor file uploads, as well as download the files when do not have access to FME Server.

Data Attribution

The data used here originates from open data made available by the City of Vancouver, British Columbia. It contains information licensed under the Open Government License - Vancouver.

Tutorial Overview: Directory Watch | Current: Directory Watch Publisher | Next: Directory Watch Publisher with Idle Time Delay (Advanced)