Purpose: The following User Guide introduces you to the File Pattern Matcher (FPM). The file pattern matcher runs the correct job from a directory where files are picked up based on the file name. For instance, you could have 25+ different Auty-mations ran automatically when you drop 25 different file formats into a drop folder. Use this Guide to understand how you can set up your file names correctly.

Description:

Please use the following script to get the history of files loaded in all environments (Dev, QA, and Production). This will give you failures for the full process and failures of the files staged in the database.

Definitions

• FPM: Is an abbr of the File Pattern Matcher Feature

Video Overview

Here is a 7-minute video showing you the basic features of the FPM end to end.

7m-AMT | Features: Enterprise: File Pattern Matcher Basic Overview - Watch Video

Getting Started with the FPM

How to Setup a new FPM

To get started with FPM, be sure you have your client setup with a File Pattern matcher Source Location, have files ready to drop, and then follow these steps.

1. Create a new Auty-mation or edit and edit an existing one and set the property File Pattern Matcher Name on the connections page as required for your use case using the naming conventions in the document below.
2. Make sure you have the Scheduler set to a time frame you want the files to be checked.
3. Note: If you are creating a new Auty-mation, the file you drop will show up in the following location NotSet/Out folder.
4. After your integration is set up, be sure to go to the Tile and set the environment to the environment you want the files picked up from Dev, QA, UAT, or Prod.
5. If you do not want to wait for the scheduler based on the schedule you set, then go to the tile and Hit the Run Now Button.

HOW TO: Control Where Files Are Picked Up or Placed as Outputs

Available Environment File Pattern Matcher Source Location Types

Setting Tile Environment Designations

The most important thing you need to know when you use the file pattern matcher is where your files will be picked up. Right now we support the following locations. You will need to work with your Rep and support to set up your file share. This cannot be set up in the autymate platform by the user.

The tile environment drop-down selection sets the environment that the integration is currently in.

The Tile has 4 Environment Drop Down Options:

For Files Generated Based on the Environment Selection, the Files will be placed in a folder on the File Share with the same name as the Environment selected. This will allow the team to control how the files are processed based on what environment is selected.

Best Practices for Using the Environments

We setup the file paths to follow best practices for Development Environment Life Cycle. Below is the purpose for each environment and how to best leverage it for testing your Auty-mation.

File Processing Pointers

1. You can drop 100’s of files at a time. They will get processed from earliest to latest and vary in the time to process depending on the file’s size.
2. The order of the files running through matters, so if you need to run 10 days’ worth of files, drop all 10 days, don’t drop previous days after you load the latest days if you want the last 10 days ran.
3. If the file fails, it will go to the Failed folder. If it runs successfully, it will go to the Success folder.
4. If you want to create files that you don’t want to get processed, then only put them in Do Not Use. Otherwise, they will get picked up.

How files get picked up (File Pattern watcher)

I order to run files, and a process runs in the background looking in certain directories specified for files that match a specific pattern. If a file is found, it will kick off the process for that folder’s environment. The process runs every scheduled interval (this is configurable) and sweeps the full directory tree from the paths’ root. Please note that the full directory tree can be swept looking for a file (See IsOnPathWalkDirectoryTree setting), so if you add files deep in the directory structure and it is in a folder you want to ignore, you can set certain paths to be ignored (SeePathToExcudeCsv setting) it will get picked up.

Final output locations

Based on the processed file status, the file gets placed in the following subdirectories based on the Environment set by the user.

Explanation of the settings

Below is an explanation of the setting changes that can be made for how files are processed. Please note only the columns below that are bold and underlined are environment-specific. So if you change to a not environment-specific field, the change will get propagated to the other environments (DEV, QA, UAT, and PROD). Some of these options may not be configurable in the user interface but can be set on request for your environment.

File Templates Formates

Autymate uses the PathToWatch and looks for all files matching the file pattern in the directory. It is crucial to set up the patterns correctly so the files will get picked up. Below is a list of how to create a file pattern with some examples.

Rules:

1. Constant File Name must be present unless you use a wild card (i.e., *) for the file name. The file pattern watcher uses the constant name as the match pattern file name. The easiest way to set the constant file name is to use the Square Brackets (I.E., []), but they can also be specified as any string, not in Angle Brackets and not a Date Format Specifier when no Square Brackets are Present. Please note that if you don’t use Square Brackets for the Constant File Name, you need to make sure you don’t use any of the Date Format Specifiers (If you have to, you can use wild cards (I.E., *) in the file name to skip these characters there is an example of this below).
2. Extra information can be set in the staging table by using Custom Date Format Specifiers or using the Angle Brackets (I.E. {}). Be sure to use underscores when you use the Angle Brackets (I.E. {})
3. Using underscores (I.E., _): When you use Angle Brackets (I.E. {}) for customer information in the file name that you want to be staged, you must use underscores to separate the custom fields. Otherwise, there is no way to determine where to split the file name.

[Constant File Name]_{ClientId}_[HISCOM]_MMMddyyyy_hhmmssfff

Constant File Name Qualifier is Square Brackets (I.E., []): This is used to explicitly set the constant file name.

• Extra Information Qualifier is Angle Brackets (i.e. {}): This is used to add extra information that you want to be picked up and put in the staging table (E.G., client name).
• Wild Card Character (I.E., *): This, when used by itself, means load every file no matter what the name is. When used in conjunction with other characters, it means skip these characters. You may want to use this when you know a set number of characters you want the file pattern matcher to ignore.
• Date Format Specifiers: When file names are loaded, Autymate will automatically add extra data to the staging table based on the file naming conventions. For instance, based on what date formats given, Autymate will add one or more of the following fields Xtra_Year,
  Xtra_Quarter, Xtra_Month, and Xtra_Date. This date is typically referred to as the File Load Schedule Type. If the File Load
  Schedule Type has enough information to determine a valid date, and then the Xtra_Date is the only field inserted.

Template Examples:

Here are some examples of file templates and the results.

* (This will pick up any file found)

HOW TO: Use Azure File Explorer

We have set up the Azure File Explorer so you can easily access the UAT File Share. Azure File Explorer is an accessible, intuitive, feature-rich graphical user interface (GUI) for full management of cloud storage resources.

How to set up the Azure File explorer?

1. Click here to download Azure File explorer

2. After installation, follows these steps:

3. Azure home page.

4. Select use a connection string from new popup.
   

5. Place the connection string and click Next. Azure File Share Connection String:
   `The key is provided as a one-time secret (https://onetimesecret.com/). You can only access this link once, so be sure to save it and store it according to your internal security policies so you can share it with other team members. If you lose your key, please feel free to reach out to your account manager, and they can send it to you again.

1. Click on connect button on the next screen

2. After that, click on storage Accounts, autymate{environmentName or client name} file Shares
   

3. Finally, click on autymate{environmentName or client name}-personal, and you will see the new window on the right side of the storage explorer. From there, double click on folder name containing {environmentName or client name}, and there will be 5 environments, and each will have 4 folders (in, out, failed, success).

4. Now, you can drop the files in each of the In Folders for the environments that you want the Auty-mation file pattern matcher to run.

5. Now Click on the environment you want to drop the files, for instance, QA.

6. Drag and drop the files you have set up for that Auty-mation Tile environment and place them in the in folder.