Setup
Data Input
Delivery
Troubleshooting
Performance Impacts
Background:
Performance issues were reviewed and programs changed at releases
8.5.3 through 8.7 which includes new routines that allow handling of
super-sized XML files, immediate deletion of archives and other
improvements for huge input file sizes.
- FormTrap is impacted by anti-virus sufficiently to warrant removing anti-virus from key folders, especially in high-performance systems or systems with occasional volume runs (such as statement runs).
- FormTrap output folders are heavily impacted by anti-virus, including the folder used by Windows print spooler to save print-ready files, and folders used by Queues, File tab for storage of archival PDFs.
- Windows Indexing Service (designed to accelerate finding a file) is not used at all by FormTrap Server and causes similar impacts to anti-virus.
- Entire archive removal is a new feature in Version 8.7, along with selection of archive delete intervals.
- FormTrap defaults to single threading via Queues, General tab, Keep replication order which when ticked processes files single-threaded, in arrival sequence. Multiple threads for formatting improve performance, however the impact is documents are not output in arrival sequence. If this is acceptable, untick Keep replication order and use multiple threads.
- For huge runs, the v7 Split function can be run as a filter to avoid a delivery thread.
- Logos are a significant factor (may be 9/10th of the processing time and more where low quality logos are used). For huge runs, logos can be uploaded to the printer and invoked as macros. Involve Support@FormTrap.com for techniques for logo handling.
- From Windows 2012, files advised as completed by Windows may in fact not be available, with Windows (in the absence of ANY other process) still blocking access as the file is considered "held with exclusive access" by some unknown Windows process. A programmed and increasing set of delays each delay doubling the previous one to blocked files is set up to overcome this (version 8.9). Reports of this have been isolated so far to huge files arriving for a banking system.
Windows and Anti-Virus
Anti-virus software will cause FormTrap Server failures through accessing and holding files required by FormTrap Server. The time to process via anti-virus may build to the point a file may not be available when FormTrap Server requires it.
- The process cannot access the file because it is being used by another process means the file is held by another process and cannot be accessed. This typically happens when FormTrap Server intends to delete temporary files or to deliver them.
On huge runs (30,000 documents), anti-virus typically adds 20% to the time taken.
Recommended:
White list the FormTrap %fthome% folder and all
subsidiaries.
White list the top folder accessed via Queues, File
tab and all subsidiaries.
White list the Windows Print Server folder (find by clicking HERE).
Windows Indexing Service keeps track of recent file and is designed to improve performance for persons operating the computer. This not a requirement for FormTrap Server. Remove this facility from all folders used by FormTrap, including destination files used by Queues, File tab.
FormTrap Performance Options
Archive is critical for huge FormTrap runs of over 30,000
documents, please see HERE,
Systems with huge volumes ... for details on how to delete
entire archive for huge runs.
FormTrap has these standard Threads:
- Input threads (3 in total)
File input thread
LPD input thread
Pipe input thread
Where the system does not use a thread type, untick Enable at startup to avoid checking that thread. - Output threads (6 in total)
SMTP (email) thread
MAPI (email) thread
Directory output thread (to Queues, File)
Process output thread (to Queues, Application)
Print thread
Fax thread - Processing threads (in Setup, Settings) where the default is 8. This is the maximum number of threads for formatting and is user-set. Recommended is a minimum of total number of cores or more for a stand-alone FormTrap Server, possibly less for a VM FormTrap Server where other tasks are running in the same computer. The queue must have General tab, Keep in replication order unticked to utilize multiple threads. Emailed output is ideal for multiple threads, particularly statement runs.
Splitting
Segregating the Split function from the Formatting function allows
multi-threading of the formatting stage.
For version 8, the split queue delivers one large XML file to Server
where the Server delivery process splits the XML file into
individual documents. These individual documents are received by the
the Formatting queue where each XML document file is formatted,
multi-threaded (General tab, Keep in replication order
unticked).
For version 7, the file is split and delivered as individual files
to the formatting queue where they are formatted, multi-threaded (General
tab, Keep in replication order unticked).
For version 8, this requires manipulation of parameters in the Queue,
Processing tab as shown below.
The Formatting queue (Delivered To queue) should be built
first, note the Split / resolve ... parameter is Already
normalized xml (resolved).
The Split queue (Delivers To queue) requires removal of the
Form and selection of the above Queue.
Logos and Images
Logos add considerably to output file sizes and to time taken.
Logos that were already scaled and embedded in the document (ie no
resizing took place) added 20% to time taken and 85%
to file sizes on a job of 30,000 documents.
Where resizing is required, time will increase by a factor of 10
times at least.
There are techniques that reduce logo generation considerably, contact Support@FormTrap.com for consulting services.