As we briefly covered in Part 1, the Forked Execution option for a Boomi Molecule has powerful advantages but must be tuned very carefully before being used in Production. In Part 2, we will review those tuning options.
Bear in mind that this document is not a sizing and tuning guide. Adjustments to the properties discussed below must be carefully considered and ideally tested in a QA environment first before going live in Production.
Enabling Forked Executions in a Molecule
If Parallel Processing is enabled on your Boomi Account, then this option can be set up by adding the “Execute Processes as Forked JVMs” property under Runtime Management → Properties → Advanced and setting its value to MULTI_PROCESS.
Memory Tuning
Usually, every action Boomi takes will operate out of the heap space allocated to the main Boomi JVM, so if you (for example) allocated 10gb of RAM to Boomi, you can be confident that Boomi will never use more than 10gb of RAM at once.
Forked Executions, however, are not drawn from Boomis allocated memory: they are new JVMs that operate in addition to Boomi’s normal operating overhead.
For this reason, you need to carefully tune two settings found under Runtime Management:
1. Your Runner Heap Size. This is the amount of memory each forked JVM will receive, and defaults to 512 MB. It is found under Forked Execution Properties:
2. Your Maximum Simultaneous Forked Executions Per Node. This controls how many forked JVMs could potentially spawn at once on a given node. It defaults to 50, and can be found under Properties → Advanced Properties:
Multiply them together to calculate the peak potential resource usage that your Forked Executions could take up on a node machine with the current settings.
Further, remember to leave some memory open for the Operating System, the Boomi node itself, and any other workload placed on the server.
Finally, calculate what your heaviest peak workload will look like and adjust these settings accordingly. You want to accommodate your work while being careful not to allow this work to overrun your total available system resources.
Remember: the Maximum Simultaneous Forked Executions Per Node setting defaults to 50. This means that if you don’t change any settings if you make heavy use of multi-processing, and if you have three nodes on your molecule, you could theoretically peak at having 150 forked JVMs running at once. At the default 512mb of Runner Memory each, that could mean an attempt to use 25gb of RAM per node!
Forked Execution Queue Tuning
Fortunately, Boomi has a queue system that will allow a higher number of Forked Execution requests to exist without attempting to run all of them at once. This is controlled by two Advanced Properties under Runtime Management:
1. Maximum Queued Forked Executions per Node. This setting controls how many Forked Execution requests you can have queued up and waiting. It defaults to 200:
2. Timeout for Queued Forked Executions per Node. This setting controls how long you’ll allow a queued-up Forked Execution to wait before getting canceled. The time is given in milliseconds and defaults to 59000 (meaning 59 seconds):
Once you hit your configured maximum number of Forked Executions, further requests will go on the queue to wait their turn. This will allow you to handle periods of exceptionally heavy load without overrunning your available resources.
Thanks to this queue system, you can successfully set your Maximum Simultaneous Forked Execution Settings Per Node property to a lower, safer setting, with less worry about execution requests being lost.
Additional Settings
There are some additional Advanced Properties you can choose to add to further customize your Forked Executions. Namely:
Maximum Forked Execution Time in Cloud: How long you’ll let a Forked Execution run for. By default, it’s limitless. Note that this property can be used in a molecule despite the word “Cloud” in the name.
Maximum Execution Threads per Forked Execution: You can perform multi-thread parallel processing even within a forked execution. This setting determines how many threads you’ll allow a forked JVM to use. By default, this is set to 50.
As you have seen, using Forked Executions on a molecule is a powerful option but one that must be handled with care. It can be very valuable to have expert advice and assistance when implementing and tuning advanced options like this one into your Boomi ecosystem. Reach out to us today for an exploratory conversation with the XTIVIA experts, and see the ways we can help you achieve your integration goals!
Additional Reading
Boomi Guides worth reading:
Help Documentation on Forked Executions
Security Policies for Forked Extensions