Architecture
Priority
When jobs are added to ScriptQ, a priority level can be assigned to each job. This permits some jobs to "move up the line". There are four priority levels: low, medium, high and urgent.
New jobs are always added behind the last job at the same priority level. The illustration below shows a new "urgent" job being added behind earlier urgent jobs but ahead of a "high" priority job.
Multi-Threading
Running the queue on a single thread ensures that jobs are executed one at a time, in linear sequence. If there are 10 jobs in the queue each taking 1 minute to execute, it will therefore take 10 minutes to clear the queue and the last job in the queue will have to wait 9 minutes before it starts executing.
Multi-threading gives ScriptQ the ability to execute jobs faster by processing them simultaneously. For example, if ScriptQ is running on 2 threads, the same 10 jobs will execute in about half the time (computer resources permitting). Multi-threading greatly increases the throughput of ScriptQ, since it prevents any single large job from blocking the execution of other jobs. The illustration below shows ScriptQ running on 2 threads.
Question: if two threads are better than one, should ScriptQ run on as many threads as possible? Answer: if too many threads are used, the benefits of a queue will be lost since it will try to execute all jobs simultaneously.
Deciding on the correct number of threads to use depends on computing resources available. Powerful servers should use more threads, whereas resource-challenged servers should use less. The exact number of threads to use depends also on the type of jobs in the queue and the computing resources involved. For example, if jobs running on a powerful server are exchanging data with a very busy second server over a slow network connection, then less threads should be used, since the overall computing resources are not capable of sustaining large numbers of jobs being executed simultaneously.
As a rule of thumb, use 2-3 threads for resource-challenged servers and 5-10 threads for high-end servers.
Grouping
ScriptQ sequences the execution of jobs based on priority. When ScriptQ is running jobs of the same priority level on multiple threads, it is still possible that one job will finish executing before jobs that were in front of it in the queue. The diagram below illustrates this point. Job 3 finishes executing before Job 1 because Job 1 takes longer to complete than both Jobs 2 and 3.
In the example above, job 3 executed before job 1 since it was not important to maintain the order of executing jobs within the given priority level. Although this is typically the case, there are situations when order of precedence must be maintained. In these circumstances, jobs can be grouped together. The illustration below for example shows how assigning the same group ID to Job 1 and Job 3 ensures that Job 3 will not start until Job 1 has completed.
Windows Service
ScriptQ runs as a Windows Service as seen in the screenshot below. Jobs will execute in the security context of the Windows Service which can be changed by specifying a different Windows user account.
Persistence
ScriptQ stores the definition of each job in a folder on the file system. If the ScriptQ service is interrupted by a server restart, jobs waiting in the queue are not lost and will start executing when the service is back online.
ScriptQ Folder Structure
ScriptQ uses the following folders to manage jobs:
| Folder | Description |
|---|
| Archive | Completed jobs are moved to the Archive folder when ScriptQ is set up to archive jobs. To enable archiving, set the registry key HKEY_LOCAL_MACHINE\Software\Belus\ScriptQ Engine\Default\Settings\Archive to yes and re-start ScriptQ service. |
| Error | Jobs that have failed are placed in the Error folder. |
| Logs | This folder is the default location for storing log files. An alternate location can be specified via the registry key HKEY_LOCAL_MACHINE\Software\Belus\ScriptQ Engine\Default\Settings\Log File. |
| Pickup | This folder is used to place jobs into the queue. An alternate method is via COM API. |
| Priority | Like the Pickup folder, this folder is used to place jobs into the queue, but jobs in the Priority folder have a higher priority than jobs placed in the "Pickup" folder. |
| Queue | This folder is where jobs are kept while waiting to be executed. |
| Temp | This is a work folder used by the queue. |
| Template | This folder stores job templates. The Instantiate() method uses these templates to instantiate jobs from these templates. An alternate path to this folder can be specified via the registry key HKEY_LOCAL_MACHINE\Software\Belus\ScriptQ Engine\Default\Settings\Template Path. |
| Test | This folder contains scripts that can be used to test various aspects of ScriptQ functionality. |
| Upload | This folder stores files/folders received from the Upload() method. An alternate path to this folder can be specified via the registry key HKEY_LOCAL_MACHINE\Software\Belus\ScriptQ Engine\Default\Settings\Upload Path. |
The screen shot below shows the folder structure.
Job Definitions
Job definitions are stored in XML files. Each file contains information about a single job. The name of the file is the ID of the job (with a .xml extension).