Changed in version 7.0.1 (April 6 2017) 9d Improved: certain issues pertaining to the design and functioning of the grid usage thread Improved: reduced server stat plugin message size that is sent to the clients to 50,000 jobs to improve performance Fixed: various Hibernate/DAO problems and table configurations Fixed: permissions problems of files uploaded through script editor Fixed: bug on Mac systems that caused the update dialog to keep appearing Fixed: transformation bug preventing double replace operation Fixed: bug sometimes preventing change requests to server library Fixed: job-based sorting bug in the grid stats window Changed in version 7.0 (March 28 2017) 5m ---[ Pipeline Client / Server ] Important: Pipeline now uses Java 1.8 Added: Remote and local file editor feature added to module definition window and Tools menu (Script editor) Added: Job management system by module name (i.e. modules named IDAGet can globally (across all users) run a maximum of 200 jobs) Added: Grid usage plugin and client interface for getting and displaying detailed information about the grid Added: New window in the help menu which summarizes how Pipeline works together with a compute cluster Added: List of program packages available to users on a server is now displayed in the module creation interface Improved: Code base updated to Java 8. Client machines should have latest version of Java installed for optimal performance Improved: Pipeline server and local client databases upgraded to Postgres from HSQLDB. Old client databases will be automatically migrated to new version upon update. Improved: Pipeline server's database access framework updated to Hibernate from JDBC Improved: User quota's calculation now factor in overlapping grid queue resources. Quota will more accurately display to users how many resources are available to them Improved: More reliable checking of module output files existence Improved: Variables can now be used in data sources with the "Number" data type Improved: Efficiency of getting data from IDA within Pipeline. There are now 2 versions of the IDAGet program. The original that expects the imageid-collectionid argument, and a new one that expects a json file with detailed subject information. The latter is used be default when creating an IDA module. The former is available in the workflow library. Improved: Workflow loading statuses are now displayed Improved: Loading times of module information Improved: XNAT can now run locally on both PCs and Macs. It previously worked only for remote downloading Fixed: Frame size and authentication window display size bugs on mac client Fixed: Bug preventing restart failed instances option from working properly Fixed: Bugs related to capital letters in server address Fixed: Bug with active sessions workflow status updating Fixed: Bug preventing a workflow from being added to the library via the "Add" action Fixed: Validation crash when invalid server is defined for a module that uses variables Webapp: Fixed bug so that server status updates properly after disconnect Changed in version 6.4 (Nov 1 2016) 6m ---[ Pipeline Client / Server ] Added: Webapp beta is now available at valve.ini.usc.edu/pipeline/ Added: Workflows that fail to validate are now editable. Ignore warnings button will become unavailable when the workflow is modified Added: Feature for users to customize server library. Requests to add, rename, update, change package, or delete workflows are submitted and approved/denied by admins Added: New icons for connection button, reset/stop/play/pause buttons, and new workflow and load workflow buttons Added: Colors in the parameters tab of modules to indicate inputs (blue) and outputs (red) Added: Error and warning colors to parameters and connection lines to indicate any validation problems Added: Server address displayed in the active sessions window when user has more than 1 connection or local sessions Added: API for requesting session and job statuses Improved: Method of querying database for invalid sessions to avoid database lockups Improved: Multiple clients open at the same time will sync their connections list Improved: XML view for workflows is now a toggle. You can view the xml while a workflow is executing reconnect to the session when switched back to workflow Improved: During validation, if the package mapped path doesn't exist, the user entered value will be used Improved: Validation errors box automatically sorts by module. Module groups, modules, sources, and sinks, can be right clicked to "Show Errors" pertaining to them Improved: Users can now use shell functions in their modules without grid engine array jobs enabled Improved: Study modules now saves user inputted entries during the module creation process Improved: The way the authentication process is displayed is more informative and clear Changed: Processes will now explicitly run from users home directory Changed: Study modules are now called "Data sources with Metadata" Changed: Max cardinality increased from 3,000 to 10,000. This is the number of total commands generated by a workflow. Changed: Transformation replace function will now replace parts of entire path including the base Changed: Validation Warnings feature enabled by default for local execution (ignorable validation errors) Changed: Dropbox account linking updated and NDAR authentication protocol updated Changed: Various popup menus have been organized and simplified Fixed: Bug where metadata wasn't being copied properly in many circumstances related to periods in file/directory names Fixed: Download file window was not displaying transfer speeds and status accurately Fixed: Server changer tool was not changing sources and sinks properly Fixed: "Copy with input" and "Copy output" were not sorting their values so that they match original order and also any paired metadata files Fixed: Various problems viewing metadata files with strict permissions set or when using relative (package mapped) paths when you have no connections Fixed: Bug where session will not appear in active sessions if you reconnect to a session, reset it and run it again Fixed: Module execution hanging issue on certain servers when a user connects to the server using it's public ip which differs from it's private ip Fixed: Issue where a module connected to a data sink that has no slashes "/" in it's path will not execute Fixed: Execution status of rotated modules not repainting properly Fixed: Module group status was not taking into account contained modules in the waiting state, would show complete despite waiting modules Fixed: Problem related to execution statuses when restarting a module connected to a module group who's contained modules are dependant on restarted module Fixed: Instance statuses in the execution info dialog were not updating properly Fixed: Bugs preventing users from closing the execution info dialog Fixed: Status of files not updated if you check them in the output files tab before they are in their final state Fixed: paths using global variables $username and $tempdir showed null Fixed: visual bugs when switching between the different toggles (i.e. Import data mode, merged/separated views) Changed in version 6.3 (April 21 2016) ---[ Pipeline Client / Server ] Added: LRUCache to decrease number of DB queries and improve overall server’s performance. Updated: IDA communication library, older versions of Pipeline won’t be able to communicate with the new version of IDA server. Fixed: Remote file browser to support filenames with spaces. Fixed: Issue when file viewer would not open if another file from the same module was downloaded before viewing the current file. Fixed: issue when data sinks were shown as incomplete, while they were already complete. Fixed: double slash appearing in output/error streams paths. Fixed: issue when some modules after conditional modules would get cancelled after module restart. Fixed: to properly transfer output parameter arguments of conditionals to child modules. Fixed: End time of conditional's instance was not properly reported. Fixed: issue with some conditional modules with metadata not being properly marked as complete. Fixed: issue related to restarting modules with data extract feature. Fixed: issue related to execution of local modules with spaces in their path. Fixed: When workflow gets modified outside of pipeline and it is in "Executing" state, reloading option won't be offered. Fixed: Removing exported parameters from Transformer modules will now properly remove them from parent module group. Fixed: When delete connections between conditional modules and modules with metadata, conditional's true/false parameters won't shift around anymore. Fixed: to not show "Executing Workflow 100% done" in GUI Client after a module gets restarted. Fixed: avoid processing messages in a new thread for CLI. Fixed: to not show reset confirmation box when workflow has validation errors. Fixed: data extract bug, which was missing some values of its instances. Changed in version 6.2.1 ---[ Pipeline Client / Server ] * Fixed: bug preventing removal of session from Active Sessions window * Fixed: conditionals with multiple parameters and multiple instances would hang forever. * Fixed: issue when reconnecting to backlogged workflow would show it as "waiting" * Fixed: issue which was preventing deletion of some invalid sessions upon server startup. * Fixed: issue when short taking file transfers to data sink would fail * Fixed: to properly show data sink statuses * Fixed: issue with conditionals which are connected to module groups. * Fixed: issue when instances would stay backlogged even after module restart * Fixed: issue related to IDA key file generation on servers where privilege escalation is disabled. * Fixed: issue when study module groups would fail to properly update * Added: a new logic to conditionals: When True and False parameters are connected to same parameter, parameter arguments will be delivered in order. Changed in version 6.2 (Nov 3 2015) ---[ Pipeline Client / Server ] * Added: An advanced option to Modules which will always force an outer product multiplication (even when all parameters have the same amount of data). * Added: Session ID field in Execution Log dialog * Added: A confirmation box for Reset Workflow action to prevent accidental removal of workflows. * Fixed: Infinite waiting issue in local execution. * Fixed: Command line and Client to properly show the workflow completeness percentage. * Fixed: An issue which was showing non-existing files as Pending in data sinks. * Fixed: to create only 1 symlink when modules have connections to multiple data sinks. * Fixed: Issues when sometimes some workflows stopped updating or some instances failed to initialize. * Fixed: Metadata augmentation to properly use user-specified values. * Fixed: Bugs related to downloading and showing metadata logs. * Fixed: issue when Pipeline was not properly post validating files with prefixes. [ Server Terminal ] * Improved: Speed of workflow list retrieval * Fixed: Issue when some Paused workflows would show up as Running Changed in version 6.1.1 ---[ Pipeline Client / Server ] * Redesigned execution events structure. Redesigned Execution Graph Controller to support multiple graphs. * Improved: Restart module speed. * Improved: Status updates * Fixed: Issue with restart modules which would hang the workflow for indefinite time * Fixed: Incorrect user management limit calculation for users within a defined group. * Fixed: The wrong name for Session History table. * Fixed: Package mapping issue so when a package has a specific version as well as a * version, it will only use the specific version of the package mapping. Changed in version 6.1 (August 11 2015) ---[ Pipeline Client / Server ] * Added: Transformer modules. A standalone module that allows complex file path transformations with simple user interface. * Added: Support for Windows 10 * Added: user group management. System admins can create custom user groups and apply quota for the users in the group as well as total quota for the group. Quota can be a percentage, a hard limit, or a fixed/reserved hard limit. * Added: unknown cardinality for string extract of output parameters. If enabled (match all occurrences), data extract will pass all matched items and the next module will have n number of instances. * Added: output files information dialog for data sinks. Similar to module execution dialog's output tab, it shows list of output files and their paths, and allows user to download/view the files. * Added: support for multiple status lines (instead of just 2) for Modules. * Fixed: when "show parameter names" is enabled, adding new parameter will have its name shown. * Fixed: for thumbnail generator to allow multiple files to be processed with 1 process, and for slicer to process the first output files instead of all files. These will improve module's post processing time. * Fixed: for restarting module with data extract, the correct cardinality for the next modules will be calculated. * Fixed: problem which would cause some modules to not have some of their arguments places in the command if they are places after conditionals or any module which changed their cardinalities. * Fixed: to only reset statuses only of modules which are children of module group receiving the event. * Fixed: to properly shutdown any child controllers when workflows stop or reset. * Fixed: for server manager's combined settings message not to do busy loop, null value settings won't be sent to client. * Fixed: bug which was changing transformation's parameter when selecting a transformation from the table. * Fixed: bugs when use study module's group as output parameter option. * Fixed: for IDA dialog when user create IDAGet module with data source, it will include subject id that's parsed from metadata. * Fixed: shortcut key conflicts for transformers and image annotations. * Fixed: for data sink output dialog to not show pending next to file names until it's confirmed pending. Also fixed to have correct parent for error dialog. * Fixed: for module's execution log dialog to not run things (like updates on labels and buttons) every 1 second when it is invisible. * Fixed: for IDA dialog when user selects file type but it's not defined (in userdata), the filetype will be created. * Fixed: for data sink output status dialog to open viewer when user double-clicks on an item. Also fixed dialog title. * Fixed: for nfs file check for non existing files or no permission files, to check recursively its parent directory. * Fixed: to clone data module's general metadata fields. * Fixed: to have "show in finder" right click menu option for downloaded output files. * Fixed: for ValidationAllowedPaths/NFS output validation check to only check files from required output parameters. * Fixed: for edit study module's matrix tab to have predefined set of columns for IDA studies, also fixed to show the last element name as matrix title (instead of full path). * Fixed: to have an environment variable for all modules submitted by guest users, with name "pl_guest_username" and the value is the guest username. * Fixed: for IDA to correctly save guest users credential file. Also fixed to check if the directory ~/.pipeline exists before running mkdir. * Fixed: to correctly identify windows 8.1 as windows 8, this fixes updater problem with win 8.1 os. * Fixed: for server to apply commands of thumbnail generator and data view (slicer) separately, as they have different sets of supported file formats. * Fixed: to automatically delete directory with port name only from libraryCache to prevent showing outdated modules and workflows in server library. * Fixed: bug which was creating a directory with port name only in library cache (but should be hostname and port). This was causing some libraries to appear outdated. * Fixed: for restart module/instance to remove output and error streams and output files of previous run. * Fixed: server preferences option to automatically check if a file is in NFS (with stat command). This option can be used in combination with ValidationAllowedPaths (renamed from NFSValidationPaths). ---[ Server Terminal ] * Added: interface for creating and editing users and quota for user group management. * Fixed: to correctly display name and selected index for server terminal launched from GUI menu. Changed in version 6.0.1 ——[ Pipeline Server ] * Added: a cache file of server library for fast access on server start up. The cache file will be regenerated/synched when server library changes. * Added: NFS file validation. If enabled (under preference tag NFSValidationPaths), any file/execution path that is not under NFS directories will cause a validation error. * Added: output file check on server (requires array jobs/submit scripts). When enabled, server will put checks on output files (in submit script) for executables which exited normally, and module will fail if output file is not found. * Fixed: SQL syntax error which was giving wrong error about database format being old. * Fixed: to have session's reset time in db when a workflow is reset. * Fixed: for crawler db's description field to be longvarchar. * Fixed: bug which was preventing restart of individual instances. * Fixed: to allow {$username} and {$tempdir} in package mapping environment variables and source scripts. * Changed: crawler db url to be the same as PL server's db url, instead of a separate db. Added preference tag EnableCrawler to allow enable/disable crawler. ---[ Pipeline Client ] * Added: for string extract the option to skip a number of lines from the top (also allows to not count empty lines). * Fixed: cloud vendor Box's authentication and other functions to their latest 2.0 version. * Fixed: to not set Outputs of node when cardinality is Zero. * Fixed: for edit study module dialog's grouping panel and matrix panel to properly edit and save. * Fixed: "Restart Failed Instances" menu option now properly shows and functions on modules with failed instances. * Fixed: bug when loading data module with inconsistent serverAddress attribute (compared to server in its own values). * Fixed: exceptions when remove metadata on study modules with string/number parameter types. * Fixed: bug on making connections with shift key pressed, connection endpoint will not jump randomly. * Fixed: when client lost connection with open active workflows, reconnecting to the server by click lower right server dropdown will properly update the status of current workflow. * Fixed: bug when open and close some active workflows, while still leave some open, disconnect and reconnect to the server will properly update the statuses of opened active workflows. * Fixed: when multiple CLI and GUI clients are connected to a workflow, reset will properly update the status of all connected clients. * Fixed: for XNAT to use the latest XDC/XnatDataClient to authenticate and download files. * Fixed: for module's output and error stream panels not to auto refresh on instance updates. This fixed constant update/refresh issue with modules that had a lot of instances. * Fixed: when right-click completed modules and select output/error stream or output files, it will correctly open execution dialog with corrected tab selected. * Fixed: when copy modules, special variables like $username and $tempdir will not be copied. * Fixed: the bug which was causing local executions to never finish. * Fixed: to automatically optimize filetypes. No duplicates will be allowed and extensions which start with dot will be fixed (for example extension declared as .img will be converted to img). * Fixed: minor bugs for data extract for string output parameters. * Changed: to send and receive 1 combined message when first connect to server instead of several separate messages. * Changed: for server terminal menu item on GUI client to hide by default and only be visible if user connects to a server and the user is an admin. ---[ Command Line Interface ] * Changed: to send and receive 1 combined message when first connect to server instead of several separate messages. * Fixed: for status option to show correct % progress. ---[ Server Terminal ] * Added: Fields in Preferences tab to allow configuring the new features like NFS file validation and Output File Check. * Fixed: when saving preferences on tabs other than Access, mapped username will not be removed. ---[ Pipeline Web Start ] * Fixed: to correctly open workflows when they are passed as argument with -url option. * Fixed: bug for parameter arguments to allow "=" in url (for example page.php?name=value will be working properly). Changed in version 6.0 ——[ Pipeline Server ] * Redesigned: The database schema, this means that databases used for previous versions won’t work with this version. It needs a fresh instance of DB. * Redesigned: Job submission thread pooling, execution management and user management * Redesigned: Module execution and workflow status events * Added: Directory Access Restrictions feature. This feature will allow administrators to set multiple directory access control with more options and should replace the DACMode feature. * Fixed: to clear old (last modified >30 days) client's log files and client's temp persistent db files (the ones with long timestamps). * Fixed: IDA to automatically renew key (if available) when user submits a workflow with IDAGet. ---[ Pipeline Client ] * Added: "add metadata" and "remove metadata" buttons. They provide functionality of deprecated Study modules. * Added: to show node errors for modules. * Added: for data extract/string output parameter the option to extract the entire output/error stream. * Added: Auto select file type button in Data Module * Added: access to Server Terminal from Tools menu * Fixed: for edit and add file type to strip leading dot on extension and needs (e.g. if user gives .img, it will save as img). * Fixed: for making connections with shift key to overwrite smartline settings, pressing shift key while holing the mouse will enable/disable smartline. * Fixed: to properly show status of module groups which contain cancelled modules. * Fixed: for validation to catch error when module has output string/number parameter that contains values (i.e. not connected to another module). * Fixed: bug which was causing some module groups to show as complete instead of waiting. * Fixed: to report an error and stop workflow submission in case there is an error during session initialization. * Fixed: not to set information about module outputs when the list is empty. * Fixed: to properly cancel modules for which all needed parents have been failed or cancelled. * Fixed: to cancel nodes for which all parents are either cancelled or failed. * Fixed: concurrency issues when multiple nodes were started together. * Fixed: a few ungroup bugs, including exceptions that could hang the ungroup action, and incorrect connections after ungroup. * Fixed: to transfer output arguments into child modules after the translation is done. This fixes the problem of some data sinks which instead of linking would try to transfer files. * Fixed: for smartline not to have memory value specified by default. * Fixed: on GUI validation with missing input, when user clicks "Ignore Warnings", module's package mapping will work as usual. * Fixed: for previewer to correctly show black blank images instead of showing Not Available error. * Fixed: for the hidden metadata output parameter to be checked at workflow loading time, in addition to connection line create/delete time. * Fixed: for workflows with study module, when a connection is removed, it checks and removes the hidden output metadata parameters recursively. * Fixed: for workflows with incomplete modules to show "errors found" instead of "completed" workflow status message, it will have orange (same as incomplete module) color. * Fixed: on GUI validation with missing input, when user clicks "Ignore Warnings", module's package mapping will work as usual. * Fixed: When server library gets corrupted for some reason, clear library cache button will correctly reset/remove existing server library cache. * Fixed: for client logging file to append instead of create new for every start. * Fixed: to only get URLs from valid URLs (not strings or numbers) when using server changer. * Fixed: bug when exported parameters are removed, the parent module group's parameter nodes will be updated. * Fixed: to cache the user permission which is used for restricted resources. * Fixed: to report error when node tries to submit more jobs than allowed * Fixed: for copy with input issue with data extraction/string output parameters. * Fixed: server exceptions when sending pre-delete email on sessions that are paused for x days. * Fixed: to show error when user runs an empty workflow, or workflow contains only data source, study, cloud source modules. * Fixed: to check multiple User type rules if there are any before restricting the user. ---[ Command Line Interface ] * Added: the options to list all active sessions, show status of a given session, and stop/reset/pause/resume a given session. * Added: the option to ignore validation warnings to submit workflows (warnings will be printed). * Fixed: to show workflows that are completed with error the correct status (instead of failed). * Fixed: not to require preferences.xml in user's home, also fixed to remove persistence*.tmp file in user's home. * Fixed: the ability to assign values to data sources and sinks. It uses "parameter_id=value" format. * Fixed: if there's parameters value assignments, the value(s) will replace the existing ones from .pipe file. Before this commit it appends the values to the existing ones. * Fixed: validation error message when using predefined variables ($tempdir and $username). * Fixed: manage options to support short session IDs in addition to complete session IDs. * Fixed: to show workflow status as complete for workflows with canceled instances (due to conditional module) but no errored ones. ---[ Server Terminal ] * Added: synchronization/update checking mechanism. When user saves preferences that are outdated (saved by another session), it will inform either overwrite or discard/refresh options. * Added: multiple selection support, which allows bulk reset/stop/pause/resume workflows. Also added highlight for long running workflows when user clicks on the label. * Added: for server terminal the option to show customized warning message when there's a server update. * Fixed: to hide "Retrieving data" label after data received from server. * Fixed: not show interface when not connected to server and added a message to users about sudden disconnection (happens when regular - non admin user tries to connect to server). * Fixed: for server terminal reset workflow button to update total counts (number of workflows, running jobs, etc). * Fixed: not append unnecessary comma when guests, shell or noshell are used with other usernames. * Fixed: reset and pause/resume buttons not to continuously call refresh after button action. Removed stop action and replaced with reset action. * Fixed: to have correct total instances count for running workflows. ---[ Pipeline Web Start ] * Added: CSV Metadata URL as argument. This will allow web start launching with import CSV file as data module with metadata. Changed in version 5.10 ---[ Pipeline Client / Server ] * Added: support so when modules are connected to multiple data sinks, links will be properly created. * Added: a log to show not created symlinks. * Added: an option for module to have a separate temporary output directory for each module instance. * Added: for print workflow the zoom slider so that user can select zoom on that dialog window. * Added: defined env. variable values to submit script. * Changed: the design of grid event listener and its handlers. There will be a variable number of threads which will change dynamically depending on number of events stored in listener's queue. Instead of processing events in bulk, all the active threads will use one queue and process events in order they arrive. * Fixed: to enable text view option for .list files in output panel. * Fixed: issue with some of the workflows with data sinks were not marked as Finished but all the modules were complete. * Fixed: bug for copy output option on completed modules. * Fixed: sql exception for some workflow with a long filename. * Fixed: a few error handling and optimization with data extraction. * Fixed: bug for data extraction on output string/number parameters to remove data extraction rules when parameter type changes, also added a Clear button to remove rules. * Fixed: wrong user usage status appearing in server stats. * Fixed: to not send huge log files to Execution Log dialog. * Fixed: to not include unnecessary counts when admins are requesting all sessions from GUI Manager's active sessions dialog. * Fixed: bug when some jobs finish but not marked as finished. * Fixed: multiple issues with crawler that caused exceptions. * Fixed: bug on Amazon S3 file browser to properly show directory contents. * Fixed: to add permissions tag in the jar manifest to avoid issues with Java 7 Update 51. * Fixed: to ignore repetitive events for non active jobs. * Fixed: transformation to interpret variables (e.g. {var}). * Fixed: a few bug on Amazon S3 cloud source/sink modules: sink with directory type will work properly, also the browser will not refresh to home directory if userdata has previous directory. * Fixed: automatic file type matching for .nii.gz files on data modules, also fixed a bug when file type is called "FILE" while extension is not empty. * Fixed: to properly handle Job deletion event, especially when the task ID is "0". * Fixed: to not terminate jobs immediately during the DB with plugin synchronization, but instead terminate after waiting a little more and verifying their non existence. * Fixed: a few bugs for cloud storage module (amazon s3), now s3 url (like https://s3.amazonaws.com/my_bucket/01.xml) will also work as file path. * Fixed: for web start to allow non-guest username to be added (passed by command line argument) in the connections. * Fixed: URL for successfully linking cloud storage accounts. * Fixed: bug that output files of instances with instance number > 1 sometimes were incorrectly assigned. * Fixed: for try it now with both guest and regular usernames, it will automatically login as guest. When launching with regular PL client, and running a workflow, the credential dialog window will have an optional button "connect as guest". * Fixed: to allow arguments to set certain web start properties for GUI, to deal with issues in Java7u45 * Fixed: to have a wrapper script on server's thumbnail generator, which will always return 0 (success) to prevent its JVM failing the whole module. * Fixed: to not to have h_vmem set up by default when creating IDA/NDAR/XNATGet modules. * Fixed: for print workflow option to be able to print canvas at current zoom scale, so that user can create high resolution images for printing/publications. * Fixed: compilation problem appeared after Java update. * Fixed: bug when editing module that connects to smartlines, sometimes edit dialog shows the smartline module. * Fixed: for network required modules, when server preferences do not have network queue specified, it will submit these jobs to the main queue. * Fixed: smartline memory allocation bug, now the memory headroom (v_hmem minus JVM xmx) is 3GB, also made server automatically calculates and fix memory allocation on smartline modules. * Fixed: a user management bug. Avoid multiplication of available quota. * Fixed: to not show 100% busy when user is not using anything but slots are available. * Fixed: to allow transformation on string and number parameters. ---[ Command Line Interface ] * Fixed: for command line validation error on invalid URLs to include module name, parameter and invalid URL in the error message. * Fixed: bugs on command line interface: for local workflows, credentials won't be needed. For remote workflows, if no credentials file is specified, or the credentials file doesn't contain the entry of that server, it will ask username and password from user's input. ---[ Server Terminal ] * Added: a new access control mode, for users with shell, it will be the same as Mode 7, not restrict; for users without shell, it will be the same as Mode 4. * Fixed: to replace spaces with undesrcored in package names when including the package name in grid native specification. * Fixed: bug on package mapping of local modules. Package mapping and environmental variables can be defined in local userdata.xml file. * Fixed: to have a GridEnabled tag on preferences.xml, which indicates if a server has grid enabled. * Fixed: to disable fields when grid is disabled under server preferences. Changed in version 5.9.1 (105.7.336) ---[ Pipeline Client / Server ] * Added: Access control for guests, it can be specified as {guests} in server preferences and terminal. * Fixed: User management bugs related to comma separated queue enumeration in GridSubmissionQueue property. * Fixed: To have the option to have all data formats in NDARGet module. * Fixed: To report usage of non used queues. * Fixed: To not put jobId as a terminating signal when job finishes. * Fixed: Bug for transformation with parameter based on itself, the transformation will correctly apply. * Fixed: Problem with some ISPs reporting the false USC move. Changed in version 5.9 (105.7.315) ---[ Pipeline Client / Server ] * Added: User Management for guests and separate queues. * Added: Metadata downloading for cloud modules * Fixed: Bug for command line to avoid duplicate transformations. * Fixed: Module restart speed for modules with many instances. * Removed: Grammar View feature, please use Workflow Miner instead. * Other bug fixes Changed in version 5.8 (105.7.276) ---[ Pipeline Client / Server ] * Added: Cloud Storage Module for Amazon S3, Box and Dropbox. The module downloads and uploads files to and from cloud storage services and the Pipeline server, without going to local machine. It lets user link their cloud accounts, browse and select files. * Added: NDAR database support. Similar to IDA/XNAT, it creates a downloading module (NDARGet) and downloads input data on server. * Added: The ability to have multiple clients (GUI and CLI) connect to the same session (with the same credential). * Added: Session (read only) sharing feature for guests. It's a server preference, and requires server admin to specify sharing username. Once enabled, guests can see all workflows by this user, but cannot modify (stop/pause/restart) them. * Changed: Server to run less SQL queries wherever found; optimized number of DB queries to improve speed of server restart and module translation. * Fixed: Java Swing issue with latest Mac update. It made Pipeline client hang at startup. * Fixed: When guest user runs workflows with metadata, metadata files will be properly copied. * Fixed: When LONI Viewer (or any local module) completes with with remote modules, the output files for those remote modules no longer disappears. * Fixed: To send status update when workflow is stopped, this update will be useful when multiple clients are connecting to the same workflow session. * Fixed: To check for expired database connections and replace them with new ones. * Fixed: Bug that caused transformation to be applied twice. * Fixed: To deregister/remove status update subscription from server for a session when user closes the workflow tab. * Fixed: For xml view mode of workflow to preserve viewport position when switching between xml mode and gui mode. * Fixed: To auto select study module import rules based on file path. * Fixed: For study module import from csv file, metadata will be properly created with guest usernames. * Fixed: To properly reset instance status when resubmitting jobs. * Fixed: Bug that was causing transformation not to be applied. * Fixed: Validation bug when {tempdir} variable is used on data sources of type string. * Fixed: To have server's command line file check to check user's group only when it's needed. * Fixed: For data sink with metadata files to have link instead of hard copy. Also for data sink to maintain a list of already linked files. * Fixed: Server bug when restarting modules, module's output files will be removed. * Fixed: Server bug where metadata generator was incorrectly added to command line. * Fixed: To save workflow xml data to file instead of db when workflows are reset. * Fixed: Server to check only for directories when cleaning up the server. * Fixed: Server bug to improve server restart restoration speed. * Fixed: Metadata augmentation when the module has cross products and it uses value from input parameters. * Fixed: Server to remove not only directories, but also files that are old in the temp file location. ---[ Command Line Interface ] * Fixed: to report missing credentials file, also updated help string. ---[ Server Terminal ] * Added: Server library testing instructions under server terminals, it provides commands to launch CLI to validate and execute workflows in server library. * Fixed: Server to not retrieve all sessions from database when administrator requests running sessions only. Changed in version 5.7 (105.7.153) ---[ Pipeline Client / Server ] * Added: "Restart failed instances" option when right-clicking modules with failed instances, it restarts all failed instances of the module. * Added: For Command line interface to show workflow status (error/success) when -sync option is enabled. * Added: Windows 8 support. * Added: Multi threaded grid job event listener. * Added: "Enable Monitored Job Verification" option for convenience. * Added: Ability to configure path to ThumbnailGenerator.jar from the server terminal GUI - this is generalized to encompass all tools that will be stored as "Pipeline utilities". * Changed: Stop button to be reset button from Pipeline control bar. * Changed: Server configuration to write PipelineUtilitiesPath to preferences.xml instead of userdata.xml and added changes aimed at fixing backwards incompatibility issues. * Changed: The way h_vmem is calculated for smartline modules to make sure that h_vmem is 2GB greater than the JVM heap size allocated using the -Xmx parameter. * Changed: Methods in URL class to report the details of an InvalidURLException instead of remaining silent and causing calling methods to throw vague exceptions. * Fixed: Bug related to handling finished events when server restarts. * Fixed: Server Sweeper to properly remove invalid sessions. * Fixed: Fixed bug in "Save as makefile/bash script" feature. * Fixed: Execution and validation will stop when user clicks the stop/reset button. * Fixed: GUI bug affecting annotation buttons on Mac. * Fixed: Initialized, backlogged, queued, and submitting module/instance can be restarted. * Fixed: To save module icons in PNG format instead of JPG for better support of transparent images. * Fixed: Fixed bug for modules require external network access and mpi, server * Fixed: Bug in redirection of standard output during local execution. * Fixed: Bug in {$tempdir} translation. * Fixed: Bug to keep sessions' temp directory if the workflow is over two times DaysToPersistStatus and it's still running. * Fixed: Module/instance restart bug, cancelled module became incomplete after restart. Now cancelled module will ignore empty updates. * Fixed: Module/instance restart, when there's a pending request, module/instance restart is disabled until a response from server is received. * Fixed: Bug when modules with smartline are copied, smartline will not be copied twice. * Fixed: Bug for smartline input parameter to be required, so that cancelled modules before smartline will make smartline module cancelled as well. * Fixed: Bug for copy with input and copy output features to disable until a pending request is done, to avoid confusion when the server is slow. * Fixed: Data sink bug: when module is remote and data sink is local with permission errors, it will properly show error message. * Fixed: Not to show total instances when data sink contains errors. * Fixed: Not to call Datastore/library when launching CLI. * Fixed: Validation exception when input is string/number type. * Fixed: Race condition that sometimes caused server not to start. * Fixed: To re-create broken database connections. * Fixed: Smartline issue with whitespace in package names of the child module. * Fixed: A CLI bug when credential was missing for submitted workflow, it caused exceptions. * Fixed: Backwards incompatibility bug and changed server configuration to write PipelineUtilitiesPath to preferences.xml instead of userdata.xml * Fixed: A bug which was leaving one executable running after the workflow was stopped. * Fixed: Backwards incompatibility bug. * Fixed: A bug in the GUI that was drawing icons on data modules when the user copied an icon to multiple modules. * Fixed: The menu options that pop up when right-clicking on a module or group of modules to exclude the grouping option for modules in active workflows, to consistently order the options, and to include the copy icon option for modules in active workflows. * Fixed: FileMonitor listener to not update the workflow with the contents of the pipe file when the file has been deleted or moved; previously, this was bug was causing the workflow to disappear upon moving or deleting the associate pipe file. * Fixed: Workflow scratch directory to be a server-specific preference for client users and improved the saving of preferences from the server terminal GUI to write changes directly to local copy of userdata.xml in addition to sending the changes to the server. * Fixed: To properly reset invalid sessions. * Fixed: To query for bulk jobs info from plugin with a limited number of jobs to avoid out of memory problem. * Fixed: To resubmit the jobs which were failed because of file stat command timeout even when getting finished job info from database or after server restart. * Fixed: To not print error from kill command in NFS Validation script. * Fixed: IDA bug when using guest accounts, metadata will be properly downloaded. * Fixed: Infinite recursion stack overflow bug that was caused by the absence of smartline parameter IDs when smartlines are drawn. * Fixed: Bug in MetadataGenerator by catching NullPointerException; the cause of the exception has not yet been determined but the failure of this thread should no longer affect the processing of the complete workflow. * Fixed: Bug that caused the transformations of an output parameter to get passed onto a connected input parameter, if the input parameter does not, itself, have any transformations. * Fixed: Transformation bug which was causing a transformation rule to be passed from an output parameter to the input parameters of downstream modules to which it was connected. * Fixed: Submit script to not wait for 10 minutes after the execution is complete. * Fixed: To not set number of total slots to Zero when total slots cmd fails. Instead keep trying to run the command. * Fixed: To avoid providing more slots than allowed to parallel started modules. * Fixed: Module statuses to not incorrectly display "Validation Passed" briefly before showing "Validation Error". * Fixed: Grid plugin version 3.0 update changed job info retrieval methods and "Killed Jobs" statuses were not updated. * Fixed: The Submission script to handle IFS Mount problems during the execution. The script will run in parallel with the actual command and will check if all files are accessible. * Fixed: To Resubmit jobs which are not found in plugin during Plugin with DB synchronization. * Fixed: Arithmetic Exception which appears when server is brand new installed. * Fixed: Bug that was preventing user from connecting to an active session that had both local (e.g., LONI viewer) and remote modules. * Fixed: Bug when crawler database is hanging, the server will not be affected (with timeout exception). * Fixed: To keep workflow title's asterisk when user submits a workflow that's modified. * Fixed: To create directory with right permissions on server update. * Fixed: For special characters in parameter transformation not be interpreted as regular expression. * Fixed: The bug in Windows which was showing "Checking for updates" message in about box but was doing nothing. ---[ Server Terminal ] * Added: Message to Users functionality which allows server admins to send message to ALL users to any connected single or multiple users. Message recipients have ability to send a reply message, but the delivery is not guaranteed if the sender is no longer connected to server. Changed in version 5.6 (105.7.4706) ---[ Pipeline Client / Server ] * Added: Validation errors UI to use balloon tips and to allow users to ignore warnings and proceed with execution in the absence of fatal errors. * Added: Support for multiple queue submission. * Added: Server feature which will detect files which are inaccessible from some of the cluster nodes and resubmit the job several times. * Added: A server configuration option which allows administrator to label missing executable and/or input parameter files as ignorable warnings, instead of validation errors. * Added: An option to send an email to server admins when free space of temp directory or home directory of Pipeline server user goes lower than the set number. * Added: Feature which will send more details for each command line argument when submitting the job via the plugin ( non-array jobs ). * Added: Update checker in About Box. It will check for available updates and provide a button for updating the Pipeline software. * Improved: Server library retrieval method with much faster initial loading time. * Improved: The speed of server restart when there are backlogged workflows * Improved: The way module icons are written to Pipe files; now, we use a list of icons and references to these icons in order to eliminate redundancy of information and optimize the size of the Pipe file. * Fixed: When module with flow control output has multiple instances, the child module that connects by the flow control did not have correct cross product arguments. * Fixed: Bug in the Pipeline that was causing a workflow that failed validation to erroneously start running upon disconnecting from and reconnecting to the server. * Fixed: Validation to catch and report any transformations that are not from a file or directory parameter to a file or directory parameter. This includes string to file, number to file, string to directory, and so on. * Fixed: Bug in Running module animation. Some of the arcs were not properly highlighted. * Fixed: Bug which was showing 0 second duration for some of the Module groups. * Fixed: Bug when setting up server on Amazon EC2, jobs failed to run on server because modules were not recognized as local. * Fixed: Bug where the circle of an exported parameter is no longer filled after deleting the smartline connected to it. * Fixed: Bug which was causing the ${username} variable to not be translated. * Fixed: Bug for metadata/study: transformed parameter with metadata will work properly to generate metadata files. Changed in build 4550 ( v5.5 ) ---[ Pipeline Client / Server ] * Added: New Auto extraction mode feature in Metadata augmentation feature. * Added: Support for Retina display MacBook Pros. * Improved: User management feature after server restart and its error correction algorithm. * Fixed: Some bugs in Usage charts. * Fixed: Some indexes in persistence database to improve performance of both server and client. * Fixed: The updater bug which was sometimes failing to launch the installer. * Fixed: Some statistical bug fixes related to Cancelled instances. * Fixed: To properly sort instances in Execution logs dialog. * Fixed: Bug in command line interface to get and update package and executable mappings. * Fixed: To prevent remote Windows URLs in output files. This was a reason for some workflows to fail. * Fixed: Bug in module suggest feature related to special characters in module definitions. * Fixed: Bugs in enumeration, study module definition dialogs and text & image annotations. * Fixed: To clean old Paused sessions. * Fixed: Bugs related to DRMAA Plugin events. Changed in build 4413 ( v5.4.3 ) ---[ Pipeline Client / Server ] * Fixed: Bug appeared in 5.4.2 which was related to file type formats. Changed in build 4411 ( v5.4.2 ) ---[ Pipeline Client / Server ] * Added: Convenient feature for data sources and sinks to auto select file types based on the values in the text area. * Added: A new server preference to set Job Retrieval method from the grid plugin. Preference allows to select how jobs will be polled from the server "One by One" or "by List". * Fixed: Study modules to support package mapping. * Fixed: Some bugs in workflow crawler. * Fixed: Bug with data sinks with need files. The total size was sometimes incorrectly calculated which resulted showing wrong completeness percentage. * Fixed: Other minor bugs. Changed in build 4333 ( v5.4.1 ) ---[ Pipeline Client / Server ] * Fixed: bug for Previewer/ThumbnailGenerator: sometimes thumbnails were not properly generated. * Fixed: open link in a browser issue on some Linux machines. * Fixed: Bugs in paused workflow timing. * Fixed: for server library search's server filter not to change selection while user types. * Fixed: bug on XNAT dialog about output file format: Analyze and Any now work properly. * Fixed: bug study module's groups are now properly saved. * Fixed: Null pointer exception at startup with locally corrupted server library. * Fixed: For module suggest window to be a simple jdialog, fixed a few exceptions and stuck window when opening multiple workflows, made it movable, also made double-click action same as confirm button. * Fixed: Bug for connection creation with try it now, it will not replace existing username with guest username but instead it will create a separate try it now if necessary. * Fixed: bug which was syncing server library multiple times at once. * Fixed: Wrong percentage issues when syncing libs. * Fixed: Infinite loop problem when loading some workflows with invalid exported parameters. * Fixed: A number of issues in the diff workflows dialog, including expanding the legend balloon to fit all the labels, changing the behavior of the Run button to load the workflows if their paths have been typed into the text field (but they have not been loaded by hitting enter), and the moving the labels for the second workflow. * Fixed: Edit enumerated parameter dialog layout bug that was introduced in revision 4114 and corrected the preferred width of the dialog to avoid horizontal scrollbar. * Fixed: bug where the parameter shape was not being updated when the parameter type was changed by right-clicking on the module, selecting Edit Parameter, and changing the parameter type. * Removed: Captcha verification from bug report dialog. Changed in build 4272 ( v5.4 ) ---[ Pipeline Client / Server ] * Fixed: The bug which was creating duplicate connections to data sinks. Changed in build 4270 ( v5.4 ) ---[ Pipeline Client / Server ] * Added: Module suggest and workflow crawler features. * Added: Runtime estimate feature. * Added: Personal library tab in parallel with Server library. * Added: A feature that allows editing pipe files in XML view. This feature is configurable and by default is OFF. To turn it on, user must go to Preferences and select the "Allow editing of workflows by XML" checkbox. * Added: Memory Usage stats for SGE. * Added: Added auto connection creation feature, with Shift key holded, moving modules toward each other, a connection between the nodes will be made if applicable. * Added: An alternate way to connect modules, after selecting the node, hold the shift key and click on other nodes, if applicable, connection will be made. * Added: "Send Workflow by Email" feature for OS X which allows to create a new message in Mail application with zipped pipe file attached to it. * Added: Line numbers to the text area of data sources, data sinks, and study modules. * Added: A search filter in server library panel. * Added: Scrolling capability to module group crumbs in the workflow toolbar. * Added: Drag and drop local CSV files to automatically create study modules. * Added: Zip file Drag & Drop functionality. Now users can drag and drop any zip file to Pipeline and Pipeline will open all supported files from it. * Added: Drag and drop of .pipe files to personal library panel. * Added: Right-click menu save selected items to personal library option. * Added: "Delete From Library" and "Delete Children From Library" options for Personal Library. * Added: Auto reload of the workflow in case of external changes. * Added: for server with guest access the feature to enable privilege escalation on regular users and to map guest users with a guest username. * Added: The option to use a variable in the Version field inside a module definition; * Added: A username column in Active Sessions dialog when multiple usernames exist in the list. * Added: Auto-fill of name and email fields in bug report dialog. * Added: a new preference field called GridSubmissionQueue, which reserved to contain value of the Pipeline's job submission queue. Grid Native Specification and Grid Total Slots Cmd field can use _pqueue to refer to the value of this new field. * Improved: The startup speed of the server * Fixed: The bug, which was disallowing some users to execute jobs on the server. * Fixed: The bug which was preventing consumption of workflows from https URLs. * Fixed: Bugs in previewer. * Fixed: Bugs in zooming of Workflow diff tools and changed its look. * Fixed: bug for some workflows validation shows more than 100% complete. * Fixed: Bug related to modules with names over 64 characters in length. * Fixed: Bug with flow control connecting local-remote modules. * Fixed: Bug related to server usage graphs. * Fixed: Bug which was preventing deletion of non-monitored jobs. * Fixed: Bug on server directory access mode when set as "same as shell". * Fixed: To properly restore jobs with Eqw state and resubmit them if needed. * Fixed: To set completion time when instance fails. * Fixed: To not ask for directory location when user saves workflow to personal library. * Fixed: Connection timeout bug during lengthy validation process. * Fixed: To properly open pipe files when multiple Pipeline instances are running. * Fixed: Package mapping white space bug. * Fixed: Other minor bug fixes. ---[ Pipeline Server Terminal ] * Added: Debug panel * Fixed: to not pop up the update notification message when reconnecting. ---[ Pipeline Server Config ] * Changed: Disabled Package and Executable mapping tabs to avoid confusion with userdata file. Changed in build 4091 ( v5.3.1 ) ---[ Pipeline Client / Server ] * Added a new action "Insert module from help file" which will open the Module Creation Utility and changed Insert New Module action to open manual module editor. * Removed: Loops and Repeat Until conditions [ Redesigned version is expected in the future ] * Removed: Preferences for grid plugins on client. * Fixed: bug introduced with diff workflows feature that wasn't allowing user to zoom in/out using Ctrl+/-. * Fixed: Problem with transformations on File or Directory which have String as the base parameter. * Fixed: module timing bug. Now module will not reset its timer with every instance execution. * Fixed: to enable/disable Insert->New Web Service action when needed. * Fixed: previewer bug when multiple instances of outputs are present, some of the thumbnails are duplication. * Fixed: some errors related to database and the problem with pipeline logs. * Fixed: the problem with opening pipe files when Pipeline software is already running on windows. * Fixed: Fixed for server submit script with thumbnail generator to exit when the main executable exits with error. * Fixed: the IDA key renewal instead of using sudo -u and redirect to file, it's using the program tee to avoid permission denied issue. * Fixed: Fixed bug which was causing the workflow viewport to incorrectly move. This was happening when multiple tabs were open and one or more of the workflows had been zoomed. * Fixed: Fixed for metadata extraction to put quotes for values that contain special characters. * Fixed: for edit module's parameter tab to keep selected sub-tab when switching between parameters. * Fixed: Improved the speed of loading of the server library on startup. ---[ Pipeline Server Terminal ] * Added: Update notification on startup Changed in build 4053 ( v5.3 ) ---[ Pipeline Client / Server ] * Added: Web Service Modules * Added: Previewer which shows thumbnail view of output image files when user hover on output parameter node or on output panel. * Added: Workflow diff functionality. The user can load two workflows and determine the differences between them. * Added: Metadata augmentation. Metadata elements can be appended, modified and deleted on any module with value from sources including user-specified, input parameter, output/error streams or metadata. * Added: Data extraction for modules. Output parameters can be specified as String or Number type, with data extracted from module's output and/or error stream. * Added: A new graphical message with buttons which appears when client is disconnected from server. It allows users to reconnect to the server or close the workflow. * Added: Cancel Instance feature. * Added: Copy Output option. It's used for copying completed module's output files as input data source/study module. * Added: Grid Variable Policy feature. This feature allows server administrators to define custom policy for grid variables used by the modules. * Added: Support for specifying env. variables for modules ( Module's advanced options). This tables can include workflow variables as their values. * Added: New attributes to workflow variables - description, required and secret. * Added: New preference GridUserIsAdmin which indicates whether the plugin needs to run sudo when deleting users jobs. * Added: New preference GridJobNamePrefix which allows server administrators to set a prefix on names for jobs submitted by pipeline server. * Added: {$tempdir} pre-define workflow variable, much like {$username} which (if configured by the server ) has a value of server's temporary directory location. * Added: New preference which will tell Pipeline server how many times to resubmit jobs in error state. * Added: to show parameter types in the same string which shows parameter names. * Added: an option to show or hide parameter names when right clicking on empty space in canvas. * Updated: The database software version * Changed: to use plugins for retrieving information about finished jobs. Removed dependency from ARCo, now ARCo support is implemented in JGDIPlugin and DRMAAPlugin. * Changed: the translation of command line arguments so a redirection symbol is not quoted, unless it is the last argument on the command line. * Fixed: The speed of drag&drop or double click of modules/workflows from server library. * Fixed: the bug which was randomly showing blank list when searching for modules in server library. * Improved: the startup time of Pipeline. * Improved: Module Restart and Instance Restart features. * Other bug fixes [ Server Configuration tool ] * Fixed: Persistence And Grid Accounting Password Text fields to Password Fields, so they won't show passwords in clear. * Updated: Some preferences Changes in build 3865 ( v5.2.4 ) ---[ Pipeline Client / Server ] * Fixed: command line prefixes to not be quoted when they contain space. This caused some already existing modules ( including smartline ) to fail because of such prefix "-jar -Xmx100m" * Fixed: several bugs related to grouping and ungrouping modules. Changes in build 3855 ( v5.2.3 ) ---[ Pipeline Client / Server ] * Added: Ctrl+D ( Command+D for mac) shortcut for Server Changer. * Added: Support for Mac OS X Lion * Added: in study module to save metadata as csv file. * Added: Support for command line redirection operators * Added: the ability to access the parameter properties in the module definition directly from the workflow by right-clicking on a parameter and selecting 'Edit Parameter'; a new menu item, 'Edit Parameter Value' still allows user to edit the parameter's value. * Changed: the zoom level of workflow area to be unlimited. * Improved: the search of server & personal libraries. Now it is much faster and responsive. * Fixed: the bug which was creating a module group GUI at incorrect location ( position in the workflow ) . * Fixed: not to show variable names in balloon hints when copying modules with variables. * Fixed: for variables dialog the ability to delete multiple entries. * Fixed: bug which was removing files in module's specified output directory. * Fixed: to give validation error for non accessible files, when the privilegeEscalation is set to false. * Fixed: not to add quotes on any commands when server is running without grid. * Fixed: several bugs related the enumerations. * Fixed: bug where the annotation collapse/expand button appears after undoing a delete annotation operation. * Fixed: to have sessions deleted from sessionhistory_table when their finish time (before was reset time) is older than 2 times the DaysToPersistStatus value. ---[ Server Monitor Terminal ] * Fixed: Some bugs in server update function. * Fixed: Preference tab to properly save TempFileLocation and SecureTempFileLocation tags when secure is checked or unchecked. Changes in build 3770 ( v5.2.2 ) ---[ Pipeline Client / Server ] * Added: for smartline module the ability to edit/change heap size. User can right click and give a number. The sge memory will be updated too accordingly. * Added: new functionality to annotations including the ability to expand/collapse one or all annotations and the ability create and resize annotations with images * Added: new annotation GUI functionality, workflow toolbar. * Added: balloon hints. * Added: a reminder popup when user tries to close a workflow that contains local modules/local files waiting to be executed, or user tries to disconnect to the server where they have open workflows with local waiting modules/files. * Added: option to preserve local input file names when staging. It is disabled by default so old workflow will have the same behavior (workflows with inputs that have same filename but different paths will continue to work). User can specify under module's advanced options. * Changed: to store Staging information in the database instead of memory. This fixes Out of memory exception for server in some cases ( for example when there is a huge local directory assigned to remote module dir type parameter ). * Fixed: the bug which was promtping for credentials for workflows with local modules. * Fixed: the bug which was trying to add Transformation base's need file to the list of output files * Fixed: to include parameter in command line when the transformation base is set but the list of transformations is empty. * Fixed: module grouping bug that caused a cyclic connection between a module group and a module. * Fixed: bug that left the prefix of a parameter off of the command line if the parameter had no arguments and the 'Include transformed parameter on command line' checkbox was unchecked. * Fixed: the bug which was showing Incomplete status for restarting module. * Fixed: IDAGet and XNATGet bugs where downloading locally failed because the smartline parameter in the call to IDAGet was not properly specified. * Fixed: the bug which was using commas at invalid places for Module tags when hovering on a module/workflow in server library. * Fixed: copy with input bugs: when one parent is connecting to multiple child modules, it will be properly copied. ---[ Server Monitor Terminal ] * Added: 'Executables' tab, which behaves much like the Packages panel, but can be used to store package-independent executable information for common tools such as java, python, perl, R; the GUI changes are accompanied by changes to the server side to properly read and write the 'Executable' information from/to userdata.xml * Added: to show total number of completed instances for all workflows. Changes in build 3665 ( v5.2.1 ) ---[ Pipeline Client / Server ] * Added: New feature for hiding/showing all annotations in a module group and all nested module groups * Added: the option to automatically export sessions and communicators information from database to a directory. When a directory is given with HistoryLogPath tag in the preferences xml, the server will save old information (that are more than two times of DaysToPersistStatus) from database to the directory and remove them from the database. * Fixed: to not create links when there are multiple data sinks connected to one module, instead files will be copied to all connected data sinks. * Fixed: "Cancelled" module bugs. In some cases when module was incomplete child modules were cancelled. Also, sometimes when the node needed to be cancelled, it wasn't. * Fixed: To properly open a file when Pipeline already is running and user double clicks on the file. * Fixed bug when user clicks OK on edit module dialog with exported parameters, the parameters will be saved properly like before (pre-5.2). * Fixed to stage local metadata files for study modules on local-remote workflows. * Fixed: bug of copy with input when input file type has needs files, the copied data source will not include those need files as input. * Fixed: bug which was not terminating those jobs which are not monitored by Pipeline. * Made optimizations and improvements ---[ Server Monitor Terminal ] * Changed: "Show Running Workflow" checkbox status to be selected by default. Changes in build 3606 ( v5.2 ) ---[ Pipeline Client / Server ] * Added: a new feature which will help to create modules from help or manual files. * Added: the ability to stage local files to remote location with data source connected to data sink. The remote files will be owned by the user. * Added: support for _ppackage, _psession, pexecauthors, pexecversion variables for Grid Engine Native specification, which will allow to change the job name to include more detailed information about the job. This can be used for gathering statistics and other kind of tracking information. * Added: For study module the option to make its groups as output parameters. * Added: Persistence for local workflows. Now the Pipeline Client will show the list of locally running workflows in active sessions list. * Added: Pages in execution logs dialog. This will make execution logs dialog to appear faster especially when the module has thousands of instances. * Added: New Splash Screen progress bar and support for dynamic version change. Added new splash screen image for Pipeline Web Start. * Added: A metadata method getElementValue for conditional module. It will take a string of XPath, and returns a string of the element value identified by the XPath. * Added: Instance restart. User can restart instances for running or completed workflows. * Added: Server status charts. This charts will show live server usage statistics. * Added: The ability to use metadata file's value as part of module's command. * Added: A feature which will handle multiple launches of Pipeline GUI client in one process but different windows. * Added: Error log dialog for data sinks. If data sink has any error, user can right-click and see a list of errors. * Added: New feature which will allow to put the command line prefix in front of each parameter's argument. * Added: new flag to allow a transformed parameter to be included on the command line. * Added: Copy with Input feature. User can right click on any completed module(s) to copy them with their intermediate input files as data sources, so that these copied modules can be pasted on a new workflow and executed easily. * Changed: Naming switch to prefix, Infinite cardinality to Unknown cardinality * Changed: Module parameter tab interface * Changed: default SGE memory to 5G for IDAGet and XNATGet modules. * Changed: the way smartline, XNATGet, and IDAGet modules are handled in the Pipeline to make them portable; this is accompanied by changes in the DPS installation tool * Changed: the 'include transformation on command line' feature to select and disable the checkbox when the transformation is applied to 'self' * Fixed: when copying module with no package name, the pasted new module will keep the empty package name. * Fixed: to correctly store the cardinalityBase string specified by the 'Based On' option. * Fixed: bug on Windows: the new Remove and Clone parameters buttons' texts are now properly displayed. * Fixed: bug that leaves a parameter off the command line if it has a prefix but no arguments * Fixed: Bug which was not saving editing column when "ENTER" was not pressed before OK button edit module dialog. * Fixed: To terminate Eqw job before resubmitting. * Fixed: Backwards compatibility issue for older java version which was not allowing Pipeline to properly start * Fixed: Study module's parameter type was sometimes inconsistent * Fixed: To not show output "flow control" parameters as output files under module's execution log > output tab. * Fixed: Bug for data sink, if one instance of data sink fails, other instances will continue to run. * Fixed: Provenance bug which was not showing Data Provenance for selected nodes. * Fixed: For study module's edit grouping and matrix panels to report undefined variables when user gives invalid variables as grouping criteria and matrix column. * Fixed: bug when starting multiple CLI processes, it will create one set of database files per process, to avoid conflicts. They will be deleted upon process termination. * Fixed: open url under help menu on Linux * Fixed: Study import from CSV will work properly for some Linux systems. ---[ Server Monitor Terminal ] * Added: Server Update feature which allows to remotely update the running server and update specified neuroimaging tools. * Added: Reset time column for session history table, it records the reset time of the workflow. It shows under workflow history. * Improved: Memory usage and thread usage graph performances * Changed: To remember user's password for reconnection within the same session. ---[ Server Configuration tool ] * Fixed: to reload userdata and preferences files when refresh button is pressed in the server terminal's preferences tab. * Fixed: the size of connection info dialog to be dynamic which will be dependent of server name length. * Fixed: to have sorted columns for package mapping table. Changes in build 3520 ( v5.1.5 ) ---[ Pipeline Client / Server ] * Added: A new feature which will allow to see all users workflows when pressing Refresh button with SHIFT key pressed in Active Sessions dialog. Of course, this will work only for server admins. * Added: web start support. New preference for server that enables guest login, new flag for client that creates server and guest username if none exists. * Added: a metadata method getElementValue for conditional module. It will take a string of XPath, and returns a string of the element value identified by the XPath. * Added "Pipeline on Facebook", "Pipeline on Twitter" and "Pipeline Homepage" items in help menu. * Added job synchronization support for those plugins, which don't support getJobList function. Instead it requests each job status one by one. Added array job submission information into PipelineGridPlugin object. * Added: support for Restartable plugin. This option can be configured from preferences. * Added: filter options in Remote File Browser * Added: to show parameter grouping Path when hovering mouse on module group's node. * Fixed: to show Module ID when hovering on the parameter of the ModuleGroup, so the user will know which module's parameter he/she is going to change if their share same name for their parameters. * Fixed: to show the description of DataModule when hovering on DataModule's parameter because DataModule doesn't allow to set description for its parameter. * Fixed: to set unique ids for modules and parameters every time user does following actions : Cut, Paste, Undo, Redo, Group, Ungroup and edits the module info because this changes can change numerical order of module ids * Fixed: the bug which was showing infinite "Waiting" state for module groups which only contain Data sources or study modules. * Fixed: to have the client to download tools from the Internet, instead of packing it with the Pipeline. It will check on startup, if it finds user doesn't have tools installed (or a new version is available), it will download and install. * Fixed: reset workflow button was not showing when user had opened a completed workflow and switch back from another empty workflow tab. * Fixed: to show "connect to server" message when server library is empty, always show the server library panel even if it's empty. * Fixed: when inserting new viewer module, user will not be prompted with edit module dialog. * Fixed: to enable multiple parameters selection on edit parameters tab, when multiple rows are selected, only remove button will be displayed, which lets user remove all selected parameters. * Fixed: Pipeline workflow viewport scrollbar to include heights of annotations as well. * Fixed: bug which was exiting Pipeline when Grid submission was not enabled. * Fixed: view text button works on remote text files under module's output tab. * Fixed: to let user to restart running modules. * Fixed: to check number of finished instances instead of current module status to enable/disable restart module button. * Fixed: to have 2.5G memory for smartline, because some large input volumes require this. * Fixed: bug on failed smartline modules: now smartline module's advanced options will be properly set when submit to the server. * Fixed: study module bug, metadata files are now properly created for module with multiple parent modules (some parent modules do not have metadata). * Fixed: when user uses shortcut key to edit module then click cancel, the workflow will not be marked as modified. ---[ Server Monitor Terminal ] * Fixed: bug which was not reactivating the User Management properly from Server Terminal. ---[ Server Configuration tool ] * Added: Restartable plugin GUI Configuration support. * Fixed: to properly display grid plugin jar's location based on the existing value. * Fixed: to have icons on fields that require server restart to take effect, instead of fields without restart. * Fixed: for server config tool's package mapping tab to enable multiple line selection, works with remove button. Changes in build 3309 ( v5.1.4 ) ---[ Pipeline Client / Server ] * Added: Option to send email notification 1 day before workflow to be automatically removed. Changed email notification functionality. * Added: for starting GUI with command line the option to give parameters and variables assignments as arguments. * Added: the option to open multiple URLs when calling GUI by giving command line arguments, also added the argument option to replace non local servers for all the files, and the ability to execute multiple workflows automatically with one authentication. * Added: to have client check for updates when client connects to a server and the server's version is newer than the client's. * Added: to check for updates when client has an "incompatible version" error when connect to a server. * Added to check in SGE accounting file for exit status when the plugin and ARCo database do not provide the exit status. * Fixed: to not validate disabled parameters * Fixed: for edit parameters tab, when user removes one parameter, the next row will be automatically selected. * Fixed: to not offer to convert parameter type of data source to "number" in data source when the data source is connected to other modules or it was not empty before. * Fixed: a bug which was sometimes causing for INVALID COMMUNICATOR ID Exception after communication birth. * Fixed: staging problem, when the client and the server are on the same machine. * Fixed: for starting GUI with command line arguments when user gives -execute and -username without password, the connection dialog will properly pop up asking for password. * Fixed: to have server's "Days to persist status" counting from the session's completion time, instead of starting time. * Fixed: data source so it doesn't unnecessarily prompt to change data type to number if no values have yet been entered. ---[ Server Monitor Terminal ] * Added: on server terminal GUI's workflow tab to have a warning message if there are running workflows started more than 21 days ago. * Added: for server terminal's user panel pipeline versions and os versions columns. ---[ Server Configuration tool ] * Fixed: for server config tool to have access mode as well as description listed in the drop down menu. Changes in build 3232 ( v5.1.3 ) ---[ Pipeline Client / Server ] * Added: updater to check for beta update. * Added: to check & automatically convert if needed the parameter type of data source to Number. * Added: to validate input of data source when parameter type is set to Number. * Fixed: bug which was causing package definitions update to fail. * Fixed: validation message for not found executable location * Fixed: bug with local file in data source, study, or input parameter. ---[ Server Monitor Terminal ] * Added: to have server's version and build number and server's OS name and version displayed when a connection is established. * Fixed: preference tab, not to show anything until data is populated, to avoid showing old data. ---[ Server Configuration tool ] * Added: option to give userdata.xml location for the server. * Fixed: to correctly get the server's userdata file based on server's port, instead of client's userdata. * Fixed: to load given preferences values instead of default values when file is specified. * Fixed: Help button URL. Changes in build 3190 ( v5.1.2 ) ---[ Pipeline Client / Server ] * Added: text file viewer on output file tab, user can click View Text button to view supported text output files. * Added: the ability to open GUI client and execute workflow by giving -execute as a command line argument and optionally give -username=user and -password=pass as credential. * Added: to have sorting on name, address and username on saved connections dialog. The sorting will always put LONI Cranium server on top. * Added: package mapping for data sources, study modules and input parameters of module. * Added: to package mapping three predefined variables to represent the values of package name, version and location that can be used in sources and variables attributes. * Fixed: the issue when the client couldn't execute modules on the locally running server but instead it was running modules locally. * Fixed: LONI Viewer now shows properly on Mac OS 10.6 with Java update 3. * Fixed: not to show needs files under output files tab for list output file. * Fixed: when multiple modules are connected to a data sink, the data sink will not show complete status until every parent is complete. * Fixed: for a saved connection with empty username (e.g. user installs PL for the first time, it has cranium server and empty username), connection dialog will ask for username as well as password from the user. * Fixed: to let user give relative path as executable location with package mapping feature. Also prepends "bin/" before the filename if both executable location and mapping is invalid. * Fixed: when connected to data sinks, the symbolic link output were sometimes overwritten as actual files. Now it preserve the link and writes to the final destination. * Fixed: to not use sudo command when creating links for Data sinks and the usePrivilegeEscalation flag is set to false. * Fixed: bug with remote file browser, it now properly lists files and directories. * Fixed: Data Sink to save its linking state to the persistence database. This caused data sink to show as failed when server restart took place after links were created. * Fixed: to put more information on more than 3000 validation error message. It will suggest users if they have a mismatch by mistake, or if they can split inputs. * Fixed: Bug in user management which resulted to show incorrect usage ---[ Server Monitor Terminal ] * Added: New Connection button to create a new server to connect to. * Added: Edit connection button to edit an existing connection. * Added: dynamic preferences support for user management. Now user management percent can be changed without requiring server restart and same for turning on/off the user management. * Added: dynamic Workflow Management configuration support. Now server admins can modify workflow management rules without restarting the server. * Added: "view workflow" which downloads and opens workflow in Pipeline client. * Added: duration column under workflow tab, changed session id to a shorter string, and fixed alignment on several columns. * Added: in server terminal's users tab to show connection history of users. It lists username, IP, start and end time of all connections made to the server. * Added: the option to save the result to a CSV file. Also added "connect time" column for connected users, which shows the time user connected to the server. * Added in server terminal's workflow tab to show history of workflows ever submitted to the server. It will show session id, start and end time, username, and lets you open the workflow. * Added: column on active sessions dialog partial session id which shows last token of session id. * Added: full session id text when one session is selected. * Fixed: server terminal on Linux and Windows, some fields were not visible because of the bigger default fonts. * Fixed: in workflow tab of server terminal gui, to have confirmation dialog when stop/reset button is pressed. Also fixed to hide end time and duration columns when "running workflow only" checkbox is selected. * Fixed: to have help button on server terminal and server pref config tool and have them link to the server guide. * Fixed: server terminal's user usage panel sorting on custom usage table, and fixed when one entry of user usage table is selected, the corresponding entry in custom usage table and workflow panel's table is also selected. * Fixed: to have dynamic update on server preference status refresh interval and timeout multiplier, and the server will restart timeout thread on update if it's dead. * Fixed: Help button to show the help section for the selected tab. ---[ Server Configuration tool ] * Fixed: to have default values displayed on some fields on server config tool. * Fixed: not to have library monitor file path required, also added default value notes for grid array job fields. * Fixed: server config tool's access tab default value for workflow management. * Fixed: to select "Grid Enabled" checkbox if there is plugin specified. It used to select based on native specification, but there can be cases when the grid is enabled without native specification. * Fixed: when open server config tool it will check for last saved preferences file if no valid argument is given. * Fixed: to show hidden files and directories for the Load and Save actions. Changes in build 3027 ( v5.1.1 ) * Added: Backlogged status for Workflows. The new preference MaximumActiveWorkflowsPerUser indicates the number of active (running, paused ) workflows the user can have at each moment. By default it is set to -1 which means Unlimited. When the user exceeds the limit, all incoming workflows get status "Backlogged" and they wait until active workflows are finished to start their execution. * Added: User Usage Panel for Server Terminal, which shows user usage when user management is enabled. * Added: Server preferences configure tool to the server terminal. Added a number of dynamic server preference fields, system admin can use server terminal to update these fields while server is running. * Added: "Last Activity" column in "Users Online" panel, which shows when was the last time user communicated with the server. * Added: in server terminal's menu "disconnect from server" function. * Fixed: bug where data sink and regular module connecting to the same data source, they will all run properly. * Fixed: to correctly show module group as staging instead of waiting when it contains staging module. * Fixed: sorting of active sessions dialog when there are backlogged workflows. * Fixed: improved workflow submission process. A new thread will be created upon server startup which will be responsible for new workflow submissions. This is very useful for batch workflows as submitting big number of workflows could halt the system until the submission is done. Improved backlogged workflows initialization, representation and behavior. * Fixed: decreased number of database calls for user management. * Fixed: to check if the workflow is running except if the workflow exists. Because if the workflow is not running there is not usage for that workflow. * Fixed: to not terminate array job's task 1 when the plugin just sends the job number without task number. For example if plugin sends job 892 pl should be intelligent enough to understand that this is the same as 892.1 * Fixed: bug where Pipeline was randomly freezing. * Fixed: pause and resume will work properly on server with array jobs enabled. * Fixed: when loading a workflow with parameters entered with one value (double-clicked on the node) and connected with a connection, it will remove the entered one value. * Fixed: when making a connection, the output parameter node already has one value (double-clicked on the node), it will remove that single value. Undo the connection will put the value back. * Fixed: bug on data sink sometimes when it errored, it showed "Failed 1 of no job". Also fixed to correctly set the status of data sink to canceled when the parent module failed. * Fixed: Server Status Label to properly show usage status when connected to multiple servers. Changed to show percentage of both User and Server utilizations. * Fixed: to create necessary directories for data sinks before the execution starts. * Fixed: to print a Warning when ARCo database is not configured instead of giving an error and exiting Pipeline. * Fixed: Switching from non array job server to array job server was not possible without restarting jobs. This fixes that problem and switching from non array to array server should not be a problem for submission. * Fixed: to double check and delete user usage of instance before inserting new one. * Fixed: the linking problem when module's output is connected to data sink and has more than one arguments. Changes in build 2967 ( v5.1 ) * Added: Array Jobs support via JGDI Plugin. * Added: User Management feature. * Added: XNAT Database support. * Added: Support for output list files. * Added: Show errors link in Pipeline control bar when there is validation error. * Added: Array instance statuses support which will provide faster delivery of statuses from server to client. * Added: Random Number Generator function for conditional modules. * Added: The module attribute to require external network access. * Added: Captcha in bug reporter dialog. User has to type in verification text in order to submit a bug. * Added: Functionality to synchronize server's package map with each connection made by client. * Added: Intelligence for module executable locations. * Added: Variable declaration and file sourcing support for array jobs. * Added: Server configuration GUI. Server preferences can be loaded, displayed and edited more easily. * Added: MPI Job support for SGE ( beta ) * Changed: Data sinks now work faster as temporary files are linked to destination files. * Changed: Colors of module statuses. Now different statuses have different colors. * Changed: Validation exceptions dialog - Increased the font size, moved the initial location in the center of Pipeline window, changed selection background color. * Changed: Server terminal's frame to center at the screen and the shortcut key for connect. * Changed: Server Status to show server business per user when user management is enabled. * Fixed: Updater to exit more gracefully. When user has open workflow, it will ask for saving options for each workflow. * Fixed: When server tries to delete old temporary directories without username, it will find out the owner and sudo remove as the owner. * Fixed: To report validation error when user specified list file which doesn't have read permission for everyone. * Fixed: Updater bug on Windows, corrupted installer files will be removed before a new download starts. * Fixed: Fixed to not start handshake when server is not found. This problem appeared when server address was correct and the port number was incorrect. * Fixed: Validation bug on recursive directory source, validation will check for match if recursive is selected, and stop right away when one match is found, or user cancels validation. * Fixed: Fixed to wait for current submission threads to complete before shutting down Pipeline server. * Fixed: Some connectivity issues with ARCo database. * Fixed: Fixed bug to correctly show the index of study matrix when number of items is less than 5. * Fixed: To copy the License information with the module when the module is copied. * Fixed: To have 4G memory for IDAGet module by default, as sometimes 2G memory caused the file conversion to fail. * Fixed: To surround with single quotes those command line arguments which contain special characters in it. * Fixed: Missing needs files for local modules/LONI Viewer in a workflow. * Fixed: Bug when a workflow has validation error, switching to another normal workflow tab and back made the validation messages disappear. * Fixed: Bug in provenance editor open workflow button now works, it also properly disables when nothing is selected. * Fixed: Bug when opening old workflow (e.g. from version 4.3.3) with smartline modules, the smartline module will properly run. * Fixed: Other minor bugs Changes in build 2877 ( v5.0.2 ) * Added: Custom variable names per execution support for Command line version of Pipeline. Now users will be able to change values of already existing variables prior validating/executing the workflow by specifying values of variables as command line arguments. * Added: Multiple workflows processing feature in command line interface (CLI). It is enabled by "-batch" option with a list of workflows or a directory of workflows. All workflows will be processed in a single call. * Added: In the command line interface (CLI) -parallel option to limit maximum number of workflows can be processed at the same time in batch mode. * Added: Functionality for loopback connections for dynamic repeat until modules. A user can connect an output parameter to an input parameter of the same type. More validation for different parameter cardinalities will be added. * Added: Module numbers. Module numbers give modules an identity * Fixed: Function F1 key was not working properly for the source code of module looping conditions. * Fixed: The looping parameters information that was shown after pressing the Function F1 key several times was misalligned and was covering the source code edit line. * Fixed: The looping parameter information from 1) and 2) was showing up after changing tabs. * Fixed: to have meaningful group names when use Generate Groups feature on study module. * Fixed: to continue job synchronization when there is/are specific error(s) while obtaining job info from DB. * Fixed: Server Connection Failed bug. * Fixed: to properly shutdown Client Engine of Pipeline server to avoid memory leaks. * Fixed: bug in local smartline module in Windows. It will now work properly without the backslash character in front of the jar path. * Fixed: bug when downloading a large IDA collection to server, it gave "argument too long" error. * Fixed: to change module status to Failed when there are node errors while staging files prior execution. * Fixed: ExceptionInInitializerError when calling command line interface with -help or -version option. * Fixed: Paste problem with copying a module another Module Group. * Fixed: validation error for existing directory in Data module's remote directory source. Fixed to return an error to client when directory content is requested but a file is given. * Fixed: bug on study and data source modules directory source mode, recursive option can properly be unchecked. * Fixed: non recursive dir source listing to not act as recursive. * Fixed: bug on LONI Viewer: now workflow with multiple LONI Viewer, they all will open files properly. * Fixed: validation bug now it will validate list files with string/number parameter type and give validation error if invalid list file was given. * Fixed: to not include semicolons in module ids. * Fixed: CLI hanging issue when user's credential was rejected from the server. Changes in build 2800 ( v5.0.1 ) * Fixed the bug which was hiding SGE stack size options from Execution Panel. Also added some GUI flexibility for the components of same panel. * Fixed bug when copying a module with exported a parameter inside a module group, the new exported parameter was not shown from the parent module group. * Fixed several validation bugs: now validation will properly report validation errors, including multiple server error and the bug reported on revision 2654. * Fixed bug on edit study module > import data's local file browser, it will show correctly the selected file by local file chooser. * Fixed the wrong number of backlogged issue reported in Pipeline client when there are stopped workflows which contain backlogged instances. * Fixed to prepend "_ " (unbderscore) to job names of those executables which names start with digit. SGE does not allow job submission with name which starts with digit. * Fixed bug on rerun module on running workflow, it will properly start from the module. Also any errorred data sink children will be reset. * Fixed to trim Filetypes when they are created from comma seperated extension type. * Fixed bug using data sink with type directory and directory dump not enabled, it will not report error. It was because sudo copy tries to preserve sticky bit on temp directory and failed. * Fixed Provenance Bug, which was showing only one module in the final module's provenance. * Fixed $username validation bug when the module is newly created. * Fixed to not include $username variable in XML file when saving the workflow. And also added to remove $username variable from workflow which has been saved before this version and includes $username variable. * Fixed bug when running local workflows using command line interface (CLI), it will complete as expected. * Fixed to report an error on validation when a file is given to Directory source. * Fixed a bug that interpreted a String parameter that happens to be a valid URL as a URL instead of a String. * Fixed to put default icon when the icon is not found for downloading files. This fixed the problem of empty dialog appearing after user tried to download some kinds of output files. * Changed: Created two subpanels for the Concrete Module's Execution panel. The Execution Environment panel is the previous Execution panel and holds information about the execution environment for the module. The new Looping Conditions panel specifies whether to run the module in a repeat until loop and the conditions for completing the loop. * Changed smartline color to have standard line color. * Changed Repeat Until looping GUI flag from icon to small circle intersecting the main concrete module GUI circle. * Changed "New loop..." string to "Loop" when on insert new loop action. Changes in build 2699 ( v5.0.0 ) * Added: Help Buttons to almost all Pipeline dialogs * Added: Search when selecting Filetypes for all module types. * Added: columns in File Type table. * Added: converter of comma seperated file types, which converts single comma seperated file type to multiple filetypes. * Added 3 rules for automatically creating/building a study module: filename matching, derived from metadata, and derived from csv file. * Added: number of backlogged jobs in server status of client. Also changed % busy calculation, now it is the total number of backlogged, queued and running jobs divided by total number of slots. * Added sortable tables under study module's matrix tab. User can define columns, and generate table that can be sorted by any column. Also added export to CSV feature, where you can export the whole table in CSV file format. * Added: support for IDA shared collections. Now user can download any shared collections as well as their own collections. * Added: bug report dialog to have a confirmation message when user doesn't provide an email address. * Added support for static loop groups. The loop index is currently initialized only through a Study module query. * Added support for dynamic repeat until looping for concrete modules. * Changed pipe to script (save as script/makefile) from Python hosted by LONI's web server to Java code. User doesn't need to send the workflow to our server to do the conversion. * Changed: to have local smartline execution if parent and child modules are both local. If either of the modules is remote, smartline will be executed remotely. * Changed smartline's color slightly. * Fixed: validation bug for conditional module when the conditional source contains source with one byte length arguments. For example String.contains("1") would not work with previous code. * Fixed: for connections dialog, when user tries to connect but it returns with error, the Connect button will not be disabled. * Fixed: when making a connection, and the input parameter already has one value (double-clicked on the node), it will remove that single value. Undo the connection will put the value back. * Fixed: missing validation error when open a workflow (with validation errors), delete some modules, then validate, it will show validation passed. But if we execute instead of validate, it will show validation error. Now validation will catch that. * Fixed: to allow string and number output parameter to connect to enumerated input parameter. Added validaiton check for each string and number values, they must be defined in the enumerated parameter's possible values. * Fixed: to not attempt to make a connection to localhost when server library refers to localhost and in that case update from local filesystem when user presses update library. * Fixed: smartline module to have parameter orders and the type of predefined input to "File" * Fixed: to hide connection line portion that runs over connection nodes. * Fixed: Export to script to support Workflow Variables * Fixed: Export to Makedile and Script to support inner & outer product dependent of module's parameter cardinalities. * Fixed: to ignore duplicated values for enumeration file type. Also fixed to generate unique enumeration value when user clicks Add. * Fixed: removed Enumerated file type for data source, study and conditional modules. Added validation error when user runs a workflow with enumerated output connected to enumerated input. Also added error message when user tries to connect these. * Fixed: multiple smartline format suggestion dialog not properly working when user clicks cancel. Changed when user holds shift key when making connections, it will override to have the opposite smartline option than user's preference. * Fixed: to keep existing grouping members and matrix table when user click OK to close study module dialog, then double-click to reopen. Also fixed Double comparison/sorting on matrix table bug. * Fixed: the issue with not visible "+" button which creates a new workflow. Now a new "+" button will be created if there will not be space for "+" Tab in workflow tabs. This change will be affected also in Pipeline Window resize. * Fixed bug on output files tab when click Download on an xml file, download manager will appear just like downloading other files. * Fixed bug now download manager will auto-scroll to the bottom to display the active download whenever user downloads a new file. Before it was fixed so user may not realize the activity below. * Fixed: Edit study grouping panel, when user changes a grouping criteria and click OK or Reload, the new criteria will be saved. * Fixed: to not include Empty Workflow in Bug Report which represents the "+" Tab. * Fixed: bug when a Pipeline client submits a workflow on a Pipeline server which is running on the same machine, it's stuck at validation forever. *Fixed: Fixed to hide workflow overview pane, when opening Pipeline menu. * Fixed: bug on edit study module dialog, if user makes changes, and switches tabs then clicks cancel, it will not save any changes. * Fixed: Fixed to create server library monitor file dirs in case they do not exist. Changes in build 2618 (v4.5.2) * Added Open From URL feature which allows to open Workflows which are located on the Web. * Added support to open Extended RDF files. * Added preference to specify manual monitoring file for Server Library. This is useful when updating multiple packages in the library at the same time. With Automatic monitoring it forces to load library multiple times during the multiple package update. Manual monitoring file solves this issue. * Fixed bug when user cancel on remote file browser, then choose remote browse and cancel again, the dialog did not show up. * Fixed a bug which was allowing to save on already existed file, when new file was created and saved. Removed fileDialog for OSX, instead it will use fileChooser as other operating systems do. * Fixed the bug which was making LONI Viewer to open JAR file instead of actual data when it has multiple instances. * Fixed bug the PMID link in module citations is updated. The old link did not work under Windows. * Fixed validation error issue when concrete module executable location is surrounded with spaces. * Fixed the issue when the command of the module was not complete when the module has multiple parents. * Fixed variables interpreted as 'null' when user copy a workflow with variables and run the original workflow. * Fixed LONI Viewer's default input parameter will have file types it supports, instead of type File(*), which caused missing header problems when trying to connect to analyze image output. * Fixed: In module's parameter type editor, when checking and unchecking random file types rapidly, then finally click OK, the saved result had more file types than it displayed. * Fixed the problem which was making Server Library to disappear when multiple clients are started together. * Fixed validation bug: when server's port is not default, validation will not give "file not exists" error on data sink. * Fixed to show the absolute path of the workflow when hovering on the workflow tab even if the path is a URL. * Fixed Out of Memory exception when loading server library for multiple servers. * Fixed bug on output files panel, when multiple instances are selected, download button is sometimes disabled. * Fixed bug which was causing "Update Library" action to not work for the first time and only after second call it was working. * Fixed other bugs Changes in build 2481 (v4.5.1) * Fixed: Remote File Browser and Directory Sources to properly recognize symbolic links. * Fixed: 'Download files' button on output files tab for local module works. * Fixed: when restart a remote-local workflow's local module, it will restart from this exact local module, not from the first local module. * Fixed: not to update the node with old events which have been occurred before the "Restart Module" action. * Fixed: not to submit new local processes when workflow is already closed. Also fixed to not submit new local processes when module's parent has been restarted. * Fixed: LONI Viewer to have valid predefined parameter and executable Location when reconnecting from another client. * Fixed: execution state of the local module when it has been interrupted and connected with another client. * Fixed: bug causing complete local modules to show incomplete after reconnection. * Fixed: validation not to give error for non-existing output files, fixed null pointer exception when there are duplicated files. * Fixed: to select the first instance in execution log dialog. * Fixed: "argument too long" error when restart a module. * Fixed: after restart module, only the statuses of the rerun module and its children are reset. Changes in build 2390 (v4.5) * Added: Smartline feature to suggest preferred output file format, if there is one. * Added: Filetype extension and needs information when selecting parameter filetypes. * Fixed: bug when populating instances execution log dialog. It improves the performance for large number of instances. * Fixed: validation to report multiple servers error when more than one remote server is used in a workflow. * Fixed: to ignore square brackets when creating unique ids. * Fixed: to set instance as errored when it is not found in ARCo database. * Fixed: bug on submission when the name of the module contains comma in it. * Fixed: to trim whitespace in front and at the end of values of parameter type file and directory. * Fixed: rerun bugs when rerun on running workflows. * Fixed: incorrect smartline output file type description when PL make preference suggestion or let user choose among multiple types. * Fixed: bug which was preventing to report local validation errors. * Fixed: missing connections when make a module group. Also fixed another grouping bug about incorrect connections. * Fixed: the design of getting info about completed jobs while server synchronizing list of jobs with plugin's list jobs. * Fixed: bug when resubmitting a job, the same job was requested many times. * Fixed: bug when resubmitting a job the grid job sync stopped working. * Fixed: bug sometimes after rerun, some modules do not show correct status. * Fixed: when user pauses the workflow and waiting for server to response, the workflow will show "pausing" instead of "paused". Also changed stop button to reset when workflow is paused. Changes in build 2345 (v4.5 beta) * Added: support for user-owned temporary files and directories. * Added: workflow pause and module rerun functions. * Added: tab User in preference that stores user information, which is used to automatically set module author when user creates a module. * Added: to show command line switch in Module Overview Box. * Added: series ID and series count information on IDA dialog. * Fixed: pipeline server bug when the server was running in local mode (No grid plugin was specified). * Fixed: remote browser bug for data sinks and sources. * Fixed: Command+E shortcut is working. * Fixed: IDA connection timeout and out of memory problems when it tries to download a large collection of data. * Fixed: validation will not give validation error about cardinality when the parameter has infinite number of arguments. * Fixed: validation will not validate the same file multiple times in a workflow. * Fixed: several memory leaks on client. * Fixed: bug that causes some module's status not shown. * Fixed: under output file tab of execution log dialog, when user downloading .img file, it will only donwload the .img file. * Fixed: edit enumeration parameter type bug. Also fixed its dialog to show a combobox for server address, instead of text field. * Fixed: to move the viewport to the selected module after the module overview box is closed. * Fixed: to show animation when module is running but has failed instances. * Fixed: disconnecting from the server will display a message on completed open workflows as well, and execution log dialog will be disabled. * Fixed: bug of Module Group status which appeared when moduleGroup had several cancelled modules. * Fixed: not show the percentage of workflow complete when it is 100% and there are errors. * Fixed: to remove from queued jobs list those restored jobs which have state "running". * Fixed: to show change instance status to queued when server restarts and find that current instance is still queued. * Fixed: when reconnecting open workflows with remote-local modules, the message "Connection to server was lost" will go away properly. * Fixed: bug when a workflow with data sink has error (the error could come from any module), the finish time is not properly set. * Fixed: When login dialog is opened from remote browser and then cancelled, it reopens that dialog which opened Remote File Browser. Changes in build 2257 (v4.4) * Added: JGDI support on server, which enables job submission as the user instead of user 'pipelnvr.' * Added: Grid Plugins support. An API is now provided to allow developers to implement their own plugin for Pipeline Server to be able to communicate with various grid managers. * Added: % Complete for running workflow in Pipeline status bar. * Added: Local and remote .scene file support for the LONI Viewer. * Added: "View Data" button to the Output Files panel of the LONI Viewer to allow the user to easily open files supported by LONIViewer. * Added: Additional information and features in Server Terminal GUI, including submitting instances report, updated time, etc. * Added: Support for grid-specific complex resource attributes. * Added: FailoverPostScript option in Preferences. This is a string argument for the path to a script to be called after a Pipeline Server failover procedure completes. * Added: Staging error visual indicator for data sinks. * Added: The ability to connect a data source to a data sink. * Added: option in server preference. If this is enabled, temporary directories older than 2 * will be deleted on Pipeline server startup. * Added: Copying modules will copy all variables defined in module group. * Optimized: Library Updater will not hold same Modules/Workflows in memory which saves time and memory. * Optimized: Queries to Pipeline's persistence Database when retrieving workflow info. This significantly speeds up workflow reconnection and overall workflow status monitoring. * Optimized: Running jobs are no longer terminated after a server restart. * Optimized: User management through SGE should now work as expected. Job priorities will be based on historical usage. * Modified: GridEngine will no longer issue multiple queries to Pipeline's Persistence database for the same data. This improves the speed of restarting workflow execution after a server restart. * Modified: Cardinality of error module when exceeding 3000 is now shown on the GUI. The Pipeline server will throw a validation error in this case. * Modified: Job submission now creates threads to submit jobs. Parallelization is improved and Pipeline server responsiveness is faster. * Modified: "Change Parameters" checkbox checked by default in Server Changer tool. * Modified: "Disable Smartline" option when user right-clicks a smartline module is now hidden IF the modules cannot be connected by a regular line. * Modified: Data Sinks panel will no longer hide file type selection area when directory dump mode is enabled. * Fixed: Bug where instance errors were not properly shown when reconnecting to workflow. * Fixed: Bug where files with filenames containing spaces weren't copied properly to data sink destination. * Fixed: Bug where the calculation of dynamic cardinality was wrong when "infinite argument" parameter exists in the node. * Fixed: Bug where copy module group with variables defined in parent module group failed. Added checking for duplicate variable name in variables dialog. * Fixed: Bug where grouping did not work when coping a module group and pasting it to another module group. * Fixed: Bug when deleting certain module groups and saving the workflow will cause it not to open in the GUI. * Fixed: Missing menu bar bug on Mac OS X Leopard, after a dialog is open and closed. This bug appeared after the recent Java update on Leopard. * Fixed: Memory leaks, when opening and closing workflows. * Fixed: Bug in Copy Citations when Data Module, Viewer Module, Smartline, Study Module, Conditional Module are copied. * Fixed: Other minor bugs Changes in build 2073 (v4.3.3) * Fixed: Selecting one concrete module with smartline modules will not enable "Group selected items" option. * Fixed server changer to have first server selected by default, and changed "All servers" to "All components". * Fixed bug where user double-clicks parameter node with directory type, and uses local file browser, directory selected will not be shown on the text field. * Fixed bug which was crashing Pipeline server when executable location or some other arguments were transferring as null to DRMAA. * Fixed problem when parameter's description is very long, edit numerator and infinite file/string/number dialog goes out of range. Added scrollbar for these dialogs. * Fixed: Pipeline updater will work in Windows Vista. Also added current version information on updater dialog. * Fixed: IDA File Conversions feature for Windows Changes in build 2037 (v4.3.2) * Added: support to specify stack size for SGE and DRMAA, user can specify from 10-256 MB of stack size per module, default will be empty (not specified). * Added: support for directory as input type of smartline, it can be a series of dicom files. * Added: to show message "Authenticating", which will appear after "Handshake done". * Changed: default order of exported parameter will be decided on the position of linked module, from left to right then top to bottom. Added "Arrange Export Order" when right click exported parameter nodes, which will list all exported parameters and let user to arrange. Also added module name column on Module Group dialog's Parameter tab. * Changed: cancellation design, now it has following rule. Node becomes canceled whenever all its parents are canceled or Failed. When it has parameter which is Optional and parent of that parameter is canceled/failed, node won't be canceled. * Changed: remote files validation design. Validation will gather all remote files information into a list and then when files are needed the list will be as a source not remote server. This will save time and make validation to pass faster. * Changed: to send an email to Pipeline Email Recipients when server fails over. * Fixed: bug for data source where "directory source" and "use filter" is checked then unchecked, some components of "use filter" did not hide properly. * Fixed: to not give validation error for paths which have spaces before or after it. * Fixed: directory staging will have correct status information displayed on GUI. * Fixed: IDA download to server will be owned by the user with proper permissions. Added feature to have one directory per series as data sink, smartline will convert the input to other image formats. * Fixed: IDA download, create data source as directory will be an option only when download as archived. * Fixed: the overlapping issue on Windows with data modules: number of input/output items count label and replace, browse, clear buttons will take two separate lines. * Fixed: the order of exported parameters for module groups will be user configurable. In addition, the default order of exported parameters will be the order of parameters within each module. * Fixed: the bug when parameter has a space at the beginning or at the end of its name, transformation base couldn't find it. * Fixed: validation will not show duplicate error messages for the same module. * Fixed: validation will get correct cardinality value for directory source of data module. And mismatched inputs producing more than 3000 instances will be caught. * Fixed: bug which was preventing to close the EditConcreteModule panel when a parameter which was a dependent of some other parameter has been removed. * Fixed: to clone Citations and Executable version as well when module is copied. * Fixed: to not redraw the connection lines which are connected to those parameters which do not exist anymore when parameter is removed from Edit Module dialog. * Fixed: to show the "Show Results" link in Control bar. * Fixed: bug when smartline is connected to data sink and user wants to disable only smartline, it was deleting whole connection line. * Fixed: bug about grouping involving smartline modules. Smartline module will be grouped to a new module group only when its parent and child are in this new group as well. * Fixed: some other minor bugs. Changes in build 1995 (v4.3.1) * Fixed update of server library bug * Fixed command line validation * Fixed IDA selected directory by remote browser to not append any type of slash because server and client OSs can be different * Fixed when parameter has a long description or large number of arguments, EditParameterValue dialog didn't render correctly, it was smaller than necessary. Now it shows proper size. Changes in build 1988 (v4.3) * Added Smartline. An automatic file conversion tool. * Added support for Directory Sources, which will allow users to specify directory from where Pipeline will read needed files and add as values for specified parameters. * Added Pipeline Updater, which will download and launch new versions of Pipeline installer automatically. * Added user can now make a screenshot of the entire workflow, it will be saved as an image file (.png) * Added support to specify maximum memory limit per module when using SGE * Added to the Pipeline functionality for retrieving the metadata of Ida collections. * Added active sessions table sorted by column values. * Added default icon and other default information for LONI Viewer. Added LONI Viewer under top menu Tools. * Added timeout for server connections * Added counts for workflows and users in Server Terminal GUI Tool, also added sorting by column. * Added new Parameter type "Flow Control" for ConcreteModules. This type of parameter will allow module to be started without transferring any data from parents * Added file conversion for local IDA download, added IDA download to include metadata. * Added validation status messages for modules and added more information in control bar's status about validation. Added support to navigate the ViewPort to those dataSinks which have 20+ instances. * Fixed "infinite waiting module" bug, where a module is completed but the child module waits forever. * Changed the GUI design of number of arguments which has more clear interface for users and will allow user to make module to have up to 99 arguments. * Updated Help Menu. Now it contains following items What's New, FAQ, Quick Start Guide, Video Tutorials, User Guide, Server Guide, Glossary, Pipe XML Overview, Pipeline Forum instead of only "Contents". * Fixed XSD document for Pipe file format. Optimized the Pipe XML file to decrease its size. * Fixed bug when a node is connected, user should not edit the parameter value by double click or right click. * Fixed to stop validation when user hits Stop button. Also added more feedback about validating modules and remote files which can be useful for big workflows which take more time for validation. * Fixed Personal Library which was freezing whole Pipeline when refreshing. * Fixed remote Browse button in Infinite dialog to open Remote File Browser instead of local file browser. * Fixed server manager to get active jobs and queued jobs count directly from drmaa engine * Fixed to moduleGroup to display accurate status instead of just showing waiting for some cases. * Fixed bug for Mac OS X Leopard which was hiding all content of dialog when Remote checkbox was clicked to be selected. Also fixed the width of browse button for Leopard. * Fixed bug where a module is incomplete, clicking "show details" in pipeline control bar returns information about first errored module. * Fixed to remove parameter's base when parameter doesn't have tranformation. * Fixed to show more than 10 argument inputs in Edit Parameter Values Dialog. Added Parameter number on appropriate labels. * Fixed user will be able to add, delete and edit annotations when workflow is in execution status. * Fixed active sessions drop down will be scrollable if the list is long, and it will be ordered by start time. * Changed textfield to combobox to help users easily select base parameter instead of typing it. * Fixed IDA will create folders as series' collection name, data sink for each download only contains files in that download; if there are more than one type of files downloaded, data sink will select all applicable types * Fixed remote browser will select the current path if no item is selected, added remote browse to IDA. * Fixed found memory leaks * Minimized number of requests to DB. In previous versions Pipeline was creating multiple times connections to DB server to put data there. Now Pipeline uses one connection for multiple columns of the same DB table. * Fixed to shutdown DRMAA Engine properly when Pipeline shuts down * Improved Server Sweeper to stop the running workflows before resetting them. Also added interrupt calls to release submitLocks if they are acquired at reset time. * Fixed cancelled event to support modules which have multiple connections and local/remote executions. * Fixed bug when save as script/makefile, the output file's line separator should be line return "\n" * Fixed bug where input of a module has multiple types, the parent module will output to the most common type between input and output, instead of the first type from output. * Fixed validation will get correct number of cardinality of modules with .list files, and validation will report error if cardinality exceeds 3000. Server changer will change the server of .list files as well. * Included in IDA: refresh list button after connected. Fixed IDA bugs: it properly gives an error when user downloads invalid series, it will delete partially downloaded files when user hit cancel, also file translation will stop sooner. Changes in build 1835 (v4.2.1) * Added: Number of queued jobs in server status * Added: Dynamic server preferences for Pipeline server * Added: Modes for directory based access control * Fixed: new provenance file format * Fixed: provenance file will be generated only if it is enabled in the workflow * Fixed: empty server status on Windows bug * Fixed: password dialog not rendered on Windows for connections window * Fixed: "Authentication done" message not changed even when it is connected * Fixed: correct staging status after reconnect * Fixed: to show Module name and ID always when opening popup description box of the module * Fixed: to show ModuleGroup Live information in PopupBox which was implemented only for ConcreteModules * Fixed: to show proper count of connections when connected to new (not saved ) connections * Fixed: to change URL compatibility when switching the Type of parameter from File to String or vice versa Changes in build 1808 (v4.2) * Added: Provenance * Added: Scripting * Added: Intelligence * Added: Remote File Browser * Added: Server status which displays server's queue utilization as a percentage * Added: Support to resume all running jobs after server restart * Added: Directory-based executable access control * Added: Server fail-over support * Added: New module statuses: Initializing, Staging, Backlogged, Cancelled * Added: Ability to show speed and estimated time when staging files * Added: Staging status message for Data Modules * Added: More detailed information for modules in their popup info boxes * Added: Hostname and port on data modules * Added: Support for multiple server libraries and its status messages. * Added: Popup menu in Data Modules dialog for Cut,Copy,Paste,Select All functions * Added: Data module usability. Added undo & redo and shortcut keys for Find & Replace. * Added: Refresh button on active sessions dialog * Added: Ability to save and open to/from Personal library * Added: Default connection to LONI's default server * Added: Ring notations for exported parameters in workflow's UI * Added: 3 state buttons in module overview box * Added: Ability to select multiple inputs when parameter has multiple arguments * Added: Link to execution logs dialog on control bar when execution is done * Added: More information about workflows in server terminal manager ( for administrators only ) * Removed: Saving password feature. Passwords will not be saved. * Fixed: Transformations * Fixed: Connections to be unique * Fixed: to not allow more than one connections with data sink when it is not dir dump * Fixed: to continue workflow execution when one of instances failed * Fixed: to not send email when user manually stops the workflow execution. * Fixed: to automatically find the file type when parameter type is any type of file * Fixed: to create personal library directory if it doesn't exist or has been removed * Fixed: to not freeze Pipeline UI while loading server library * Fixed: to send server library updates to all connected users when there is a library change * Fixed: variables to work properly on local executions * Fixed: increased maximum number of simultaneous IDA download * Fixed: leading spaces in executable path of modules to execute properly * Fixed: to reconnect properly when user changed IP address. * Fixed: server changer tool to keep user defined variables as they are * Fixed: connections to exported parameter to be copied * Fixed: to show job id when job is queued * Fixed: to show appropriate icon for connected and disconnected status * Fixed: command line run locally only if executable location is localhost, sync is enabled automatically * Fixed: active sessions quick reconnect to reload open workflows with lost connections Changes in build 1545 (v4.1) * Fixed: Fixed command line switch and multiple arguments bug * Fixed: When user decreases argument count extra filled values will be removed * Fixed: Changing to List file when parameter type is changed from File to String. * Fixed: Ctrl+E (Command+E) to open edit Module window Changes in build 1528 (v4.1 Alpha) * Added: New visualization tool called "Viewer" * Fixed: Stop workflow command stops local executions too, but not Viewers * Fixed: Workflow continues execution on reconnect, even if it has local and remote executions * Fixed: Execution logs dialog updates Changes in build 1500 (v4.1 Alpha) * Added: Find and Replace functions in Data Modules dialog * Changed: Variables are now inherited to child module groups * Fixed some bugs Changes in build 1476 (v4.1 Alpha) * Added: New tool Server Changer at Tools menu which helps to change server addresses of multiple modules and parameters * Fixed: Module groups wasn't showing execution time * Fixed: Data modules now have correct cardinality value * Fixed: Wrap validation error message * Fixed: Execution of local module with remote parameters and otherwise Changes in build 1455 (v4.1 Alpha) * Added: Workflows may involve modules that run either on grid server or on local host * Added: Pipelines validates global environmental variables * Added: Validation now includes acceptance of the workflow * Added: "Save connection" button in the Connection Credentials dialog * Added: Font styles in annotations * Added: Data module editor now shows argument count for input/output data * Added: Cut,copy, paste operations for annotations * Added: Alias for Redo operation. Now Redo has Ctrl+Y and Ctrl+Shift+Z shortcuts on all platforms * Added: Remove icon button in Edit module dialog which appears only when icon is set to module * Added: Copy icon, paste icon, remove icon functions in module's popup menu * Changed: Pipeline messaging. This version can't connect to old version servers * Changed: Make module group darker when it is selected * Changed: Create module group command now includes annotations too * Changed: Top level menus * Changed: Modules which execution stopped by stop button show Incomplete state * Changed: Select all command now selects annotations too * Changed: Pause button replaced with grey (not clickable) play button, because pause function is not still supported * Changed: Cloning of module groups to non XML dependent * Changed: In Output files panel paths became selectable, users are able to select them * Changed: selectionBox includes annotations too * Changed: Now pipeline uses system's temporary location for its temp files * Changed: Module's parameter tooltip and description box to show parameter values and file types * Changed: default port from 8000 to 8001 * Changed: Pipeline's XML files will be smaller because they keep module icon resized info not the whole icon * Fixed bug: Main workflow timer roll-back problem when finishing file transfers * Fixed bug: Undo for the group from library now works properly in case if user was inside the group at undo time * Fixed bug: Show validation error message for every (not only for one) module with cardinality error * Fixed bug: Active sessions dialog now updates active sessions at runtime * Fixed bug: Execution log dialog now responds to keyboard shortcuts such as Command + W * Fixed bug: Get Files button in Output files dialog becomes enabled after execution completes for every instance * Fixed bug: Update passing messages to allow data transfers for Server Terminal GUI * Fixed bug: Changed default execution status times to be 0 instaed of -1 at XML serialization * Fixed some minor bugs as well Changes in build 1359 (v4.0.5) * IDA library has been updated * Added support for serializing/deserializing binary provenance information into workflows * updated execution engine to support DRMAA 1.0 * removed a thread race in the execution engine * the Pipeline log file location can be changed in the preferences.xml file * privilege escalation copying now uses a single cp command instead of many smaller ones * fixed a bug that sometimes prevented a user from connecting two parameters Changes in build 1322 (v4.0.4) * empty modules do not appear in the personal library anymore * fixed a bug where selecting a different workflow tab would try to open a recent workflow * fixed a copy/paste issue on module groups with connections * fixed a bug where modules saved with exported params, were still exported when dragged in from the personal lib. * updated the hsqldb library Changes in build 1314 (v4.0.3) * revamped IDA back end * data sources/sinks are now separated from regular modules in the personal/server library Changes in build 1307 (v4.0.2) * changed the IDA hostname Changes in build 1296 (v4.0.1) * fixed a bug in the active sessions dialog when trying to remove multiple sessions at once * fixed a translation issue with side effect outputs * fixed a bug in the execution log window * added the Pipeline logo to the main frame icon Changes in build 1255 (V4.0) * fixed a bug with hyper links that was introduced in the last build * new JavaApplicationStub for OS X Changes in build 1242 * Minor GUI fixes everywhere * The Personal Library is loaded dynamically * Valication now checks for invalid user defined variables * Program start time should be faster in all OSes * Added more support for Pipeline server administration * Validation now checks for misuse of base parameters * Command Line Interface rewrite Changes in build 1216 * You can set environment variables in your preferences * Reconnecting to workflows is much easier now. (They won't all explode onto your screen) * Reduced memory footprint * Commands and output files are loaded dynamically in the GUI * You can now download outputs that are directories after execution * Some validation bug fixes * Validation now checks that base and cardinality parameters are valid * Bug fixes for GUI drawing * Fixed a couple issues with the Connections Dialog * Updates to the command line interface * Added support for OS X 10.5 (Leopard) * Fixed a bug that would sometimes affect list generation * Fixed a race condition that occurred when hitting stop and reset back to back * Local system execution errors are now displayed in the error stream tab (in red) * Lots of back end work Changes in build 1156 * Sever library now animates during execution or when its empty (can be disabled) * Validation is again more thorough than before * Bug fixes in Copy/Paste for data sinks and sources * Fixed where data modules show up in the library panels * Personal Library and Variables Dialog remain on top of the main frame * A bug fix when staging local files for remote execution on Windows * Server side authentication is now synchronized to 1 thread at a time * Fixed a problem with too many open file descriptors for non-grid execution * Also fixed an issue with execution stops Changes in build 1127 * Great improvements in network speed and stability * Validation is much faster, and more thorough * Fixed several GUI bugs relating to execution, validation and general screen updates * Fixed a bug that wouldn't allow a specific filetype to connect to a 'File' input * Resolved some issues with Windows execution * Icons are now stored inside the .pipe files * Fixed a bug in selecting all modules through the edit menu * Added some keyboard shortcuts * Fixed prestaging bugs * Fixed a bug in execution when a command only had 1 argument Changes in build 1062 * Added a splash screen * Progress of file copying is now drawn to the screen when a module completes * Another revision of the variables dialog * Many dialogs now have keyboard listeners, and the personal library and variables dialogs remember their positions * Validation catches many more errors now (including file type checking) * This may cause problems for workflows that had improperly connected input/outputs before * Validation status is drawn to the screen more accurately * Added support for dragging and dropping .pipe files from the desktop to the workflow area * Fixed several issues with the server and personal libraries * Known issue * Some executables that are executed locally in Windows Vista can be problematic. Changes in build 1027 * Fixed bug with communicator not sending problem messages * DownloadManager is more informative on failure * PipelineControlBar no longer defaults to "Validating Workflow" during execution * Preference dialog tweaks * Changed the dmg mounting name to LONI Pipeline, added optimization * Fixed memory leak when workflows close * Fixed bug in UNIX release scripts * Changed the transformations list in the edit advanced parameter dialog to be drag-able * Optimizations in the Java2D rendering * Fixed high CPU usage bug * Increased WorkflowAnimator action period to save CPU cycles * Moved update checker off Swing thread * Animator now stops processing updates after workflow is complete * Validation error is now thrown for input parameters that have no file types selected * Added JDOM to About Box * Pipeline Server Utility GUI added Changes in build 980 * many small bug fixes Changes in build 967 * FEATURE: New searchable Server and Personal Library * FEATURE: Tagging, Modules can be tag with any information which is searchable * FEATURE: The Pipeline will check for program updates at startup * FEATURE: Workflow can now be paused and stopped at anytime * FEATURE: User definable file types * FEATURE: Inputs can now accept multiple file types * FEATURE: Bug Reporter, please submit your bugs * FEATURE: Feedback for pre-execution events * validation now has a more robust check for the correct number of files going from one module to another * module groups resizes correctly after having a parameter exported * fixed annotations loading * fixed a bug with editing infinite cardinality file parameters * module groups edit dialog can be reached by right clicking * fixed a bug where some needs file did not get uploaded * added a new download download dialog * status message is optimized to save the amount of information going across the network * fixed a bug where connections can be made even if that exact one exist already * file transfers can now be stopped while in progress * data sinks can be marked as directory dumps, all outputs connect to a directory dump will be placed at the dump location * Recent files now displays the correct number of files defined in the Preferences * fixed the file finder on windows * validation will now check to make sure that all input values in an enumerated list are defined in the module definition * job times now displays the correct time after a reconnect * commands.log file is now in XML format and is printed out in order * annotations will now load properly * new Log Viewer Dialog * data sent across the network will now be compressed to increase transfer speed * validation now checks for malformed variables * fixed a bug where validation hangs if the server you're trying to connect to is down * added a GUI for selecting user defined filetypes * improved variables dialog * the about box is fixed * personal and server library has new icons to identify: data modules, modules, and module groups * some keyboard shortcuts has been updated Changes in build 822 * FEATURE: New icons for OS X builds and integration with Finder * FEATURE: New icons for pipe files * fixed a bug where directories before and after workflow execution were not being copied over correctly * fixed a bug where filetypes with multiple dots, eg someFileName.nifty.gz, were not being sent over to the server correctly * validation bug fix (validation will throw an error if the user puts in a directory for the executable path Changes in build 800 * FEATURE: IDA access * FEATURE: Validation can be called without executing the program * FEATURE: Module Groups * FEATURE: Data sources and sinks * FEATURE: Packages for OS X and Windows * FEATURE: Launch workflows in the GUI from the command line, example: java -jar Pipeline.jar myworkflow.pipe * FEATURE: Module definitions have support for two types of authors now (module and executable) and citation info as well * FEATURE: Once you submit a workflow to the server for execution, you can close the client. Reconnect later, and your workflow will reopen showing you its current status * FEATURE: The Pipeline now works on Windows Vista too * fixed parameters that output strings * more thorough validation including cardinality checks between modules * DRMAA native grid spec is now a preference (used to be hardcoded) * new preference for deleting workflow data after user resets workflow execution * commands.log file is now closed after writing * added citations to the module panel (removed metadata attributes) * fixed bug when trying to convert a parameter from an enumerated type to a string (bug 001693) * the browse button is back in the edit concrete module panel * validation checks enabled parameters in addition to required parameters * validation throws an error if a required parameter is not enabled * made copy/paste a little more robust * more detailed author information for modules * fixed a bug where users couldn't remove a base from the advanced parameter dialog * added a dialog that captures output and error streams and displays them to the user in the program * preferences dialog lets client users adjust thread pool size for execution * validation no longer checks output directories of type File for existence * validation fix, disabled parameters with values are no longer validated * fixed a regression that caused problems when editing enumerated parameter types * fixed the column size in the parameter table when editing module parameters * validation bug fix (no longer checks for needs files on List parameters) Changes in build 695 * bug fix for file transfer post processing * added logic to handle cycle detection in Validation * added several filetypes including some for the Brains2 application * fixed a Validation bug when checking extension that contain a dot * race condition in GUI updates at the end of workflows is now fixed * FEATURE: New tools available for monitoring a server's health * FEATURE: Annotations (notes) in a workflow. Use these to describe what different parts of your workflow do. * FEATURE: Program remembers its last size and position upon closing and opens up there (if possible) on the next run * Tons of bug fixes in the workflow editing area * FEATURE: You can download output files of parameters through the output log viewer * FEATURE: Thread pooling which allows limiting of the number of simultaneous threads/jobs that are running * FEATURE: Parameters added in the Edit Module dialog will now show up below the currently selected parameter * FEATURE: You can save output files after execution from the Log Viewer without having to bind a value. * personal library and variables dialog are now positioned inside the main frame (so they don't draw off screen) * removed old code and old libraries for a smaller binary and total package * bug fixed when trying to reorder parameters Known issue(s) in build 694 * Repainting of the workflow may leave artifacts sometimes Changes in build 661 * Many improvements in validation. It is much more strict now as well. * New dialogs for editing modules. Provenance attributes can be edited on workflows and module descriptions * Directory copying has been improved * Lots of little nice feature additions * Many bug fixes throughout the code. Changes in build 8 * Fixed a bug that would sometimes drop a network connection * Permissions on remote files are now handled by the server (no more chmod 777'ing destination folders) * Reduced the size of status updates * Improved the way status updates drawn to the GUI * Module and workflow timers that show you how long execution has taken * new log viewer * reorganized menu items * added Afni Brick, Afni Head and Nii.gz filetypes * the program will prompt you to make a connection to a server when you hit play, if the connection hasn't already been made * the workflow repaints in a smarter way (reduces CPU usage) * lots of bug fixes and robustness added throughout * the server's memory footprint has been significantly reduced Known issues in build 8 * Repainting of the workflow may leave artifacts sometimes * If a module has an error during execution, it will incorrectly turn green once the entire workflow completes