Hotfix 5.6-HF001
Fix the compatibility with BIOVIA Pipeline Pilot server 2021 for the modules "Client Automation & SWAPP" and "Pipeline Pilot Data Functions".
New in 5.6
SWAPP
New input type: REST API Resource
A new input type REST API Resource has been added to the SWAPP. It can be used with paremeters of type String, Number or Arrays of String or Number. With this input type, you can populate the options using an array of values returned from a REST API endpoint. Furthermore, other parameters of the same protocol form can be specified as dependency. When those dependencies' values will change, a new call to the API Endpoint will be triggered, passing along the dependencies' values.
The final display can be a select dropdown, a multiple checkbox input for arrays or a group of radio buttons for single values.
Read the tutorial to learn how to implement it using a Pipeline Pilot protocol as API Resource.
New option for Molecule Input: MarvinJS
Until now, only Ketcher and Biovia Draw (in Analyst or IE) were available as molecule Sketcher. You can now also use MarvinJS, provided it is deployed on your Pipeline Pilot server. To do so, simply specify the relative path to the editor in the Pipeline Pilot Global Properties.
Simplified Menu
The three settings in the SWAPP Menu (Register a protocol, Edit the menu, Toggle Synchronisation) have been merged in a single "Settings" button. When registering a new protocol, you can now navigate back a step, for example to select another protocol, without having to close the window.
XMLDB Root option
A new Global Option SWAPPProtocolsRoot has been added and can be defined in Pipeline Pilot's Global Properties. It allows configuring the root folder when browsing the Pipeline Pilot's XMLDB from the SWAPP.
This is especially usefull if all the protocols you want to register in the SWAPP are located in the same folder in Pipeline Pilot. Specifying this folder, only its protocols and subfolders will be listed when registering a new protocol in the SWAPP.
New in 5.5
Client Automation & SWAPP module
Legend
The Legend properties are now accessible for each visualization, using the JavaScript API or the Pipeline Pilot visualization components:
The modification of legend item parameters is done via an array in which the legend item is simply referenced by its name (e.g. 'Data table' or 'Color by').
New Pipeline Pilot component: Open Document
You can now open an existing document in your Pipeline Pilot protocol, then modify it by adding data tables, pages, visualizations, ...
The component 'Open Document', available in the Discngine/TIBCO Spotfire Connector/Client Automation/Document/ folder, allows you to open document from the library, or from a local folder if you are not using the Web Player, possibly applying a configuration block, and trigger the execution of a function once the document is successfully opened. The opening of the document is either done when the page is loaded (using the Pipeline Pilot startup functions), or is added to the body of the web page, or is embedded in a function that can be called on demand.
Refer to the 'Discngine/TIBCO Spotfire Enabled Protocols/Client Automation/Miscellaneous Examples/Open an existing document and modify it' protocol example to see an application of this component:
Force the range of scale axis
WARNING: The data type of the min and max range parameters were Double in PP, it is now string in order to accept more data types (DateTime particularly), so if you set visualization components with double or integer range values in some protocols, you will have to specify the data type 'Integer' or 'Real' to force the formatting.
Minor changes
Client Automation & SWAPP module
Limit data using expression
This property was already available when using the JavaScript API, but was not exposed in the Pipeline Pilot components. It is now the case for all the visualization components:
Bug fixes
- Client Automation & SWAPP module:
- When configuring a visualization, it was not possible to set the Fixed color mode for markers without defining a color rule.
New in 5.4
Client Automation & SWAPP module
IronPython scripting
With IronPython scripts it is possible to configure and automate a lot of different aspects of a TIBCO Spotfire® analysis and add behaviors to it. All the Spotfire Analyst API is accessible from IronPython scripts. In TIBCO Spotfire® an IronPython script can be added to several places in the analysis. Scripts can be executed from action controls in the Text Area, from the Graphical Table or the KPI Chart or be triggered by Document Property changes.
For this version 5.4 of the Connector, efforts have been made to generalize the use of IronPython scripts in Pipeline Pilot protocols. These scripts can be added at the visualization level using the dedicated parameter of the visualization component, or at the protocol level using either the Add Custom Script component or the Execute IronPython Script component.
Note that IronPython scripts executed from a Pipeline Pilot protocol do not need to be trusted in TIBCO Spotfire®, and the user does not need the Author Scripts license.
At the visualization level
All the visualization types are now available as Pipeline Pilot components. They do not expose all the Spotfire properties, but they all have a Custom Script parameter that allows you to complete the visualization setup with an IronPython script.
So, if the property you need is not yet exposed as a parameter of the visualization component, just write the IronPython script in this 'Custom Script' parameter. For example, if you want to define error bars for your visualization like this:
Simply write the following script in the 'Custom Script' parameter of the component of your visualization:
dataviz.YAxis.ErrorBars.Enabled = True
dataviz.YAxis.ErrorBars.UpperExpression = "StdErr([LogD])"
dataviz.YAxis.ErrorBars.FixedColor = Color.Red
dataviz.YAxis.ErrorBars.ShowEndCaps = True
At the protocol level
You may also need to run a script at the protocol level, for example to apply the same change to multiple visualizations, or to manipulate objects other than visualizations like data tables, document properties, ...
To do this, you will use the component Execute IronPython Script in which you will have access to all the objects in the document (pages, visualizations, data tables, document properties, ...).
Specifically, you can also use the Add Custom Script component, in which you are able to easily reference the last data table, the last page and the last visualization created.
A few example protocols have been added to the Connector collection to make it easier for you to get to work with these features:
Helpful links for writing IronPython scripts
IronPython scripting in TIBCO Spotfire® does not necessarily require a lot of python skills, however it does involve using their API. Here are a few links that you may find helpful:
- TIBCO Spotfire® Community - IronPython Scripting in TIBCO Spotfire®
- Spotfire Developer
- The Spotfire IronPython Quick Reference
Minor changes
Client Automation & SWAPP module
Coloring in visualizations
Coloring properties have been added for part of the visualization components:
A new example protocol showing the coloring of a scatter plot is available in TIBCO Spotfire Enabled Protocols/Client Automation/Miscellaneous Examples.
Bug fixes
- Client Automation & SWAPP module:
- The 'HTML TIBCO Spotfire® Viewer' component has been fixed: it did not take into account the 'On Success Function' and 'On Error Function' parameters of the 'Apply Document Editor Changes' components present in the flow.
- In the Web Player, when you switched the Edit mode (Viewing/Editing), the visualizations were redrawn.
- Pipeline Pilot icons were provided in the collection to distinguish TIBCO Spotfire® specific components from deprecated Connector 4.x components from other components of the collection. But this icon customization was not really stable, so we chose to limit it to icons for deprecated Connector 4.x components and to the classical Discngine icons.
- Data Functions module:
- An error blocked the opening of a document containing columns calculated by a Pipeline Pilot protocol when the TIBCO Spotfire® Analyst client was launched at the same time (i.e. when the Analyst client was launched by double-clicking on the DXP file). There are now no more blocking errors, but the data table needs to be refreshed for these columns to be calculated. A notification will inform you about this.
New in 5.3
Data Functions module
Pipeline Pilot Calculated Columns - beta
In this new version 5.3 we have extended the TIBCO Spotfire® Calculated Columns concept to use the Pipeline Pilot protocols as new column calculation engines. It is supplied with the Data Functions module.
We provide four generic functions (one per data type), taking as argument the Pipeline Pilot protocol path and all the required data (records and parameters):
More information can be found in the section dedicated to this topic in our User Guide.
Client Automation & SWAPP module
TIBCO Spotfire® Filters handling
We also have integrated Pipeline Pilot components allowing you to manipulate the filters in your TIBCO Spotfire® documents.
In the Add Filter component, all the filter types and their parameters are available, except the hierarchy filter. The components Reset Filter and Set Filter Visible are applicable irrespective of the type of filter.
Known limitation: depending on the number of decimal digits displayed for the filtered column, there may be differences between the value displayed in the TIBCO Spotfire® data table and the value to be set (due to rounding).
Execute IronPython Script component: new parameters
The Execute IronPython Script component allows you to add a custom IronPython script in your protocol, for instance because you need a feature not yet implemented in the Connector collection.
This component already existed in previous versions, but in this new version 5.3 we have improved it by adding two parameters: One Call per Record and Add Script at the end.
One Call per Record: when the component is piped into a multi-record branch, it can generate either one call for each record or one call for all.
Add Script at the end: this parameter defines where the IronPython Script will be added in the call stack. It is only meant to be used in very specific cases and should likely be left unchanged.
Minor changes
Client Automation & SWAPP module: new Tools menu options
In the TIBCO Spotfire Analyst client (Tools > Client Automation), you can now either create a new document containing the Client Automation panel (Create new document option), add the Client Automation panel to an open document (Display panel option), or reset the panel URL to the home page (Reset to home page option):
Bug fixes in the JavaScript API
- When using spotfireDocumentEditor.setDocumentProperty on an existing property, the property was removed before being added again. Now this function only modifies the value, without touching the property, which makes it possible to keep the possible links with scripts, in Text Areas, ...
- When the SpotfireDocumentEditor was modified, the active page layout was set to Vertical, aligning all visualizations vertically, instead of keeping the layout defined by the user.
- The function getRunStdOut always returned undefined.
Documentation
- New tutorial: How to set up a Reverse Proxy for Pipeline Pilot and TIBCO Spotfire.
- New tutorial: How to update license keys.
New in 5.2
TIBCO Spotfire® Library Upload REST API integration
Available from TIBCO Spotfire® version 10.2, the Library REST API lets external applications upload Spotfire Binary Data Format (SBDF) files to the Spotfire library. In this new version 5.2 of the Connector we have integrated the use of this REST API into the Pipeline Pilot collection. In this way, you can now write SBDF files from your Pipeline Pilot protocol directly to the TIBCO Spotfire® library:
You will find in our installation guide all the necessary information to configure the use of this REST API. The official documentation is available here.
TIBCO Spotfire® Automation Services REST API integration
Until this new version of the Connector, the only solution to execute Automation Services Jobs from a Pipeline Pilot protocol was to use the Client Job Sender tool provided by TIBCO Spotfire®. The main disadvantage of this solution is that the Client Job Sender tool can only be used by a Pipeline Pilot hosted on a Windows server.
From TIBCO Spotfire® version 7.13 a REST API is available to trigger Automation Services jobs from an external application. We have therefore integrated the use of this REST API into the Pipeline Pilot collection of the Connector 5.2, creating the new component "Automation Services/Run Job with REST API":
The protocols developed with this component are therefore compatible with Pipeline Pilot servers other than Windows, another significant advantage of this new solution is that it is easier to configure and more secure.
You will find in our installation guide all the necessary information to configure the use of this REST API. The official documentation is available here.
New in the SWAPP
Protocol shortcut link
A new section "Shortcut link" is now available in the SWAPP when running a protocol. From there you can generate a link to the current protocol that will allow you to reload the form with the exact same parameters you have when clicking the "Get Shortcut Link" button. This can be useful if you have a protocol that you always want to use with the same parameters, or if you want to share the filled form with a colleague. Furthermore, ticking the "Auto Submit" option will trigger the submit of the form as soon as the page loads from the link.
File(s) to upload as input
A new input type has been added which allows you to give the user the ability to upload file to Pipeline Pilot. The uploaded file(s) will be placed in $(JobDir)/temp/$(FileName)
. Optionally you can define a list of allowed extensions.
Real-time Synchronisation
Real-time Synchronisation (see below) has been implemented in the SWAPP and can be toggled on/off to match your needs.
Templates
A template is a TIBCO Spotfire® document (DXP file) from which we have removed the records of all the data tables. We provide in this new version two components allowing you to manipulate the templates: "Client Automation/Save Template" and "Client Automation/Apply Template".
With the "Save Template" component, you can save a DXP from your current analysis. The DXP will be identical to your analysis, except that the data tables are empty.
With the "Apply Template" component, you can apply a saved template on your current analysis. In this way, the data tables of this analysis are exported and re-injected into the template, and you will have your data with all the parameters of the template (pages, visualizations, ...).
New Pipeline Pilot icons
In order to better distinguish the difference between the components coming from Connector 4.1 and those coming from Connector 5+, we have added new Pipeline Pilot icons in 5.2.
Once the new collection is installed, all your protocols will contain components that display the Component 4.1 icon. We therefore also provide in the collection a protocol to update the icons in your protocols already developed with the Connector 5+. This protocol is available here: Utilities/Change warning icons to v5 icons
.
Minor changes
New
- New component "Data Management/Add Data Table from Information Link" (and its JavaScript function SpotfireDocumentEditor.loadDataTableFromInformationLink).
- SpotfireDocumentEditor.addCustomScript: allows to add a custom python script to the editor.
- SpotfireDocument.setSpinnerState: allows to add a spinner containing a custom message to indicate when Spotfire is working.
- SpotfireDocument.getActiveMarkingAsync: gets the active marking name of the active visual.
- SpotfireDocumentEditor.removeRowsFromDataTable: removes all rows from a data table.
New in visualizations
In the JavaScript API and the Pipeline Pilot collection:
- Pie Chart: size by properties added.
- Heat Map: creation of the visualization, x-axis, y-axis, cell values and trellis properties added.
Modified
- SpotfireDocument.markRowsAsync: now exposes the operation to use when combining this selection with previously set selections.
- SpotfireDocument.openDocument: can now open a document from a local file; only available if you use the TIBCO Spotfire® Analyst client.
- "Web Panel" properties and preferences are renamed by "Client Automation".
Bug fixes
- Fix the versionning of TIBCO Spotfire® extensions (.dll files) in order to keep the backward compatibility with saved DXP files after the upgrade of the Connector.
- Include correct permissions for the Pipeline Pilot collection provided in the package (for Linux servers).
Documentation
- New tutorial: How to migrate 4.1 to 5.x Pipeline Pilot protocols.
- Warnings regarding trust in the Data Functions appearing in TIBCO Spotfire® 10.3.
- SWAPP example protocols now documented.
New in 5.1
Client Automation home page
We now provide with the Pipeline Pilot collection a protocol that generates a home page for the Client Automation. It will give you access to the SWAPP, as well as to the examples and tutorials available.
This home page also lists the protocols saved on the Pipeline Pilot server in the folder
Protocols/Discngine/TIBCO Spotfire Enabled Protocols/Client Automation/
, so feel free to add
your development protocols to test them in the Web Player or Analyst.
Real Time Synchronization
When developing Pipeline Pilot workflows for TIBCO Spotfire®, developers can now activate the "Real Time Synchronization". Thanks to this feature, it is easier to share data tables between Pipeline Pilot and TIBCO Spotfire®: the upload and persistence processes are automated, and you can focus on the required data without sending data back and forth when they already have been sent.
On a technical note, the Real Time Synchronization takes care of tracking changes in your Data Tables and updates the caches on Pipeline Pilot based on your settings (whether you use a manual trigger or the form submission). The caches are scoped to the user and the session.
This feature is available on demand through the Real Time Synchronization parameter of the HTML TIBCO Spotfire Viewer component. A tutorial is available at the following location:
Protocols/TIBCO Spotfire Enabled Protocols/Client Automation/Real Time Synchronization/Real Time Synchronization Examples Home
.
The Real Time Synchronization has also been implemented on the SWAPP and can be toggled in the menu bar. Note that it is active by default.
Pipeline Pilot Data Function with multiple files as results
Previously, to have multiple data tables generated by data functions in your TIBCO document, you had to store as many data functions as data tables and call them separately. This required a lot of manual intervention. Moreover, it was far from optimal in terms of performance because the "Pipeline Pilot Data Function" tool was called as many times as there were data tables to load, while some steps could be shared.
There is now a way to define the expected results files in the protocol and therefore to have a data function with multiple data tables as results.
Here is an example of how to configure a protocol with 3 output data tables:
As you can see, you just need to add the file name as the output property. The file name (without the extension) will be used to name the data table in TIBCO Spotfire®.
Improvement of the Pipeline Pilot reporting in the SWAPP
You can now use the Pipeline Pilot components from the Reporting collection that redirect to another protocol. This allows you for example to develop a sequence of protocols, each redirecting to the next, and execute them via the SWAPP.
New Pipeline Pilot Visualization components
The existing components have been improved. In addition, the Box Plot and the Pie Chart components have been added.
New in 5.0.1
This minor new version offers some minor bug fixes, improves the components of the Pipeline Pilot collection, and provides a quick start protocol to easily start with the collection.
New in 5.0 - Release Candidate
The Connector 5.0 is a major new version. The Client Automation, new name of the Discngine Web Panel, has been completely redesigned. The Pipeline Pilot Data Functions and the Automation Services Tasks have been improved in a minor way.
The very big change for this version is that the Connector 5.0 is compatible with TIBCO Spotfire® Analyst and TIBCO Spotfire® Web Player. Thus, the same Pipeline Pilot protocol can be executed either in the Analyst or in the Web Player.
This huge advantage requires modifying existing Pipeline Pilot protocols to replace Connector 4.1 components with Connector 5.0 components. Indeed, the Pipeline Pilot collection has also been entirely redesigned, and the collection 4.1 is now deprecated (components and example protocols) although backwards compatibility with the Connector 4.1 is ensured. Protocols developed in 4.1 will continue to work using Web Panel 4.1 embedded into the 5.0, but they will not be compatible with both Analyst and Web Player.
Client Automation
All is new for this product in the Connector 5.0: new TIBCO Spotfire® extensions, new JavaScript API, new Pipeline Pilot collections, and even new name! All is new, but as said before, the Connector 4.1 is included into the Connector 5.0, so the backwards compatibility is ensured.
SWAPP - Spotfire Web Application for Pipeline Pilot
We provide also with this new version 5.0 the SWAPP, a standalone web application designed to be used in the Web Panel and developped in collaboration with one of the top Pharmaceutical Companies. This application allows you to register Pipeline Pilot protocols and run them seamlessly as web forms from TIBCO Spotfire® Analyst or Web Player.
Pipeline Pilot Data Functions
Improvement of the Data Function registration
The connection with the Pipeline Pilot server is now manageable in the "Register Data Function" view. A status bar on the bottom of the screen informs you about your connection status. Also, "Login" and "Logout" buttons have been added in the screen, and the registering is not available until you are logged in.
Pipeline Pilot Automation Services Tasks
For this new release, efforts regarding Automation Services tasks have been focused on improving visualization tasks. Indeed, label properties have been added for Scatter Plots and Bar Charts, and you can now also add Pie Charts, Line Charts, Box Plots and Heat Maps.
Here is the list of all Pipeline Pilot tasks now available: