Changelog

If Pipeline Pilot 2021 is installed on your Pipeline Pilot server, an additionnal configuration step may be performed. Open the Pipeline Pilot server administration page, navigate to Setup > Server Configuration in the left panel. For the "SSL/TLS Security Level" setting, choose the "Older" option and press the "Save" button.
Warning: important changes have been made in TIBCO Spotfire® release 10.3 and later on how authorized users trust data functions. We strongly recommend that you read the official documentation and follow the TIBCO Spotfire® guidelines if you are concerned.

Hotfix 5.6-HF001

Fix the compatibility with BIOVIA Pipeline Pilot server 2021 for the modules "Client Automation & SWAPP" and "Pipeline Pilot Data Functions".

New in 5.6

A new input type REST API Resource has been added to the SWAPP. It can be used with paremeters of type String, Number or Arrays of String or Number. With this input type, you can populate the options using an array of values returned from a REST API endpoint. Furthermore, other parameters of the same protocol form can be specified as dependency. When those dependencies' values will change, a new call to the API Endpoint will be triggered, passing along the dependencies' values.

The final display can be a select dropdown, a multiple checkbox input for arrays or a group of radio buttons for single values.

Read the tutorial to learn how to implement it using a Pipeline Pilot protocol as API Resource.

REST API Resource

Until now, only Ketcher and Biovia Draw (in Analyst or IE) were available as molecule Sketcher. You can now also use MarvinJS, provided it is deployed on your Pipeline Pilot server. To do so, simply specify the relative path to the editor in the Pipeline Pilot Global Properties.

The three settings in the SWAPP Menu (Register a protocol, Edit the menu, Toggle Synchronisation) have been merged in a single "Settings" button. When registering a new protocol, you can now navigate back a step, for example to select another protocol, without having to close the window.

SWAPP Settings

A new Global Option SWAPPProtocolsRoot has been added and can be defined in Pipeline Pilot's Global Properties. It allows configuring the root folder when browsing the Pipeline Pilot's XMLDB from the SWAPP.

This is especially usefull if all the protocols you want to register in the SWAPP are located in the same folder in Pipeline Pilot. Specifying this folder, only its protocols and subfolders will be listed when registering a new protocol in the SWAPP.

SWAPP Protocols Root

New in 5.5

The Legend properties are now accessible for each visualization, using the JavaScript API or the Pipeline Pilot visualization components:

Legend parameters

The modification of legend item parameters is done via an array in which the legend item is simply referenced by its name (e.g. 'Data table' or 'Color by').

You can now open an existing document in your Pipeline Pilot protocol, then modify it by adding data tables, pages, visualizations, ...

The component 'Open Document', available in the Discngine/TIBCO Spotfire Connector/Client Automation/Document/ folder, allows you to open document from the library, or from a local folder if you are not using the Web Player, possibly applying a configuration block, and trigger the execution of a function once the document is successfully opened. The opening of the document is either done when the page is loaded (using the Pipeline Pilot startup functions), or is added to the body of the web page, or is embedded in a function that can be called on demand.

Open Document component

Refer to the 'Discngine/TIBCO Spotfire Enabled Protocols/Client Automation/Miscellaneous Examples/Open an existing document and modify it' protocol example to see an application of this component:

Open Document example

WARNING: The data type of the min and max range parameters were Double in PP, it is now string in order to accept more data types (DateTime particularly), so if you set visualization components with double or integer range values in some protocols, you will have to specify the data type 'Integer' or 'Real' to force the formatting.

This property was already available when using the JavaScript API, but was not exposed in the Pipeline Pilot components. It is now the case for all the visualization components:

Limit data using expression

  • Client Automation & SWAPP module:
    • When configuring a visualization, it was not possible to set the Fixed color mode for markers without defining a color rule.

New in 5.4

With IronPython scripts it is possible to configure and automate a lot of different aspects of a TIBCO Spotfire® analysis and add behaviors to it. All the Spotfire Analyst API is accessible from IronPython scripts. In TIBCO Spotfire® an IronPython script can be added to several places in the analysis. Scripts can be executed from action controls in the Text Area, from the Graphical Table or the KPI Chart or be triggered by Document Property changes.

For this version 5.4 of the Connector, efforts have been made to generalize the use of IronPython scripts in Pipeline Pilot protocols. These scripts can be added at the visualization level using the dedicated parameter of the visualization component, or at the protocol level using either the Add Custom Script component or the Execute IronPython Script component.

Note that IronPython scripts executed from a Pipeline Pilot protocol do not need to be trusted in TIBCO Spotfire®, and the user does not need the Author Scripts license.

All the visualization types are now available as Pipeline Pilot components. They do not expose all the Spotfire properties, but they all have a Custom Script parameter that allows you to complete the visualization setup with an IronPython script.

Visualization components - Custom script parameter

So, if the property you need is not yet exposed as a parameter of the visualization component, just write the IronPython script in this 'Custom Script' parameter. For example, if you want to define error bars for your visualization like this:

Spotfire Error bars properties

Simply write the following script in the 'Custom Script' parameter of the component of your visualization:

dataviz.YAxis.ErrorBars.Enabled = True
dataviz.YAxis.ErrorBars.UpperExpression = "StdErr([LogD])"
dataviz.YAxis.ErrorBars.FixedColor = Color.Red
dataviz.YAxis.ErrorBars.ShowEndCaps = True

Custom Script example - Pipeline Pilot protocol

You may also need to run a script at the protocol level, for example to apply the same change to multiple visualizations, or to manipulate objects other than visualizations like data tables, document properties, ...

To do this, you will use the component Execute IronPython Script in which you will have access to all the objects in the document (pages, visualizations, data tables, document properties, ...).

Specifically, you can also use the Add Custom Script component, in which you are able to easily reference the last data table, the last page and the last visualization created.

A few example protocols have been added to the Connector collection to make it easier for you to get to work with these features:

Example protocols

IronPython scripting in TIBCO Spotfire® does not necessarily require a lot of python skills, however it does involve using their API. Here are a few links that you may find helpful:

Coloring properties have been added for part of the visualization components:

Coloring parameters

A new example protocol showing the coloring of a scatter plot is available in TIBCO Spotfire Enabled Protocols/Client Automation/Miscellaneous Examples.

  • Client Automation & SWAPP module:
    • The 'HTML TIBCO Spotfire® Viewer' component has been fixed: it did not take into account the 'On Success Function' and 'On Error Function' parameters of the 'Apply Document Editor Changes' components present in the flow.
    • In the Web Player, when you switched the Edit mode (Viewing/Editing), the visualizations were redrawn.
    • Pipeline Pilot icons were provided in the collection to distinguish TIBCO Spotfire® specific components from deprecated Connector 4.x components from other components of the collection. But this icon customization was not really stable, so we chose to limit it to icons for deprecated Connector 4.x components and to the classical Discngine icons.
  • Data Functions module:
    • An error blocked the opening of a document containing columns calculated by a Pipeline Pilot protocol when the TIBCO Spotfire® Analyst client was launched at the same time (i.e. when the Analyst client was launched by double-clicking on the DXP file). There are now no more blocking errors, but the data table needs to be refreshed for these columns to be calculated. A notification will inform you about this.

New in 5.3

In this new version 5.3 we have extended the TIBCO Spotfire® Calculated Columns concept to use the Pipeline Pilot protocols as new column calculation engines. It is supplied with the Data Functions module.

We provide four generic functions (one per data type), taking as argument the Pipeline Pilot protocol path and all the required data (records and parameters):

Add calculated column

More information can be found in the section dedicated to this topic in our User Guide.

We also have integrated Pipeline Pilot components allowing you to manipulate the filters in your TIBCO Spotfire® documents.

Filter components

In the Add Filter component, all the filter types and their parameters are available, except the hierarchy filter. The components Reset Filter and Set Filter Visible are applicable irrespective of the type of filter.

Known limitation: depending on the number of decimal digits displayed for the filtered column, there may be differences between the value displayed in the TIBCO Spotfire® data table and the value to be set (due to rounding).

Execute IronPython Script component: new parameters

The Execute IronPython Script component allows you to add a custom IronPython script in your protocol, for instance because you need a feature not yet implemented in the Connector collection.

This component already existed in previous versions, but in this new version 5.3 we have improved it by adding two parameters: One Call per Record and Add Script at the end.

Execute IronPython Script component

One Call per Record: when the component is piped into a multi-record branch, it can generate either one call for each record or one call for all.

Add Script at the end: this parameter defines where the IronPython Script will be added in the call stack. It is only meant to be used in very specific cases and should likely be left unchanged.

Client Automation & SWAPP module: new Tools menu options

In the TIBCO Spotfire Analyst client (Tools > Client Automation), you can now either create a new document containing the Client Automation panel (Create new document option), add the Client Automation panel to an open document (Display panel option), or reset the panel URL to the home page (Reset to home page option):

Client Automation menu

  • When using spotfireDocumentEditor.setDocumentProperty on an existing property, the property was removed before being added again. Now this function only modifies the value, without touching the property, which makes it possible to keep the possible links with scripts, in Text Areas, ...
  • When the SpotfireDocumentEditor was modified, the active page layout was set to Vertical, aligning all visualizations vertically, instead of keeping the layout defined by the user.
  • The function getRunStdOut always returned undefined.

New in 5.2

Available from TIBCO Spotfire® version 10.2, the Library REST API lets external applications upload Spotfire Binary Data Format (SBDF) files to the Spotfire library. In this new version 5.2 of the Connector we have integrated the use of this REST API into the Pipeline Pilot collection. In this way, you can now write SBDF files from your Pipeline Pilot protocol directly to the TIBCO Spotfire® library:

Library Upload REST API

You will find in our installation guide all the necessary information to configure the use of this REST API. The official documentation is available here.

Until this new version of the Connector, the only solution to execute Automation Services Jobs from a Pipeline Pilot protocol was to use the Client Job Sender tool provided by TIBCO Spotfire®. The main disadvantage of this solution is that the Client Job Sender tool can only be used by a Pipeline Pilot hosted on a Windows server.

From TIBCO Spotfire® version 7.13 a REST API is available to trigger Automation Services jobs from an external application. We have therefore integrated the use of this REST API into the Pipeline Pilot collection of the Connector 5.2, creating the new component "Automation Services/Run Job with REST API":

Automation Services REST API

The protocols developed with this component are therefore compatible with Pipeline Pilot servers other than Windows, another significant advantage of this new solution is that it is easier to configure and more secure.

You will find in our installation guide all the necessary information to configure the use of this REST API. The official documentation is available here.

A new section "Shortcut link" is now available in the SWAPP when running a protocol. From there you can generate a link to the current protocol that will allow you to reload the form with the exact same parameters you have when clicking the "Get Shortcut Link" button. This can be useful if you have a protocol that you always want to use with the same parameters, or if you want to share the filled form with a colleague. Furthermore, ticking the "Auto Submit" option will trigger the submit of the form as soon as the page loads from the link.

SWAPP Protocol shortcut link

A new input type has been added which allows you to give the user the ability to upload file to Pipeline Pilot. The uploaded file(s) will be placed in $(JobDir)/temp/$(FileName). Optionally you can define a list of allowed extensions.

SWAPP File input

Real-time Synchronisation (see below) has been implemented in the SWAPP and can be toggled on/off to match your needs.

A template is a TIBCO Spotfire® document (DXP file) from which we have removed the records of all the data tables. We provide in this new version two components allowing you to manipulate the templates: "Client Automation/Save Template" and "Client Automation/Apply Template".

Template components

With the "Save Template" component, you can save a DXP from your current analysis. The DXP will be identical to your analysis, except that the data tables are empty.

With the "Apply Template" component, you can apply a saved template on your current analysis. In this way, the data tables of this analysis are exported and re-injected into the template, and you will have your data with all the parameters of the template (pages, visualizations, ...).

In order to better distinguish the difference between the components coming from Connector 4.1 and those coming from Connector 5+, we have added new Pipeline Pilot icons in 5.2.

New icons in 5.2

Once the new collection is installed, all your protocols will contain components that display the Component 4.1 icon. We therefore also provide in the collection a protocol to update the icons in your protocols already developed with the Connector 5+. This protocol is available here: Utilities/Change warning icons to v5 icons.

In the JavaScript API and the Pipeline Pilot collection:

  • Pie Chart: size by properties added.
  • Heat Map: creation of the visualization, x-axis, y-axis, cell values and trellis properties added.
  • SpotfireDocument.markRowsAsync: now exposes the operation to use when combining this selection with previously set selections.
  • SpotfireDocument.openDocument: can now open a document from a local file; only available if you use the TIBCO Spotfire® Analyst client.
  • "Web Panel" properties and preferences are renamed by "Client Automation".
  • Fix the versionning of TIBCO Spotfire® extensions (.dll files) in order to keep the backward compatibility with saved DXP files after the upgrade of the Connector.
  • Include correct permissions for the Pipeline Pilot collection provided in the package (for Linux servers).

New in 5.1

We now provide with the Pipeline Pilot collection a protocol that generates a home page for the Client Automation. It will give you access to the SWAPP, as well as to the examples and tutorials available.

This home page also lists the protocols saved on the Pipeline Pilot server in the folder Protocols/Discngine/TIBCO Spotfire Enabled Protocols/Client Automation/, so feel free to add your development protocols to test them in the Web Player or Analyst.

Client Automation home page

When developing Pipeline Pilot workflows for TIBCO Spotfire®, developers can now activate the "Real Time Synchronization". Thanks to this feature, it is easier to share data tables between Pipeline Pilot and TIBCO Spotfire®: the upload and persistence processes are automated, and you can focus on the required data without sending data back and forth when they already have been sent.

On a technical note, the Real Time Synchronization takes care of tracking changes in your Data Tables and updates the caches on Pipeline Pilot based on your settings (whether you use a manual trigger or the form submission). The caches are scoped to the user and the session.

Real Time Synchronization

This feature is available on demand through the Real Time Synchronization parameter of the HTML TIBCO Spotfire Viewer component. A tutorial is available at the following location: Protocols/TIBCO Spotfire Enabled Protocols/Client Automation/Real Time Synchronization/Real Time Synchronization Examples Home.

The Real Time Synchronization has also been implemented on the SWAPP and can be toggled in the menu bar. Note that it is active by default.

Real Time Synchronization Collection

Previously, to have multiple data tables generated by data functions in your TIBCO document, you had to store as many data functions as data tables and call them separately. This required a lot of manual intervention. Moreover, it was far from optimal in terms of performance because the "Pipeline Pilot Data Function" tool was called as many times as there were data tables to load, while some steps could be shared.

There is now a way to define the expected results files in the protocol and therefore to have a data function with multiple data tables as results.

Here is an example of how to configure a protocol with 3 output data tables:

Multi file results in a protocol

As you can see, you just need to add the file name as the output property. The file name (without the extension) will be used to name the data table in TIBCO Spotfire®.

You can now use the Pipeline Pilot components from the Reporting collection that redirect to another protocol. This allows you for example to develop a sequence of protocols, each redirecting to the next, and execute them via the SWAPP.

The existing components have been improved. In addition, the Box Plot and the Pie Chart components have been added.

Pipeline Pilot Visualization components

New in 5.0.1

This minor new version offers some minor bug fixes, improves the components of the Pipeline Pilot collection, and provides a quick start protocol to easily start with the collection.

New in 5.0 - Release Candidate

The Connector 5.0 is a major new version. The Client Automation, new name of the Discngine Web Panel, has been completely redesigned. The Pipeline Pilot Data Functions and the Automation Services Tasks have been improved in a minor way.

The very big change for this version is that the Connector 5.0 is compatible with TIBCO Spotfire® Analyst and TIBCO Spotfire® Web Player. Thus, the same Pipeline Pilot protocol can be executed either in the Analyst or in the Web Player.

Architecture overview

This huge advantage requires modifying existing Pipeline Pilot protocols to replace Connector 4.1 components with Connector 5.0 components. Indeed, the Pipeline Pilot collection has also been entirely redesigned, and the collection 4.1 is now deprecated (components and example protocols) although backwards compatibility with the Connector 4.1 is ensured. Protocols developed in 4.1 will continue to work using Web Panel 4.1 embedded into the 5.0, but they will not be compatible with both Analyst and Web Player.

All is new for this product in the Connector 5.0: new TIBCO Spotfire® extensions, new JavaScript API, new Pipeline Pilot collections, and even new name! All is new, but as said before, the Connector 4.1 is included into the Connector 5.0, so the backwards compatibility is ensured.

We provide also with this new version 5.0 the SWAPP, a standalone web application designed to be used in the Web Panel and developped in collaboration with one of the top Pharmaceutical Companies. This application allows you to register Pipeline Pilot protocols and run them seamlessly as web forms from TIBCO Spotfire® Analyst or Web Player.

SWAPP Form

The connection with the Pipeline Pilot server is now manageable in the "Register Data Function" view. A status bar on the bottom of the screen informs you about your connection status. Also, "Login" and "Logout" buttons have been added in the screen, and the registering is not available until you are logged in.

Register Pipeline Pilot Data Functions

For this new release, efforts regarding Automation Services tasks have been focused on improving visualization tasks. Indeed, label properties have been added for Scatter Plots and Bar Charts, and you can now also add Pie Charts, Line Charts, Box Plots and Heat Maps.

Here is the list of all Pipeline Pilot tasks now available:

Automation Services Tasks