This article provides an overview of the performance reports generated for individual experiments across email, push, and SMS tests that have used a classic testing methodology (i.e. straight splits or test and deploy).

Experiment-level performance reports summarise the performance of your Jacquard experiments based on the results received from the test.

You can find the performance report for individual experiments under the 'results' tab for any experiment in Jacquard that has result data.

Experiment-level performance reporting

Jacquard automatically generates the report once it receives the results of an experiment, either via one of our technology integrations or by uploading them to the platform.

The exact metrics the report displays will depend on the channel and testing methodology of the experiment.

The experiment's winning copy variant is highlighted at the top of your performance report. This copy variant was determined to be the winner based on the optimisation metric selected during the experiment's initial set-up (Step 1). For example, if you chose to optimise your copy variants based on the open rate, the winning variant is declared based on which received the highest open rate.

For experiments that used a test and deploy methodology, the winning copy variant displayed is the copy variant that was determined to be the winning variant when the testing window completed.

Note: In some cases, engagement observed after a testing period has ended may cause a different variant to overtake the variant selected as the winner at the end of the test. This isn't cause for concern, but it is possible for the variant that is declared as the winner in the report to be different to the variant ranked as number one in the results table below the performance report.

All subsequent data displayed under the winning variant is based on this variant. You will see your experiment's uplift and incremental event results directly below the winning variant. And below that are the results from the winning variant's final send.

Uplift calculations will be displayed for any email, push or SMS experiments that have used either a test and deploy or straight splits testing methodology.

Uplift in this report view is calculated based on the percentage change between the key metrics for the winning copy variant (e.g. opens, clicks) and the copy variant that ranked the lowest in the experiment.

The exact metric of the uplift will depend on the channel the experiment was deployed on and the result data available. These include:

Incremental events are only calculated for any experiments that have used a Test and Deploy methodology.

They are a cumulative number and represent the additional number of times people have opened or clicked your email, push notification or SMS message due to your Jacquard experiment.

Incremental events are calculated by multiplying the percentage difference of the relevant metric (e.g. opens, clicks) between the winner and the human control. The difference is then divided by the final audience size.

Depending on the channel and result data available, you will see slightly different incremental event metrics:

Below the uplift and incremental event results, the results for your winning variant are displayed in orange.

For straight split experiments, these are the results from the winning variant as determined by your selection optimisation metric during the initial set-up of the experiment (Step 1).

For test and deploy experiments, the performance metrics are based on the results from the final send. The results achieved during the testing window are displayed in the variants result table.

The variant result table summarises the key results for each variant. You can click on the arrows in each column header to sort the variant table.

The results displayed here and in the winning variant summary will depend on the channel and result data available to Jacquard: