Build an integration

This is for:

Developer

Creating a new integration

Step 1

Select Data tools and then Integrations from the side menu. Select New integration

Step 2

Choose an integration from the available templates, then select Choose

Step 3

Enter a name (must be unique) and description in the fields provided and select a trigger:

  • Experience - Where you want the integration to be invoked when a visitor is exposed to an experience

  • HTTP - Where you want the integration to be invoked by an HTTP request

Tip
Leading-practice

You might use an HTTP trigger, for example, where you wanted to execute an integration outside of an experience or from an internal system.

  • QP event - Where you want the integration to be invoked by a QP event. For a description of each QP event, see QP Triggers

Warning

Contact Coveo Support or your Customer Success Manager if you would like to use a QP event as your integration trigger.
Tip
Leading-practice

You might use a QP event trigger, for example, where you wanted to trigger a slack message to your Customer Service team every time one of your VIP visitors logs in, filtering the QP event ecUser for loyalty tier or tier points.

  • Recurring - Where you want an integration to run on a schedule, independently from experiences or other integration triggers

Warning

You cannot change the trigger type once the integration has been created.
Tip
Leading-practice

Schedules can be set on a per minute, per hour, per day, or per week schedule. You can change the schedule in Settings

Select Create to continue

Using the integration editor

The integration editor is the central location where you can develop your integration code, add dependencies, manage secrets, and customize settings.

editor

The main body of the editor contains 2 tabs, integration.js and package.json:

  • integration.js will be where you can develop the core functionality of your integration. It is divided into 3 separate panels:

    • The main panel contains the source code of your integration, running under NodeJS v8.10 (LTS)

    • The Sample payload and Logs panels allow you to test your integration during development, you can read more about this in Testing your integration

  • package.json contains the dependencies for your integration, allowing you to make use of the rich ecosystem of open source NPM packages, and reduce substantially the amount of code needed to develop an integration

    There are many packages available for common integrations use cases:

Additional controls are located on the footer:

  • Manage secrets - Opens up the secrets manager and allows you to set secrets

  • Add a package - Will bring you to the package explorer where you can browse and manage dependencies

  • settings opens Settings where you can configure timeouts, rate-limits, and recurring schedules

  • Run - Allows you to preview the integration immediately so you can test your code before releasing it, learn more in Testing your Integration

  • Save - Will save any changes you’ve made to your integration code or dependencies

integration.js

As discussed earlier, integration.js contains the source code of your integration, which will be run in NodeJS v8.10 (LTS). This is the default handler when your integration is first created:

module.exports = async function handler ({ payload, secrets, cache }) {
  console.info(payload)
}

As you can see, a single function is exported. This will be called when your integration is executed. We call this handler with an object that contains:

  • payload - The payload that is passed by the executor

  • secrets - An object containing the secrets defined in Managing secrets, which are decrypted and injected when the function is executed

  • cache - Our caching service that helps you to reduce process heavy operations or long HTTP requests

package.json

As integrations run on NodeJS, we have provided support for using packages from the public NPM and also Qubit’s own registry. You can add dependencies by manually declaring them in the package.json file through the dependency object (more info here), but we recommend you use our package explorer, as this will ensure that dependencies are declared correctly.

To add a new dependency, select the package.json tab and select Add a package on the footer bar. This will open up the package explorer:

package explorer

The package explorer shows your most recently used packages first, but you can search for specific packages by typing into the search bar. If you want to see the documentation for a package, you can hover over it and select package explorer.

When you have found the package you would like to use, select the + and it will be added to the dependencies in your package.json file

package.json

Managing secrets

Following secure software development guidelines, secrets, which include keys, passwords, tokens, etc, should be kept separate from the application logic.

Qubit Integrations has built in support for encrypting and storing your secrets which will be available to your integration during execution.

Selecting Manage secrets in the bottom-left hand corner of the page will open up the secrets manager:

secrets

Each secret is a key-value pair with the value being encrypted securely through Google KMS before being stored. Once set, a secret can no longer be retrieved through the UI, although you can still update the secret to a new value.

When your integration is executed, the secrets and decrypted and injected into your code through the secrets object:

module.exports = async function handler ({ payload, secrets, cache }) {
  await axios.post('https://example.com/authenticated-endpoint', {
    headers: {
      Authorization: `Bearer ${secrets.API_TOKEN}`
    }
  })
}

Precautions have been put in place to catch accidental logging of secrets, but you should ensure you treat these values according to best practices.

Settings

In Settings you can configure timeouts, rate-limits, and recurring schedules.

To open, select settings

settings

Timeout

This controls the maximum time that your integration can run for. You may want to set different timeouts for different situations:

  • If the integration is called by an experience to fetch data before modifying the site, having a short timeout could prevent spikes in response times from negatively impacting the user’s experience. In this case it may be better to not fire your experience if the call is taking too long

  • If the integration is used as a background processing job, long execution times will not have a negative impact on the user’s experience, so you may want a longer timeout to give your integration more time to complete

Scheduling

If you chose to build an integration with a recurring schedule, you can change the schedule frequency:

  • Every minute - Execute every minute, starting from the specified time

  • Every hour - Execute every hour, starting from the specified time

  • Every day - Execute every day, starting from the specified time

  • Every week - Execute every week, starting from the specified time on the specified day(s)

Note

All times are in UTC.

Rate-limiting

When developing integrations, especially those scheduled from an experience, we recommend using key object, as shown in the following example.

The key object functions as a unique Id that you can use to refer back to a scheduled invocation in a later browser session to cancel or reschedule it. You can also use the key to rate-limit invocations.

When adding a rate limit you can choose whether to target the whole integration or to target it on a per key basis.

Note

Each rate limit consists of a limit (frequency), interval (period), and a target—​whole invocation or a specific key combination.

To get started, select Add rate limit. You can then define your limit, interval, and target.

In the following example, the user has selected to rate-limit on a per-key basis, to restrict the sending of emails to once per week per email address:

rate-limit-key

In another example, the user has selected to rate-limit on a per-key basis, to restrict the sending of emails to twice a week for any particular campaign, per email address:

rate-limit-key-campaign

In the following two snippets, we show how key is used. Firstly in an integration executed immediately and another executed according to a schedule:

Example:

const md5 = require('md5')

module.exports = async function variation (options) {
  const integration = options.integration('slack')

  // Wait for ecUser QP event so we have the user's email address
  options.uv.once('ecUser', (userEvent) => {

    // The key will be used to rate limit the number of executions per user
    const key = {
      // Hash the user's email so we don't expose PII
      email: md5(userEvent.user.email)
    }

    // Wait for ecBasketTransactionSummary event so we
    // know when someone has completed a purchase
    options.uv.on('ecBasketTransactionSummary', (transactionEvent) => {

      const { currency, value } = transactionEvent.basket.total

      if (currency === 'GBP' && value > 100) {

        // The payload passed to the integration
        const payload = `A user has just spent £${value}!`

         // Passing through the key allows the integration to
         // rate-limit the number of executions
        const options = { key }

        // Execute function returns a promise
        integration.execute(payload, options)
          .then(() => {
            options.log.info('Successfully pinged slack!')
          })
          .catch(options.log.error)
      }
    }).replay()

  }).replay()
}
Note

You can more information about executing integrations immediately in Executing integrations immediately.

Example:

module.exports = function variation (options) {
  // Slack integration that posts messages to a specific channel
  const integration = options.integration('slack-update')

  // Payload is same as calling an integration directly
  const payload = { message: 'Hello!' }
  const options = {
    key: {
      foo: 'bar'
    },
    delay: 60 * 60 * 1000, // Amount of time to wait before executing
    window: { // Window of time the integration is allowed to execute
      day: 'monday-friday', // Range of days allowed
      time: '09:00-17:00' // Time range allowed
    }
  }

  integration.schedule(payload, options)
    .catch((err) => {
      options.log.error(`Could not schedule execution: ${err.message}`)
    })
}
Note

You can find more information about scheduling in Scheduling Integrations.

Use static IP

Enabling this will ensure that all outgoing network requests originate from fixed IP addresses. This is useful for protected services that require IP whitelisting.

All requests will originate from one of these IP addresses:

  • 18.202.52.194

  • 54.72.231.165

Testing your integration

The editor provides the ability to test your code in a production environment directly from the editor. In the integration.js tab, there are two panels at the bottom that allow you to do this:

  • Sample payload allows you to set a sample JSON payload which will be used when executing the integration

  • Logs will show the logs from executing the integration with the sample payload.

Let’s test this out with simple integration that makes an external HTTP request.

We’re going to use axios and make a call to randomuser.me to generate a random user:

// package.json
{
  "dependencies": {
    "axios": "^0.18.0"
  }
}
// integrations.js
const axios = require('axios')

module.exports = async function handler ({ payload }) {
  // We pass a seed through the payload object
  const { seed } = payload
  console.log(`Generating random user with seed ${seed}`)

  // Request some random users through an external API
  const { data } = await axios.get(`https://randomuser.me/api/?seed=${seed}`)
  const [ person ] = data.results

  // And log the person's name
  console.log(`Random name: ${person.name.first} ${person.name.last}`)
}

Now let’s test this. In the Sample Payload box, we’ll pass in a seed:

// Sample Payload
{
  "seed": "foo"
}

Then select Run. This will deploy your integration as a preview to a production environment, and execute the integration immediately.

Note

Execution times in preview mode will take longer than in production as we need to fully redeploy the integration when there are changes

After execution finishes, the resulting logs will be displayed in the Logs panel:

logs

Let’s break these down line by line:

BOOT Using cached dependencies

This describes where we sourced dependencies for each build. Since each publish package version is immutable, we cache installed packages, which are reused when there is a build that require the same dependencies. This is used mainly for debugging purposes. If you select the line, it will expand to show the the exact version of all the packages that were installed.

INFO Generating random user with seed foo

This is a log we implemented ourselves in the code above. All console.debug(), console.log(), console.error(), and console.warn() calls are allowed.

HTTP GET https://randomuser.me/api/?seed=foo - 200 OK

All HTTP requests are automatically logged. On this line, we show the url that was requested and the method used, along with the response code returned. Selecting this line will expand the log to show more information that is generally useful for debugging long running requests, the main cause of integrations timing out.

Execution finished in 626ms

This indicates that the execution was successful, expanding this line will also show the response payload.

When you are happy with your integration, you can now release it. Head over to Releasing your integration to learn how to do this.

Editing an integration

To edit an integration, open it from your list of integrations, and select edit