Manage the Crawling Module using the REST API

This is for:

Developer System Administrator

Maestro is driven using a REST API and listens on service port 5000 by default, unless you chose a different port while installing Maestro. Check your Maestro settings to confirm the selected port.

Since no UI is available yet to interact with Maestro and manage the Crawling Module, you must use the Maestro Swagger at http://localhost:5000/api/swagger/. If you decided to use a different service port for Maestro, go to the corresponding address instead (for example, http://localhost:5001/api/swagger/ if you chose to use port 5001).

Note

Coveo only supports Swagger for Crawling Module management. If you want to use a different tool (for example, PowerShell), keep in mind that the Coveo Support team offers help with Swagger only.

This article is an overview of the calls you will use to manage the Coveo Crawling Module. Details on each API call are provided under Crawling Module REST API reference.

Typical Crawling Module use

Once deployed, the Crawling Module can retrieve content from several on-premises sources. You must however ensure adjust the Crawling Module configuration so that you have enough workers to crawl your content optimally.

  1. Use the /api/config GET call to view your Crawling Module configuration.

  2. Use the /api/config PUT call to edit your Crawling Module configuration and your number of workers if needed.

  3. Should you also edit the Maestro settings file (maestro_settings.ini), use /api/service/restart to restart Maestro and apply your changes.

  4. Create a source.

Status and troubleshooting

The following calls can be used to troubleshoot your Crawling Module instance. You don’t need to make them in the order in which they’re presented below. Make those that are relevant to your investigation of the issue.

We recommend that you contact Coveo Sales when you encounter issues.

Updates

Crawling Module updates are automatic. This allows you to benefit from the latest features and bug fixes, and prevents the Crawling Module from becoming incompatible with the most recent Coveo update. To stay up to date, the Crawling Module periodically polls Coveo for a new version of Maestro. If an update is available, it will be downloaded and installed within 24 hours, at the scheduled update time. The default time is 11:00 PM in the host server’s time zone.

Disconnecting your server from the Internet or shutting it down will result in the Crawling Module not polling Coveo for updates. Upon the next successful call for an update, if Maestro is two or more versions behind, the workers will stop until it’s up to date again.

The /api/status/version allows you to see which version of Maestro is installed on your server. Alternatively, you can find this information in the Crawling Modules (platform-ca | platform-eu | platform-au) Administration Console page, which also shows the latest version available, and on the component dashboard.

The /api/update/maestro call is available in case you ever need to update Maestro manually, for instance if automatic updates fail. You can also contact the Coveo Support team for help.

Password update

If you ever need to change the password of your proxy or database, there’s more than one way to do so, depending on whether you can store your password in clear text.

Storing the password in clear text in the Maestro settings

With this method, your password will appear in clear text in the Maestro settings file. Coveo therefore recommends avoiding this method if you’re not the only Crawling Module administrator.

  1. Edit the Maestro settings file.

  2. Use the /api/service/restart call to restart Maestro.

Storing the password in the Windows Credential store

If you want to avoid storing your password in clear text in the Maestro settings file, you can store it in the Windows Credentials Manager instead.

If you’re using this method to change your database password, you must follow these steps before actually changing your database password. In other words, you must first provide Maestro with the desired new password, and then change your old database password to the new one.

  1. Edit the Maestro settings file and set your password to ${credential:key=maestro/<PASSWORD>} where <PASSWORD> is an identifier for the password you want to use (for example, ${credential:key=maestro/proxypassword}). Before you save and close the file, copy this identifier.

  2. Make the api/config/sensitive call to provide the ${credential:key=maestro/<PASSWORD>} password entry from the previous step.

  3. Use the /api/service/restart call to restart Maestro.

Miscellaneous

Linking the Crawling Module to an organization

Before you start create a source, you must specify in which organization you want to index your content. Typically, you’re prompted to make this decision at the end of the Maestro installation process. However, if you have not done so, you must follow the steps below.

You should also follow them if you want to change the organization with which your Crawling Module instance is linked.

  1. Edit the Maestro settings and then restart Maestro if:

    • You want the Crawling Module to communicate with Coveo through a proxy

    • You want to switch from a regular production organization to a HIPAA environment or vice versa

  2. Use /api/authorize/url to get a URL at which you will be able to select the organization to link with your Crawling Module instance.

  3. Access this URL and log in to Coveo with an account that has the privilege to create API keys.

  4. On the Grant Access page, click the dropdown menu to select the organization to link with the Crawling Module, and then click Next.

  5. Click Authorize.

  6. Back on the Crawling Module Swagger page, use the /api/authorize/verify call to confirm the success of the linking process.

  7. Optionally, restrict the IP addresses from which the Crawling modules API key can be used in requests to Coveo.