Create a Basic Public Web Source

This is for:

In this article

Sample request

A Web source typically crawls the content of a single website, starting from the specified URLs, and then recursively following any hyperlink it finds in pages to discover the entire site. By default, a Web source schedule is set to rescan the website at midnight everyday. For details on web sources, see Add a Web source.

Use the Create a source from simple configuration operation to create a basic public Web source in a Coveo organization. Alternatively, you can also create a Web source from the Coveo Administration Console.

Request template

POST https://platform.cloud.coveo.com/rest/organizations/<MyOrganizationId>/sources HTTP/1.1

Content-Type: application/json
Accept: application/json
Authorization: Bearer <MyAccessToken>

Payload

{
  "sourceType": "WEB2",
  "name": <MyWebSourceName>,
  "sourceVisibility": "SHARED",
  "urls": [
    <MyURL>*
  ]
}

Note

This introductory article explains how create a SHARED (public) Web source, which is why the sourceType property is set to WEB2 and the sourceVisibility property is set to SHARED in the above request template.

The Coveo Platform allows you to create sources based on various other connectors with different sourceType values. It also supports three different types of source visibility: PRIVATE (specific users and groups), SHARED (everyone), and SECURED (based on the repository’s permission system). See Content security for details on this topic.

In the request path:

Replace <MyOrganizationId> with the ID of the target Coveo organization.

In the Authorization HTTP header:

Replace <MyAccessToken> with an access token that grants privileges to edit sources (that is, the Edit all access level on the Sources domain) in the target Coveo organization. See Create an API key and Manage privileges for details on API key privileges. You may also want to read more about the Get the privileges of an access token and Get your Coveo access token API calls.

In the request body:

Replace <MyWebSourceName> with a meaningful name property value. You can’t change the name of a source once it has been created, so make sure the name you choose fits the content you intend to index with that source.
Replace <MyURL>* with one or more urls from which to start crawling the pages to index with your source.

Leading practice

Although you can provide more than one value in the urls array, you should normally only index one website per Web source.

The body of a successful response (201 Created) contains information about the source you just created. The id property value is important, as it’s a required parameter in many Coveo REST API operations. You can always retrieve this value later.

Sample request

Creating a basic public Web source

POST https://platform.cloud.coveo.com/rest/organizations/mycoveocloudv2organizationg8tp8wu3/sources HTTP/1.1

Content-Type: application/json
Accept: application/json
Authorization: Bearer **********-****-****-****-************

Payload

{
  "sourceType": "WEB2",
  "name": "My web source",
  "sourceVisibility": "SHARED",
  "urls": [
    "http://www.example.com/"
  ]
}

Successful response - 201 Created

{
  "sourceType": "WEB2",
  "id": "mycoveocloudv2organizationg8tp8wu3-xtyq5ljb65btzsx2miknabccru",
  "name": "My web source",
  "owner": "asmith@example.com",
  "sourceVisibility": "SHARED",
  "information": {
    ...
  },
  ...
}