Create a Basic Public Web Source

In this article

A Web source typically crawls the content of a single website, starting from the specified URL (or URLs), and then recursively following any hyperlink it finds in pages to discover the entire site (see Add a Web source). By default, a Web source schedule is set to rescan the website at midnight everyday.

Use the Create a source from simple configuration operation to create a basic public Web source in a Coveo organization.

Request template

POST https://platform.cloud.coveo.com/rest/organizations/<MyOrganizationId>/sources HTTP/1.1
 
Content-Type: application/json
Accept: application/json
Authorization: Bearer <MyAccessToken>

Payload

{
  "sourceType": "WEB2",
  "name": <MyWebSourceName>,
  "sourceVisibility": "SHARED",
  "urls": [
    <MyURL>*
  ]
}

This introductory article explains how create a SHARED (public) Web source, which is why the sourceType property is set to WEB2 and the sourceVisibility property is set to SHARED in the above request template.

the Coveo Platform allows you to create sources based on various other connectors (see Possible sourceType Values), and supports three different types of source visibility: PRIVATE (specific users and groups), SHARED (everyone), and SECURED (based on the repository’s permission system) (see Content Security).

In the request path:

In the Authorization HTTP header:

In the request body:

  • Replace <MyWebSourceName> with a meaningful name property value.

    You can’t change the name of a source once it has been created, so make sure the name you choose fits the content you intend to index with that source.

  • Replace <MyURL>* with one or more urls from which to start crawling the pages to index with your source.

    Although you can provide more than one value in the urls array, you should normally only index one website per Web source.

The body of a successful response (201 Created) contains information about the source you just created. The id property value is important, as it’s a required parameter in many Coveo REST API operations. You can always retrieve this value later.

Sample Request

Creating a basic public Web source

POST https://platform.cloud.coveo.com/rest/organizations/mycoveocloudv2organizationg8tp8wu3/sources HTTP/1.1
 
Content-Type: application/json
Accept: application/json
Authorization: Bearer **********-****-****-****-************

Payload

{
  "sourceType": "WEB2",
  "name": "My web source",
  "sourceVisibility": "SHARED",
  "urls": [
    "http://www.example.com/"
  ]
}

Successful response - 201 Created

{
  "sourceType": "WEB2",
  "id": "mycoveocloudv2organizationg8tp8wu3-xtyq5ljb65btzsx2miknabccru",
  "name": "My web source",
  "owner": "asmith@example.com",
  "sourceVisibility": "SHARED",
  "information": {
    ...
  },
  ...
}