Creating a Basic Public Web Source

A Web source typically crawls the content of a single website, starting from the specified URL (or URLs), and then recursively following any hyperlink it finds in pages to discover the entire site (see Add or Edit a Web Source). By default, a Web source schedule is set to rescan the website at midnight everyday.

Use the Create a source from simple configuration operation to create a basic public Web source in a Coveo organization.

Request template

POST https://platform.cloud.coveo.com/rest/organizations/<MyOrganizationId>/sources HTTP/1.1
 
Content-Type: application/json
Accept: application/json
Authorization: Bearer <MyAccessToken>

Payload

{
  "sourceType": "WEB2",
  "name": <MyWebSourceName>,
  "sourceVisibility": "SHARED",
  "urls": [
    <MyURL>*
  ]
}

This introductory article explains how create a SHARED (public) Web source, which is why the sourceType property is set to WEB2 and the sourceVisibility property is set to SHARED in the above request template.

The Coveo Platform allows you to create sources based on various other connectors (see Possible sourceType Values), and supports three different types of source visibility: PRIVATE (source creator), SHARED (everyone), and SECURED (determined by source permissions) (see Content Security).

In the request path:

In the Authorization HTTP header:

In the request body:

  • Replace <MyWebSourceName> with a meaningful name property value.

    You can’t change the name of a source once it has been created, so make sure the name you choose fits the content you intend to index with that source.

  • Replace <MyURL>* with one or more urls from which to start crawling the pages to index with your source.

    Although you can provide more than one value in the urls array, you should normally only index one website per Web source.

The body of a successful response (201 Created) contains information about the source you just created. The id property value is important, as it’s a required parameter in many Coveo Platform REST API operations. You can always retrieve this value later (see Retrieve the ID of a Source).

Sample Request

Creating a basic public Web source

POST https://platform.cloud.coveo.com/rest/organizations/mycoveocloudv2organizationg8tp8wu3/sources HTTP/1.1
 
Content-Type: application/json
Accept: application/json
Authorization: Bearer **********-****-****-****-************

Payload

{
  "sourceType": "WEB2",
  "name": "My web source",
  "sourceVisibility": "SHARED",
  "urls": [
    "http://www.example.com/"
  ]
}

Successful response - 201 Created

{
  "sourceType": "WEB2",
  "id": "mycoveocloudv2organizationg8tp8wu3-xtyq5ljb65btzsx2miknabccru",
  "name": "My web source",
  "owner": "asmith@example.com",
  "sourceVisibility": "SHARED",
  "information": {
    ...
  },
  ...
}
Recommended Articles