Create a Basic Public Web Source

This is for:

In this article

Sample Request

A Web source typically crawls the content of a single website, starting from the specified URL (or URLs), and then recursively following any hyperlink it finds in pages to discover the entire site (see Add a Web source). By default, a Web source schedule is set to rescan the website at midnight everyday.

Use the Create a source from simple configuration operation to create a basic public Web source in a Coveo organization.

You can also create a source from the Coveo Administration Console.

Request template

POST https://platform.cloud.coveo.com/rest/organizations/<MyOrganizationId>/sources HTTP/1.1
 
Content-Type: application/json
Accept: application/json
Authorization: Bearer <MyAccessToken>

Payload

{
  "sourceType": "WEB2",
  "name": <MyWebSourceName>,
  "sourceVisibility": "SHARED",
  "urls": [
    <MyURL>*
  ]
}

This introductory article explains how create a SHARED (public) Web source, which is why the sourceType property is set to WEB2 and the sourceVisibility property is set to SHARED in the above request template.

the Coveo Platform allows you to create sources based on various other connectors (see Possible sourceType Values), and supports three different types of source visibility: PRIVATE (specific users and groups), SHARED (everyone), and SECURED (based on the repository’s permission system) (see Content Security).

In the request path:

Replace <MyOrganizationId> with the actual ID of the target Coveo organization (see Retrieve the organization ID).

In the Authorization HTTP header:

Replace <MyAccessToken> with an access token that grants privilege to edit sources (that is, the Edit all access level on the Sources domain) in the target Coveo organization (see Create an API key, Manage privileges, Sources Domain, Get the privileges of an access token, and Get your Coveo access token).

In the request body:

Replace <MyWebSourceName> with a meaningful name property value.

You can’t change the name of a source once it has been created, so make sure the name you choose fits the content you intend to index with that source.
Replace <MyURL>* with one or more urls from which to start crawling the pages to index with your source.

Although you can provide more than one value in the urls array, you should normally only index one website per Web source.

The body of a successful response (201 Created) contains information about the source you just created. The id property value is important, as it’s a required parameter in many Coveo REST API operations. You can always retrieve this value later.

Sample Request

Creating a basic public Web source

POST https://platform.cloud.coveo.com/rest/organizations/mycoveocloudv2organizationg8tp8wu3/sources HTTP/1.1
 
Content-Type: application/json
Accept: application/json
Authorization: Bearer **********-****-****-****-************

Payload

{
  "sourceType": "WEB2",
  "name": "My web source",
  "sourceVisibility": "SHARED",
  "urls": [
    "http://www.example.com/"
  ]
}

Successful response - 201 Created

{
  "sourceType": "WEB2",
  "id": "mycoveocloudv2organizationg8tp8wu3-xtyq5ljb65btzsx2miknabccru",
  "name": "My web source",
  "owner": "asmith@example.com",
  "sourceVisibility": "SHARED",
  "information": {
    ...
  },
  ...
}