Changes to API Pagination of Search Requests

Currently, the search APIs allow pagination using two parameters in the request: page and size. We have introduced a new method of pagination, and going forward, this should be used for API pagination. This new API parameter will reduce the load that search requests place on the platform.

Customers should start to convert scripts, targeting completion for the Q1 2024 release on January 14th. Users can start to test their new scripts immediately.

The new method for pagination involves adding a parameter called searchAfter in the search API request. When using searchAfter, set the page parameter to 0 in the request. The size parameter can contain any integer up to 1000. The size parameter indicates the number of records to be fetched in a single API response.

When making requests via the older method, users will start to see an error if they paginate past 10,000 results. Customers can request that we disable this error temporarily. However, customers that request an exemption should finish converting scripts by March 1, 2024.

Examples

To retrieve multiple pages of results, follow the process below:

  1. In the first request, the searchAfter parameter is optional. If used, it should contain an empty array [].
  2. The response of the first request will include a parameter called searchAfter.
  3. The searchAfter value received in the response of the first request should be used as the searchAfter value in the second request.
  4. The searchAfter value received in the response of the second request should be used as the searchAfter value in the third request and so on.
  5. If there are 0 records in the response, it indicates that there are no more records to be retrieved, and the pagination has reached the last page.

Search APIs will typically take the following form.

Endpoint Request Body

POST https://{platformName}.risksense.com/api
/v1/client/{clientId}/{pageType}/search

{

 "filters": [],

 "sort": [],

  "searchAfter": [],

 "page": 0,

 "size": 100

}

URL Parameters

Name

Description

Additional Information

platformName

Name of the platform that you are trying to connect to.

Examples: “platform”, “platform4”, “platform-eu”

clientId

The ID of the client that you are searching.

Finding Your Client ID

pageType

The pageType parameter can be assigned values based on specific requirements, with potential options such as “host”, “hostFinding”, “application”, and “applicationFinding”.

 

Request Body Parameters

Name Description

filters

The filters that need to be applied to the search request.

sort

The sort order that should be applied to the search results. The setting should be fixed for every API request in the same pagination sequence.

searchAfter

The parameter used for retrieving additional pages of results.

For the initial request, there's no need to include this in the body. It can be an empty array ([]). However, for subsequent requests, this parameter should be populated with the searchAfter value received in the most recent response.

page

A parameter that should not be used in conjunction with searchAfter. It should be set to 0 for all searchAfter requests.

size

The number of documents to be fetched (up to 1000).

To retrieve the entire dataset, you only need to change only the searchAfter value. The searchAfter value received in the most recent response is the searchAfter value for the next consecutive request. Requests should be stopped when the response has 0 records.

Sample Requests and Responses

Request 1:

Copy
{

  "filters": [],

  "projection": "internal",

  "sort": [

    {

      "field": "riskRating",

      "direction": "DESC"

    }

  ],

  "page": 0,

  "size": 50

}

Response1:

Copy
{

  "_embedded": {

    "findings": [

      ….

    ]

  },

  "_links": {

    "first": {

      "href": "http://localhost/api/v1/client/222/hostFinding/search?page=0&size=3&sort=riskRating,desc"

    },

    "self": {

      "href": "http://localhost/api/v1/client/222/ hostFinding /search?page=0&size=3&sort= riskRating,desc"

    },

    "next": {

      "href": "http://localhost/api/v1/client/222/hostFinding/search?page=1&size=3&sort= riskRating,desc"

    },

    "last": {

      "href": "http://localhost/api/v1/client/222/hostFinding/search?page=1&size=3&sort= riskRating,desc"

    }

  },

  "errors": [],

  "page": {

    "size": 50,

    "totalElements": 50,

    "totalPages": 100,

    "number": 0

  },

  "searchAfter": ["9", "123456”]

}

Request 2:

The searchAfter value from the previous response should be used in the second request.

Copy

{

{

  "filters": [],

  "projection": "internal",

  "sort": [

    {

      "field": "riskRating",

      "direction": "DESC"

    }

  ],

  "searchAfter": ["9", "123456”],

  "page": 0,

  "size": 50

}

Response2:

Copy
{

  "_embedded": {

    "findings": [

      ….

    ]

  },

  "_links": {

    "first": {

      "href": "http://localhost/api/v1/client/222/hostFinding/search?page=0&size=3&sort=riskRating,desc"

    },

    "self": {

      "href": "http://localhost/api/v1/client/222/ hostFinding /search?page=0&size=3&sort= riskRating,desc"

    },

    "next": {

      "href": "http://localhost/api/v1/client/222/hostFinding/search?page=1&size=3&sort= riskRating,desc"

    },

    "last": {

      "href": "http://localhost/api/v1/client/222/hostFinding/search?page=1&size=3&sort= riskRating,desc"

    }

  },

  "errors": [],

  "page": {

    "size": 50,

    "totalElements": 50,

    "totalPages": 100,

    "number": 0

  },

  "searchAfter": ["7", "3456789”]

}

Request 3:

The searchAfter value from previous response should be used in the third request.

Copy
{

  "filters": [],

  "projection": "internal",

  "sort": [

    {

      "field": "riskRating",

      "direction": "DESC"

    }

  ],

  "searchAfter": ["7", "3456789”],

  "page": 0,

  "size": 50

}