Conversations missing on next pages /search v2.0

Hi,

We’re trying to fetch conversation data for an hour with API version 2.0. We’ve noticed that when we fetch data for an hour the conversation counts seem to be suspiciously N x 150 (we have per_page = 150).
While investigating this we found that sometimes when we get a successful page with pages object including next object (e.g. {"type":"pages","next":{"page":4,"starting_after":"WzE1OTA5MjM3NTQwMDAsMjczMDUwNjUwNjIsNF0="}, and metadata also saying that there is 9 pages total, and conversation count is 1327), when we try to fetch the next page (with "starting_after":"WzE1OTA5MjM3NTQwMDAsMjczMDUwNjUwNjIsNF0=") we get no conversations ({"type":"conversation.list","pages":{"type":"pages","page":4,"per_page":150,"total_pages":9},"total_count":1327,"conversations":[]}).

I understand from the docs that

Consequently, if items are updated between 2 paginated queries, this can lead to duplicate or missed records.

and we would have no issues with getting extra data, but missing the whole 5 pages? How can we retrieve all of the 9 pages?

I have tested, and querying the first page again and taking the “new” starting_after I sometimes reach into further pages, but it still results in "conversations":[] at one point while metadata saying that there are still more pages to go but no starting_after value. I have not yet reached the 9-th page conversations.

I’m just going to replicate a typical session of trying to fetch conversations via 2.0 API here:

First I try to query conversations for a 1h interval (I only show the “pages” object of the response):

$ curl -g 'https://api.intercom.io/conversations/search' -d '{"query":{"operator":"AND","value":[{"field":"updated_at","operator":">","value":1590991200},{"field":"updated_at","operator":"<","value":1590994800}]},"pagination":{"per_page":150}}' -H 'Accept: application/json' -H 'Content-Type: application/json' -H 'Intercom-Version: 2.0' -H 'Authorization: Bearer ***' | jq '.pages'
{
  "type": "pages",
  "next": {
    "page": 2,
    "starting_after": "WzE1OTA5OTMwOTkwMDAsMjczMTI3MjU4NjAsMl0="
  },
  "page": 1,
  "per_page": 150,
  "total_pages": 5
}

It says wer’re on page 1, there are 150 conversations per page and in total there are 5 pages. Now I take the “starting_after” value and add that to the query:

$ curl -g 'https://api.intercom.io/conversations/search' -d '{"query":{"operator":"AND","value":[{"field":"updated_at","operator":">","value":1590991200},{"field":"updated_at","operator":"<","value":1590994800}]},"pagination":{"per_page":150, "starting_after": "WzE1OTA5OTMwOTkwMDAsMjczMTI3MjU4NjAsMl0="}}' -H 'Accept: application/json' -H 'Content-Type: application/json' -H 'Intercom-Version: 2.0' -H 'Authorization: Bearer ***' | jq '.pages'
{
  "type": "pages",
  "next": {
    "page": 3,
    "starting_after": "WzE1OTA4NzIxMTYwMDAsMjczMDA1NDIwMjQsM10="
  },
  "page": 2,
  "per_page": 150,
  "total_pages": 5
}

All good. We’re now on the 2-nd page of the result, still 150 conversations per page and in total 5 pages (there are also conversations in the response). Now, I again take the “starting_after” value from the response and subsitute that to the next query:

$ curl -g 'https://api.intercom.io/conversations/search' -d '{"query":{"operator":"AND","value":[{"field":"updated_at","operator":">","value":1590991200},{"field":"updated_at","operator":"<","value":1590994800}]},"pagination":{"per_page":150, "starting_after": "WzE1OTA4NzIxMTYwMDAsMjczMDA1NDIwMjQsM10="}}' -H 'Accept: application/json' -H 'Content-Type: application/json' -H 'Intercom-Version: 2.0' -H 'Authorization: Bearer ***' | jq '.pages'
{
  "type": "pages",
  "page": 3,
  "per_page": 150,
  "total_pages": 5
}

And here it happens. We’re on page 3, there are no conversations (conversations:[]), it still says there should be 5 pages, but there is no “starting_after” value which I could take to fetch the next page. It is not the case either that everything has changed in the background and that all the conversations have “moved” to different “updated_at” value, because when I do the request now without “starting_after” value for the same query (the same hour interval, fetching 1-st page), I still get 5 pages.

Is there something I am missing here?