你们是怎么一次查询超过500条的数据来进行统计的？我循环调用分页查询接口获取所有数据会无规律出现某一页查询异常

问题现象：
1、定时任务从13万条数据中查找最近一周的数据，数据量约8千条，循环根据时间分页每页500条的方式获取数据，会随机出现接口调用的未错误。出错原因是分页查询出现未知规律的异常，有时是第5页查询异常，有时是第15页查询异常，没有错误提示。
2、将出现异常的查询语句放到JQL执行，正常执行。
以下是分页代码片段示例
1、分页获取数据接口

//分页获取数据  
async function _getAnsCard_page(query, fieldGet, pageSize = 30,  afterTime = 0,firstTime = 0) {  
       const dbcmd = db.command;  
       var defaultQuery ={startTime: dbcmd.lt(afterTime)}  
        //若不过滤起始时间  
        if(afterTime==0&&firstTime==0){  
            afterTime = new Date().getTime();  
            defaultQuery.startTime = dbcmd.lt(afterTime);  
        }//过滤起始时间     
        else if(afterTime>0 && firstTime>0){  
            defaultQuery.startTime = dbcmd.gte(firstTime).and(dbcmd.lt(afterTime))  
        }  

        var newQuery =Object.assign(  
            defaultQuery,  
            query  
        )   

        var newFieldGet =Object.assign(  
            {   
                _id:true ,  
            },  
            fieldGet  
        )  
        console.log("newQuery")  
        console.log(`afterTime=${afterTime},firstTime=${firstTime}`)  
        let resdb = await examdb_ansCard.where(newQuery).limit(pageSize).orderBy("startTime","desc").get();  
        return {  
            errCode: 0,  
            errMsg: '查询成功',  
            data: resdb.data,  
            pageSize:pageSize  
        }  
}

2、获取所有数据的接口
async function _getAnsCard_all(query, fieldGet,afterTime,firstTime){

var haveMore= false;  
var allAnsCardList = [];  
var pageSize = 500;  

do{  
        console.log("开始获取所有数据query")  
    var res =await _getAnsCard_page(query,fieldGet,pageSize,afterTime,firstTime);  
    console.log("获取到数据")  
    if(res.data.length>0){  
        allAnsCardList = allAnsCardList.concat(res.data);  
    }     
    console.log("获取到数据")  
    //还有下一页,则继续查询  
    if(res.data.length==pageSize){  
        haveMore = true;  
        var lastItem = res.data[pageSize-1];  
        afterTime = lastItem.startTime;  
    }  
    else {  
        haveMore = false;  
    }  
    console.log("还有下一页"+haveMore)  
}while(haveMore);  

return allAnsCardList;  

}
3、问题日志

[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/73ms/DEBUG] -------------执行检查_sentScore_type_1：start=Mon Mar 27 2023 00:00:00 GMT+0000 (UTC);end=Mon Apr 03 2023 00:00:00 GMT+0000 (UTC)
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/124ms/DEBUG] 开始获取所有数据query
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/125ms/DEBUG] newQuery
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/125ms/DEBUG] afterTime=1680480000000,firstTime=1679875200000
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/535ms/DEBUG] 获取到数据
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/536ms/DEBUG] 获取到数据
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/536ms/DEBUG] 还有下一页true
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/536ms/DEBUG] 开始获取所有数据query
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/536ms/DEBUG] newQuery
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/536ms/DEBUG] afterTime=1680438718812,firstTime=1679875200000
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/870ms/DEBUG] 获取到数据
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/870ms/DEBUG] 获取到数据
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/871ms/DEBUG] 还有下一页true
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/871ms/DEBUG] 开始获取所有数据query
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/871ms/DEBUG] newQuery
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/871ms/DEBUG] afterTime=1680419924274,firstTime=1679875200000
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/1162ms/DEBUG] 获取到数据
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/1163ms/DEBUG] 获取到数据
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/1163ms/DEBUG] 还有下一页true
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/1163ms/DEBUG] 开始获取所有数据query
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/1163ms/DEBUG] newQuery
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/1163ms/DEBUG] afterTime=1680399978023,firstTime=1679875200000
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/1439ms/DEBUG] 获取到数据
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/1439ms/DEBUG] 获取到数据
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/1440ms/DEBUG] 还有下一页true
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/1440ms/DEBUG] 开始获取所有数据query
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/1440ms/DEBUG] newQuery
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/1440ms/DEBUG] afterTime=1680360507198,firstTime=1679875200000
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/1764ms/DEBUG] 获取到数据
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/1765ms/DEBUG] 获取到数据
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/1765ms/DEBUG] 还有下一页true
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/1765ms/DEBUG] 开始获取所有数据query
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/1765ms/DEBUG] newQuery
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/1765ms/DEBUG] afterTime=1680341869432,firstTime=1679875200000
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/2067ms/DEBUG] 获取到数据
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/2067ms/DEBUG] 获取到数据
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/2067ms/DEBUG] 还有下一页true
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/2067ms/DEBUG] 开始获取所有数据query
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/2068ms/DEBUG] newQuery
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/2068ms/DEBUG] afterTime=1680326262861,firstTime=1679875200000
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/2336ms/DEBUG] 获取到数据
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/2336ms/DEBUG] 获取到数据
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/2336ms/DEBUG] 还有下一页true
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/2336ms/DEBUG] 开始获取所有数据query
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/2336ms/DEBUG] newQuery
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/2336ms/DEBUG] afterTime=1680294510340,firstTime=1679875200000
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/2617ms/DEBUG] 获取到数据
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/2617ms/DEBUG] 获取到数据
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/2618ms/DEBUG] 还有下一页true
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/2618ms/DEBUG] 开始获取所有数据query
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/2618ms/DEBUG] newQuery
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/2618ms/DEBUG] afterTime=1680270507033,firstTime=1679875200000
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/2909ms/DEBUG] 获取到数据
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/2909ms/DEBUG] 获取到数据
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/2909ms/DEBUG] 还有下一页true
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/2909ms/DEBUG] 开始获取所有数据query
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/2909ms/DEBUG] newQuery
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/2909ms/DEBUG] afterTime=1680253650817,firstTime=1679875200000
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/3172ms/DEBUG] 获取到数据
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/3172ms/DEBUG] 获取到数据
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/3172ms/DEBUG] 还有下一页true
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/3172ms/DEBUG] 开始获取所有数据query
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/3172ms/DEBUG] newQuery
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/3172ms/DEBUG] afterTime=1680244234093,firstTime=1679875200000
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/3513ms/DEBUG] 获取到数据
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/3513ms/DEBUG] 获取到数据
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/3513ms/DEBUG] 还有下一页true
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/3513ms/DEBUG] 开始获取所有数据query
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/3513ms/DEBUG] newQuery
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/3513ms/DEBUG] afterTime=1680233715832,firstTime=1679875200000
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/3790ms/DEBUG] 获取到数据
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/3790ms/DEBUG] 获取到数据
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/3790ms/DEBUG] 还有下一页true
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/3790ms/DEBUG] 开始获取所有数据query
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/3790ms/DEBUG] newQuery
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/3790ms/DEBUG] afterTime=1680227649105,firstTime=1679875200000
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/4159ms/DEBUG] 获取到数据
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/4159ms/DEBUG] 获取到数据
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/4159ms/DEBUG] 还有下一页true
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/4159ms/DEBUG] 开始获取所有数据query
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/4159ms/DEBUG] newQuery
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/4159ms/DEBUG] afterTime=1680222081825,firstTime=1679875200000
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/4427ms/DEBUG] 获取到数据
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/4427ms/DEBUG] 获取到数据
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/4427ms/DEBUG] 还有下一页true
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/4427ms/DEBUG] 开始获取所有数据query
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/4427ms/DEBUG] newQuery
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/4427ms/DEBUG] afterTime=1680179527785,firstTime=1679875200000
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/4690ms/DEBUG] 获取到数据
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/4690ms/DEBUG] 获取到数据
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/4690ms/DEBUG] 还有下一页true
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/4690ms/DEBUG] 开始获取所有数据query
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/4690ms/DEBUG] newQuery
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/4690ms/DEBUG] afterTime=1680156905158,firstTime=1679875200000
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/4984ms/DEBUG] 获取到数据
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/4984ms/DEBUG] 获取到数据
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/4984ms/DEBUG] 还有下一页true
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/4984ms/DEBUG] 开始获取所有数据query
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/4984ms/DEBUG] newQuery
[cron-server/2D3DBF86-EC44-5CD9-B2A2-EC85982E1439/4984ms/DEBUG] afterTime=1680105806625,firstTime=1679875200000

随机在查询的某页时，无任何日志，接口执行失败，执行状态是未知；

各位大佬，你们怎么获取超过500条数据的？有没有碰到这个问题？