(enhance)[s3load] Add file list numbers check in S3 load

[0AyanamiRei]

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

add a config to limit the s3 list files number.

Release note

None

Check List (For Author)

• Test

  • Regression test
  • Unit Test
  • Manual test (add detailed scripts or steps below)
  • No need to test or manual test. Explain why:
    • This is a refactor/code format and no logic has been changed.
    • Previous test can cover this change.
    • No code files have been changed.
    • Other reason

• Behavior changed:

  • No.
  • Yes.

• Does this need documentation?

  • No.
  • Yes.

Check List (For Reviewer who merge this PR)

• Confirm the release note
• Confirm test cases
• Confirm document
• Add branch pick label

[hello-stephen]

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
3. What features were added. Why was this function added?
4. Which code was refactored and why was this part of the code refactored?
5. Which functions were optimized and what is the difference before and after the optimization?

[0AyanamiRei]

run buildall

[doris-robot]

TPC-H: Total hot run time: 32778 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 2cdfc0499587e487b5405ba729f013a5f1afdae0, data reload: false

------ Round 1 ----------------------------------
q1	17634	4205	4098	4098
q2	2129	392	272	272
q3	10032	1269	774	774
q4	10209	838	320	320
q5	7553	2137	1859	1859
q6	191	170	141	141
q7	945	810	679	679
q8	9321	1423	1234	1234
q9	4924	4747	4605	4605
q10	6833	1809	1441	1441
q11	576	341	311	311
q12	711	775	630	630
q13	17805	3909	3131	3131
q14	307	311	282	282
q15	604	519	551	519
q16	737	713	682	682
q17	797	908	462	462
q18	6692	6445	7082	6445
q19	1216	1120	706	706
q20	454	413	285	285
q21	3518	2821	2824	2821
q22	1201	1081	1098	1081
Total cold run time: 104389 ms
Total hot run time: 32778 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4528	4407	4371	4371
q2	349	450	354	354
q3	2408	2866	2487	2487
q4	1484	1915	1614	1614
q5	4516	4473	4450	4450
q6	234	175	139	139
q7	1991	1914	1781	1781
q8	2594	2440	2403	2403
q9	7283	7270	7314	7270
q10	2609	2698	2407	2407
q11	570	520	474	474
q12	734	738	607	607
q13	3488	3856	3219	3219
q14	277	303	268	268
q15	534	504	497	497
q16	657	705	635	635
q17	1169	1304	1339	1304
q18	7613	7581	7185	7185
q19	956	923	914	914
q20	1976	1990	1874	1874
q21	4695	4384	4338	4338
q22	1125	1102	1003	1003
Total cold run time: 51790 ms
Total hot run time: 49594 ms

[doris-robot]

TPC-DS: Total hot run time: 181868 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 2cdfc0499587e487b5405ba729f013a5f1afdae0, data reload: false

query5	4445	665	544	544
query6	353	244	225	225
query7	4251	508	296	296
query8	375	276	295	276
query9	8699	3381	3349	3349
query10	558	413	353	353
query11	15560	15354	14987	14987
query12	187	125	118	118
query13	1254	575	477	477
query14	6174	3139	2941	2941
query14_1	2794	2772	2784	2772
query15	218	206	188	188
query16	1014	541	499	499
query17	1141	721	629	629
query18	2599	470	377	377
query19	245	249	217	217
query20	135	124	124	124
query21	225	152	137	137
query22	4204	4435	4271	4271
query23	16369	15883	15622	15622
query23_1	15752	15737	15704	15704
query24	7103	1653	1236	1236
query24_1	1237	1262	1210	1210
query25	605	505	472	472
query26	1258	311	175	175
query27	2707	489	315	315
query28	4548	2536	2498	2498
query29	854	606	495	495
query30	331	246	224	224
query31	834	689	618	618
query32	95	85	83	83
query33	550	404	364	364
query34	970	957	592	592
query35	762	798	698	698
query36	1035	1049	905	905
query37	149	110	98	98
query38	2823	2840	2758	2758
query39	805	793	776	776
query39_1	744	742	733	733
query40	240	155	138	138
query41	78	79	74	74
query42	120	114	122	114
query43	504	541	521	521
query44	1496	857	864	857
query45	201	192	184	184
query46	936	1008	626	626
query47	1527	1601	1478	1478
query48	370	374	292	292
query49	648	474	385	385
query50	724	313	233	233
query51	3874	3817	3952	3817
query52	120	118	110	110
query53	332	340	282	282
query54	346	334	348	334
query55	99	94	85	85
query56	349	327	335	327
query57	1095	1048	944	944
query58	296	277	282	277
query59	2509	2565	2275	2275
query60	363	375	351	351
query61	164	169	168	168
query62	432	394	336	336
query63	313	278	281	278
query64	5023	1438	1083	1083
query65	3871	3810	3844	3810
query66	1448	450	344	344
query67	15917	15940	15842	15842
query68	2464	1243	868	868
query69	481	398	357	357
query70	1097	1100	1094	1094
query71	371	346	321	321
query72	5505	3591	3593	3591
query73	734	794	360	360
query74	8856	8894	8682	8682
query75	2805	2864	2487	2487
query76	2278	1111	723	723
query77	400	425	343	343
query78	9841	10073	9223	9223
query79	1324	989	673	673
query80	1331	632	537	537
query81	556	280	240	240
query82	996	160	129	129
query83	358	289	274	274
query84	270	137	115	115
query85	951	584	500	500
query86	429	335	335	335
query87	2979	2950	2826	2826
query88	4127	3051	3034	3034
query89	419	370	348	348
query90	1981	193	175	175
query91	200	185	158	158
query92	85	81	78	78
query93	1184	1010	616	616
query94	680	358	343	343
query95	624	367	412	367
query96	764	554	266	266
query97	2372	2425	2373	2373
query98	237	220	216	216
query99	650	663	563	563
Total cold run time: 254360 ms
Total hot run time: 181868 ms

[doris-robot]

ClickBench: Total hot run time: 27.93 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 2cdfc0499587e487b5405ba729f013a5f1afdae0, data reload: false

query1	0.06	0.05	0.05
query2	0.11	0.05	0.05
query3	0.27	0.09	0.10
query4	1.61	0.13	0.12
query5	0.29	0.28	0.27
query6	1.16	0.68	0.68
query7	0.03	0.03	0.04
query8	0.07	0.05	0.05
query9	0.60	0.54	0.52
query10	0.58	0.58	0.58
query11	0.16	0.11	0.11
query12	0.16	0.12	0.12
query13	0.62	0.59	0.59
query14	0.96	0.97	0.98
query15	0.82	0.81	0.82
query16	0.40	0.45	0.42
query17	1.19	1.18	1.16
query18	0.24	0.22	0.22
query19	2.10	1.96	1.93
query20	0.02	0.01	0.01
query21	15.37	0.31	0.16
query22	4.79	0.07	0.06
query23	15.65	0.32	0.11
query24	1.58	0.29	0.21
query25	0.10	0.07	0.07
query26	0.16	0.14	0.15
query27	0.07	0.07	0.05
query28	4.06	1.12	0.89
query29	12.61	4.53	3.49
query30	0.29	0.16	0.15
query31	2.82	0.70	0.41
query32	3.25	0.59	0.47
query33	3.04	3.07	3.16
query34	16.21	5.21	4.48
query35	4.43	4.50	4.47
query36	0.69	0.52	0.52
query37	0.12	0.08	0.08
query38	0.08	0.04	0.05
query39	0.05	0.03	0.04
query40	0.17	0.15	0.14
query41	0.10	0.04	0.04
query42	0.05	0.03	0.03
query43	0.05	0.04	0.04
Total cold run time: 97.19 s
Total hot run time: 27.93 s

[0AyanamiRei]

run buildall

[hello-stephen]

FE UT Coverage Report

Increment line coverage 50.67% (38/75) 🎉
Increment coverage report
Complete coverage report

[doris-robot]

TPC-H: Total hot run time: 31338 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 910a36a4d0f647d5b4ba78e1722d94cfe8e6045e, data reload: false

------ Round 1 ----------------------------------
q1	17646	4286	4065	4065
q2	2064	349	244	244
q3	10133	1250	731	731
q4	10228	867	306	306
q5	7554	2027	1854	1854
q6	191	171	137	137
q7	921	783	656	656
q8	9291	1439	1079	1079
q9	4892	4526	4605	4526
q10	6787	1798	1416	1416
q11	493	297	287	287
q12	701	730	593	593
q13	17778	3839	3097	3097
q14	286	290	283	283
q15	595	510	508	508
q16	691	679	637	637
q17	661	814	462	462
q18	6899	6418	6480	6418
q19	1093	975	621	621
q20	376	358	239	239
q21	2960	2415	2192	2192
q22	1044	1020	987	987
Total cold run time: 103284 ms
Total hot run time: 31338 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4093	4062	4056	4056
q2	340	379	314	314
q3	2100	2581	2219	2219
q4	1319	1737	1325	1325
q5	4093	4049	4136	4049
q6	203	167	126	126
q7	1848	1822	1685	1685
q8	2847	2494	2369	2369
q9	7206	7315	7199	7199
q10	2600	2696	2362	2362
q11	604	515	496	496
q12	703	741	657	657
q13	3794	4281	3582	3582
q14	316	322	295	295
q15	548	505	495	495
q16	709	709	677	677
q17	1224	1413	1504	1413
q18	7940	8008	7660	7660
q19	893	848	852	848
q20	2181	2060	1916	1916
q21	4885	4361	4149	4149
q22	1060	1038	986	986
Total cold run time: 51506 ms
Total hot run time: 48878 ms

[doris-robot]

TPC-DS: Total hot run time: 174713 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 910a36a4d0f647d5b4ba78e1722d94cfe8e6045e, data reload: false

query5	4433	615	481	481
query6	338	240	224	224
query7	4223	452	266	266
query8	351	255	266	255
query9	8707	2872	2855	2855
query10	529	386	343	343
query11	15222	15121	15029	15029
query12	185	122	117	117
query13	1263	493	371	371
query14	6351	3030	2746	2746
query14_1	2688	2673	2680	2673
query15	197	195	176	176
query16	1017	479	493	479
query17	1089	642	522	522
query18	2522	438	316	316
query19	239	211	186	186
query20	117	113	112	112
query21	216	138	112	112
query22	3801	4200	3908	3908
query23	16038	15758	15387	15387
query23_1	15429	15682	15330	15330
query24	7221	1534	1195	1195
query24_1	1160	1158	1167	1158
query25	514	426	392	392
query26	1246	262	150	150
query27	2785	434	276	276
query28	4596	2122	2117	2117
query29	771	517	409	409
query30	317	245	205	205
query31	786	629	579	579
query32	82	71	74	71
query33	517	372	307	307
query34	896	877	521	521
query35	711	772	664	664
query36	878	898	873	873
query37	130	94	81	81
query38	2664	2770	2671	2671
query39	783	744	739	739
query39_1	717	728	728	728
query40	223	137	119	119
query41	65	63	59	59
query42	105	102	108	102
query43	472	443	449	443
query44	1300	726	712	712
query45	186	186	175	175
query46	807	968	590	590
query47	1370	1476	1403	1403
query48	309	320	240	240
query49	595	415	348	348
query50	611	260	201	201
query51	3875	3799	3763	3763
query52	103	108	97	97
query53	280	321	276	276
query54	319	262	251	251
query55	82	75	79	75
query56	300	299	300	299
query57	1008	1018	919	919
query58	273	255	250	250
query59	1965	2237	2203	2203
query60	334	328	322	322
query61	178	150	149	149
query62	394	356	292	292
query63	301	272	263	263
query64	4942	1237	937	937
query65	3833	3737	3822	3737
query66	1421	412	322	322
query67	15579	15647	15515	15515
query68	2455	1080	745	745
query69	439	371	331	331
query70	1021	919	882	882
query71	330	311	287	287
query72	5334	3393	3481	3393
query73	631	722	307	307
query74	8822	8743	8607	8607
query75	2763	2825	2504	2504
query76	2318	1043	650	650
query77	364	391	322	322
query78	9847	10001	9230	9230
query79	1126	835	591	591
query80	654	596	523	523
query81	462	264	244	244
query82	1328	153	118	118
query83	368	259	248	248
query84	260	126	98	98
query85	922	535	428	428
query86	363	290	291	290
query87	2835	2933	2751	2751
query88	3482	2580	2550	2550
query89	383	347	325	325
query90	1901	174	165	165
query91	168	157	135	135
query92	72	74	72	72
query93	909	871	534	534
query94	453	317	290	290
query95	566	388	319	319
query96	638	496	226	226
query97	2337	2380	2317	2317
query98	218	199	219	199
query99	588	578	534	534
Total cold run time: 246173 ms
Total hot run time: 174713 ms

[doris-robot]

ClickBench: Total hot run time: 26.58 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 910a36a4d0f647d5b4ba78e1722d94cfe8e6045e, data reload: false

query1	0.06	0.05	0.04
query2	0.10	0.05	0.05
query3	0.26	0.09	0.09
query4	1.60	0.12	0.11
query5	0.28	0.26	0.25
query6	1.15	0.66	0.65
query7	0.04	0.03	0.03
query8	0.05	0.04	0.04
query9	0.58	0.50	0.51
query10	0.56	0.56	0.54
query11	0.14	0.09	0.10
query12	0.14	0.11	0.11
query13	0.60	0.58	0.60
query14	0.96	0.93	0.94
query15	0.79	0.76	0.78
query16	0.39	0.39	0.40
query17	1.04	1.05	0.98
query18	0.22	0.21	0.21
query19	1.90	1.91	1.76
query20	0.01	0.01	0.02
query21	15.41	0.28	0.14
query22	5.30	0.05	0.04
query23	16.04	0.28	0.10
query24	1.59	0.33	0.26
query25	0.08	0.06	0.06
query26	0.15	0.13	0.13
query27	0.07	0.06	0.05
query28	3.34	1.10	0.89
query29	12.54	3.91	3.15
query30	0.28	0.13	0.12
query31	2.81	0.61	0.39
query32	3.24	0.55	0.45
query33	3.06	3.00	3.00
query34	16.06	5.14	4.43
query35	4.45	4.44	4.42
query36	0.64	0.50	0.48
query37	0.11	0.07	0.07
query38	0.07	0.04	0.04
query39	0.04	0.03	0.03
query40	0.16	0.14	0.13
query41	0.09	0.03	0.02
query42	0.04	0.03	0.03
query43	0.05	0.03	0.04
Total cold run time: 96.49 s
Total hot run time: 26.58 s

[hello-stephen]

FE Regression Coverage Report

Increment line coverage 66.67% (50/75) 🎉
Increment coverage report
Complete coverage report