Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[opt](nereids) optimize limit on distinct aggregate #47570

Merged
merged 4 commits into from
Feb 12, 2025
Merged

[opt](nereids) optimize limit on distinct aggregate #47570

merged 4 commits into from
Feb 12, 2025

Conversation

englefly
Copy link
Contributor

@englefly englefly commented Feb 7, 2025 •
edited
Loading

What problem does this PR solve?

if there is no aggregate-functions in aggregation output, the limit could be pushed down inside this aggregate.
example： select a from T group by a limit 1
original plan: limit(1)->distinct_aggregate(global, limit=1)->distinct_aggregate(local)->scan
after optimize: limit(1) -> distinct_aggregate(global, limit=1)->distinct_aggregate(local, limit=1)->scan

this work is implemented in PhysicalPlanTranslator

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason

  • Behavior changed:

    • No.
    • Yes.

  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Feb 7, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@englefly
Copy link
Contributor Author

englefly commented Feb 7, 2025

run buildall

@englefly englefly changed the title opt limit push agg [opt](nereids) optimize limit on distinct aggregate Feb 7, 2025
@doris-robot
Copy link

TPC-H: Total hot run time: 31725 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit c1d847fbdc2ae60e196e82e4789469915306bbb6, data reload: false

------ Round 1 ----------------------------------
q1	17573	5106	5068	5068
q2	2047	298	166	166
q3	10409	1224	739	739
q4	10201	1020	537	537
q5	7508	2342	2367	2342
q6	182	163	136	136
q7	896	739	591	591
q8	9293	1220	1046	1046
q9	4741	4873	4770	4770
q10	6859	2315	1901	1901
q11	474	270	252	252
q12	347	354	215	215
q13	17804	3670	3113	3113
q14	229	222	214	214
q15	510	473	466	466
q16	628	612	579	579
q17	553	862	337	337
q18	7186	6475	6311	6311
q19	1363	960	532	532
q20	320	320	192	192
q21	2736	2107	1914	1914
q22	366	326	304	304
Total cold run time: 102225 ms
Total hot run time: 31725 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5118	5112	5118	5112
q2	234	328	234	234
q3	2142	2657	2338	2338
q4	1395	1798	1369	1369
q5	4222	4133	4132	4132
q6	215	164	124	124
q7	1900	1830	1683	1683
q8	2610	2628	2496	2496
q9	7203	7250	7096	7096
q10	3050	3201	2808	2808
q11	569	518	504	504
q12	703	760	619	619
q13	3433	3927	3335	3335
q14	278	299	267	267
q15	508	460	468	460
q16	635	689	659	659
q17	1136	1624	1329	1329
q18	7578	7415	7342	7342
q19	792	773	798	773
q20	1974	1988	1878	1878
q21	5460	5107	4789	4789
q22	613	570	549	549
Total cold run time: 51768 ms
Total hot run time: 49896 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190829 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit c1d847fbdc2ae60e196e82e4789469915306bbb6, data reload: false

query1	1358	960	932	932
query2	6310	1946	1932	1932
query3	11138	4678	4614	4614
query4	53927	24912	23557	23557
query5	5192	583	503	503
query6	352	233	177	177
query7	4922	498	302	302
query8	309	252	244	244
query9	5824	2518	2491	2491
query10	445	321	261	261
query11	15169	15261	14892	14892
query12	166	111	106	106
query13	1064	508	399	399
query14	10631	6371	6497	6371
query15	200	212	175	175
query16	7085	648	495	495
query17	1093	756	584	584
query18	1541	420	318	318
query19	202	210	176	176
query20	128	133	130	130
query21	209	128	107	107
query22	4389	4497	4445	4445
query23	34030	33408	33410	33408
query24	5817	2452	2462	2452
query25	459	484	390	390
query26	717	295	164	164
query27	1766	467	339	339
query28	2744	2403	2379	2379
query29	570	556	418	418
query30	210	185	155	155
query31	867	865	792	792
query32	75	64	67	64
query33	482	359	297	297
query34	817	867	494	494
query35	836	858	782	782
query36	955	1006	928	928
query37	120	95	70	70
query38	4196	4357	4336	4336
query39	1545	1438	1456	1438
query40	214	113	102	102
query41	50	49	49	49
query42	121	105	109	105
query43	519	518	483	483
query44	1295	800	802	800
query45	181	177	165	165
query46	899	1091	666	666
query47	1859	1917	1788	1788
query48	387	426	309	309
query49	696	498	423	423
query50	705	773	415	415
query51	4262	4344	4305	4305
query52	111	105	97	97
query53	229	258	197	197
query54	472	485	405	405
query55	87	80	77	77
query56	276	272	246	246
query57	1203	1206	1154	1154
query58	257	246	250	246
query59	2860	3027	2653	2653
query60	286	278	261	261
query61	121	117	118	117
query62	772	757	687	687
query63	235	198	200	198
query64	1838	1037	667	667
query65	3226	3120	3133	3120
query66	727	382	291	291
query67	15994	15674	15491	15491
query68	5590	777	523	523
query69	525	301	264	264
query70	1195	1133	1056	1056
query71	430	289	306	289
query72	6333	3661	3689	3661
query73	1266	750	353	353
query74	8924	9223	8847	8847
query75	3273	3167	2677	2677
query76	3840	1195	739	739
query77	531	379	278	278
query78	10030	10141	9339	9339
query79	2426	810	596	596
query80	696	541	489	489
query81	512	272	237	237
query82	405	134	95	95
query83	176	174	158	158
query84	287	91	79	79
query85	735	358	311	311
query86	370	330	301	301
query87	4457	4469	4405	4405
query88	3585	2173	2154	2154
query89	391	311	282	282
query90	1733	194	193	193
query91	129	141	108	108
query92	70	59	58	58
query93	2678	1016	593	593
query94	693	370	298	298
query95	349	276	265	265
query96	475	558	268	268
query97	2769	2875	2785	2785
query98	235	206	210	206
query99	1342	1382	1288	1288
Total cold run time: 294192 ms
Total hot run time: 190829 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.88 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit c1d847fbdc2ae60e196e82e4789469915306bbb6, data reload: false

query1	0.03	0.03	0.04
query2	0.07	0.04	0.03
query3	0.24	0.06	0.06
query4	1.62	0.10	0.10
query5	0.44	0.43	0.41
query6	1.17	0.65	0.66
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.58	0.52	0.53
query10	0.58	0.60	0.57
query11	0.15	0.11	0.11
query12	0.14	0.11	0.11
query13	0.62	0.60	0.60
query14	2.69	2.68	2.70
query15	0.92	0.85	0.85
query16	0.38	0.38	0.38
query17	1.03	1.04	1.05
query18	0.22	0.20	0.20
query19	1.90	1.76	2.03
query20	0.02	0.01	0.01
query21	15.35	0.90	0.53
query22	0.77	1.16	0.67
query23	14.95	1.31	0.58
query24	7.22	1.71	1.17
query25	0.48	0.30	0.09
query26	0.58	0.17	0.14
query27	0.06	0.05	0.05
query28	10.09	0.84	0.43
query29	12.53	4.00	3.29
query30	0.25	0.08	0.06
query31	2.82	0.57	0.38
query32	3.23	0.55	0.47
query33	2.98	3.05	3.02
query34	15.65	5.14	4.52
query35	4.60	4.57	4.55
query36	0.66	0.51	0.48
query37	0.08	0.06	0.06
query38	0.05	0.04	0.03
query39	0.03	0.03	0.02
query40	0.16	0.13	0.13
query41	0.08	0.03	0.02
query42	0.04	0.02	0.02
query43	0.03	0.03	0.02
Total cold run time: 105.55 s
Total hot run time: 30.88 s

@englefly
Copy link
Contributor Author

englefly commented Feb 8, 2025

run p0

@englefly
Copy link
Contributor Author

englefly commented Feb 8, 2025

run cloud_p0

@englefly
Copy link
Contributor Author

englefly commented Feb 8, 2025

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 31611 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 6138a06c99e1963cebc4b3d5ef02e52bedba0be5, data reload: false

------ Round 1 ----------------------------------
q1	17615	5159	5059	5059
q2	2057	303	169	169
q3	10476	1265	766	766
q4	10228	1005	538	538
q5	7704	2392	2348	2348
q6	192	170	135	135
q7	892	756	592	592
q8	9299	1310	1127	1127
q9	4956	4753	4536	4536
q10	6832	2319	1938	1938
q11	481	272	258	258
q12	346	349	222	222
q13	17786	3714	3109	3109
q14	235	225	220	220
q15	517	485	457	457
q16	618	625	581	581
q17	574	863	358	358
q18	6769	6346	6247	6247
q19	1468	968	565	565
q20	319	324	191	191
q21	2845	2325	1899	1899
q22	359	338	296	296
Total cold run time: 102568 ms
Total hot run time: 31611 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5156	5563	5127	5127
q2	236	329	234	234
q3	2186	2673	2315	2315
q4	1514	1865	1423	1423
q5	4236	4078	4154	4078
q6	209	168	125	125
q7	1864	1814	1755	1755
q8	2641	2651	2557	2557
q9	7238	7234	7197	7197
q10	3060	3232	2829	2829
q11	587	517	483	483
q12	689	794	660	660
q13	3417	3920	3308	3308
q14	280	284	283	283
q15	517	473	469	469
q16	653	683	632	632
q17	1157	1658	1284	1284
q18	7725	7536	7421	7421
q19	811	776	945	776
q20	1979	2045	1877	1877
q21	5547	5067	4839	4839
q22	652	588	560	560
Total cold run time: 52354 ms
Total hot run time: 50232 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190252 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 6138a06c99e1963cebc4b3d5ef02e52bedba0be5, data reload: false

query1	1328	964	935	935
query2	6221	1944	1881	1881
query3	11129	4647	4671	4647
query4	54714	25239	22919	22919
query5	5102	535	486	486
query6	329	195	188	188
query7	4895	514	294	294
query8	314	254	233	233
query9	5882	2567	2571	2567
query10	415	323	246	246
query11	15249	15073	14925	14925
query12	152	114	107	107
query13	1055	487	370	370
query14	10863	6613	7092	6613
query15	200	206	170	170
query16	7034	642	467	467
query17	1054	699	546	546
query18	1537	409	302	302
query19	204	193	156	156
query20	124	126	124	124
query21	210	133	103	103
query22	4601	4462	4425	4425
query23	34163	33375	33391	33375
query24	5730	2420	2421	2420
query25	456	448	410	410
query26	673	282	163	163
query27	1807	500	341	341
query28	2706	2455	2435	2435
query29	571	541	422	422
query30	221	188	153	153
query31	898	894	801	801
query32	78	67	65	65
query33	485	345	297	297
query34	763	879	492	492
query35	801	831	758	758
query36	942	1004	904	904
query37	123	98	71	71
query38	4279	4270	4358	4270
query39	1482	1431	1419	1419
query40	202	116	102	102
query41	50	50	50	50
query42	125	103	106	103
query43	511	520	490	490
query44	1296	804	805	804
query45	182	178	176	176
query46	869	1073	669	669
query47	1870	1899	1847	1847
query48	391	415	305	305
query49	687	509	406	406
query50	725	742	423	423
query51	4293	4278	4298	4278
query52	106	103	104	103
query53	238	255	182	182
query54	486	486	406	406
query55	83	85	81	81
query56	259	281	297	281
query57	1162	1199	1120	1120
query58	248	240	243	240
query59	2816	2921	2788	2788
query60	271	282	282	282
query61	120	115	117	115
query62	781	757	710	710
query63	228	191	191	191
query64	1685	1015	743	743
query65	3189	3129	3127	3127
query66	756	377	285	285
query67	15862	15509	15417	15417
query68	2993	761	532	532
query69	450	291	259	259
query70	1194	1172	1096	1096
query71	322	296	259	259
query72	5215	3562	3771	3562
query73	630	715	345	345
query74	8970	9138	8945	8945
query75	3127	3165	2674	2674
query76	1597	1155	737	737
query77	411	360	287	287
query78	9900	10118	9238	9238
query79	988	880	583	583
query80	773	544	443	443
query81	519	276	238	238
query82	398	131	94	94
query83	173	171	151	151
query84	278	96	117	96
query85	742	350	296	296
query86	371	321	283	283
query87	4605	4597	4486	4486
query88	2850	2170	2157	2157
query89	387	308	280	280
query90	1657	186	185	185
query91	131	150	109	109
query92	65	66	56	56
query93	1410	997	569	569
query94	633	410	304	304
query95	345	259	250	250
query96	491	550	267	267
query97	2810	2887	2789	2789
query98	222	202	216	202
query99	1297	1416	1256	1256
Total cold run time: 284235 ms
Total hot run time: 190252 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.7 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 6138a06c99e1963cebc4b3d5ef02e52bedba0be5, data reload: false

query1	0.03	0.04	0.03
query2	0.09	0.04	0.03
query3	0.23	0.07	0.06
query4	1.62	0.10	0.10
query5	0.42	0.43	0.42
query6	1.18	0.68	0.68
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.59	0.52	0.52
query10	0.57	0.58	0.58
query11	0.16	0.10	0.10
query12	0.14	0.11	0.12
query13	0.62	0.60	0.60
query14	2.68	2.78	2.67
query15	0.93	0.85	0.86
query16	0.37	0.37	0.37
query17	1.01	1.00	1.03
query18	0.22	0.20	0.20
query19	1.90	1.79	1.98
query20	0.01	0.01	0.01
query21	15.35	0.93	0.55
query22	0.74	1.15	0.71
query23	14.94	1.40	0.63
query24	6.97	1.53	0.82
query25	0.50	0.17	0.09
query26	0.59	0.16	0.14
query27	0.06	0.05	0.04
query28	9.69	0.81	0.44
query29	12.53	4.02	3.34
query30	0.25	0.08	0.06
query31	2.82	0.59	0.37
query32	3.22	0.56	0.46
query33	3.06	2.99	3.08
query34	15.90	5.16	4.56
query35	4.60	4.56	4.52
query36	0.65	0.49	0.49
query37	0.09	0.06	0.06
query38	0.05	0.04	0.03
query39	0.04	0.02	0.02
query40	0.16	0.13	0.13
query41	0.09	0.03	0.02
query42	0.04	0.03	0.03
query43	0.03	0.03	0.03
Total cold run time: 105.2 s
Total hot run time: 30.7 s

@englefly
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 31938 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e8b8dd1b033225ee3a423bc2bb07d1350eef76a7, data reload: false

------ Round 1 ----------------------------------
q1	17606	5251	5144	5144
q2	2044	312	177	177
q3	10381	1273	767	767
q4	10236	1031	538	538
q5	7653	2438	2391	2391
q6	204	169	132	132
q7	914	748	603	603
q8	9307	1436	1228	1228
q9	4865	4950	4695	4695
q10	7023	2349	1896	1896
q11	519	286	256	256
q12	351	382	215	215
q13	17812	3725	3049	3049
q14	227	225	206	206
q15	521	474	455	455
q16	635	611	582	582
q17	591	912	339	339
q18	6580	6266	6217	6217
q19	1708	1030	588	588
q20	325	331	187	187
q21	2874	2178	1979	1979
q22	358	330	294	294
Total cold run time: 102734 ms
Total hot run time: 31938 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5403	5314	5268	5268
q2	252	332	228	228
q3	2135	2701	2248	2248
q4	1441	1869	1406	1406
q5	4254	4167	4141	4141
q6	227	165	125	125
q7	1896	1897	1799	1799
q8	2671	2761	2689	2689
q9	7105	7104	7101	7101
q10	3036	3238	2780	2780
q11	576	514	493	493
q12	714	800	651	651
q13	3523	4003	3250	3250
q14	292	307	264	264
q15	519	473	463	463
q16	655	712	631	631
q17	1158	1656	1361	1361
q18	7629	7399	7314	7314
q19	868	796	841	796
q20	2000	2009	1877	1877
q21	5387	4984	4677	4677
q22	641	597	545	545
Total cold run time: 52382 ms
Total hot run time: 50107 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190696 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e8b8dd1b033225ee3a423bc2bb07d1350eef76a7, data reload: false

query1	1313	972	937	937
query2	6351	1922	1903	1903
query3	10926	4325	4460	4325
query4	52977	24321	23715	23715
query5	5114	567	482	482
query6	341	190	187	187
query7	4878	538	300	300
query8	307	253	233	233
query9	5450	2505	2496	2496
query10	432	322	265	265
query11	15130	15025	14934	14934
query12	158	108	108	108
query13	1027	510	373	373
query14	10477	6337	6977	6337
query15	213	215	188	188
query16	7044	664	482	482
query17	1072	731	560	560
query18	1528	411	312	312
query19	194	189	167	167
query20	131	132	121	121
query21	212	136	105	105
query22	4437	4561	4336	4336
query23	34090	33521	33231	33231
query24	5666	2450	2517	2450
query25	518	494	441	441
query26	725	282	165	165
query27	1798	497	340	340
query28	2861	2460	2406	2406
query29	614	601	452	452
query30	215	194	160	160
query31	932	853	864	853
query32	77	68	64	64
query33	478	379	305	305
query34	783	895	505	505
query35	805	828	753	753
query36	973	1006	923	923
query37	124	97	75	75
query38	4296	4457	4326	4326
query39	1505	1438	1429	1429
query40	212	117	102	102
query41	51	50	52	50
query42	125	109	119	109
query43	515	577	503	503
query44	1332	807	819	807
query45	186	176	170	170
query46	892	1064	674	674
query47	1803	1878	1783	1783
query48	383	416	309	309
query49	684	516	453	453
query50	719	735	432	432
query51	4355	4289	4264	4264
query52	107	102	92	92
query53	238	261	189	189
query54	479	499	415	415
query55	85	76	77	76
query56	270	272	278	272
query57	1141	1159	1133	1133
query58	254	243	248	243
query59	2818	3090	2893	2893
query60	302	278	273	273
query61	122	116	119	116
query62	753	756	684	684
query63	249	203	194	194
query64	1462	1029	680	680
query65	3351	3194	3127	3127
query66	724	443	309	309
query67	15845	15616	15400	15400
query68	5824	783	537	537
query69	530	306	264	264
query70	1192	1166	1126	1126
query71	457	287	258	258
query72	6014	3659	3753	3659
query73	1362	761	359	359
query74	8969	9090	8951	8951
query75	3189	3163	2686	2686
query76	3873	1173	745	745
query77	516	392	285	285
query78	9999	10170	9279	9279
query79	1994	840	601	601
query80	591	552	445	445
query81	493	275	241	241
query82	201	128	100	100
query83	179	171	154	154
query84	284	93	73	73
query85	794	377	304	304
query86	364	326	275	275
query87	4443	4620	4374	4374
query88	3349	2227	2291	2227
query89	412	335	284	284
query90	1683	197	191	191
query91	135	144	107	107
query92	78	62	54	54
query93	1665	1046	592	592
query94	685	407	297	297
query95	349	280	263	263
query96	477	555	274	274
query97	2765	2873	2739	2739
query98	225	210	206	206
query99	1362	1392	1250	1250
Total cold run time: 290082 ms
Total hot run time: 190696 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.21 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e8b8dd1b033225ee3a423bc2bb07d1350eef76a7, data reload: false

query1	0.04	0.04	0.03
query2	0.07	0.03	0.03
query3	0.23	0.07	0.06
query4	1.61	0.10	0.11
query5	0.43	0.42	0.42
query6	1.14	0.65	0.66
query7	0.02	0.01	0.02
query8	0.04	0.04	0.03
query9	0.60	0.52	0.51
query10	0.58	0.58	0.58
query11	0.15	0.10	0.10
query12	0.15	0.12	0.11
query13	0.63	0.59	0.60
query14	2.78	2.71	2.71
query15	0.91	0.86	0.85
query16	0.37	0.38	0.38
query17	1.04	1.03	1.05
query18	0.22	0.20	0.20
query19	1.91	1.77	2.02
query20	0.02	0.01	0.02
query21	15.35	0.92	0.55
query22	0.76	1.22	0.66
query23	14.87	1.38	0.62
query24	6.76	1.97	1.41
query25	0.42	0.30	0.14
query26	0.62	0.16	0.13
query27	0.05	0.06	0.04
query28	9.85	0.76	0.42
query29	12.55	3.90	3.32
query30	0.25	0.09	0.07
query31	2.82	0.59	0.38
query32	3.22	0.55	0.46
query33	2.95	3.01	3.02
query34	15.81	5.08	4.50
query35	4.53	4.52	4.55
query36	0.67	0.49	0.49
query37	0.10	0.06	0.06
query38	0.05	0.04	0.03
query39	0.03	0.02	0.03
query40	0.16	0.13	0.13
query41	0.09	0.03	0.03
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 104.92 s
Total hot run time: 31.21 s

@englefly
Copy link
Contributor Author

run cloud_p0

@englefly
Copy link
Contributor Author

run feut

@englefly
Copy link
Contributor Author

run cloud_p0

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Feb 12, 2025
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

HappenLee
HappenLee approved these changes Feb 12, 2025
View reviewed changes
Copy link
Contributor

@HappenLee HappenLee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@HappenLee HappenLee merged commit f21bcc9 into apache:master Feb 12, 2025
26 of 27 checks passed
github-actions bot pushed a commit that referenced this pull request Feb 12, 2025
@englefly
[opt](nereids) optimize limit on distinct aggregate (#47570)
2c2dff8 
### What problem does this PR solve?
if there is no aggregate-functions in aggregation output, the limit
could be pushed down inside this aggregate.
example： select a from T group by a limit 1
original plan: limit(1)->distinct_aggregate(global,
limit=1)->distinct_aggregate(local)->scan
after optimize: limit(1) -> distinct_aggregate(global,
limit=1)->distinct_aggregate(local, limit=1)->scan

this work is implemented in PhysicalPlanTranslator
@github-actions github-actions bot mentioned this pull request Feb 12, 2025
Merged
lzyy2024 pushed a commit to lzyy2024/doris that referenced this pull request Feb 21, 2025
@englefly @lzyy2024
[opt](nereids) optimize limit on distinct aggregate (apache#47570)
13c232f 
### What problem does this PR solve?
if there is no aggregate-functions in aggregation output, the limit
could be pushed down inside this aggregate.
example： select a from T group by a limit 1
original plan: limit(1)->distinct_aggregate(global,
limit=1)->distinct_aggregate(local)->scan
after optimize: limit(1) -> distinct_aggregate(global,
limit=1)->distinct_aggregate(local, limit=1)->scan

this work is implemented in PhysicalPlanTranslator
@morrySnow morrySnow added the p0_b label Feb 24, 2025
dataroaring pushed a commit that referenced this pull request Feb 24, 2025
@github-actions @englefly
branch-3.0: [opt](nereids) optimize limit on distinct aggregate #47570 (
197869e 
#47816)

Cherry-picked from #47570

Co-authored-by: minghong <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@HappenLee HappenLee HappenLee approved these changes

@starocean999 starocean999 starocean999 approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.5-merged p0_b reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@englefly @Thearas @doris-robot @HappenLee @starocean999 @dataroaring @morrySnow