oracle return all rows from a table if
Sorry, my description sucks, but the problem is easy to explain. I have a massive table (over 2bn rows) called BigTable which is partitioned by date that looks like this:
ID Object Date
--- ------- ----------
1 A 2018-10-01
1 B 2018-10-01
1 C 2018-10-01
1 D 2018-10-01
2 M 2018-10-01
2 N 2018-10-01
2 O 2018-10-01
3 X 2018-10-01
3 B 2018-10-01
I need a SQL that returns ALL rows for the ID where an Object = "B" and the date = '2018-10-01'
Therefore the result would be:
ID Object Date
--- ------- ----------
1 A 2018-10-01
1 B 2018-10-01
1 C 2018-10-01
1 D 2018-10-01
3 X 2018-10-01
3 B 2018-10-01
The SQL is easy enough, just join the table to itself:
SELECT t2.id,t2.object, t2.date
FROM BigTable T1
join BigTable T2 on t2.id = t1.id
AND t2.transaction_date = '01-OCT-2018'
AND t2.object = 'B'
where t1.date = '01-OCT-2018'
This however runs for HOURS. Subquery does not make it any better.
What is the most efficient way to return all rows for an ID that contains an object = 'B'?
sql oracle performance join
add a comment |
Sorry, my description sucks, but the problem is easy to explain. I have a massive table (over 2bn rows) called BigTable which is partitioned by date that looks like this:
ID Object Date
--- ------- ----------
1 A 2018-10-01
1 B 2018-10-01
1 C 2018-10-01
1 D 2018-10-01
2 M 2018-10-01
2 N 2018-10-01
2 O 2018-10-01
3 X 2018-10-01
3 B 2018-10-01
I need a SQL that returns ALL rows for the ID where an Object = "B" and the date = '2018-10-01'
Therefore the result would be:
ID Object Date
--- ------- ----------
1 A 2018-10-01
1 B 2018-10-01
1 C 2018-10-01
1 D 2018-10-01
3 X 2018-10-01
3 B 2018-10-01
The SQL is easy enough, just join the table to itself:
SELECT t2.id,t2.object, t2.date
FROM BigTable T1
join BigTable T2 on t2.id = t1.id
AND t2.transaction_date = '01-OCT-2018'
AND t2.object = 'B'
where t1.date = '01-OCT-2018'
This however runs for HOURS. Subquery does not make it any better.
What is the most efficient way to return all rows for an ID that contains an object = 'B'?
sql oracle performance join
2
Uset2.transaction_date = DATE '2018-10-01'
andt1.date = DATE '2018-10-01'
to avoid implicit conversion. Create a local Index on(date, id, transaction_date, object)
– Kaushik Nayak
Nov 14 '18 at 17:35
@KaushikNayak You should put that in an answer, because it's very likely that the implicit type conversion is his real problem.
– Matthew McPeak
Nov 14 '18 at 19:26
Please check my answer and accept/upvote it if it worked for you so that it would also help others seeking answers.Please read : stackoverflow.com/help/someone-answers
– Kaushik Nayak
Dec 19 '18 at 15:40
Unfortunately, index already existed and suggested changes did not improve the performance.
– SRev
Dec 20 '18 at 17:59
add a comment |
Sorry, my description sucks, but the problem is easy to explain. I have a massive table (over 2bn rows) called BigTable which is partitioned by date that looks like this:
ID Object Date
--- ------- ----------
1 A 2018-10-01
1 B 2018-10-01
1 C 2018-10-01
1 D 2018-10-01
2 M 2018-10-01
2 N 2018-10-01
2 O 2018-10-01
3 X 2018-10-01
3 B 2018-10-01
I need a SQL that returns ALL rows for the ID where an Object = "B" and the date = '2018-10-01'
Therefore the result would be:
ID Object Date
--- ------- ----------
1 A 2018-10-01
1 B 2018-10-01
1 C 2018-10-01
1 D 2018-10-01
3 X 2018-10-01
3 B 2018-10-01
The SQL is easy enough, just join the table to itself:
SELECT t2.id,t2.object, t2.date
FROM BigTable T1
join BigTable T2 on t2.id = t1.id
AND t2.transaction_date = '01-OCT-2018'
AND t2.object = 'B'
where t1.date = '01-OCT-2018'
This however runs for HOURS. Subquery does not make it any better.
What is the most efficient way to return all rows for an ID that contains an object = 'B'?
sql oracle performance join
Sorry, my description sucks, but the problem is easy to explain. I have a massive table (over 2bn rows) called BigTable which is partitioned by date that looks like this:
ID Object Date
--- ------- ----------
1 A 2018-10-01
1 B 2018-10-01
1 C 2018-10-01
1 D 2018-10-01
2 M 2018-10-01
2 N 2018-10-01
2 O 2018-10-01
3 X 2018-10-01
3 B 2018-10-01
I need a SQL that returns ALL rows for the ID where an Object = "B" and the date = '2018-10-01'
Therefore the result would be:
ID Object Date
--- ------- ----------
1 A 2018-10-01
1 B 2018-10-01
1 C 2018-10-01
1 D 2018-10-01
3 X 2018-10-01
3 B 2018-10-01
The SQL is easy enough, just join the table to itself:
SELECT t2.id,t2.object, t2.date
FROM BigTable T1
join BigTable T2 on t2.id = t1.id
AND t2.transaction_date = '01-OCT-2018'
AND t2.object = 'B'
where t1.date = '01-OCT-2018'
This however runs for HOURS. Subquery does not make it any better.
What is the most efficient way to return all rows for an ID that contains an object = 'B'?
sql oracle performance join
sql oracle performance join
asked Nov 14 '18 at 16:45
SRevSRev
195
195
2
Uset2.transaction_date = DATE '2018-10-01'
andt1.date = DATE '2018-10-01'
to avoid implicit conversion. Create a local Index on(date, id, transaction_date, object)
– Kaushik Nayak
Nov 14 '18 at 17:35
@KaushikNayak You should put that in an answer, because it's very likely that the implicit type conversion is his real problem.
– Matthew McPeak
Nov 14 '18 at 19:26
Please check my answer and accept/upvote it if it worked for you so that it would also help others seeking answers.Please read : stackoverflow.com/help/someone-answers
– Kaushik Nayak
Dec 19 '18 at 15:40
Unfortunately, index already existed and suggested changes did not improve the performance.
– SRev
Dec 20 '18 at 17:59
add a comment |
2
Uset2.transaction_date = DATE '2018-10-01'
andt1.date = DATE '2018-10-01'
to avoid implicit conversion. Create a local Index on(date, id, transaction_date, object)
– Kaushik Nayak
Nov 14 '18 at 17:35
@KaushikNayak You should put that in an answer, because it's very likely that the implicit type conversion is his real problem.
– Matthew McPeak
Nov 14 '18 at 19:26
Please check my answer and accept/upvote it if it worked for you so that it would also help others seeking answers.Please read : stackoverflow.com/help/someone-answers
– Kaushik Nayak
Dec 19 '18 at 15:40
Unfortunately, index already existed and suggested changes did not improve the performance.
– SRev
Dec 20 '18 at 17:59
2
2
Use
t2.transaction_date = DATE '2018-10-01'
and t1.date = DATE '2018-10-01'
to avoid implicit conversion. Create a local Index on (date, id, transaction_date, object)
– Kaushik Nayak
Nov 14 '18 at 17:35
Use
t2.transaction_date = DATE '2018-10-01'
and t1.date = DATE '2018-10-01'
to avoid implicit conversion. Create a local Index on (date, id, transaction_date, object)
– Kaushik Nayak
Nov 14 '18 at 17:35
@KaushikNayak You should put that in an answer, because it's very likely that the implicit type conversion is his real problem.
– Matthew McPeak
Nov 14 '18 at 19:26
@KaushikNayak You should put that in an answer, because it's very likely that the implicit type conversion is his real problem.
– Matthew McPeak
Nov 14 '18 at 19:26
Please check my answer and accept/upvote it if it worked for you so that it would also help others seeking answers.Please read : stackoverflow.com/help/someone-answers
– Kaushik Nayak
Dec 19 '18 at 15:40
Please check my answer and accept/upvote it if it worked for you so that it would also help others seeking answers.Please read : stackoverflow.com/help/someone-answers
– Kaushik Nayak
Dec 19 '18 at 15:40
Unfortunately, index already existed and suggested changes did not improve the performance.
– SRev
Dec 20 '18 at 17:59
Unfortunately, index already existed and suggested changes did not improve the performance.
– SRev
Dec 20 '18 at 17:59
add a comment |
3 Answers
3
active
oldest
votes
I assume you have good indexing in place... No guarantees this will help, but have you tried a semi-join:
select
t1.*
from BigTable T1
where
t1.transaction_date = '01-OCT-2018' and
exists (
select null
from BigTable T2
where
t2.transaction_date = t1.transaction_date and
t2.transaction_date = '01-OCT-2018'
t2.object = 'B'
)
The advantage of the semi-join is that once it finds a match it should "stop looking."
The join on transaction date may seem superfluous, but try it both ways.
add a comment |
You can try window functions:
SELECT t.id, t.object, t.date
FROM (SELECT t.*,
SUM(CASE WHEN t.object = 'B' THEN 1 ELSE 0 END) OVER (PARTITION BY t.id, t.transaction_date) as cnt
FROM BigTable t
WHERE t.transaction_date = DATE '2018-10-01'
) t
WHERE cnt > 0;
One possibility is that you misunderstand the partitioning, and the entire contents of the table are being read in your query.
add a comment |
You should avoid using date strings directly while comparing it with dates. The reason being implicit conversion may not always use the available partitions on date. The documentation for Data type conversion states that
Algorithms for implicit conversion are subject to change across
software releases and among Oracle products. Behavior of explicit
conversions is more predictable.
If implicit data type conversion occurs in an index expression, then
Oracle Database might not use the index because it is defined for the
pre-conversion data type. This can have a negative impact on
performance.
Thus, it is preferable to use the ANSI standard date literals
SELECT t2.id,t2.object, t2.date
FROM BigTable T1
join BigTable T2 on t2.id = t1.id
AND t2.transaction_date = DATE '2018-10-01'
AND t2.object = 'B'
where t1.date = DATE '2018-10-01'
Also, as already suggested in comments, create a LOCAL INDEX
on (date, id, transaction_date, object)
to further improve the speed of the scan using the index along with partition pruning.
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53305046%2foracle-return-all-rows-from-a-table-if%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
3 Answers
3
active
oldest
votes
3 Answers
3
active
oldest
votes
active
oldest
votes
active
oldest
votes
I assume you have good indexing in place... No guarantees this will help, but have you tried a semi-join:
select
t1.*
from BigTable T1
where
t1.transaction_date = '01-OCT-2018' and
exists (
select null
from BigTable T2
where
t2.transaction_date = t1.transaction_date and
t2.transaction_date = '01-OCT-2018'
t2.object = 'B'
)
The advantage of the semi-join is that once it finds a match it should "stop looking."
The join on transaction date may seem superfluous, but try it both ways.
add a comment |
I assume you have good indexing in place... No guarantees this will help, but have you tried a semi-join:
select
t1.*
from BigTable T1
where
t1.transaction_date = '01-OCT-2018' and
exists (
select null
from BigTable T2
where
t2.transaction_date = t1.transaction_date and
t2.transaction_date = '01-OCT-2018'
t2.object = 'B'
)
The advantage of the semi-join is that once it finds a match it should "stop looking."
The join on transaction date may seem superfluous, but try it both ways.
add a comment |
I assume you have good indexing in place... No guarantees this will help, but have you tried a semi-join:
select
t1.*
from BigTable T1
where
t1.transaction_date = '01-OCT-2018' and
exists (
select null
from BigTable T2
where
t2.transaction_date = t1.transaction_date and
t2.transaction_date = '01-OCT-2018'
t2.object = 'B'
)
The advantage of the semi-join is that once it finds a match it should "stop looking."
The join on transaction date may seem superfluous, but try it both ways.
I assume you have good indexing in place... No guarantees this will help, but have you tried a semi-join:
select
t1.*
from BigTable T1
where
t1.transaction_date = '01-OCT-2018' and
exists (
select null
from BigTable T2
where
t2.transaction_date = t1.transaction_date and
t2.transaction_date = '01-OCT-2018'
t2.object = 'B'
)
The advantage of the semi-join is that once it finds a match it should "stop looking."
The join on transaction date may seem superfluous, but try it both ways.
answered Nov 14 '18 at 16:51
HamboneHambone
10.3k52849
10.3k52849
add a comment |
add a comment |
You can try window functions:
SELECT t.id, t.object, t.date
FROM (SELECT t.*,
SUM(CASE WHEN t.object = 'B' THEN 1 ELSE 0 END) OVER (PARTITION BY t.id, t.transaction_date) as cnt
FROM BigTable t
WHERE t.transaction_date = DATE '2018-10-01'
) t
WHERE cnt > 0;
One possibility is that you misunderstand the partitioning, and the entire contents of the table are being read in your query.
add a comment |
You can try window functions:
SELECT t.id, t.object, t.date
FROM (SELECT t.*,
SUM(CASE WHEN t.object = 'B' THEN 1 ELSE 0 END) OVER (PARTITION BY t.id, t.transaction_date) as cnt
FROM BigTable t
WHERE t.transaction_date = DATE '2018-10-01'
) t
WHERE cnt > 0;
One possibility is that you misunderstand the partitioning, and the entire contents of the table are being read in your query.
add a comment |
You can try window functions:
SELECT t.id, t.object, t.date
FROM (SELECT t.*,
SUM(CASE WHEN t.object = 'B' THEN 1 ELSE 0 END) OVER (PARTITION BY t.id, t.transaction_date) as cnt
FROM BigTable t
WHERE t.transaction_date = DATE '2018-10-01'
) t
WHERE cnt > 0;
One possibility is that you misunderstand the partitioning, and the entire contents of the table are being read in your query.
You can try window functions:
SELECT t.id, t.object, t.date
FROM (SELECT t.*,
SUM(CASE WHEN t.object = 'B' THEN 1 ELSE 0 END) OVER (PARTITION BY t.id, t.transaction_date) as cnt
FROM BigTable t
WHERE t.transaction_date = DATE '2018-10-01'
) t
WHERE cnt > 0;
One possibility is that you misunderstand the partitioning, and the entire contents of the table are being read in your query.
answered Nov 14 '18 at 20:19
Gordon LinoffGordon Linoff
786k35311416
786k35311416
add a comment |
add a comment |
You should avoid using date strings directly while comparing it with dates. The reason being implicit conversion may not always use the available partitions on date. The documentation for Data type conversion states that
Algorithms for implicit conversion are subject to change across
software releases and among Oracle products. Behavior of explicit
conversions is more predictable.
If implicit data type conversion occurs in an index expression, then
Oracle Database might not use the index because it is defined for the
pre-conversion data type. This can have a negative impact on
performance.
Thus, it is preferable to use the ANSI standard date literals
SELECT t2.id,t2.object, t2.date
FROM BigTable T1
join BigTable T2 on t2.id = t1.id
AND t2.transaction_date = DATE '2018-10-01'
AND t2.object = 'B'
where t1.date = DATE '2018-10-01'
Also, as already suggested in comments, create a LOCAL INDEX
on (date, id, transaction_date, object)
to further improve the speed of the scan using the index along with partition pruning.
add a comment |
You should avoid using date strings directly while comparing it with dates. The reason being implicit conversion may not always use the available partitions on date. The documentation for Data type conversion states that
Algorithms for implicit conversion are subject to change across
software releases and among Oracle products. Behavior of explicit
conversions is more predictable.
If implicit data type conversion occurs in an index expression, then
Oracle Database might not use the index because it is defined for the
pre-conversion data type. This can have a negative impact on
performance.
Thus, it is preferable to use the ANSI standard date literals
SELECT t2.id,t2.object, t2.date
FROM BigTable T1
join BigTable T2 on t2.id = t1.id
AND t2.transaction_date = DATE '2018-10-01'
AND t2.object = 'B'
where t1.date = DATE '2018-10-01'
Also, as already suggested in comments, create a LOCAL INDEX
on (date, id, transaction_date, object)
to further improve the speed of the scan using the index along with partition pruning.
add a comment |
You should avoid using date strings directly while comparing it with dates. The reason being implicit conversion may not always use the available partitions on date. The documentation for Data type conversion states that
Algorithms for implicit conversion are subject to change across
software releases and among Oracle products. Behavior of explicit
conversions is more predictable.
If implicit data type conversion occurs in an index expression, then
Oracle Database might not use the index because it is defined for the
pre-conversion data type. This can have a negative impact on
performance.
Thus, it is preferable to use the ANSI standard date literals
SELECT t2.id,t2.object, t2.date
FROM BigTable T1
join BigTable T2 on t2.id = t1.id
AND t2.transaction_date = DATE '2018-10-01'
AND t2.object = 'B'
where t1.date = DATE '2018-10-01'
Also, as already suggested in comments, create a LOCAL INDEX
on (date, id, transaction_date, object)
to further improve the speed of the scan using the index along with partition pruning.
You should avoid using date strings directly while comparing it with dates. The reason being implicit conversion may not always use the available partitions on date. The documentation for Data type conversion states that
Algorithms for implicit conversion are subject to change across
software releases and among Oracle products. Behavior of explicit
conversions is more predictable.
If implicit data type conversion occurs in an index expression, then
Oracle Database might not use the index because it is defined for the
pre-conversion data type. This can have a negative impact on
performance.
Thus, it is preferable to use the ANSI standard date literals
SELECT t2.id,t2.object, t2.date
FROM BigTable T1
join BigTable T2 on t2.id = t1.id
AND t2.transaction_date = DATE '2018-10-01'
AND t2.object = 'B'
where t1.date = DATE '2018-10-01'
Also, as already suggested in comments, create a LOCAL INDEX
on (date, id, transaction_date, object)
to further improve the speed of the scan using the index along with partition pruning.
answered Nov 15 '18 at 11:43
Kaushik NayakKaushik Nayak
20.4k41332
20.4k41332
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53305046%2foracle-return-all-rows-from-a-table-if%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
2
Use
t2.transaction_date = DATE '2018-10-01'
andt1.date = DATE '2018-10-01'
to avoid implicit conversion. Create a local Index on(date, id, transaction_date, object)
– Kaushik Nayak
Nov 14 '18 at 17:35
@KaushikNayak You should put that in an answer, because it's very likely that the implicit type conversion is his real problem.
– Matthew McPeak
Nov 14 '18 at 19:26
Please check my answer and accept/upvote it if it worked for you so that it would also help others seeking answers.Please read : stackoverflow.com/help/someone-answers
– Kaushik Nayak
Dec 19 '18 at 15:40
Unfortunately, index already existed and suggested changes did not improve the performance.
– SRev
Dec 20 '18 at 17:59