oracle return all rows from a table if










2















Sorry, my description sucks, but the problem is easy to explain. I have a massive table (over 2bn rows) called BigTable which is partitioned by date that looks like this:



ID Object Date
--- ------- ----------
1 A 2018-10-01
1 B 2018-10-01
1 C 2018-10-01
1 D 2018-10-01
2 M 2018-10-01
2 N 2018-10-01
2 O 2018-10-01
3 X 2018-10-01
3 B 2018-10-01


I need a SQL that returns ALL rows for the ID where an Object = "B" and the date = '2018-10-01'
Therefore the result would be:




ID Object Date
--- ------- ----------
1 A 2018-10-01
1 B 2018-10-01
1 C 2018-10-01
1 D 2018-10-01
3 X 2018-10-01
3 B 2018-10-01


The SQL is easy enough, just join the table to itself:



SELECT t2.id,t2.object, t2.date
FROM BigTable T1
join BigTable T2 on t2.id = t1.id
AND t2.transaction_date = '01-OCT-2018'
AND t2.object = 'B'
where t1.date = '01-OCT-2018'


This however runs for HOURS. Subquery does not make it any better.
What is the most efficient way to return all rows for an ID that contains an object = 'B'?










share|improve this question

















  • 2





    Use t2.transaction_date = DATE '2018-10-01' and t1.date = DATE '2018-10-01' to avoid implicit conversion. Create a local Index on (date, id, transaction_date, object)

    – Kaushik Nayak
    Nov 14 '18 at 17:35











  • @KaushikNayak You should put that in an answer, because it's very likely that the implicit type conversion is his real problem.

    – Matthew McPeak
    Nov 14 '18 at 19:26











  • Please check my answer and accept/upvote it if it worked for you so that it would also help others seeking answers.Please read : stackoverflow.com/help/someone-answers

    – Kaushik Nayak
    Dec 19 '18 at 15:40











  • Unfortunately, index already existed and suggested changes did not improve the performance.

    – SRev
    Dec 20 '18 at 17:59















2















Sorry, my description sucks, but the problem is easy to explain. I have a massive table (over 2bn rows) called BigTable which is partitioned by date that looks like this:



ID Object Date
--- ------- ----------
1 A 2018-10-01
1 B 2018-10-01
1 C 2018-10-01
1 D 2018-10-01
2 M 2018-10-01
2 N 2018-10-01
2 O 2018-10-01
3 X 2018-10-01
3 B 2018-10-01


I need a SQL that returns ALL rows for the ID where an Object = "B" and the date = '2018-10-01'
Therefore the result would be:




ID Object Date
--- ------- ----------
1 A 2018-10-01
1 B 2018-10-01
1 C 2018-10-01
1 D 2018-10-01
3 X 2018-10-01
3 B 2018-10-01


The SQL is easy enough, just join the table to itself:



SELECT t2.id,t2.object, t2.date
FROM BigTable T1
join BigTable T2 on t2.id = t1.id
AND t2.transaction_date = '01-OCT-2018'
AND t2.object = 'B'
where t1.date = '01-OCT-2018'


This however runs for HOURS. Subquery does not make it any better.
What is the most efficient way to return all rows for an ID that contains an object = 'B'?










share|improve this question

















  • 2





    Use t2.transaction_date = DATE '2018-10-01' and t1.date = DATE '2018-10-01' to avoid implicit conversion. Create a local Index on (date, id, transaction_date, object)

    – Kaushik Nayak
    Nov 14 '18 at 17:35











  • @KaushikNayak You should put that in an answer, because it's very likely that the implicit type conversion is his real problem.

    – Matthew McPeak
    Nov 14 '18 at 19:26











  • Please check my answer and accept/upvote it if it worked for you so that it would also help others seeking answers.Please read : stackoverflow.com/help/someone-answers

    – Kaushik Nayak
    Dec 19 '18 at 15:40











  • Unfortunately, index already existed and suggested changes did not improve the performance.

    – SRev
    Dec 20 '18 at 17:59













2












2








2








Sorry, my description sucks, but the problem is easy to explain. I have a massive table (over 2bn rows) called BigTable which is partitioned by date that looks like this:



ID Object Date
--- ------- ----------
1 A 2018-10-01
1 B 2018-10-01
1 C 2018-10-01
1 D 2018-10-01
2 M 2018-10-01
2 N 2018-10-01
2 O 2018-10-01
3 X 2018-10-01
3 B 2018-10-01


I need a SQL that returns ALL rows for the ID where an Object = "B" and the date = '2018-10-01'
Therefore the result would be:




ID Object Date
--- ------- ----------
1 A 2018-10-01
1 B 2018-10-01
1 C 2018-10-01
1 D 2018-10-01
3 X 2018-10-01
3 B 2018-10-01


The SQL is easy enough, just join the table to itself:



SELECT t2.id,t2.object, t2.date
FROM BigTable T1
join BigTable T2 on t2.id = t1.id
AND t2.transaction_date = '01-OCT-2018'
AND t2.object = 'B'
where t1.date = '01-OCT-2018'


This however runs for HOURS. Subquery does not make it any better.
What is the most efficient way to return all rows for an ID that contains an object = 'B'?










share|improve this question














Sorry, my description sucks, but the problem is easy to explain. I have a massive table (over 2bn rows) called BigTable which is partitioned by date that looks like this:



ID Object Date
--- ------- ----------
1 A 2018-10-01
1 B 2018-10-01
1 C 2018-10-01
1 D 2018-10-01
2 M 2018-10-01
2 N 2018-10-01
2 O 2018-10-01
3 X 2018-10-01
3 B 2018-10-01


I need a SQL that returns ALL rows for the ID where an Object = "B" and the date = '2018-10-01'
Therefore the result would be:




ID Object Date
--- ------- ----------
1 A 2018-10-01
1 B 2018-10-01
1 C 2018-10-01
1 D 2018-10-01
3 X 2018-10-01
3 B 2018-10-01


The SQL is easy enough, just join the table to itself:



SELECT t2.id,t2.object, t2.date
FROM BigTable T1
join BigTable T2 on t2.id = t1.id
AND t2.transaction_date = '01-OCT-2018'
AND t2.object = 'B'
where t1.date = '01-OCT-2018'


This however runs for HOURS. Subquery does not make it any better.
What is the most efficient way to return all rows for an ID that contains an object = 'B'?







sql oracle performance join






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked Nov 14 '18 at 16:45









SRevSRev

195




195







  • 2





    Use t2.transaction_date = DATE '2018-10-01' and t1.date = DATE '2018-10-01' to avoid implicit conversion. Create a local Index on (date, id, transaction_date, object)

    – Kaushik Nayak
    Nov 14 '18 at 17:35











  • @KaushikNayak You should put that in an answer, because it's very likely that the implicit type conversion is his real problem.

    – Matthew McPeak
    Nov 14 '18 at 19:26











  • Please check my answer and accept/upvote it if it worked for you so that it would also help others seeking answers.Please read : stackoverflow.com/help/someone-answers

    – Kaushik Nayak
    Dec 19 '18 at 15:40











  • Unfortunately, index already existed and suggested changes did not improve the performance.

    – SRev
    Dec 20 '18 at 17:59












  • 2





    Use t2.transaction_date = DATE '2018-10-01' and t1.date = DATE '2018-10-01' to avoid implicit conversion. Create a local Index on (date, id, transaction_date, object)

    – Kaushik Nayak
    Nov 14 '18 at 17:35











  • @KaushikNayak You should put that in an answer, because it's very likely that the implicit type conversion is his real problem.

    – Matthew McPeak
    Nov 14 '18 at 19:26











  • Please check my answer and accept/upvote it if it worked for you so that it would also help others seeking answers.Please read : stackoverflow.com/help/someone-answers

    – Kaushik Nayak
    Dec 19 '18 at 15:40











  • Unfortunately, index already existed and suggested changes did not improve the performance.

    – SRev
    Dec 20 '18 at 17:59







2




2





Use t2.transaction_date = DATE '2018-10-01' and t1.date = DATE '2018-10-01' to avoid implicit conversion. Create a local Index on (date, id, transaction_date, object)

– Kaushik Nayak
Nov 14 '18 at 17:35





Use t2.transaction_date = DATE '2018-10-01' and t1.date = DATE '2018-10-01' to avoid implicit conversion. Create a local Index on (date, id, transaction_date, object)

– Kaushik Nayak
Nov 14 '18 at 17:35













@KaushikNayak You should put that in an answer, because it's very likely that the implicit type conversion is his real problem.

– Matthew McPeak
Nov 14 '18 at 19:26





@KaushikNayak You should put that in an answer, because it's very likely that the implicit type conversion is his real problem.

– Matthew McPeak
Nov 14 '18 at 19:26













Please check my answer and accept/upvote it if it worked for you so that it would also help others seeking answers.Please read : stackoverflow.com/help/someone-answers

– Kaushik Nayak
Dec 19 '18 at 15:40





Please check my answer and accept/upvote it if it worked for you so that it would also help others seeking answers.Please read : stackoverflow.com/help/someone-answers

– Kaushik Nayak
Dec 19 '18 at 15:40













Unfortunately, index already existed and suggested changes did not improve the performance.

– SRev
Dec 20 '18 at 17:59





Unfortunately, index already existed and suggested changes did not improve the performance.

– SRev
Dec 20 '18 at 17:59












3 Answers
3






active

oldest

votes


















0














I assume you have good indexing in place... No guarantees this will help, but have you tried a semi-join:



select
t1.*
from BigTable T1
where
t1.transaction_date = '01-OCT-2018' and
exists (
select null
from BigTable T2
where
t2.transaction_date = t1.transaction_date and
t2.transaction_date = '01-OCT-2018'
t2.object = 'B'
)


The advantage of the semi-join is that once it finds a match it should "stop looking."



The join on transaction date may seem superfluous, but try it both ways.






share|improve this answer






























    0














    You can try window functions:



    SELECT t.id, t.object, t.date
    FROM (SELECT t.*,
    SUM(CASE WHEN t.object = 'B' THEN 1 ELSE 0 END) OVER (PARTITION BY t.id, t.transaction_date) as cnt
    FROM BigTable t
    WHERE t.transaction_date = DATE '2018-10-01'
    ) t
    WHERE cnt > 0;


    One possibility is that you misunderstand the partitioning, and the entire contents of the table are being read in your query.






    share|improve this answer






























      0














      You should avoid using date strings directly while comparing it with dates. The reason being implicit conversion may not always use the available partitions on date. The documentation for Data type conversion states that




      Algorithms for implicit conversion are subject to change across
      software releases and among Oracle products. Behavior of explicit
      conversions is more predictable.



      If implicit data type conversion occurs in an index expression, then
      Oracle Database might not use the index because it is defined for the
      pre-conversion data type. This can have a negative impact on
      performance.




      Thus, it is preferable to use the ANSI standard date literals



      SELECT t2.id,t2.object, t2.date
      FROM BigTable T1
      join BigTable T2 on t2.id = t1.id
      AND t2.transaction_date = DATE '2018-10-01'
      AND t2.object = 'B'
      where t1.date = DATE '2018-10-01'


      Also, as already suggested in comments, create a LOCAL INDEX on (date, id, transaction_date, object) to further improve the speed of the scan using the index along with partition pruning.






      share|improve this answer






















        Your Answer






        StackExchange.ifUsing("editor", function ()
        StackExchange.using("externalEditor", function ()
        StackExchange.using("snippets", function ()
        StackExchange.snippets.init();
        );
        );
        , "code-snippets");

        StackExchange.ready(function()
        var channelOptions =
        tags: "".split(" "),
        id: "1"
        ;
        initTagRenderer("".split(" "), "".split(" "), channelOptions);

        StackExchange.using("externalEditor", function()
        // Have to fire editor after snippets, if snippets enabled
        if (StackExchange.settings.snippets.snippetsEnabled)
        StackExchange.using("snippets", function()
        createEditor();
        );

        else
        createEditor();

        );

        function createEditor()
        StackExchange.prepareEditor(
        heartbeatType: 'answer',
        autoActivateHeartbeat: false,
        convertImagesToLinks: true,
        noModals: true,
        showLowRepImageUploadWarning: true,
        reputationToPostImages: 10,
        bindNavPrevention: true,
        postfix: "",
        imageUploader:
        brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
        contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
        allowUrls: true
        ,
        onDemand: true,
        discardSelector: ".discard-answer"
        ,immediatelyShowMarkdownHelp:true
        );



        );













        draft saved

        draft discarded


















        StackExchange.ready(
        function ()
        StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53305046%2foracle-return-all-rows-from-a-table-if%23new-answer', 'question_page');

        );

        Post as a guest















        Required, but never shown

























        3 Answers
        3






        active

        oldest

        votes








        3 Answers
        3






        active

        oldest

        votes









        active

        oldest

        votes






        active

        oldest

        votes









        0














        I assume you have good indexing in place... No guarantees this will help, but have you tried a semi-join:



        select
        t1.*
        from BigTable T1
        where
        t1.transaction_date = '01-OCT-2018' and
        exists (
        select null
        from BigTable T2
        where
        t2.transaction_date = t1.transaction_date and
        t2.transaction_date = '01-OCT-2018'
        t2.object = 'B'
        )


        The advantage of the semi-join is that once it finds a match it should "stop looking."



        The join on transaction date may seem superfluous, but try it both ways.






        share|improve this answer



























          0














          I assume you have good indexing in place... No guarantees this will help, but have you tried a semi-join:



          select
          t1.*
          from BigTable T1
          where
          t1.transaction_date = '01-OCT-2018' and
          exists (
          select null
          from BigTable T2
          where
          t2.transaction_date = t1.transaction_date and
          t2.transaction_date = '01-OCT-2018'
          t2.object = 'B'
          )


          The advantage of the semi-join is that once it finds a match it should "stop looking."



          The join on transaction date may seem superfluous, but try it both ways.






          share|improve this answer

























            0












            0








            0







            I assume you have good indexing in place... No guarantees this will help, but have you tried a semi-join:



            select
            t1.*
            from BigTable T1
            where
            t1.transaction_date = '01-OCT-2018' and
            exists (
            select null
            from BigTable T2
            where
            t2.transaction_date = t1.transaction_date and
            t2.transaction_date = '01-OCT-2018'
            t2.object = 'B'
            )


            The advantage of the semi-join is that once it finds a match it should "stop looking."



            The join on transaction date may seem superfluous, but try it both ways.






            share|improve this answer













            I assume you have good indexing in place... No guarantees this will help, but have you tried a semi-join:



            select
            t1.*
            from BigTable T1
            where
            t1.transaction_date = '01-OCT-2018' and
            exists (
            select null
            from BigTable T2
            where
            t2.transaction_date = t1.transaction_date and
            t2.transaction_date = '01-OCT-2018'
            t2.object = 'B'
            )


            The advantage of the semi-join is that once it finds a match it should "stop looking."



            The join on transaction date may seem superfluous, but try it both ways.







            share|improve this answer












            share|improve this answer



            share|improve this answer










            answered Nov 14 '18 at 16:51









            HamboneHambone

            10.3k52849




            10.3k52849























                0














                You can try window functions:



                SELECT t.id, t.object, t.date
                FROM (SELECT t.*,
                SUM(CASE WHEN t.object = 'B' THEN 1 ELSE 0 END) OVER (PARTITION BY t.id, t.transaction_date) as cnt
                FROM BigTable t
                WHERE t.transaction_date = DATE '2018-10-01'
                ) t
                WHERE cnt > 0;


                One possibility is that you misunderstand the partitioning, and the entire contents of the table are being read in your query.






                share|improve this answer



























                  0














                  You can try window functions:



                  SELECT t.id, t.object, t.date
                  FROM (SELECT t.*,
                  SUM(CASE WHEN t.object = 'B' THEN 1 ELSE 0 END) OVER (PARTITION BY t.id, t.transaction_date) as cnt
                  FROM BigTable t
                  WHERE t.transaction_date = DATE '2018-10-01'
                  ) t
                  WHERE cnt > 0;


                  One possibility is that you misunderstand the partitioning, and the entire contents of the table are being read in your query.






                  share|improve this answer

























                    0












                    0








                    0







                    You can try window functions:



                    SELECT t.id, t.object, t.date
                    FROM (SELECT t.*,
                    SUM(CASE WHEN t.object = 'B' THEN 1 ELSE 0 END) OVER (PARTITION BY t.id, t.transaction_date) as cnt
                    FROM BigTable t
                    WHERE t.transaction_date = DATE '2018-10-01'
                    ) t
                    WHERE cnt > 0;


                    One possibility is that you misunderstand the partitioning, and the entire contents of the table are being read in your query.






                    share|improve this answer













                    You can try window functions:



                    SELECT t.id, t.object, t.date
                    FROM (SELECT t.*,
                    SUM(CASE WHEN t.object = 'B' THEN 1 ELSE 0 END) OVER (PARTITION BY t.id, t.transaction_date) as cnt
                    FROM BigTable t
                    WHERE t.transaction_date = DATE '2018-10-01'
                    ) t
                    WHERE cnt > 0;


                    One possibility is that you misunderstand the partitioning, and the entire contents of the table are being read in your query.







                    share|improve this answer












                    share|improve this answer



                    share|improve this answer










                    answered Nov 14 '18 at 20:19









                    Gordon LinoffGordon Linoff

                    786k35311416




                    786k35311416





















                        0














                        You should avoid using date strings directly while comparing it with dates. The reason being implicit conversion may not always use the available partitions on date. The documentation for Data type conversion states that




                        Algorithms for implicit conversion are subject to change across
                        software releases and among Oracle products. Behavior of explicit
                        conversions is more predictable.



                        If implicit data type conversion occurs in an index expression, then
                        Oracle Database might not use the index because it is defined for the
                        pre-conversion data type. This can have a negative impact on
                        performance.




                        Thus, it is preferable to use the ANSI standard date literals



                        SELECT t2.id,t2.object, t2.date
                        FROM BigTable T1
                        join BigTable T2 on t2.id = t1.id
                        AND t2.transaction_date = DATE '2018-10-01'
                        AND t2.object = 'B'
                        where t1.date = DATE '2018-10-01'


                        Also, as already suggested in comments, create a LOCAL INDEX on (date, id, transaction_date, object) to further improve the speed of the scan using the index along with partition pruning.






                        share|improve this answer



























                          0














                          You should avoid using date strings directly while comparing it with dates. The reason being implicit conversion may not always use the available partitions on date. The documentation for Data type conversion states that




                          Algorithms for implicit conversion are subject to change across
                          software releases and among Oracle products. Behavior of explicit
                          conversions is more predictable.



                          If implicit data type conversion occurs in an index expression, then
                          Oracle Database might not use the index because it is defined for the
                          pre-conversion data type. This can have a negative impact on
                          performance.




                          Thus, it is preferable to use the ANSI standard date literals



                          SELECT t2.id,t2.object, t2.date
                          FROM BigTable T1
                          join BigTable T2 on t2.id = t1.id
                          AND t2.transaction_date = DATE '2018-10-01'
                          AND t2.object = 'B'
                          where t1.date = DATE '2018-10-01'


                          Also, as already suggested in comments, create a LOCAL INDEX on (date, id, transaction_date, object) to further improve the speed of the scan using the index along with partition pruning.






                          share|improve this answer

























                            0












                            0








                            0







                            You should avoid using date strings directly while comparing it with dates. The reason being implicit conversion may not always use the available partitions on date. The documentation for Data type conversion states that




                            Algorithms for implicit conversion are subject to change across
                            software releases and among Oracle products. Behavior of explicit
                            conversions is more predictable.



                            If implicit data type conversion occurs in an index expression, then
                            Oracle Database might not use the index because it is defined for the
                            pre-conversion data type. This can have a negative impact on
                            performance.




                            Thus, it is preferable to use the ANSI standard date literals



                            SELECT t2.id,t2.object, t2.date
                            FROM BigTable T1
                            join BigTable T2 on t2.id = t1.id
                            AND t2.transaction_date = DATE '2018-10-01'
                            AND t2.object = 'B'
                            where t1.date = DATE '2018-10-01'


                            Also, as already suggested in comments, create a LOCAL INDEX on (date, id, transaction_date, object) to further improve the speed of the scan using the index along with partition pruning.






                            share|improve this answer













                            You should avoid using date strings directly while comparing it with dates. The reason being implicit conversion may not always use the available partitions on date. The documentation for Data type conversion states that




                            Algorithms for implicit conversion are subject to change across
                            software releases and among Oracle products. Behavior of explicit
                            conversions is more predictable.



                            If implicit data type conversion occurs in an index expression, then
                            Oracle Database might not use the index because it is defined for the
                            pre-conversion data type. This can have a negative impact on
                            performance.




                            Thus, it is preferable to use the ANSI standard date literals



                            SELECT t2.id,t2.object, t2.date
                            FROM BigTable T1
                            join BigTable T2 on t2.id = t1.id
                            AND t2.transaction_date = DATE '2018-10-01'
                            AND t2.object = 'B'
                            where t1.date = DATE '2018-10-01'


                            Also, as already suggested in comments, create a LOCAL INDEX on (date, id, transaction_date, object) to further improve the speed of the scan using the index along with partition pruning.







                            share|improve this answer












                            share|improve this answer



                            share|improve this answer










                            answered Nov 15 '18 at 11:43









                            Kaushik NayakKaushik Nayak

                            20.4k41332




                            20.4k41332



























                                draft saved

                                draft discarded
















































                                Thanks for contributing an answer to Stack Overflow!


                                • Please be sure to answer the question. Provide details and share your research!

                                But avoid


                                • Asking for help, clarification, or responding to other answers.

                                • Making statements based on opinion; back them up with references or personal experience.

                                To learn more, see our tips on writing great answers.




                                draft saved


                                draft discarded














                                StackExchange.ready(
                                function ()
                                StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53305046%2foracle-return-all-rows-from-a-table-if%23new-answer', 'question_page');

                                );

                                Post as a guest















                                Required, but never shown





















































                                Required, but never shown














                                Required, but never shown












                                Required, but never shown







                                Required, but never shown

































                                Required, but never shown














                                Required, but never shown












                                Required, but never shown







                                Required, but never shown







                                Popular posts from this blog

                                Use pre created SQLite database for Android project in kotlin

                                Darth Vader #20

                                Ondo