Numpy Aggregate Rows and Sum









up vote
1
down vote

favorite
2












I have a Numpy matrix:



M = [[55, 5],
[56, 3],
[57, 7],
[58, 9],
[59, 3],
[60, 8],
[61, 1]]


I want to aggregate by group_size (for example into 3 groups):



group_size = math.ceil(M.size/groups) # math.ceil(7/3) = 3


Each aggregated row has a left value being the first left value from the group, and the right value being the sum of all right values from the group.



Expected output:



R = [[55, 15], # 55 first left column value of first group, 15 sum of all right values in group 
[58, 20], # 58 first left column value of second group, 20 sum of all right values in group
[61, 1]] # Third group consist only of one row, remainder


Is there an efficient way to solve this with Numpy without looping?










share|improve this question

























    up vote
    1
    down vote

    favorite
    2












    I have a Numpy matrix:



    M = [[55, 5],
    [56, 3],
    [57, 7],
    [58, 9],
    [59, 3],
    [60, 8],
    [61, 1]]


    I want to aggregate by group_size (for example into 3 groups):



    group_size = math.ceil(M.size/groups) # math.ceil(7/3) = 3


    Each aggregated row has a left value being the first left value from the group, and the right value being the sum of all right values from the group.



    Expected output:



    R = [[55, 15], # 55 first left column value of first group, 15 sum of all right values in group 
    [58, 20], # 58 first left column value of second group, 20 sum of all right values in group
    [61, 1]] # Third group consist only of one row, remainder


    Is there an efficient way to solve this with Numpy without looping?










    share|improve this question























      up vote
      1
      down vote

      favorite
      2









      up vote
      1
      down vote

      favorite
      2






      2





      I have a Numpy matrix:



      M = [[55, 5],
      [56, 3],
      [57, 7],
      [58, 9],
      [59, 3],
      [60, 8],
      [61, 1]]


      I want to aggregate by group_size (for example into 3 groups):



      group_size = math.ceil(M.size/groups) # math.ceil(7/3) = 3


      Each aggregated row has a left value being the first left value from the group, and the right value being the sum of all right values from the group.



      Expected output:



      R = [[55, 15], # 55 first left column value of first group, 15 sum of all right values in group 
      [58, 20], # 58 first left column value of second group, 20 sum of all right values in group
      [61, 1]] # Third group consist only of one row, remainder


      Is there an efficient way to solve this with Numpy without looping?










      share|improve this question













      I have a Numpy matrix:



      M = [[55, 5],
      [56, 3],
      [57, 7],
      [58, 9],
      [59, 3],
      [60, 8],
      [61, 1]]


      I want to aggregate by group_size (for example into 3 groups):



      group_size = math.ceil(M.size/groups) # math.ceil(7/3) = 3


      Each aggregated row has a left value being the first left value from the group, and the right value being the sum of all right values from the group.



      Expected output:



      R = [[55, 15], # 55 first left column value of first group, 15 sum of all right values in group 
      [58, 20], # 58 first left column value of second group, 20 sum of all right values in group
      [61, 1]] # Third group consist only of one row, remainder


      Is there an efficient way to solve this with Numpy without looping?







      python pandas numpy






      share|improve this question













      share|improve this question











      share|improve this question




      share|improve this question










      asked Nov 10 at 13:03









      Franc Weser

      16317




      16317






















          4 Answers
          4






          active

          oldest

          votes

















          up vote
          3
          down vote



          accepted










          Here's one way with NumPy:



          n = 3
          x = M[::n, 0]
          y = np.add.reduceat(M[:, 1], np.arange(0, M.shape[0], n))

          R = np.vstack((x, y)).T

          print(R)

          array([[55, 15],
          [58, 20],
          [61, 1]])





          share|improve this answer



























            up vote
            3
            down vote













            pandas solution should be use agg with first and sum:



            group_size = 3
            df = pd.DataFrame(M).groupby(np.arange(len(M)) // group_size).agg(0:'first',1:'sum')
            print (df)
            0 1
            0 55 15
            1 58 20
            2 61 1

            a = np.array(df.values.tolist())
            print(a)
            [[55 15]
            [58 20]
            [61 1]]





            share|improve this answer



























              up vote
              1
              down vote













              A solution using Python:



              from operator import itemgetter

              M = [[55, 5],
              [56, 3],
              [57, 7],
              [58, 9],
              [59, 3],
              [60, 8],
              [61, 1]]
              it = (M[e:e+3] for e in range(0, len(M), 3))
              print([[e[0][0], sum(map(itemgetter(1), e))] for e in it])


              Output



              [[55, 15], [58, 20], [61, 1]]





              share|improve this answer



























                up vote
                0
                down vote













                a = np.array([[2, 3],[5, 6],[7, 9]])
                b = numpy.zeros(shape=(len(a[0])))
                for i in a:
                b=b+i
                print(b)





                share|improve this answer
















                • 2




                  An explanation, what a code does and how this addresses the problem in the question, rarely fails to improve an answer.
                  – blue-phoenox
                  Nov 10 at 16:57










                Your Answer






                StackExchange.ifUsing("editor", function ()
                StackExchange.using("externalEditor", function ()
                StackExchange.using("snippets", function ()
                StackExchange.snippets.init();
                );
                );
                , "code-snippets");

                StackExchange.ready(function()
                var channelOptions =
                tags: "".split(" "),
                id: "1"
                ;
                initTagRenderer("".split(" "), "".split(" "), channelOptions);

                StackExchange.using("externalEditor", function()
                // Have to fire editor after snippets, if snippets enabled
                if (StackExchange.settings.snippets.snippetsEnabled)
                StackExchange.using("snippets", function()
                createEditor();
                );

                else
                createEditor();

                );

                function createEditor()
                StackExchange.prepareEditor(
                heartbeatType: 'answer',
                convertImagesToLinks: true,
                noModals: true,
                showLowRepImageUploadWarning: true,
                reputationToPostImages: 10,
                bindNavPrevention: true,
                postfix: "",
                imageUploader:
                brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
                contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
                allowUrls: true
                ,
                onDemand: true,
                discardSelector: ".discard-answer"
                ,immediatelyShowMarkdownHelp:true
                );



                );













                draft saved

                draft discarded


















                StackExchange.ready(
                function ()
                StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53239224%2fnumpy-aggregate-rows-and-sum%23new-answer', 'question_page');

                );

                Post as a guest















                Required, but never shown

























                4 Answers
                4






                active

                oldest

                votes








                4 Answers
                4






                active

                oldest

                votes









                active

                oldest

                votes






                active

                oldest

                votes








                up vote
                3
                down vote



                accepted










                Here's one way with NumPy:



                n = 3
                x = M[::n, 0]
                y = np.add.reduceat(M[:, 1], np.arange(0, M.shape[0], n))

                R = np.vstack((x, y)).T

                print(R)

                array([[55, 15],
                [58, 20],
                [61, 1]])





                share|improve this answer
























                  up vote
                  3
                  down vote



                  accepted










                  Here's one way with NumPy:



                  n = 3
                  x = M[::n, 0]
                  y = np.add.reduceat(M[:, 1], np.arange(0, M.shape[0], n))

                  R = np.vstack((x, y)).T

                  print(R)

                  array([[55, 15],
                  [58, 20],
                  [61, 1]])





                  share|improve this answer






















                    up vote
                    3
                    down vote



                    accepted







                    up vote
                    3
                    down vote



                    accepted






                    Here's one way with NumPy:



                    n = 3
                    x = M[::n, 0]
                    y = np.add.reduceat(M[:, 1], np.arange(0, M.shape[0], n))

                    R = np.vstack((x, y)).T

                    print(R)

                    array([[55, 15],
                    [58, 20],
                    [61, 1]])





                    share|improve this answer












                    Here's one way with NumPy:



                    n = 3
                    x = M[::n, 0]
                    y = np.add.reduceat(M[:, 1], np.arange(0, M.shape[0], n))

                    R = np.vstack((x, y)).T

                    print(R)

                    array([[55, 15],
                    [58, 20],
                    [61, 1]])






                    share|improve this answer












                    share|improve this answer



                    share|improve this answer










                    answered Nov 10 at 13:13









                    jpp

                    87.1k194999




                    87.1k194999






















                        up vote
                        3
                        down vote













                        pandas solution should be use agg with first and sum:



                        group_size = 3
                        df = pd.DataFrame(M).groupby(np.arange(len(M)) // group_size).agg(0:'first',1:'sum')
                        print (df)
                        0 1
                        0 55 15
                        1 58 20
                        2 61 1

                        a = np.array(df.values.tolist())
                        print(a)
                        [[55 15]
                        [58 20]
                        [61 1]]





                        share|improve this answer
























                          up vote
                          3
                          down vote













                          pandas solution should be use agg with first and sum:



                          group_size = 3
                          df = pd.DataFrame(M).groupby(np.arange(len(M)) // group_size).agg(0:'first',1:'sum')
                          print (df)
                          0 1
                          0 55 15
                          1 58 20
                          2 61 1

                          a = np.array(df.values.tolist())
                          print(a)
                          [[55 15]
                          [58 20]
                          [61 1]]





                          share|improve this answer






















                            up vote
                            3
                            down vote










                            up vote
                            3
                            down vote









                            pandas solution should be use agg with first and sum:



                            group_size = 3
                            df = pd.DataFrame(M).groupby(np.arange(len(M)) // group_size).agg(0:'first',1:'sum')
                            print (df)
                            0 1
                            0 55 15
                            1 58 20
                            2 61 1

                            a = np.array(df.values.tolist())
                            print(a)
                            [[55 15]
                            [58 20]
                            [61 1]]





                            share|improve this answer












                            pandas solution should be use agg with first and sum:



                            group_size = 3
                            df = pd.DataFrame(M).groupby(np.arange(len(M)) // group_size).agg(0:'first',1:'sum')
                            print (df)
                            0 1
                            0 55 15
                            1 58 20
                            2 61 1

                            a = np.array(df.values.tolist())
                            print(a)
                            [[55 15]
                            [58 20]
                            [61 1]]






                            share|improve this answer












                            share|improve this answer



                            share|improve this answer










                            answered Nov 10 at 13:10









                            jezrael

                            314k21250328




                            314k21250328




















                                up vote
                                1
                                down vote













                                A solution using Python:



                                from operator import itemgetter

                                M = [[55, 5],
                                [56, 3],
                                [57, 7],
                                [58, 9],
                                [59, 3],
                                [60, 8],
                                [61, 1]]
                                it = (M[e:e+3] for e in range(0, len(M), 3))
                                print([[e[0][0], sum(map(itemgetter(1), e))] for e in it])


                                Output



                                [[55, 15], [58, 20], [61, 1]]





                                share|improve this answer
























                                  up vote
                                  1
                                  down vote













                                  A solution using Python:



                                  from operator import itemgetter

                                  M = [[55, 5],
                                  [56, 3],
                                  [57, 7],
                                  [58, 9],
                                  [59, 3],
                                  [60, 8],
                                  [61, 1]]
                                  it = (M[e:e+3] for e in range(0, len(M), 3))
                                  print([[e[0][0], sum(map(itemgetter(1), e))] for e in it])


                                  Output



                                  [[55, 15], [58, 20], [61, 1]]





                                  share|improve this answer






















                                    up vote
                                    1
                                    down vote










                                    up vote
                                    1
                                    down vote









                                    A solution using Python:



                                    from operator import itemgetter

                                    M = [[55, 5],
                                    [56, 3],
                                    [57, 7],
                                    [58, 9],
                                    [59, 3],
                                    [60, 8],
                                    [61, 1]]
                                    it = (M[e:e+3] for e in range(0, len(M), 3))
                                    print([[e[0][0], sum(map(itemgetter(1), e))] for e in it])


                                    Output



                                    [[55, 15], [58, 20], [61, 1]]





                                    share|improve this answer












                                    A solution using Python:



                                    from operator import itemgetter

                                    M = [[55, 5],
                                    [56, 3],
                                    [57, 7],
                                    [58, 9],
                                    [59, 3],
                                    [60, 8],
                                    [61, 1]]
                                    it = (M[e:e+3] for e in range(0, len(M), 3))
                                    print([[e[0][0], sum(map(itemgetter(1), e))] for e in it])


                                    Output



                                    [[55, 15], [58, 20], [61, 1]]






                                    share|improve this answer












                                    share|improve this answer



                                    share|improve this answer










                                    answered Nov 10 at 13:22









                                    Daniel Mesejo

                                    10.2k1923




                                    10.2k1923




















                                        up vote
                                        0
                                        down vote













                                        a = np.array([[2, 3],[5, 6],[7, 9]])
                                        b = numpy.zeros(shape=(len(a[0])))
                                        for i in a:
                                        b=b+i
                                        print(b)





                                        share|improve this answer
















                                        • 2




                                          An explanation, what a code does and how this addresses the problem in the question, rarely fails to improve an answer.
                                          – blue-phoenox
                                          Nov 10 at 16:57














                                        up vote
                                        0
                                        down vote













                                        a = np.array([[2, 3],[5, 6],[7, 9]])
                                        b = numpy.zeros(shape=(len(a[0])))
                                        for i in a:
                                        b=b+i
                                        print(b)





                                        share|improve this answer
















                                        • 2




                                          An explanation, what a code does and how this addresses the problem in the question, rarely fails to improve an answer.
                                          – blue-phoenox
                                          Nov 10 at 16:57












                                        up vote
                                        0
                                        down vote










                                        up vote
                                        0
                                        down vote









                                        a = np.array([[2, 3],[5, 6],[7, 9]])
                                        b = numpy.zeros(shape=(len(a[0])))
                                        for i in a:
                                        b=b+i
                                        print(b)





                                        share|improve this answer












                                        a = np.array([[2, 3],[5, 6],[7, 9]])
                                        b = numpy.zeros(shape=(len(a[0])))
                                        for i in a:
                                        b=b+i
                                        print(b)






                                        share|improve this answer












                                        share|improve this answer



                                        share|improve this answer










                                        answered Nov 10 at 13:42









                                        Mohammad reza Kashi

                                        194




                                        194







                                        • 2




                                          An explanation, what a code does and how this addresses the problem in the question, rarely fails to improve an answer.
                                          – blue-phoenox
                                          Nov 10 at 16:57












                                        • 2




                                          An explanation, what a code does and how this addresses the problem in the question, rarely fails to improve an answer.
                                          – blue-phoenox
                                          Nov 10 at 16:57







                                        2




                                        2




                                        An explanation, what a code does and how this addresses the problem in the question, rarely fails to improve an answer.
                                        – blue-phoenox
                                        Nov 10 at 16:57




                                        An explanation, what a code does and how this addresses the problem in the question, rarely fails to improve an answer.
                                        – blue-phoenox
                                        Nov 10 at 16:57

















                                        draft saved

                                        draft discarded
















































                                        Thanks for contributing an answer to Stack Overflow!


                                        • Please be sure to answer the question. Provide details and share your research!

                                        But avoid


                                        • Asking for help, clarification, or responding to other answers.

                                        • Making statements based on opinion; back them up with references or personal experience.

                                        To learn more, see our tips on writing great answers.





                                        Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


                                        Please pay close attention to the following guidance:


                                        • Please be sure to answer the question. Provide details and share your research!

                                        But avoid


                                        • Asking for help, clarification, or responding to other answers.

                                        • Making statements based on opinion; back them up with references or personal experience.

                                        To learn more, see our tips on writing great answers.




                                        draft saved


                                        draft discarded














                                        StackExchange.ready(
                                        function ()
                                        StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53239224%2fnumpy-aggregate-rows-and-sum%23new-answer', 'question_page');

                                        );

                                        Post as a guest















                                        Required, but never shown





















































                                        Required, but never shown














                                        Required, but never shown












                                        Required, but never shown







                                        Required, but never shown

































                                        Required, but never shown














                                        Required, but never shown












                                        Required, but never shown







                                        Required, but never shown







                                        Popular posts from this blog

                                        How to how show current date and time by default on contact form 7 in WordPress without taking input from user in datetimepicker

                                        Syphilis

                                        Darth Vader #20