recycle vector values with data.table shift instead of padding with fill=NA










0














I would like to use the shift function from data.table to lead/lag a new column, but I would like to recycle values from the lagged vector that was added to the data.table. From what I can see, fill must be a vector of length 1, and so the values that are lagged must be populated with a constant value (ie NA here).



Please see the MWE below.



dt1 is the resulting data.table using the shift function as is. The new b column has NA values where 4, 5, and 6 should be.



dt2 is the desired data.table result. If my thinking is correct, the output requires R recycling rules but with a lead/lag value specified where the vector should begin.



I could have added a new vector (see below in x_to_avoid) but that requires more manual work that I hope to avoid.



Thanks,



library(data.table)
library(magrittr)

# vector to lead/lag when updating datatable
x = c(1:6)

# leaves NA where 4, 5, 6 "should" have gone for my purposes
dt1 <- data.table(a = c(1:10)) %>%
.[, b := shift(x,
n = 3L,
fill = NA,
type = c("lag"))]
dt1

# desired output
dt2 <- dt1[, .(a)] %>%
.[, b := c(4,5,6,1,2,3,4,5,6,1)]

# could use another vector, but my actual use is more complicated and I prefer to avoid this (if possible)
x_to_avoid = c(4,5,6,1:6,1)









share|improve this question

















  • 1




    Perhaps you could define your own function, something like shift2 <- function(x, n, type = "lag") if(type == "lag") c(tail(x, -(length(x) - n)), head(x, -n)) else c(tail(x, -n), head(x, -(length(x) - n))). Then you could do dt1 <- data.table(a = 1:10, b = shift2(x, n = 3L)). Though n here isn't vectorized like in data.table::shift
    – David Arenburg
    Nov 12 '18 at 14:41










  • @DavidArenburg Thanks for that. Was not familiar with head and tail function. Also didn't know about defining a function without { parenthesis! Speed is not a big issue here as it is a one-off and I will likely create a CSV file to load from (and so others can use it). This is awesome, thanks! If you post it as an answer I will mark it correct (if you want)...
    – Prevost
    Dec 13 '18 at 21:35











  • I wouldn't make some hack as an official data.table answer. I think you could perhaps add a feature request on the GH issues page instead.
    – David Arenburg
    Dec 16 '18 at 11:42
















0














I would like to use the shift function from data.table to lead/lag a new column, but I would like to recycle values from the lagged vector that was added to the data.table. From what I can see, fill must be a vector of length 1, and so the values that are lagged must be populated with a constant value (ie NA here).



Please see the MWE below.



dt1 is the resulting data.table using the shift function as is. The new b column has NA values where 4, 5, and 6 should be.



dt2 is the desired data.table result. If my thinking is correct, the output requires R recycling rules but with a lead/lag value specified where the vector should begin.



I could have added a new vector (see below in x_to_avoid) but that requires more manual work that I hope to avoid.



Thanks,



library(data.table)
library(magrittr)

# vector to lead/lag when updating datatable
x = c(1:6)

# leaves NA where 4, 5, 6 "should" have gone for my purposes
dt1 <- data.table(a = c(1:10)) %>%
.[, b := shift(x,
n = 3L,
fill = NA,
type = c("lag"))]
dt1

# desired output
dt2 <- dt1[, .(a)] %>%
.[, b := c(4,5,6,1,2,3,4,5,6,1)]

# could use another vector, but my actual use is more complicated and I prefer to avoid this (if possible)
x_to_avoid = c(4,5,6,1:6,1)









share|improve this question

















  • 1




    Perhaps you could define your own function, something like shift2 <- function(x, n, type = "lag") if(type == "lag") c(tail(x, -(length(x) - n)), head(x, -n)) else c(tail(x, -n), head(x, -(length(x) - n))). Then you could do dt1 <- data.table(a = 1:10, b = shift2(x, n = 3L)). Though n here isn't vectorized like in data.table::shift
    – David Arenburg
    Nov 12 '18 at 14:41










  • @DavidArenburg Thanks for that. Was not familiar with head and tail function. Also didn't know about defining a function without { parenthesis! Speed is not a big issue here as it is a one-off and I will likely create a CSV file to load from (and so others can use it). This is awesome, thanks! If you post it as an answer I will mark it correct (if you want)...
    – Prevost
    Dec 13 '18 at 21:35











  • I wouldn't make some hack as an official data.table answer. I think you could perhaps add a feature request on the GH issues page instead.
    – David Arenburg
    Dec 16 '18 at 11:42














0












0








0







I would like to use the shift function from data.table to lead/lag a new column, but I would like to recycle values from the lagged vector that was added to the data.table. From what I can see, fill must be a vector of length 1, and so the values that are lagged must be populated with a constant value (ie NA here).



Please see the MWE below.



dt1 is the resulting data.table using the shift function as is. The new b column has NA values where 4, 5, and 6 should be.



dt2 is the desired data.table result. If my thinking is correct, the output requires R recycling rules but with a lead/lag value specified where the vector should begin.



I could have added a new vector (see below in x_to_avoid) but that requires more manual work that I hope to avoid.



Thanks,



library(data.table)
library(magrittr)

# vector to lead/lag when updating datatable
x = c(1:6)

# leaves NA where 4, 5, 6 "should" have gone for my purposes
dt1 <- data.table(a = c(1:10)) %>%
.[, b := shift(x,
n = 3L,
fill = NA,
type = c("lag"))]
dt1

# desired output
dt2 <- dt1[, .(a)] %>%
.[, b := c(4,5,6,1,2,3,4,5,6,1)]

# could use another vector, but my actual use is more complicated and I prefer to avoid this (if possible)
x_to_avoid = c(4,5,6,1:6,1)









share|improve this question













I would like to use the shift function from data.table to lead/lag a new column, but I would like to recycle values from the lagged vector that was added to the data.table. From what I can see, fill must be a vector of length 1, and so the values that are lagged must be populated with a constant value (ie NA here).



Please see the MWE below.



dt1 is the resulting data.table using the shift function as is. The new b column has NA values where 4, 5, and 6 should be.



dt2 is the desired data.table result. If my thinking is correct, the output requires R recycling rules but with a lead/lag value specified where the vector should begin.



I could have added a new vector (see below in x_to_avoid) but that requires more manual work that I hope to avoid.



Thanks,



library(data.table)
library(magrittr)

# vector to lead/lag when updating datatable
x = c(1:6)

# leaves NA where 4, 5, 6 "should" have gone for my purposes
dt1 <- data.table(a = c(1:10)) %>%
.[, b := shift(x,
n = 3L,
fill = NA,
type = c("lag"))]
dt1

# desired output
dt2 <- dt1[, .(a)] %>%
.[, b := c(4,5,6,1,2,3,4,5,6,1)]

# could use another vector, but my actual use is more complicated and I prefer to avoid this (if possible)
x_to_avoid = c(4,5,6,1:6,1)






r data.table






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked Nov 12 '18 at 2:20









Prevost

15710




15710







  • 1




    Perhaps you could define your own function, something like shift2 <- function(x, n, type = "lag") if(type == "lag") c(tail(x, -(length(x) - n)), head(x, -n)) else c(tail(x, -n), head(x, -(length(x) - n))). Then you could do dt1 <- data.table(a = 1:10, b = shift2(x, n = 3L)). Though n here isn't vectorized like in data.table::shift
    – David Arenburg
    Nov 12 '18 at 14:41










  • @DavidArenburg Thanks for that. Was not familiar with head and tail function. Also didn't know about defining a function without { parenthesis! Speed is not a big issue here as it is a one-off and I will likely create a CSV file to load from (and so others can use it). This is awesome, thanks! If you post it as an answer I will mark it correct (if you want)...
    – Prevost
    Dec 13 '18 at 21:35











  • I wouldn't make some hack as an official data.table answer. I think you could perhaps add a feature request on the GH issues page instead.
    – David Arenburg
    Dec 16 '18 at 11:42













  • 1




    Perhaps you could define your own function, something like shift2 <- function(x, n, type = "lag") if(type == "lag") c(tail(x, -(length(x) - n)), head(x, -n)) else c(tail(x, -n), head(x, -(length(x) - n))). Then you could do dt1 <- data.table(a = 1:10, b = shift2(x, n = 3L)). Though n here isn't vectorized like in data.table::shift
    – David Arenburg
    Nov 12 '18 at 14:41










  • @DavidArenburg Thanks for that. Was not familiar with head and tail function. Also didn't know about defining a function without { parenthesis! Speed is not a big issue here as it is a one-off and I will likely create a CSV file to load from (and so others can use it). This is awesome, thanks! If you post it as an answer I will mark it correct (if you want)...
    – Prevost
    Dec 13 '18 at 21:35











  • I wouldn't make some hack as an official data.table answer. I think you could perhaps add a feature request on the GH issues page instead.
    – David Arenburg
    Dec 16 '18 at 11:42








1




1




Perhaps you could define your own function, something like shift2 <- function(x, n, type = "lag") if(type == "lag") c(tail(x, -(length(x) - n)), head(x, -n)) else c(tail(x, -n), head(x, -(length(x) - n))). Then you could do dt1 <- data.table(a = 1:10, b = shift2(x, n = 3L)). Though n here isn't vectorized like in data.table::shift
– David Arenburg
Nov 12 '18 at 14:41




Perhaps you could define your own function, something like shift2 <- function(x, n, type = "lag") if(type == "lag") c(tail(x, -(length(x) - n)), head(x, -n)) else c(tail(x, -n), head(x, -(length(x) - n))). Then you could do dt1 <- data.table(a = 1:10, b = shift2(x, n = 3L)). Though n here isn't vectorized like in data.table::shift
– David Arenburg
Nov 12 '18 at 14:41












@DavidArenburg Thanks for that. Was not familiar with head and tail function. Also didn't know about defining a function without { parenthesis! Speed is not a big issue here as it is a one-off and I will likely create a CSV file to load from (and so others can use it). This is awesome, thanks! If you post it as an answer I will mark it correct (if you want)...
– Prevost
Dec 13 '18 at 21:35





@DavidArenburg Thanks for that. Was not familiar with head and tail function. Also didn't know about defining a function without { parenthesis! Speed is not a big issue here as it is a one-off and I will likely create a CSV file to load from (and so others can use it). This is awesome, thanks! If you post it as an answer I will mark it correct (if you want)...
– Prevost
Dec 13 '18 at 21:35













I wouldn't make some hack as an official data.table answer. I think you could perhaps add a feature request on the GH issues page instead.
– David Arenburg
Dec 16 '18 at 11:42





I wouldn't make some hack as an official data.table answer. I think you could perhaps add a feature request on the GH issues page instead.
– David Arenburg
Dec 16 '18 at 11:42













1 Answer
1






active

oldest

votes


















0














I think that binhf::shift does the job. First you will need to lengthen the vector using rep.len and then you can cycle it using binhf::shift. I have no idea about the performance though.






share|improve this answer




















    Your Answer






    StackExchange.ifUsing("editor", function ()
    StackExchange.using("externalEditor", function ()
    StackExchange.using("snippets", function ()
    StackExchange.snippets.init();
    );
    );
    , "code-snippets");

    StackExchange.ready(function()
    var channelOptions =
    tags: "".split(" "),
    id: "1"
    ;
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function()
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled)
    StackExchange.using("snippets", function()
    createEditor();
    );

    else
    createEditor();

    );

    function createEditor()
    StackExchange.prepareEditor(
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader:
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    ,
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    );



    );













    draft saved

    draft discarded


















    StackExchange.ready(
    function ()
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53255242%2frecycle-vector-values-with-data-table-shift-instead-of-padding-with-fill-na%23new-answer', 'question_page');

    );

    Post as a guest















    Required, but never shown

























    1 Answer
    1






    active

    oldest

    votes








    1 Answer
    1






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    0














    I think that binhf::shift does the job. First you will need to lengthen the vector using rep.len and then you can cycle it using binhf::shift. I have no idea about the performance though.






    share|improve this answer

























      0














      I think that binhf::shift does the job. First you will need to lengthen the vector using rep.len and then you can cycle it using binhf::shift. I have no idea about the performance though.






      share|improve this answer























        0












        0








        0






        I think that binhf::shift does the job. First you will need to lengthen the vector using rep.len and then you can cycle it using binhf::shift. I have no idea about the performance though.






        share|improve this answer












        I think that binhf::shift does the job. First you will need to lengthen the vector using rep.len and then you can cycle it using binhf::shift. I have no idea about the performance though.







        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered Nov 14 '18 at 19:02









        Alp Arıbal

        324




        324



























            draft saved

            draft discarded
















































            Thanks for contributing an answer to Stack Overflow!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid


            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.

            To learn more, see our tips on writing great answers.





            Some of your past answers have not been well-received, and you're in danger of being blocked from answering.


            Please pay close attention to the following guidance:


            • Please be sure to answer the question. Provide details and share your research!

            But avoid


            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.

            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53255242%2frecycle-vector-values-with-data-table-shift-instead-of-padding-with-fill-na%23new-answer', 'question_page');

            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            Use pre created SQLite database for Android project in kotlin

            Darth Vader #20

            Ondo