recycle vector values with data.table shift instead of padding with fill=NA
I would like to use the shift
function from data.table
to lead/lag a new column, but I would like to recycle values from the lagged vector that was added to the data.table
. From what I can see, fill must be a vector of length 1
, and so the values that are lagged must be populated with a constant value (ie NA
here).
Please see the MWE below.
dt1
is the resulting data.table
using the shift
function as is. The new b
column has NA
values where 4, 5, and 6 should be.
dt2
is the desired data.table result. If my thinking is correct, the output requires R
recycling rules but with a lead/lag value specified where the vector should begin.
I could have added a new vector (see below in x_to_avoid
) but that requires more manual work that I hope to avoid.
Thanks,
library(data.table)
library(magrittr)
# vector to lead/lag when updating datatable
x = c(1:6)
# leaves NA where 4, 5, 6 "should" have gone for my purposes
dt1 <- data.table(a = c(1:10)) %>%
.[, b := shift(x,
n = 3L,
fill = NA,
type = c("lag"))]
dt1
# desired output
dt2 <- dt1[, .(a)] %>%
.[, b := c(4,5,6,1,2,3,4,5,6,1)]
# could use another vector, but my actual use is more complicated and I prefer to avoid this (if possible)
x_to_avoid = c(4,5,6,1:6,1)
r data.table
add a comment |
I would like to use the shift
function from data.table
to lead/lag a new column, but I would like to recycle values from the lagged vector that was added to the data.table
. From what I can see, fill must be a vector of length 1
, and so the values that are lagged must be populated with a constant value (ie NA
here).
Please see the MWE below.
dt1
is the resulting data.table
using the shift
function as is. The new b
column has NA
values where 4, 5, and 6 should be.
dt2
is the desired data.table result. If my thinking is correct, the output requires R
recycling rules but with a lead/lag value specified where the vector should begin.
I could have added a new vector (see below in x_to_avoid
) but that requires more manual work that I hope to avoid.
Thanks,
library(data.table)
library(magrittr)
# vector to lead/lag when updating datatable
x = c(1:6)
# leaves NA where 4, 5, 6 "should" have gone for my purposes
dt1 <- data.table(a = c(1:10)) %>%
.[, b := shift(x,
n = 3L,
fill = NA,
type = c("lag"))]
dt1
# desired output
dt2 <- dt1[, .(a)] %>%
.[, b := c(4,5,6,1,2,3,4,5,6,1)]
# could use another vector, but my actual use is more complicated and I prefer to avoid this (if possible)
x_to_avoid = c(4,5,6,1:6,1)
r data.table
1
Perhaps you could define your own function, something likeshift2 <- function(x, n, type = "lag") if(type == "lag") c(tail(x, -(length(x) - n)), head(x, -n)) else c(tail(x, -n), head(x, -(length(x) - n)))
. Then you could dodt1 <- data.table(a = 1:10, b = shift2(x, n = 3L))
. Thoughn
here isn't vectorized like indata.table::shift
– David Arenburg
Nov 12 '18 at 14:41
@DavidArenburg Thanks for that. Was not familiar withhead
andtail
function. Also didn't know about defining a function without{
parenthesis! Speed is not a big issue here as it is a one-off and I will likely create a CSV file to load from (and so others can use it). This is awesome, thanks! If you post it as an answer I will mark it correct (if you want)...
– Prevost
Dec 13 '18 at 21:35
I wouldn't make some hack as an official data.table answer. I think you could perhaps add a feature request on the GH issues page instead.
– David Arenburg
Dec 16 '18 at 11:42
add a comment |
I would like to use the shift
function from data.table
to lead/lag a new column, but I would like to recycle values from the lagged vector that was added to the data.table
. From what I can see, fill must be a vector of length 1
, and so the values that are lagged must be populated with a constant value (ie NA
here).
Please see the MWE below.
dt1
is the resulting data.table
using the shift
function as is. The new b
column has NA
values where 4, 5, and 6 should be.
dt2
is the desired data.table result. If my thinking is correct, the output requires R
recycling rules but with a lead/lag value specified where the vector should begin.
I could have added a new vector (see below in x_to_avoid
) but that requires more manual work that I hope to avoid.
Thanks,
library(data.table)
library(magrittr)
# vector to lead/lag when updating datatable
x = c(1:6)
# leaves NA where 4, 5, 6 "should" have gone for my purposes
dt1 <- data.table(a = c(1:10)) %>%
.[, b := shift(x,
n = 3L,
fill = NA,
type = c("lag"))]
dt1
# desired output
dt2 <- dt1[, .(a)] %>%
.[, b := c(4,5,6,1,2,3,4,5,6,1)]
# could use another vector, but my actual use is more complicated and I prefer to avoid this (if possible)
x_to_avoid = c(4,5,6,1:6,1)
r data.table
I would like to use the shift
function from data.table
to lead/lag a new column, but I would like to recycle values from the lagged vector that was added to the data.table
. From what I can see, fill must be a vector of length 1
, and so the values that are lagged must be populated with a constant value (ie NA
here).
Please see the MWE below.
dt1
is the resulting data.table
using the shift
function as is. The new b
column has NA
values where 4, 5, and 6 should be.
dt2
is the desired data.table result. If my thinking is correct, the output requires R
recycling rules but with a lead/lag value specified where the vector should begin.
I could have added a new vector (see below in x_to_avoid
) but that requires more manual work that I hope to avoid.
Thanks,
library(data.table)
library(magrittr)
# vector to lead/lag when updating datatable
x = c(1:6)
# leaves NA where 4, 5, 6 "should" have gone for my purposes
dt1 <- data.table(a = c(1:10)) %>%
.[, b := shift(x,
n = 3L,
fill = NA,
type = c("lag"))]
dt1
# desired output
dt2 <- dt1[, .(a)] %>%
.[, b := c(4,5,6,1,2,3,4,5,6,1)]
# could use another vector, but my actual use is more complicated and I prefer to avoid this (if possible)
x_to_avoid = c(4,5,6,1:6,1)
r data.table
r data.table
asked Nov 12 '18 at 2:20
Prevost
15710
15710
1
Perhaps you could define your own function, something likeshift2 <- function(x, n, type = "lag") if(type == "lag") c(tail(x, -(length(x) - n)), head(x, -n)) else c(tail(x, -n), head(x, -(length(x) - n)))
. Then you could dodt1 <- data.table(a = 1:10, b = shift2(x, n = 3L))
. Thoughn
here isn't vectorized like indata.table::shift
– David Arenburg
Nov 12 '18 at 14:41
@DavidArenburg Thanks for that. Was not familiar withhead
andtail
function. Also didn't know about defining a function without{
parenthesis! Speed is not a big issue here as it is a one-off and I will likely create a CSV file to load from (and so others can use it). This is awesome, thanks! If you post it as an answer I will mark it correct (if you want)...
– Prevost
Dec 13 '18 at 21:35
I wouldn't make some hack as an official data.table answer. I think you could perhaps add a feature request on the GH issues page instead.
– David Arenburg
Dec 16 '18 at 11:42
add a comment |
1
Perhaps you could define your own function, something likeshift2 <- function(x, n, type = "lag") if(type == "lag") c(tail(x, -(length(x) - n)), head(x, -n)) else c(tail(x, -n), head(x, -(length(x) - n)))
. Then you could dodt1 <- data.table(a = 1:10, b = shift2(x, n = 3L))
. Thoughn
here isn't vectorized like indata.table::shift
– David Arenburg
Nov 12 '18 at 14:41
@DavidArenburg Thanks for that. Was not familiar withhead
andtail
function. Also didn't know about defining a function without{
parenthesis! Speed is not a big issue here as it is a one-off and I will likely create a CSV file to load from (and so others can use it). This is awesome, thanks! If you post it as an answer I will mark it correct (if you want)...
– Prevost
Dec 13 '18 at 21:35
I wouldn't make some hack as an official data.table answer. I think you could perhaps add a feature request on the GH issues page instead.
– David Arenburg
Dec 16 '18 at 11:42
1
1
Perhaps you could define your own function, something like
shift2 <- function(x, n, type = "lag") if(type == "lag") c(tail(x, -(length(x) - n)), head(x, -n)) else c(tail(x, -n), head(x, -(length(x) - n)))
. Then you could do dt1 <- data.table(a = 1:10, b = shift2(x, n = 3L))
. Though n
here isn't vectorized like in data.table::shift
– David Arenburg
Nov 12 '18 at 14:41
Perhaps you could define your own function, something like
shift2 <- function(x, n, type = "lag") if(type == "lag") c(tail(x, -(length(x) - n)), head(x, -n)) else c(tail(x, -n), head(x, -(length(x) - n)))
. Then you could do dt1 <- data.table(a = 1:10, b = shift2(x, n = 3L))
. Though n
here isn't vectorized like in data.table::shift
– David Arenburg
Nov 12 '18 at 14:41
@DavidArenburg Thanks for that. Was not familiar with
head
and tail
function. Also didn't know about defining a function without {
parenthesis! Speed is not a big issue here as it is a one-off and I will likely create a CSV file to load from (and so others can use it). This is awesome, thanks! If you post it as an answer I will mark it correct (if you want)...– Prevost
Dec 13 '18 at 21:35
@DavidArenburg Thanks for that. Was not familiar with
head
and tail
function. Also didn't know about defining a function without {
parenthesis! Speed is not a big issue here as it is a one-off and I will likely create a CSV file to load from (and so others can use it). This is awesome, thanks! If you post it as an answer I will mark it correct (if you want)...– Prevost
Dec 13 '18 at 21:35
I wouldn't make some hack as an official data.table answer. I think you could perhaps add a feature request on the GH issues page instead.
– David Arenburg
Dec 16 '18 at 11:42
I wouldn't make some hack as an official data.table answer. I think you could perhaps add a feature request on the GH issues page instead.
– David Arenburg
Dec 16 '18 at 11:42
add a comment |
1 Answer
1
active
oldest
votes
I think that binhf::shift does the job. First you will need to lengthen the vector using rep.len
and then you can cycle it using binhf::shift
. I have no idea about the performance though.
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53255242%2frecycle-vector-values-with-data-table-shift-instead-of-padding-with-fill-na%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
I think that binhf::shift does the job. First you will need to lengthen the vector using rep.len
and then you can cycle it using binhf::shift
. I have no idea about the performance though.
add a comment |
I think that binhf::shift does the job. First you will need to lengthen the vector using rep.len
and then you can cycle it using binhf::shift
. I have no idea about the performance though.
add a comment |
I think that binhf::shift does the job. First you will need to lengthen the vector using rep.len
and then you can cycle it using binhf::shift
. I have no idea about the performance though.
I think that binhf::shift does the job. First you will need to lengthen the vector using rep.len
and then you can cycle it using binhf::shift
. I have no idea about the performance though.
answered Nov 14 '18 at 19:02
Alp Arıbal
324
324
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53255242%2frecycle-vector-values-with-data-table-shift-instead-of-padding-with-fill-na%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
Perhaps you could define your own function, something like
shift2 <- function(x, n, type = "lag") if(type == "lag") c(tail(x, -(length(x) - n)), head(x, -n)) else c(tail(x, -n), head(x, -(length(x) - n)))
. Then you could dodt1 <- data.table(a = 1:10, b = shift2(x, n = 3L))
. Thoughn
here isn't vectorized like indata.table::shift
– David Arenburg
Nov 12 '18 at 14:41
@DavidArenburg Thanks for that. Was not familiar with
head
andtail
function. Also didn't know about defining a function without{
parenthesis! Speed is not a big issue here as it is a one-off and I will likely create a CSV file to load from (and so others can use it). This is awesome, thanks! If you post it as an answer I will mark it correct (if you want)...– Prevost
Dec 13 '18 at 21:35
I wouldn't make some hack as an official data.table answer. I think you could perhaps add a feature request on the GH issues page instead.
– David Arenburg
Dec 16 '18 at 11:42