How to do left join on several dataframe
I have a several dataframes with the same name. Each dataframe has one row and two columns. One column is common in all of dataframes. I would like to left-join them together. Assuming the name of dataframes is same. I have no plan on differing their names from each other as they are so many of them and I am just putting a few of them here. Is there any way that i can do left join them and generate the desired output mentioned below?
Here is the dataframes:
col1 col2_4
0 1 2
col1 col2_9
0 1 10
col1 col2_1
0 1 12
col1 col2_3
0 1 5
Output:
col1 col2_4 col2_9 col2_1 col_3
0 1 2 10 12 5
Code:
group = df.groupby([randomcolumnname])
for name, groups in group:
#do some stuff for groups
print(groups)
#I want to join the groups dataframes after this line(some groups dataframes are given above)
Thanks in advance!
python pandas join left-join
add a comment |
I have a several dataframes with the same name. Each dataframe has one row and two columns. One column is common in all of dataframes. I would like to left-join them together. Assuming the name of dataframes is same. I have no plan on differing their names from each other as they are so many of them and I am just putting a few of them here. Is there any way that i can do left join them and generate the desired output mentioned below?
Here is the dataframes:
col1 col2_4
0 1 2
col1 col2_9
0 1 10
col1 col2_1
0 1 12
col1 col2_3
0 1 5
Output:
col1 col2_4 col2_9 col2_1 col_3
0 1 2 10 12 5
Code:
group = df.groupby([randomcolumnname])
for name, groups in group:
#do some stuff for groups
print(groups)
#I want to join the groups dataframes after this line(some groups dataframes are given above)
Thanks in advance!
python pandas join left-join
add a comment |
I have a several dataframes with the same name. Each dataframe has one row and two columns. One column is common in all of dataframes. I would like to left-join them together. Assuming the name of dataframes is same. I have no plan on differing their names from each other as they are so many of them and I am just putting a few of them here. Is there any way that i can do left join them and generate the desired output mentioned below?
Here is the dataframes:
col1 col2_4
0 1 2
col1 col2_9
0 1 10
col1 col2_1
0 1 12
col1 col2_3
0 1 5
Output:
col1 col2_4 col2_9 col2_1 col_3
0 1 2 10 12 5
Code:
group = df.groupby([randomcolumnname])
for name, groups in group:
#do some stuff for groups
print(groups)
#I want to join the groups dataframes after this line(some groups dataframes are given above)
Thanks in advance!
python pandas join left-join
I have a several dataframes with the same name. Each dataframe has one row and two columns. One column is common in all of dataframes. I would like to left-join them together. Assuming the name of dataframes is same. I have no plan on differing their names from each other as they are so many of them and I am just putting a few of them here. Is there any way that i can do left join them and generate the desired output mentioned below?
Here is the dataframes:
col1 col2_4
0 1 2
col1 col2_9
0 1 10
col1 col2_1
0 1 12
col1 col2_3
0 1 5
Output:
col1 col2_4 col2_9 col2_1 col_3
0 1 2 10 12 5
Code:
group = df.groupby([randomcolumnname])
for name, groups in group:
#do some stuff for groups
print(groups)
#I want to join the groups dataframes after this line(some groups dataframes are given above)
Thanks in advance!
python pandas join left-join
python pandas join left-join
edited Nov 11 at 10:32
asked Nov 11 at 9:22
user8034918
53110
53110
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
I believe you need for left join merge with list of DataFrames by column col1:
dfs = [df1, df2, df3, df4]
from functools import reduce
df = df_final = reduce(lambda left,right: pd.merge(left,right,on='col1', how='left'), dfs)
print (df)
col1 col2_1 col2_2 col2_3 col2_4
0 1 2 10 12 5
Or for outer join create index by set_index and concat:
df = pd.concat([x.set_index('col1') for x in dfs], axis=1).reset_index()
print (df)
col1 col2_1 col2_2 col2_3 col2_4
0 1 2 10 12 5
EDIT:
I think better is use custom function with GroupBy.apply:
def func(x):
print (x)
#do some stuff for groups
return x
group = df.groupby([randomcolumnname]).apply(func)
If not possible, for lsit of DataFrames use:
dfs =
group = df.groupby([randomcolumnname])
for name, groups in group:
#do some stuff for groups
print(groups)
dfs.append(groups)
What aboutpd.concat([df1,df2, df3], axis=1)
– pygo
Nov 11 at 9:36
@pygo - it align by indices, not by columncol1, so not
– jezrael
Nov 11 at 9:40
1
Okay , thanks for clarifying,,😊
– pygo
Nov 11 at 9:43
@jezrael the problem is i can't make a list out of dataframes. There are too many of them. And the way they have been created is with the for loop and they don't have separate name!
– user8034918
Nov 11 at 9:47
@user8034918 - it is no problem, only in loop append data to list.
– jezrael
Nov 11 at 9:47
|
show 7 more comments
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53247361%2fhow-to-do-left-join-on-several-dataframe%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
I believe you need for left join merge with list of DataFrames by column col1:
dfs = [df1, df2, df3, df4]
from functools import reduce
df = df_final = reduce(lambda left,right: pd.merge(left,right,on='col1', how='left'), dfs)
print (df)
col1 col2_1 col2_2 col2_3 col2_4
0 1 2 10 12 5
Or for outer join create index by set_index and concat:
df = pd.concat([x.set_index('col1') for x in dfs], axis=1).reset_index()
print (df)
col1 col2_1 col2_2 col2_3 col2_4
0 1 2 10 12 5
EDIT:
I think better is use custom function with GroupBy.apply:
def func(x):
print (x)
#do some stuff for groups
return x
group = df.groupby([randomcolumnname]).apply(func)
If not possible, for lsit of DataFrames use:
dfs =
group = df.groupby([randomcolumnname])
for name, groups in group:
#do some stuff for groups
print(groups)
dfs.append(groups)
What aboutpd.concat([df1,df2, df3], axis=1)
– pygo
Nov 11 at 9:36
@pygo - it align by indices, not by columncol1, so not
– jezrael
Nov 11 at 9:40
1
Okay , thanks for clarifying,,😊
– pygo
Nov 11 at 9:43
@jezrael the problem is i can't make a list out of dataframes. There are too many of them. And the way they have been created is with the for loop and they don't have separate name!
– user8034918
Nov 11 at 9:47
@user8034918 - it is no problem, only in loop append data to list.
– jezrael
Nov 11 at 9:47
|
show 7 more comments
I believe you need for left join merge with list of DataFrames by column col1:
dfs = [df1, df2, df3, df4]
from functools import reduce
df = df_final = reduce(lambda left,right: pd.merge(left,right,on='col1', how='left'), dfs)
print (df)
col1 col2_1 col2_2 col2_3 col2_4
0 1 2 10 12 5
Or for outer join create index by set_index and concat:
df = pd.concat([x.set_index('col1') for x in dfs], axis=1).reset_index()
print (df)
col1 col2_1 col2_2 col2_3 col2_4
0 1 2 10 12 5
EDIT:
I think better is use custom function with GroupBy.apply:
def func(x):
print (x)
#do some stuff for groups
return x
group = df.groupby([randomcolumnname]).apply(func)
If not possible, for lsit of DataFrames use:
dfs =
group = df.groupby([randomcolumnname])
for name, groups in group:
#do some stuff for groups
print(groups)
dfs.append(groups)
What aboutpd.concat([df1,df2, df3], axis=1)
– pygo
Nov 11 at 9:36
@pygo - it align by indices, not by columncol1, so not
– jezrael
Nov 11 at 9:40
1
Okay , thanks for clarifying,,😊
– pygo
Nov 11 at 9:43
@jezrael the problem is i can't make a list out of dataframes. There are too many of them. And the way they have been created is with the for loop and they don't have separate name!
– user8034918
Nov 11 at 9:47
@user8034918 - it is no problem, only in loop append data to list.
– jezrael
Nov 11 at 9:47
|
show 7 more comments
I believe you need for left join merge with list of DataFrames by column col1:
dfs = [df1, df2, df3, df4]
from functools import reduce
df = df_final = reduce(lambda left,right: pd.merge(left,right,on='col1', how='left'), dfs)
print (df)
col1 col2_1 col2_2 col2_3 col2_4
0 1 2 10 12 5
Or for outer join create index by set_index and concat:
df = pd.concat([x.set_index('col1') for x in dfs], axis=1).reset_index()
print (df)
col1 col2_1 col2_2 col2_3 col2_4
0 1 2 10 12 5
EDIT:
I think better is use custom function with GroupBy.apply:
def func(x):
print (x)
#do some stuff for groups
return x
group = df.groupby([randomcolumnname]).apply(func)
If not possible, for lsit of DataFrames use:
dfs =
group = df.groupby([randomcolumnname])
for name, groups in group:
#do some stuff for groups
print(groups)
dfs.append(groups)
I believe you need for left join merge with list of DataFrames by column col1:
dfs = [df1, df2, df3, df4]
from functools import reduce
df = df_final = reduce(lambda left,right: pd.merge(left,right,on='col1', how='left'), dfs)
print (df)
col1 col2_1 col2_2 col2_3 col2_4
0 1 2 10 12 5
Or for outer join create index by set_index and concat:
df = pd.concat([x.set_index('col1') for x in dfs], axis=1).reset_index()
print (df)
col1 col2_1 col2_2 col2_3 col2_4
0 1 2 10 12 5
EDIT:
I think better is use custom function with GroupBy.apply:
def func(x):
print (x)
#do some stuff for groups
return x
group = df.groupby([randomcolumnname]).apply(func)
If not possible, for lsit of DataFrames use:
dfs =
group = df.groupby([randomcolumnname])
for name, groups in group:
#do some stuff for groups
print(groups)
dfs.append(groups)
edited Nov 11 at 10:47
answered Nov 11 at 9:28
jezrael
319k22258336
319k22258336
What aboutpd.concat([df1,df2, df3], axis=1)
– pygo
Nov 11 at 9:36
@pygo - it align by indices, not by columncol1, so not
– jezrael
Nov 11 at 9:40
1
Okay , thanks for clarifying,,😊
– pygo
Nov 11 at 9:43
@jezrael the problem is i can't make a list out of dataframes. There are too many of them. And the way they have been created is with the for loop and they don't have separate name!
– user8034918
Nov 11 at 9:47
@user8034918 - it is no problem, only in loop append data to list.
– jezrael
Nov 11 at 9:47
|
show 7 more comments
What aboutpd.concat([df1,df2, df3], axis=1)
– pygo
Nov 11 at 9:36
@pygo - it align by indices, not by columncol1, so not
– jezrael
Nov 11 at 9:40
1
Okay , thanks for clarifying,,😊
– pygo
Nov 11 at 9:43
@jezrael the problem is i can't make a list out of dataframes. There are too many of them. And the way they have been created is with the for loop and they don't have separate name!
– user8034918
Nov 11 at 9:47
@user8034918 - it is no problem, only in loop append data to list.
– jezrael
Nov 11 at 9:47
What about
pd.concat([df1,df2, df3], axis=1)– pygo
Nov 11 at 9:36
What about
pd.concat([df1,df2, df3], axis=1)– pygo
Nov 11 at 9:36
@pygo - it align by indices, not by column
col1, so not– jezrael
Nov 11 at 9:40
@pygo - it align by indices, not by column
col1, so not– jezrael
Nov 11 at 9:40
1
1
Okay , thanks for clarifying,,😊
– pygo
Nov 11 at 9:43
Okay , thanks for clarifying,,😊
– pygo
Nov 11 at 9:43
@jezrael the problem is i can't make a list out of dataframes. There are too many of them. And the way they have been created is with the for loop and they don't have separate name!
– user8034918
Nov 11 at 9:47
@jezrael the problem is i can't make a list out of dataframes. There are too many of them. And the way they have been created is with the for loop and they don't have separate name!
– user8034918
Nov 11 at 9:47
@user8034918 - it is no problem, only in loop append data to list.
– jezrael
Nov 11 at 9:47
@user8034918 - it is no problem, only in loop append data to list.
– jezrael
Nov 11 at 9:47
|
show 7 more comments
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53247361%2fhow-to-do-left-join-on-several-dataframe%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown