Merging of Excel Files with Pandas









up vote
-2
down vote

favorite












I have about 19 different excel files that all have the same columns and I want to merge them all together into one excel file. When I go merge it, half way down, the columns shift one to the right.



For example all the columns are A through U but when merged some will be A through U and some will be A through V. Everything lines up except the U column it either skips the U column and put everything in V or adds something random to the U column and pushes what is supposed to be in U into V.



The code I have to merge them is as follow:



def mergingExcel():
sharedDocs = "C:\Users\CHI\Documents\SPSharedDocuments\*.xlsx"
invoices = "C:\UsersCHI\Documents\SPInvoices\*.xlsx"
formsCerts = "C:\UsersCHI\Documents\SPForms&Certificates\*.xlsx"
mgmt = "C:\UsersCHI\Documents\SPManagement\*.xlsx"

files = [sharedDocs, invoices, formsCerts, mgmt] #contains variables that point to the path of files

for docs in files: #cycles through the files array
excel =
for file in glob.glob(docs): #cycles through the excel files at each path
excel.append(file) #add excel doc to excel array

excels = [pd.ExcelFile(name) for name in excel]
frames = [x.parse(x.sheet_names[0], header=None, index_col=None) for x in excels]
frames[1:] = [df[1:] for df in frames[1:]]
combined = pd.concat(frames)

if sharedDocs == docs:
combined.to_excel("SharedDocsMerged.xlsx", header = False, index = False)
shutil.move(os.path.join("C:\Users\CHI\source\repos\PythonApplication1\PythonApplication1\SharedDocsMerged.xlsx","C:\Users\CHI\Documents\SPSharedDocuments"),os.path.join("C:\Users\CHI\source\repos\PythonApplication1\PythonApplication1\SharedDocsMerged.xlsx","C:\Users\CHI\Documents\SPSharedDocuments"))
elif invoices == docs:
combined.to_excel("InvoicesMerged.xlsx", header = False, index = False)
shutil.move(os.path.join("C:\Users\CHI\source\repos\PythonApplication1\PythonApplication1\InvoicesMerged.xlsx","C:\Users\CHI\Documents\SPInvoices"),os.path.join("C:\Users\CHI\source\repos\PythonApplication1\PythonApplication1\InvoicesMerged.xlsx","C:\Users\CHI\Documents\SPInvoices"))
elif formsCerts == docs:
combined.to_excel("FormsCertsMerged.xlsx", header = False, index = False)
shutil.move(os.path.join("C:\Users\CHI\source\repos\PythonApplication1\PythonApplication1\FormsCertsMerged.xlsx","C:\Users\CHI\Documents\SPForms&Certificates"),os.path.join("C:\Users\CHI\source\repos\PythonApplication1\PythonApplication1\FormsCertsMerged.xlsx","C:\Users\CHI\Documents\SPForms&Certificates"))
else:
combined.to_excel("MGMTMerged.xlsx", header = False, index = False)
shutil.move(os.path.join("C:\Users\CHI\source\repos\PythonApplication1\PythonApplication1\MGMTMerged.xlsx","C:\Users\CHI\Documents\SPManagement"),os.path.join("C:\Users\CHI\source\repos\PythonApplication1\PythonApplication1\MGMTMerged.xlsx","C:\Users\CHI\Documents\SPManagement"))
return;


Thank in advance for any advice










share|improve this question



















  • 5




    it's really hard to know what the issue here is without some sample data
    – wpercy
    Nov 9 at 21:05














up vote
-2
down vote

favorite












I have about 19 different excel files that all have the same columns and I want to merge them all together into one excel file. When I go merge it, half way down, the columns shift one to the right.



For example all the columns are A through U but when merged some will be A through U and some will be A through V. Everything lines up except the U column it either skips the U column and put everything in V or adds something random to the U column and pushes what is supposed to be in U into V.



The code I have to merge them is as follow:



def mergingExcel():
sharedDocs = "C:\Users\CHI\Documents\SPSharedDocuments\*.xlsx"
invoices = "C:\UsersCHI\Documents\SPInvoices\*.xlsx"
formsCerts = "C:\UsersCHI\Documents\SPForms&Certificates\*.xlsx"
mgmt = "C:\UsersCHI\Documents\SPManagement\*.xlsx"

files = [sharedDocs, invoices, formsCerts, mgmt] #contains variables that point to the path of files

for docs in files: #cycles through the files array
excel =
for file in glob.glob(docs): #cycles through the excel files at each path
excel.append(file) #add excel doc to excel array

excels = [pd.ExcelFile(name) for name in excel]
frames = [x.parse(x.sheet_names[0], header=None, index_col=None) for x in excels]
frames[1:] = [df[1:] for df in frames[1:]]
combined = pd.concat(frames)

if sharedDocs == docs:
combined.to_excel("SharedDocsMerged.xlsx", header = False, index = False)
shutil.move(os.path.join("C:\Users\CHI\source\repos\PythonApplication1\PythonApplication1\SharedDocsMerged.xlsx","C:\Users\CHI\Documents\SPSharedDocuments"),os.path.join("C:\Users\CHI\source\repos\PythonApplication1\PythonApplication1\SharedDocsMerged.xlsx","C:\Users\CHI\Documents\SPSharedDocuments"))
elif invoices == docs:
combined.to_excel("InvoicesMerged.xlsx", header = False, index = False)
shutil.move(os.path.join("C:\Users\CHI\source\repos\PythonApplication1\PythonApplication1\InvoicesMerged.xlsx","C:\Users\CHI\Documents\SPInvoices"),os.path.join("C:\Users\CHI\source\repos\PythonApplication1\PythonApplication1\InvoicesMerged.xlsx","C:\Users\CHI\Documents\SPInvoices"))
elif formsCerts == docs:
combined.to_excel("FormsCertsMerged.xlsx", header = False, index = False)
shutil.move(os.path.join("C:\Users\CHI\source\repos\PythonApplication1\PythonApplication1\FormsCertsMerged.xlsx","C:\Users\CHI\Documents\SPForms&Certificates"),os.path.join("C:\Users\CHI\source\repos\PythonApplication1\PythonApplication1\FormsCertsMerged.xlsx","C:\Users\CHI\Documents\SPForms&Certificates"))
else:
combined.to_excel("MGMTMerged.xlsx", header = False, index = False)
shutil.move(os.path.join("C:\Users\CHI\source\repos\PythonApplication1\PythonApplication1\MGMTMerged.xlsx","C:\Users\CHI\Documents\SPManagement"),os.path.join("C:\Users\CHI\source\repos\PythonApplication1\PythonApplication1\MGMTMerged.xlsx","C:\Users\CHI\Documents\SPManagement"))
return;


Thank in advance for any advice










share|improve this question



















  • 5




    it's really hard to know what the issue here is without some sample data
    – wpercy
    Nov 9 at 21:05












up vote
-2
down vote

favorite









up vote
-2
down vote

favorite











I have about 19 different excel files that all have the same columns and I want to merge them all together into one excel file. When I go merge it, half way down, the columns shift one to the right.



For example all the columns are A through U but when merged some will be A through U and some will be A through V. Everything lines up except the U column it either skips the U column and put everything in V or adds something random to the U column and pushes what is supposed to be in U into V.



The code I have to merge them is as follow:



def mergingExcel():
sharedDocs = "C:\Users\CHI\Documents\SPSharedDocuments\*.xlsx"
invoices = "C:\UsersCHI\Documents\SPInvoices\*.xlsx"
formsCerts = "C:\UsersCHI\Documents\SPForms&Certificates\*.xlsx"
mgmt = "C:\UsersCHI\Documents\SPManagement\*.xlsx"

files = [sharedDocs, invoices, formsCerts, mgmt] #contains variables that point to the path of files

for docs in files: #cycles through the files array
excel =
for file in glob.glob(docs): #cycles through the excel files at each path
excel.append(file) #add excel doc to excel array

excels = [pd.ExcelFile(name) for name in excel]
frames = [x.parse(x.sheet_names[0], header=None, index_col=None) for x in excels]
frames[1:] = [df[1:] for df in frames[1:]]
combined = pd.concat(frames)

if sharedDocs == docs:
combined.to_excel("SharedDocsMerged.xlsx", header = False, index = False)
shutil.move(os.path.join("C:\Users\CHI\source\repos\PythonApplication1\PythonApplication1\SharedDocsMerged.xlsx","C:\Users\CHI\Documents\SPSharedDocuments"),os.path.join("C:\Users\CHI\source\repos\PythonApplication1\PythonApplication1\SharedDocsMerged.xlsx","C:\Users\CHI\Documents\SPSharedDocuments"))
elif invoices == docs:
combined.to_excel("InvoicesMerged.xlsx", header = False, index = False)
shutil.move(os.path.join("C:\Users\CHI\source\repos\PythonApplication1\PythonApplication1\InvoicesMerged.xlsx","C:\Users\CHI\Documents\SPInvoices"),os.path.join("C:\Users\CHI\source\repos\PythonApplication1\PythonApplication1\InvoicesMerged.xlsx","C:\Users\CHI\Documents\SPInvoices"))
elif formsCerts == docs:
combined.to_excel("FormsCertsMerged.xlsx", header = False, index = False)
shutil.move(os.path.join("C:\Users\CHI\source\repos\PythonApplication1\PythonApplication1\FormsCertsMerged.xlsx","C:\Users\CHI\Documents\SPForms&Certificates"),os.path.join("C:\Users\CHI\source\repos\PythonApplication1\PythonApplication1\FormsCertsMerged.xlsx","C:\Users\CHI\Documents\SPForms&Certificates"))
else:
combined.to_excel("MGMTMerged.xlsx", header = False, index = False)
shutil.move(os.path.join("C:\Users\CHI\source\repos\PythonApplication1\PythonApplication1\MGMTMerged.xlsx","C:\Users\CHI\Documents\SPManagement"),os.path.join("C:\Users\CHI\source\repos\PythonApplication1\PythonApplication1\MGMTMerged.xlsx","C:\Users\CHI\Documents\SPManagement"))
return;


Thank in advance for any advice










share|improve this question















I have about 19 different excel files that all have the same columns and I want to merge them all together into one excel file. When I go merge it, half way down, the columns shift one to the right.



For example all the columns are A through U but when merged some will be A through U and some will be A through V. Everything lines up except the U column it either skips the U column and put everything in V or adds something random to the U column and pushes what is supposed to be in U into V.



The code I have to merge them is as follow:



def mergingExcel():
sharedDocs = "C:\Users\CHI\Documents\SPSharedDocuments\*.xlsx"
invoices = "C:\UsersCHI\Documents\SPInvoices\*.xlsx"
formsCerts = "C:\UsersCHI\Documents\SPForms&Certificates\*.xlsx"
mgmt = "C:\UsersCHI\Documents\SPManagement\*.xlsx"

files = [sharedDocs, invoices, formsCerts, mgmt] #contains variables that point to the path of files

for docs in files: #cycles through the files array
excel =
for file in glob.glob(docs): #cycles through the excel files at each path
excel.append(file) #add excel doc to excel array

excels = [pd.ExcelFile(name) for name in excel]
frames = [x.parse(x.sheet_names[0], header=None, index_col=None) for x in excels]
frames[1:] = [df[1:] for df in frames[1:]]
combined = pd.concat(frames)

if sharedDocs == docs:
combined.to_excel("SharedDocsMerged.xlsx", header = False, index = False)
shutil.move(os.path.join("C:\Users\CHI\source\repos\PythonApplication1\PythonApplication1\SharedDocsMerged.xlsx","C:\Users\CHI\Documents\SPSharedDocuments"),os.path.join("C:\Users\CHI\source\repos\PythonApplication1\PythonApplication1\SharedDocsMerged.xlsx","C:\Users\CHI\Documents\SPSharedDocuments"))
elif invoices == docs:
combined.to_excel("InvoicesMerged.xlsx", header = False, index = False)
shutil.move(os.path.join("C:\Users\CHI\source\repos\PythonApplication1\PythonApplication1\InvoicesMerged.xlsx","C:\Users\CHI\Documents\SPInvoices"),os.path.join("C:\Users\CHI\source\repos\PythonApplication1\PythonApplication1\InvoicesMerged.xlsx","C:\Users\CHI\Documents\SPInvoices"))
elif formsCerts == docs:
combined.to_excel("FormsCertsMerged.xlsx", header = False, index = False)
shutil.move(os.path.join("C:\Users\CHI\source\repos\PythonApplication1\PythonApplication1\FormsCertsMerged.xlsx","C:\Users\CHI\Documents\SPForms&Certificates"),os.path.join("C:\Users\CHI\source\repos\PythonApplication1\PythonApplication1\FormsCertsMerged.xlsx","C:\Users\CHI\Documents\SPForms&Certificates"))
else:
combined.to_excel("MGMTMerged.xlsx", header = False, index = False)
shutil.move(os.path.join("C:\Users\CHI\source\repos\PythonApplication1\PythonApplication1\MGMTMerged.xlsx","C:\Users\CHI\Documents\SPManagement"),os.path.join("C:\Users\CHI\source\repos\PythonApplication1\PythonApplication1\MGMTMerged.xlsx","C:\Users\CHI\Documents\SPManagement"))
return;


Thank in advance for any advice







python excel pandas






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Nov 9 at 21:08









petezurich

3,37881633




3,37881633










asked Nov 9 at 21:03









jordan23

237




237







  • 5




    it's really hard to know what the issue here is without some sample data
    – wpercy
    Nov 9 at 21:05












  • 5




    it's really hard to know what the issue here is without some sample data
    – wpercy
    Nov 9 at 21:05







5




5




it's really hard to know what the issue here is without some sample data
– wpercy
Nov 9 at 21:05




it's really hard to know what the issue here is without some sample data
– wpercy
Nov 9 at 21:05

















active

oldest

votes











Your Answer






StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);



);













 

draft saved


draft discarded


















StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53233242%2fmerging-of-excel-files-with-pandas%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown






























active

oldest

votes













active

oldest

votes









active

oldest

votes






active

oldest

votes















 

draft saved


draft discarded















































 


draft saved


draft discarded














StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53233242%2fmerging-of-excel-files-with-pandas%23new-answer', 'question_page');

);

Post as a guest















Required, but never shown





















































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown

































Required, but never shown














Required, but never shown












Required, but never shown







Required, but never shown







Popular posts from this blog

How to how show current date and time by default on contact form 7 in WordPress without taking input from user in datetimepicker

Syphilis

Darth Vader #20