Error when trying to fetch rows from mysql and using multi threading to process them
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty height:90px;width:728px;box-sizing:border-box;
1
Code that tries to read from mysql db and process them :
try:
sema = threading.Semaphore(value=100)
threads =
connection = pymysql.connect(host= host, user=user, passwd=pwd, db="Golden")
cursor = connection.cursor()
cursor.execute("SELECT query FROM golden_Queries");
rows = cursor.fetchall()
while rows is not None:
for line in rows:
t = Thread(target=fetchAndStore, args=(line,))
t.daemon = True
threads.append(t)
for t in threads:
t.start()
[t.join() for t in threads]
except Exception,e:
print str(e)
finally:
connection.close()
fetchAndStore will then process these queries one by one. That module works fine.
Output :
Exception in thread Thread-82:
Traceback (most recent call last):
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 810, in __bootstrap_inner
self.run()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 763, in run
self.__target(*self.__args, **self.__kwargs)
File "fetchFromDB.py", line 59, in fetchAndStore
query_string = urllib.urlencode(query)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib.py", line 1333, in urlencode
raise TypeError
TypeError: not a valid non-string sequence or mapping object
can't start new thread
Any help will be really nice :)
python mysql multithreading
asked Nov 15 '18 at 11:14
ArchieArchie
244
add a comment |
1
Code that tries to read from mysql db and process them :
try:
sema = threading.Semaphore(value=100)
threads =
connection = pymysql.connect(host= host, user=user, passwd=pwd, db="Golden")
cursor = connection.cursor()
cursor.execute("SELECT query FROM golden_Queries");
rows = cursor.fetchall()
while rows is not None:
for line in rows:
t = Thread(target=fetchAndStore, args=(line,))
t.daemon = True
threads.append(t)
for t in threads:
t.start()
[t.join() for t in threads]
except Exception,e:
print str(e)
finally:
connection.close()
fetchAndStore will then process these queries one by one. That module works fine.
Output :
Exception in thread Thread-82:
Traceback (most recent call last):
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 810, in __bootstrap_inner
self.run()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 763, in run
self.__target(*self.__args, **self.__kwargs)
File "fetchFromDB.py", line 59, in fetchAndStore
query_string = urllib.urlencode(query)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib.py", line 1333, in urlencode
raise TypeError
TypeError: not a valid non-string sequence or mapping object
can't start new thread
Any help will be really nice :)
python mysql multithreading
asked Nov 15 '18 at 11:14
ArchieArchie
244
The error clearly complains about the linequery_string = urllib.urlencode(query)infetchAndStorewhich wasn't posted, nor is it related to multithreading. The error line showsFile "fetchFromDB.py", line 59, in fetchAndStore query_string = urllib.urlencode(query)
– Panagiotis Kanavos
Nov 15 '18 at 11:22
Wonder how i missed that. Thanks.
– Archie
Nov 16 '18 at 10:12
ensue your query is string, and your error does not occur on the code you provided.
– kcorlidy
Nov 18 '18 at 3:55
add a comment |
1
1
1
Code that tries to read from mysql db and process them :
try:
sema = threading.Semaphore(value=100)
threads =
connection = pymysql.connect(host= host, user=user, passwd=pwd, db="Golden")
cursor = connection.cursor()
cursor.execute("SELECT query FROM golden_Queries");
rows = cursor.fetchall()
while rows is not None:
for line in rows:
t = Thread(target=fetchAndStore, args=(line,))
t.daemon = True
threads.append(t)
for t in threads:
t.start()
[t.join() for t in threads]
except Exception,e:
print str(e)
finally:
connection.close()
fetchAndStore will then process these queries one by one. That module works fine.
Output :
Exception in thread Thread-82:
Traceback (most recent call last):
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 810, in __bootstrap_inner
self.run()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 763, in run
self.__target(*self.__args, **self.__kwargs)
File "fetchFromDB.py", line 59, in fetchAndStore
query_string = urllib.urlencode(query)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib.py", line 1333, in urlencode
raise TypeError
TypeError: not a valid non-string sequence or mapping object
can't start new thread
Any help will be really nice :)
python mysql multithreading
asked Nov 15 '18 at 11:14
ArchieArchie
244
Code that tries to read from mysql db and process them :
try:
sema = threading.Semaphore(value=100)
threads =
connection = pymysql.connect(host= host, user=user, passwd=pwd, db="Golden")
cursor = connection.cursor()
cursor.execute("SELECT query FROM golden_Queries");
rows = cursor.fetchall()
while rows is not None:
for line in rows:
t = Thread(target=fetchAndStore, args=(line,))
t.daemon = True
threads.append(t)
for t in threads:
t.start()
[t.join() for t in threads]
except Exception,e:
print str(e)
finally:
connection.close()
fetchAndStore will then process these queries one by one. That module works fine.
Output :
Exception in thread Thread-82:
Traceback (most recent call last):
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 810, in __bootstrap_inner
self.run()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 763, in run
self.__target(*self.__args, **self.__kwargs)
File "fetchFromDB.py", line 59, in fetchAndStore
query_string = urllib.urlencode(query)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib.py", line 1333, in urlencode
raise TypeError
TypeError: not a valid non-string sequence or mapping object
can't start new thread
Any help will be really nice :)
python mysql multithreading
python mysql multithreading
asked Nov 15 '18 at 11:14
ArchieArchie
244
asked Nov 15 '18 at 11:14
ArchieArchie
244
asked Nov 15 '18 at 11:14
ArchieArchie
244
asked Nov 15 '18 at 11:14
ArchieArchie
244
asked Nov 15 '18 at 11:14
ArchieArchie
244
244
The error clearly complains about the linequery_string = urllib.urlencode(query)infetchAndStorewhich wasn't posted, nor is it related to multithreading. The error line showsFile "fetchFromDB.py", line 59, in fetchAndStore query_string = urllib.urlencode(query)
– Panagiotis Kanavos
Nov 15 '18 at 11:22
Wonder how i missed that. Thanks.
– Archie
Nov 16 '18 at 10:12
ensue your query is string, and your error does not occur on the code you provided.
– kcorlidy
Nov 18 '18 at 3:55
add a comment |
The error clearly complains about the linequery_string = urllib.urlencode(query)infetchAndStorewhich wasn't posted, nor is it related to multithreading. The error line showsFile "fetchFromDB.py", line 59, in fetchAndStore query_string = urllib.urlencode(query)
– Panagiotis Kanavos
Nov 15 '18 at 11:22
Wonder how i missed that. Thanks.
– Archie
Nov 16 '18 at 10:12
ensue your query is string, and your error does not occur on the code you provided.
– kcorlidy
Nov 18 '18 at 3:55
The error clearly complains about the line
query_string = urllib.urlencode(query) in fetchAndStore which wasn't posted, nor is it related to multithreading. The error line shows File "fetchFromDB.py", line 59, in fetchAndStore query_string = urllib.urlencode(query)– Panagiotis Kanavos
Nov 15 '18 at 11:22
The error clearly complains about the line
query_string = urllib.urlencode(query) in fetchAndStore which wasn't posted, nor is it related to multithreading. The error line shows File "fetchFromDB.py", line 59, in fetchAndStore query_string = urllib.urlencode(query)– Panagiotis Kanavos
Nov 15 '18 at 11:22
Wonder how i missed that. Thanks.
– Archie
Nov 16 '18 at 10:12
Wonder how i missed that. Thanks.
– Archie
Nov 16 '18 at 10:12
ensue your query is string, and your error does not occur on the code you provided.
– kcorlidy
Nov 18 '18 at 3:55
ensue your query is string, and your error does not occur on the code you provided.
– kcorlidy
Nov 18 '18 at 3:55
add a comment |
0
active
oldest
votes
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53318231%2ferror-when-trying-to-fetch-rows-from-mysql-and-using-multi-threading-to-process%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
0
active
oldest
votes
0
active
oldest
votes
active
oldest
votes
active
oldest
votes
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53318231%2ferror-when-trying-to-fetch-rows-from-mysql-and-using-multi-threading-to-process%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
The error clearly complains about the line
query_string = urllib.urlencode(query)infetchAndStorewhich wasn't posted, nor is it related to multithreading. The error line showsFile "fetchFromDB.py", line 59, in fetchAndStore query_string = urllib.urlencode(query)– Panagiotis Kanavos
Nov 15 '18 at 11:22
Wonder how i missed that. Thanks.
– Archie
Nov 16 '18 at 10:12
ensue your query is string, and your error does not occur on the code you provided.
– kcorlidy
Nov 18 '18 at 3:55