Undefined variable in lambda function
I have the code :
from functools import reduce
public_ids = [1,2,3,4,5]
filepath = '/path/to/file/'
rdd = sc.textFile(
filepath
)
new_rdd = reduce(
lambda a, b: a.filter(
lambda x: b not in x
),
public_ids,
rdd
)
This code is suppose to filter lines in a rdd according to a list of ids. The rdd is created from files located in filepath using spark context sc's textFile method.
This code works fine, but pylint raises the error :
E: Undefined variable 'b' (undefined-variable)
I believe the way I coded it is not the proper way. How can I change it so pylint does not raise the error again ? Or is it just a structure that pylint does not recognize properly ?
python pyspark pylint
asked Nov 14 '18 at 16:45
StevenSteven
2,75911335
add a comment |
I have the code :
from functools import reduce
public_ids = [1,2,3,4,5]
filepath = '/path/to/file/'
rdd = sc.textFile(
filepath
)
new_rdd = reduce(
lambda a, b: a.filter(
lambda x: b not in x
),
public_ids,
rdd
)
This code is suppose to filter lines in a rdd according to a list of ids. The rdd is created from files located in filepath using spark context sc's textFile method.
This code works fine, but pylint raises the error :
E: Undefined variable 'b' (undefined-variable)
I believe the way I coded it is not the proper way. How can I change it so pylint does not raise the error again ? Or is it just a structure that pylint does not recognize properly ?
python pyspark pylint
asked Nov 14 '18 at 16:45
StevenSteven
2,75911335
1
To answer the spirit of your question, perhaps you can try actually naming your functions instead of nesting the lambdas:def func(a, b): return a.filter(lambda x: b not in x). And then callreduce(func, ...)instead. I'm guessing it had to do with scoping within anonymous functions, so naming it might help.
– Idlehands
Nov 14 '18 at 17:05
I thought that could be the issue indeed, but creating a function used only once is against my religion.
– Steven
Nov 14 '18 at 17:07
1
From my point of view, disposable named functions are evil! Well then you're lost! ...sorry
– Idlehands
Nov 14 '18 at 17:16
add a comment |
I have the code :
from functools import reduce
public_ids = [1,2,3,4,5]
filepath = '/path/to/file/'
rdd = sc.textFile(
filepath
)
new_rdd = reduce(
lambda a, b: a.filter(
lambda x: b not in x
),
public_ids,
rdd
)
This code is suppose to filter lines in a rdd according to a list of ids. The rdd is created from files located in filepath using spark context sc's textFile method.
This code works fine, but pylint raises the error :
E: Undefined variable 'b' (undefined-variable)
I believe the way I coded it is not the proper way. How can I change it so pylint does not raise the error again ? Or is it just a structure that pylint does not recognize properly ?
python pyspark pylint
asked Nov 14 '18 at 16:45
StevenSteven
2,75911335
I have the code :
from functools import reduce
public_ids = [1,2,3,4,5]
filepath = '/path/to/file/'
rdd = sc.textFile(
filepath
)
new_rdd = reduce(
lambda a, b: a.filter(
lambda x: b not in x
),
public_ids,
rdd
)
This code is suppose to filter lines in a rdd according to a list of ids. The rdd is created from files located in filepath using spark context sc's textFile method.
This code works fine, but pylint raises the error :
E: Undefined variable 'b' (undefined-variable)
I believe the way I coded it is not the proper way. How can I change it so pylint does not raise the error again ? Or is it just a structure that pylint does not recognize properly ?
python pyspark pylint
python pyspark pylint
asked Nov 14 '18 at 16:45
StevenSteven
2,75911335
asked Nov 14 '18 at 16:45
StevenSteven
2,75911335
asked Nov 14 '18 at 16:45
StevenSteven
2,75911335
asked Nov 14 '18 at 16:45
StevenSteven
2,75911335
asked Nov 14 '18 at 16:45
StevenSteven
2,75911335
2,75911335
1
To answer the spirit of your question, perhaps you can try actually naming your functions instead of nesting the lambdas:def func(a, b): return a.filter(lambda x: b not in x). And then callreduce(func, ...)instead. I'm guessing it had to do with scoping within anonymous functions, so naming it might help.
– Idlehands
Nov 14 '18 at 17:05
I thought that could be the issue indeed, but creating a function used only once is against my religion.
– Steven
Nov 14 '18 at 17:07
1
From my point of view, disposable named functions are evil! Well then you're lost! ...sorry
– Idlehands
Nov 14 '18 at 17:16
add a comment |
1
To answer the spirit of your question, perhaps you can try actually naming your functions instead of nesting the lambdas:def func(a, b): return a.filter(lambda x: b not in x). And then callreduce(func, ...)instead. I'm guessing it had to do with scoping within anonymous functions, so naming it might help.
– Idlehands
Nov 14 '18 at 17:05
I thought that could be the issue indeed, but creating a function used only once is against my religion.
– Steven
Nov 14 '18 at 17:07
1
From my point of view, disposable named functions are evil! Well then you're lost! ...sorry
– Idlehands
Nov 14 '18 at 17:16
1
1
To answer the spirit of your question, perhaps you can try actually naming your functions instead of nesting the lambdas:
def func(a, b): return a.filter(lambda x: b not in x). And then call reduce(func, ...) instead. I'm guessing it had to do with scoping within anonymous functions, so naming it might help.– Idlehands
Nov 14 '18 at 17:05
To answer the spirit of your question, perhaps you can try actually naming your functions instead of nesting the lambdas:
def func(a, b): return a.filter(lambda x: b not in x). And then call reduce(func, ...) instead. I'm guessing it had to do with scoping within anonymous functions, so naming it might help.– Idlehands
Nov 14 '18 at 17:05
I thought that could be the issue indeed, but creating a function used only once is against my religion.
– Steven
Nov 14 '18 at 17:07
I thought that could be the issue indeed, but creating a function used only once is against my religion.
– Steven
Nov 14 '18 at 17:07
1
1
From my point of view, disposable named functions are evil! Well then you're lost! ...sorry
– Idlehands
Nov 14 '18 at 17:16
From my point of view, disposable named functions are evil! Well then you're lost! ...sorry
– Idlehands
Nov 14 '18 at 17:16
add a comment |
1 Answer
1
active
oldest
votes
Most likely it's a bug in pylint.
Here is a similar bug report from 2 years ago
foo = lambda x: lambda: x + 1 print(foo(1)())
correctly prints 2 when run, but pylint incorrectly reports
E: 1,24: Undefined variable 'x' (undefined-variable)
This is a regression from pylint 1.4.x.
And here is a recent issue reporting the same issue on 11/14/2018
The issue has been reported at #760 and fixed by #2274. However, the
fix is merged only into pylint 2.x which supports only python >= 3.4
leaving us with the bug unresolved in the pylint 1.x series for python
2.
EDIT
Looks like your false positive might be slightly different than the issue above, nevertheless, I would still consider this a bug.
I would try creating an issue on their repo and see what happens
(if you decide to do so, please post link in comments so we can follow it.)
answered Nov 14 '18 at 16:52
gtalaricogtalarico
1,154719
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53305053%2fundefined-variable-in-lambda-function%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Most likely it's a bug in pylint.
Here is a similar bug report from 2 years ago
foo = lambda x: lambda: x + 1 print(foo(1)())
correctly prints 2 when run, but pylint incorrectly reports
E: 1,24: Undefined variable 'x' (undefined-variable)
This is a regression from pylint 1.4.x.
And here is a recent issue reporting the same issue on 11/14/2018
The issue has been reported at #760 and fixed by #2274. However, the
fix is merged only into pylint 2.x which supports only python >= 3.4
leaving us with the bug unresolved in the pylint 1.x series for python
2.
EDIT
Looks like your false positive might be slightly different than the issue above, nevertheless, I would still consider this a bug.
I would try creating an issue on their repo and see what happens
(if you decide to do so, please post link in comments so we can follow it.)
answered Nov 14 '18 at 16:52
gtalaricogtalarico
1,154719
add a comment |
Most likely it's a bug in pylint.
Here is a similar bug report from 2 years ago
foo = lambda x: lambda: x + 1 print(foo(1)())
correctly prints 2 when run, but pylint incorrectly reports
E: 1,24: Undefined variable 'x' (undefined-variable)
This is a regression from pylint 1.4.x.
And here is a recent issue reporting the same issue on 11/14/2018
The issue has been reported at #760 and fixed by #2274. However, the
fix is merged only into pylint 2.x which supports only python >= 3.4
leaving us with the bug unresolved in the pylint 1.x series for python
2.
EDIT
Looks like your false positive might be slightly different than the issue above, nevertheless, I would still consider this a bug.
I would try creating an issue on their repo and see what happens
(if you decide to do so, please post link in comments so we can follow it.)
answered Nov 14 '18 at 16:52
gtalaricogtalarico
1,154719
add a comment |
Most likely it's a bug in pylint.
Here is a similar bug report from 2 years ago
foo = lambda x: lambda: x + 1 print(foo(1)())
correctly prints 2 when run, but pylint incorrectly reports
E: 1,24: Undefined variable 'x' (undefined-variable)
This is a regression from pylint 1.4.x.
And here is a recent issue reporting the same issue on 11/14/2018
The issue has been reported at #760 and fixed by #2274. However, the
fix is merged only into pylint 2.x which supports only python >= 3.4
leaving us with the bug unresolved in the pylint 1.x series for python
2.
EDIT
Looks like your false positive might be slightly different than the issue above, nevertheless, I would still consider this a bug.
I would try creating an issue on their repo and see what happens
(if you decide to do so, please post link in comments so we can follow it.)
answered Nov 14 '18 at 16:52
gtalaricogtalarico
1,154719
Most likely it's a bug in pylint.
Here is a similar bug report from 2 years ago
foo = lambda x: lambda: x + 1 print(foo(1)())
correctly prints 2 when run, but pylint incorrectly reports
E: 1,24: Undefined variable 'x' (undefined-variable)
This is a regression from pylint 1.4.x.
And here is a recent issue reporting the same issue on 11/14/2018
The issue has been reported at #760 and fixed by #2274. However, the
fix is merged only into pylint 2.x which supports only python >= 3.4
leaving us with the bug unresolved in the pylint 1.x series for python
2.
EDIT
Looks like your false positive might be slightly different than the issue above, nevertheless, I would still consider this a bug.
I would try creating an issue on their repo and see what happens
(if you decide to do so, please post link in comments so we can follow it.)
answered Nov 14 '18 at 16:52
gtalaricogtalarico
1,154719
edited Nov 14 '18 at 16:57
answered Nov 14 '18 at 16:52
gtalaricogtalarico
1,154719
answered Nov 14 '18 at 16:52
gtalaricogtalarico
1,154719
answered Nov 14 '18 at 16:52
gtalaricogtalarico
1,154719
1,154719
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53305053%2fundefined-variable-in-lambda-function%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
1
To answer the spirit of your question, perhaps you can try actually naming your functions instead of nesting the lambdas:
def func(a, b): return a.filter(lambda x: b not in x). And then callreduce(func, ...)instead. I'm guessing it had to do with scoping within anonymous functions, so naming it might help.– Idlehands
Nov 14 '18 at 17:05
I thought that could be the issue indeed, but creating a function used only once is against my religion.
– Steven
Nov 14 '18 at 17:07
1
From my point of view, disposable named functions are evil! Well then you're lost! ...sorry
– Idlehands
Nov 14 '18 at 17:16