Pfthb

I am getting the following error when i ran this:

df['initial_referrer'].apply(lambda x: value.split("utm_campaign=",1)[1] if 'utm_campaign' in value else np.nan for value in x.split('&'))

TypeError: 'generator' object is not callable

I am not sure the meaning of the error and how to modify my this to get rid of this. I have read couple of similar questions here but could not figure our what could be the issue.

So I have values in df['initial_referrer'] be like:

df['initial_referrer'].head()
0 /login/index.php
1 /login/index.php?utm_source=INTERNAL&utm_medium=EMAIL&utm_campaign=login-day1
2 /login/index.php
3 /login/index.php?utm_source=INTERNAL&utm_medium=EMAIL&utm_campaign=login-day1
4 /login/index.php

And in this, I wanted to extract the value of utm_campaign which is login-day1 thats why I was using the for loop and then if statement it was taking a lot of time/days to process 20mil rows. Therefore I wanted to use generator expression or list compression to process it faster.

It's instructive to first use apply with a regular function:

def func(x):
 return [value.split("utm_campaign=",1)[1] if 'utm_campaign' in value else np.nan 
 for value in x.split('&')]

df['initial_referrer'].apply(func)

Notice the square brackets representing the list comprehension. You need to translate this to your lambda function:

df['initial_referrer'].apply(lambda x: [value.split("utm_campaign=",1)[1] if 'utm_campaign' in value else np.nan for value in x.split('&')])

But the latter is unreadable. You are better off writing a regular function.

Note pd.Series.apply is a Python-level loop. You can use map instead and will likely see a performance improvement:

df['initial_referrer'] = list(map(func, df['initial_referrer'].values))

Or even a list comprehension:

df['initial_referrer'] = [func(x) for x in df['initial_referrer'].values]