Click on 'Next' button using Selenium until all pages are parsed: Python3
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty height:90px;width:728px;box-sizing:border-box;
I am trying to click on the "Next" button of a webpage, but I'm receiving an error.
The steps I took to reach the table:
- Go to ibl.mdanderson.org/fasmic/#!
- Type and select AKT1 (3 mutations) (NOTE:'GO' button doesn't work, please click the option from the drop-down)
- Click on the green button with the text 'MS', a new table will appear.
- In this new table, there will be a tab called literature.
- At the end of the literature tab, there will be a "Next" button to go to the next page.
I need to parse data from the literature tab(which works perfectly). However, I cant go to the next page to parse data from the same table.
Following is the code:
driver.find_element_by_xpath("//*[contains(text(),'Literature')]").click()
for elements in driver.find_elements_by_css_selector("#literature_div [ng-repeat]"):
print(elements.text,"n")
driver.implicitly_wait(5)
driver.find_element_by_xpath('//a[@ng-click="selectPage(page + 1, $event)"]').click()
After executing this, I get the following error:
"Message: The element reference of is stale; either the element is no
longer attached to the DOM, it is not in the current frame context, or
the document has been refreshed"
python python-3.x selenium-webdriver web-scraping
add a comment |
I am trying to click on the "Next" button of a webpage, but I'm receiving an error.
The steps I took to reach the table:
- Go to ibl.mdanderson.org/fasmic/#!
- Type and select AKT1 (3 mutations) (NOTE:'GO' button doesn't work, please click the option from the drop-down)
- Click on the green button with the text 'MS', a new table will appear.
- In this new table, there will be a tab called literature.
- At the end of the literature tab, there will be a "Next" button to go to the next page.
I need to parse data from the literature tab(which works perfectly). However, I cant go to the next page to parse data from the same table.
Following is the code:
driver.find_element_by_xpath("//*[contains(text(),'Literature')]").click()
for elements in driver.find_elements_by_css_selector("#literature_div [ng-repeat]"):
print(elements.text,"n")
driver.implicitly_wait(5)
driver.find_element_by_xpath('//a[@ng-click="selectPage(page + 1, $event)"]').click()
After executing this, I get the following error:
"Message: The element reference of is stale; either the element is no
longer attached to the DOM, it is not in the current frame context, or
the document has been refreshed"
python python-3.x selenium-webdriver web-scraping
I see those items are getting logged in the console. Here's a thread where someone captures the log. Probably easier than troubleshooting that message.
– pguardiario
Nov 15 '18 at 22:55
add a comment |
I am trying to click on the "Next" button of a webpage, but I'm receiving an error.
The steps I took to reach the table:
- Go to ibl.mdanderson.org/fasmic/#!
- Type and select AKT1 (3 mutations) (NOTE:'GO' button doesn't work, please click the option from the drop-down)
- Click on the green button with the text 'MS', a new table will appear.
- In this new table, there will be a tab called literature.
- At the end of the literature tab, there will be a "Next" button to go to the next page.
I need to parse data from the literature tab(which works perfectly). However, I cant go to the next page to parse data from the same table.
Following is the code:
driver.find_element_by_xpath("//*[contains(text(),'Literature')]").click()
for elements in driver.find_elements_by_css_selector("#literature_div [ng-repeat]"):
print(elements.text,"n")
driver.implicitly_wait(5)
driver.find_element_by_xpath('//a[@ng-click="selectPage(page + 1, $event)"]').click()
After executing this, I get the following error:
"Message: The element reference of is stale; either the element is no
longer attached to the DOM, it is not in the current frame context, or
the document has been refreshed"
python python-3.x selenium-webdriver web-scraping
I am trying to click on the "Next" button of a webpage, but I'm receiving an error.
The steps I took to reach the table:
- Go to ibl.mdanderson.org/fasmic/#!
- Type and select AKT1 (3 mutations) (NOTE:'GO' button doesn't work, please click the option from the drop-down)
- Click on the green button with the text 'MS', a new table will appear.
- In this new table, there will be a tab called literature.
- At the end of the literature tab, there will be a "Next" button to go to the next page.
I need to parse data from the literature tab(which works perfectly). However, I cant go to the next page to parse data from the same table.
Following is the code:
driver.find_element_by_xpath("//*[contains(text(),'Literature')]").click()
for elements in driver.find_elements_by_css_selector("#literature_div [ng-repeat]"):
print(elements.text,"n")
driver.implicitly_wait(5)
driver.find_element_by_xpath('//a[@ng-click="selectPage(page + 1, $event)"]').click()
After executing this, I get the following error:
"Message: The element reference of is stale; either the element is no
longer attached to the DOM, it is not in the current frame context, or
the document has been refreshed"
python python-3.x selenium-webdriver web-scraping
python python-3.x selenium-webdriver web-scraping
edited Nov 15 '18 at 16:42
RRg
asked Nov 15 '18 at 15:37
RRgRRg
496
496
I see those items are getting logged in the console. Here's a thread where someone captures the log. Probably easier than troubleshooting that message.
– pguardiario
Nov 15 '18 at 22:55
add a comment |
I see those items are getting logged in the console. Here's a thread where someone captures the log. Probably easier than troubleshooting that message.
– pguardiario
Nov 15 '18 at 22:55
I see those items are getting logged in the console. Here's a thread where someone captures the log. Probably easier than troubleshooting that message.
– pguardiario
Nov 15 '18 at 22:55
I see those items are getting logged in the console. Here's a thread where someone captures the log. Probably easier than troubleshooting that message.
– pguardiario
Nov 15 '18 at 22:55
add a comment |
2 Answers
2
active
oldest
votes
At the end of literature tab try
driver.find_element_by_xpath('/html/body/div/div[2]/div[2]/div[1]/div/div/div[3]/div/div/div[5]/div/ul/li[7]/a').click()
I still get the same error: Message: The element reference of <div class="ng-scope"> is stale; either the element is no longer attached to the DOM, it is not in the current frame context, or the document has been refreshed
– RRg
Nov 15 '18 at 16:10
add a comment |
Have you tried
driver.find_element_by_css_selector('[ng-click="selectPage(page + 1, $event)"]').click()
You can improve but this works for the click (Assuming I am targeting the right Next)
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
url = "https://ibl.mdanderson.org/fasmic/#!/"
d = webdriver.Chrome()
wait = WebDriverWait(d, 10)
d.get(url)
d.find_element_by_css_selector('[type=text]').send_keys('AKT1 (3 mutations)')
d.find_element_by_css_selector("input[type='text']").send_keys(Keys.RETURN)
btn = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, ".btn.btn-default.btn-tab-avail")))
btn.click()
d.find_element_by_css_selector("[heading=Literature]").click()
ele = wait.until(EC.text_to_be_present_in_element((By.CSS_SELECTOR, "#literature_div [ng-repeat]"), "PMID"))
eles = d.find_elements_by_css_selector("#literature_div [ng-repeat]")
d.find_element_by_css_selector('[ng-click="selectPage(page + 1, $event)"]').click()
#d.quit()
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53322893%2fclick-on-next-button-using-selenium-until-all-pages-are-parsed-python3%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
At the end of literature tab try
driver.find_element_by_xpath('/html/body/div/div[2]/div[2]/div[1]/div/div/div[3]/div/div/div[5]/div/ul/li[7]/a').click()
I still get the same error: Message: The element reference of <div class="ng-scope"> is stale; either the element is no longer attached to the DOM, it is not in the current frame context, or the document has been refreshed
– RRg
Nov 15 '18 at 16:10
add a comment |
At the end of literature tab try
driver.find_element_by_xpath('/html/body/div/div[2]/div[2]/div[1]/div/div/div[3]/div/div/div[5]/div/ul/li[7]/a').click()
I still get the same error: Message: The element reference of <div class="ng-scope"> is stale; either the element is no longer attached to the DOM, it is not in the current frame context, or the document has been refreshed
– RRg
Nov 15 '18 at 16:10
add a comment |
At the end of literature tab try
driver.find_element_by_xpath('/html/body/div/div[2]/div[2]/div[1]/div/div/div[3]/div/div/div[5]/div/ul/li[7]/a').click()
At the end of literature tab try
driver.find_element_by_xpath('/html/body/div/div[2]/div[2]/div[1]/div/div/div[3]/div/div/div[5]/div/ul/li[7]/a').click()
answered Nov 15 '18 at 16:03
Florian BordFlorian Bord
54
54
I still get the same error: Message: The element reference of <div class="ng-scope"> is stale; either the element is no longer attached to the DOM, it is not in the current frame context, or the document has been refreshed
– RRg
Nov 15 '18 at 16:10
add a comment |
I still get the same error: Message: The element reference of <div class="ng-scope"> is stale; either the element is no longer attached to the DOM, it is not in the current frame context, or the document has been refreshed
– RRg
Nov 15 '18 at 16:10
I still get the same error: Message: The element reference of <div class="ng-scope"> is stale; either the element is no longer attached to the DOM, it is not in the current frame context, or the document has been refreshed
– RRg
Nov 15 '18 at 16:10
I still get the same error: Message: The element reference of <div class="ng-scope"> is stale; either the element is no longer attached to the DOM, it is not in the current frame context, or the document has been refreshed
– RRg
Nov 15 '18 at 16:10
add a comment |
Have you tried
driver.find_element_by_css_selector('[ng-click="selectPage(page + 1, $event)"]').click()
You can improve but this works for the click (Assuming I am targeting the right Next)
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
url = "https://ibl.mdanderson.org/fasmic/#!/"
d = webdriver.Chrome()
wait = WebDriverWait(d, 10)
d.get(url)
d.find_element_by_css_selector('[type=text]').send_keys('AKT1 (3 mutations)')
d.find_element_by_css_selector("input[type='text']").send_keys(Keys.RETURN)
btn = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, ".btn.btn-default.btn-tab-avail")))
btn.click()
d.find_element_by_css_selector("[heading=Literature]").click()
ele = wait.until(EC.text_to_be_present_in_element((By.CSS_SELECTOR, "#literature_div [ng-repeat]"), "PMID"))
eles = d.find_elements_by_css_selector("#literature_div [ng-repeat]")
d.find_element_by_css_selector('[ng-click="selectPage(page + 1, $event)"]').click()
#d.quit()
add a comment |
Have you tried
driver.find_element_by_css_selector('[ng-click="selectPage(page + 1, $event)"]').click()
You can improve but this works for the click (Assuming I am targeting the right Next)
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
url = "https://ibl.mdanderson.org/fasmic/#!/"
d = webdriver.Chrome()
wait = WebDriverWait(d, 10)
d.get(url)
d.find_element_by_css_selector('[type=text]').send_keys('AKT1 (3 mutations)')
d.find_element_by_css_selector("input[type='text']").send_keys(Keys.RETURN)
btn = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, ".btn.btn-default.btn-tab-avail")))
btn.click()
d.find_element_by_css_selector("[heading=Literature]").click()
ele = wait.until(EC.text_to_be_present_in_element((By.CSS_SELECTOR, "#literature_div [ng-repeat]"), "PMID"))
eles = d.find_elements_by_css_selector("#literature_div [ng-repeat]")
d.find_element_by_css_selector('[ng-click="selectPage(page + 1, $event)"]').click()
#d.quit()
add a comment |
Have you tried
driver.find_element_by_css_selector('[ng-click="selectPage(page + 1, $event)"]').click()
You can improve but this works for the click (Assuming I am targeting the right Next)
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
url = "https://ibl.mdanderson.org/fasmic/#!/"
d = webdriver.Chrome()
wait = WebDriverWait(d, 10)
d.get(url)
d.find_element_by_css_selector('[type=text]').send_keys('AKT1 (3 mutations)')
d.find_element_by_css_selector("input[type='text']").send_keys(Keys.RETURN)
btn = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, ".btn.btn-default.btn-tab-avail")))
btn.click()
d.find_element_by_css_selector("[heading=Literature]").click()
ele = wait.until(EC.text_to_be_present_in_element((By.CSS_SELECTOR, "#literature_div [ng-repeat]"), "PMID"))
eles = d.find_elements_by_css_selector("#literature_div [ng-repeat]")
d.find_element_by_css_selector('[ng-click="selectPage(page + 1, $event)"]').click()
#d.quit()
Have you tried
driver.find_element_by_css_selector('[ng-click="selectPage(page + 1, $event)"]').click()
You can improve but this works for the click (Assuming I am targeting the right Next)
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
url = "https://ibl.mdanderson.org/fasmic/#!/"
d = webdriver.Chrome()
wait = WebDriverWait(d, 10)
d.get(url)
d.find_element_by_css_selector('[type=text]').send_keys('AKT1 (3 mutations)')
d.find_element_by_css_selector("input[type='text']").send_keys(Keys.RETURN)
btn = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, ".btn.btn-default.btn-tab-avail")))
btn.click()
d.find_element_by_css_selector("[heading=Literature]").click()
ele = wait.until(EC.text_to_be_present_in_element((By.CSS_SELECTOR, "#literature_div [ng-repeat]"), "PMID"))
eles = d.find_elements_by_css_selector("#literature_div [ng-repeat]")
d.find_element_by_css_selector('[ng-click="selectPage(page + 1, $event)"]').click()
#d.quit()
answered Nov 15 '18 at 17:08
QHarrQHarr
38.8k82245
38.8k82245
add a comment |
add a comment |
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53322893%2fclick-on-next-button-using-selenium-until-all-pages-are-parsed-python3%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
I see those items are getting logged in the console. Here's a thread where someone captures the log. Probably easier than troubleshooting that message.
– pguardiario
Nov 15 '18 at 22:55