Click on 'Next' button using Selenium until all pages are parsed: Python3



.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty height:90px;width:728px;box-sizing:border-box;








0















I am trying to click on the "Next" button of a webpage, but I'm receiving an error.
The steps I took to reach the table:



  1. Go to ibl.mdanderson.org/fasmic/#!

  2. Type and select AKT1 (3 mutations) (NOTE:'GO' button doesn't work, please click the option from the drop-down)

  3. Click on the green button with the text 'MS', a new table will appear.

  4. In this new table, there will be a tab called literature.

  5. At the end of the literature tab, there will be a "Next" button to go to the next page.

I need to parse data from the literature tab(which works perfectly). However, I cant go to the next page to parse data from the same table.



Following is the code:



driver.find_element_by_xpath("//*[contains(text(),'Literature')]").click()
for elements in driver.find_elements_by_css_selector("#literature_div [ng-repeat]"):
print(elements.text,"n")
driver.implicitly_wait(5)
driver.find_element_by_xpath('//a[@ng-click="selectPage(page + 1, $event)"]').click()


After executing this, I get the following error:




"Message: The element reference of is stale; either the element is no
longer attached to the DOM, it is not in the current frame context, or
the document has been refreshed"











share|improve this question
























  • I see those items are getting logged in the console. Here's a thread where someone captures the log. Probably easier than troubleshooting that message.

    – pguardiario
    Nov 15 '18 at 22:55


















0















I am trying to click on the "Next" button of a webpage, but I'm receiving an error.
The steps I took to reach the table:



  1. Go to ibl.mdanderson.org/fasmic/#!

  2. Type and select AKT1 (3 mutations) (NOTE:'GO' button doesn't work, please click the option from the drop-down)

  3. Click on the green button with the text 'MS', a new table will appear.

  4. In this new table, there will be a tab called literature.

  5. At the end of the literature tab, there will be a "Next" button to go to the next page.

I need to parse data from the literature tab(which works perfectly). However, I cant go to the next page to parse data from the same table.



Following is the code:



driver.find_element_by_xpath("//*[contains(text(),'Literature')]").click()
for elements in driver.find_elements_by_css_selector("#literature_div [ng-repeat]"):
print(elements.text,"n")
driver.implicitly_wait(5)
driver.find_element_by_xpath('//a[@ng-click="selectPage(page + 1, $event)"]').click()


After executing this, I get the following error:




"Message: The element reference of is stale; either the element is no
longer attached to the DOM, it is not in the current frame context, or
the document has been refreshed"











share|improve this question
























  • I see those items are getting logged in the console. Here's a thread where someone captures the log. Probably easier than troubleshooting that message.

    – pguardiario
    Nov 15 '18 at 22:55














0












0








0








I am trying to click on the "Next" button of a webpage, but I'm receiving an error.
The steps I took to reach the table:



  1. Go to ibl.mdanderson.org/fasmic/#!

  2. Type and select AKT1 (3 mutations) (NOTE:'GO' button doesn't work, please click the option from the drop-down)

  3. Click on the green button with the text 'MS', a new table will appear.

  4. In this new table, there will be a tab called literature.

  5. At the end of the literature tab, there will be a "Next" button to go to the next page.

I need to parse data from the literature tab(which works perfectly). However, I cant go to the next page to parse data from the same table.



Following is the code:



driver.find_element_by_xpath("//*[contains(text(),'Literature')]").click()
for elements in driver.find_elements_by_css_selector("#literature_div [ng-repeat]"):
print(elements.text,"n")
driver.implicitly_wait(5)
driver.find_element_by_xpath('//a[@ng-click="selectPage(page + 1, $event)"]').click()


After executing this, I get the following error:




"Message: The element reference of is stale; either the element is no
longer attached to the DOM, it is not in the current frame context, or
the document has been refreshed"











share|improve this question
















I am trying to click on the "Next" button of a webpage, but I'm receiving an error.
The steps I took to reach the table:



  1. Go to ibl.mdanderson.org/fasmic/#!

  2. Type and select AKT1 (3 mutations) (NOTE:'GO' button doesn't work, please click the option from the drop-down)

  3. Click on the green button with the text 'MS', a new table will appear.

  4. In this new table, there will be a tab called literature.

  5. At the end of the literature tab, there will be a "Next" button to go to the next page.

I need to parse data from the literature tab(which works perfectly). However, I cant go to the next page to parse data from the same table.



Following is the code:



driver.find_element_by_xpath("//*[contains(text(),'Literature')]").click()
for elements in driver.find_elements_by_css_selector("#literature_div [ng-repeat]"):
print(elements.text,"n")
driver.implicitly_wait(5)
driver.find_element_by_xpath('//a[@ng-click="selectPage(page + 1, $event)"]').click()


After executing this, I get the following error:




"Message: The element reference of is stale; either the element is no
longer attached to the DOM, it is not in the current frame context, or
the document has been refreshed"








python python-3.x selenium-webdriver web-scraping






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Nov 15 '18 at 16:42







RRg

















asked Nov 15 '18 at 15:37









RRgRRg

496




496












  • I see those items are getting logged in the console. Here's a thread where someone captures the log. Probably easier than troubleshooting that message.

    – pguardiario
    Nov 15 '18 at 22:55


















  • I see those items are getting logged in the console. Here's a thread where someone captures the log. Probably easier than troubleshooting that message.

    – pguardiario
    Nov 15 '18 at 22:55

















I see those items are getting logged in the console. Here's a thread where someone captures the log. Probably easier than troubleshooting that message.

– pguardiario
Nov 15 '18 at 22:55






I see those items are getting logged in the console. Here's a thread where someone captures the log. Probably easier than troubleshooting that message.

– pguardiario
Nov 15 '18 at 22:55













2 Answers
2






active

oldest

votes


















0














At the end of literature tab try



driver.find_element_by_xpath('/html/body/div/div[2]/div[2]/div[1]/div/div/div[3]/div/div/div[5]/div/ul/li[7]/a').click()





share|improve this answer























  • I still get the same error: Message: The element reference of <div class="ng-scope"> is stale; either the element is no longer attached to the DOM, it is not in the current frame context, or the document has been refreshed

    – RRg
    Nov 15 '18 at 16:10


















0














Have you tried



driver.find_element_by_css_selector('[ng-click="selectPage(page + 1, $event)"]').click()


You can improve but this works for the click (Assuming I am targeting the right Next)



from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

url = "https://ibl.mdanderson.org/fasmic/#!/"
d = webdriver.Chrome()
wait = WebDriverWait(d, 10)
d.get(url)
d.find_element_by_css_selector('[type=text]').send_keys('AKT1 (3 mutations)')
d.find_element_by_css_selector("input[type='text']").send_keys(Keys.RETURN)
btn = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, ".btn.btn-default.btn-tab-avail")))
btn.click()
d.find_element_by_css_selector("[heading=Literature]").click()

ele = wait.until(EC.text_to_be_present_in_element((By.CSS_SELECTOR, "#literature_div [ng-repeat]"), "PMID"))
eles = d.find_elements_by_css_selector("#literature_div [ng-repeat]")

d.find_element_by_css_selector('[ng-click="selectPage(page + 1, $event)"]').click()

#d.quit()





share|improve this answer























    Your Answer






    StackExchange.ifUsing("editor", function ()
    StackExchange.using("externalEditor", function ()
    StackExchange.using("snippets", function ()
    StackExchange.snippets.init();
    );
    );
    , "code-snippets");

    StackExchange.ready(function()
    var channelOptions =
    tags: "".split(" "),
    id: "1"
    ;
    initTagRenderer("".split(" "), "".split(" "), channelOptions);

    StackExchange.using("externalEditor", function()
    // Have to fire editor after snippets, if snippets enabled
    if (StackExchange.settings.snippets.snippetsEnabled)
    StackExchange.using("snippets", function()
    createEditor();
    );

    else
    createEditor();

    );

    function createEditor()
    StackExchange.prepareEditor(
    heartbeatType: 'answer',
    autoActivateHeartbeat: false,
    convertImagesToLinks: true,
    noModals: true,
    showLowRepImageUploadWarning: true,
    reputationToPostImages: 10,
    bindNavPrevention: true,
    postfix: "",
    imageUploader:
    brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
    contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
    allowUrls: true
    ,
    onDemand: true,
    discardSelector: ".discard-answer"
    ,immediatelyShowMarkdownHelp:true
    );



    );













    draft saved

    draft discarded


















    StackExchange.ready(
    function ()
    StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53322893%2fclick-on-next-button-using-selenium-until-all-pages-are-parsed-python3%23new-answer', 'question_page');

    );

    Post as a guest















    Required, but never shown

























    2 Answers
    2






    active

    oldest

    votes








    2 Answers
    2






    active

    oldest

    votes









    active

    oldest

    votes






    active

    oldest

    votes









    0














    At the end of literature tab try



    driver.find_element_by_xpath('/html/body/div/div[2]/div[2]/div[1]/div/div/div[3]/div/div/div[5]/div/ul/li[7]/a').click()





    share|improve this answer























    • I still get the same error: Message: The element reference of <div class="ng-scope"> is stale; either the element is no longer attached to the DOM, it is not in the current frame context, or the document has been refreshed

      – RRg
      Nov 15 '18 at 16:10















    0














    At the end of literature tab try



    driver.find_element_by_xpath('/html/body/div/div[2]/div[2]/div[1]/div/div/div[3]/div/div/div[5]/div/ul/li[7]/a').click()





    share|improve this answer























    • I still get the same error: Message: The element reference of <div class="ng-scope"> is stale; either the element is no longer attached to the DOM, it is not in the current frame context, or the document has been refreshed

      – RRg
      Nov 15 '18 at 16:10













    0












    0








    0







    At the end of literature tab try



    driver.find_element_by_xpath('/html/body/div/div[2]/div[2]/div[1]/div/div/div[3]/div/div/div[5]/div/ul/li[7]/a').click()





    share|improve this answer













    At the end of literature tab try



    driver.find_element_by_xpath('/html/body/div/div[2]/div[2]/div[1]/div/div/div[3]/div/div/div[5]/div/ul/li[7]/a').click()






    share|improve this answer












    share|improve this answer



    share|improve this answer










    answered Nov 15 '18 at 16:03









    Florian BordFlorian Bord

    54




    54












    • I still get the same error: Message: The element reference of <div class="ng-scope"> is stale; either the element is no longer attached to the DOM, it is not in the current frame context, or the document has been refreshed

      – RRg
      Nov 15 '18 at 16:10

















    • I still get the same error: Message: The element reference of <div class="ng-scope"> is stale; either the element is no longer attached to the DOM, it is not in the current frame context, or the document has been refreshed

      – RRg
      Nov 15 '18 at 16:10
















    I still get the same error: Message: The element reference of <div class="ng-scope"> is stale; either the element is no longer attached to the DOM, it is not in the current frame context, or the document has been refreshed

    – RRg
    Nov 15 '18 at 16:10





    I still get the same error: Message: The element reference of <div class="ng-scope"> is stale; either the element is no longer attached to the DOM, it is not in the current frame context, or the document has been refreshed

    – RRg
    Nov 15 '18 at 16:10













    0














    Have you tried



    driver.find_element_by_css_selector('[ng-click="selectPage(page + 1, $event)"]').click()


    You can improve but this works for the click (Assuming I am targeting the right Next)



    from selenium import webdriver
    from selenium.webdriver.common.keys import Keys
    from selenium.webdriver.common.by import By
    from selenium.webdriver.support.ui import WebDriverWait
    from selenium.webdriver.support import expected_conditions as EC

    url = "https://ibl.mdanderson.org/fasmic/#!/"
    d = webdriver.Chrome()
    wait = WebDriverWait(d, 10)
    d.get(url)
    d.find_element_by_css_selector('[type=text]').send_keys('AKT1 (3 mutations)')
    d.find_element_by_css_selector("input[type='text']").send_keys(Keys.RETURN)
    btn = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, ".btn.btn-default.btn-tab-avail")))
    btn.click()
    d.find_element_by_css_selector("[heading=Literature]").click()

    ele = wait.until(EC.text_to_be_present_in_element((By.CSS_SELECTOR, "#literature_div [ng-repeat]"), "PMID"))
    eles = d.find_elements_by_css_selector("#literature_div [ng-repeat]")

    d.find_element_by_css_selector('[ng-click="selectPage(page + 1, $event)"]').click()

    #d.quit()





    share|improve this answer



























      0














      Have you tried



      driver.find_element_by_css_selector('[ng-click="selectPage(page + 1, $event)"]').click()


      You can improve but this works for the click (Assuming I am targeting the right Next)



      from selenium import webdriver
      from selenium.webdriver.common.keys import Keys
      from selenium.webdriver.common.by import By
      from selenium.webdriver.support.ui import WebDriverWait
      from selenium.webdriver.support import expected_conditions as EC

      url = "https://ibl.mdanderson.org/fasmic/#!/"
      d = webdriver.Chrome()
      wait = WebDriverWait(d, 10)
      d.get(url)
      d.find_element_by_css_selector('[type=text]').send_keys('AKT1 (3 mutations)')
      d.find_element_by_css_selector("input[type='text']").send_keys(Keys.RETURN)
      btn = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, ".btn.btn-default.btn-tab-avail")))
      btn.click()
      d.find_element_by_css_selector("[heading=Literature]").click()

      ele = wait.until(EC.text_to_be_present_in_element((By.CSS_SELECTOR, "#literature_div [ng-repeat]"), "PMID"))
      eles = d.find_elements_by_css_selector("#literature_div [ng-repeat]")

      d.find_element_by_css_selector('[ng-click="selectPage(page + 1, $event)"]').click()

      #d.quit()





      share|improve this answer

























        0












        0








        0







        Have you tried



        driver.find_element_by_css_selector('[ng-click="selectPage(page + 1, $event)"]').click()


        You can improve but this works for the click (Assuming I am targeting the right Next)



        from selenium import webdriver
        from selenium.webdriver.common.keys import Keys
        from selenium.webdriver.common.by import By
        from selenium.webdriver.support.ui import WebDriverWait
        from selenium.webdriver.support import expected_conditions as EC

        url = "https://ibl.mdanderson.org/fasmic/#!/"
        d = webdriver.Chrome()
        wait = WebDriverWait(d, 10)
        d.get(url)
        d.find_element_by_css_selector('[type=text]').send_keys('AKT1 (3 mutations)')
        d.find_element_by_css_selector("input[type='text']").send_keys(Keys.RETURN)
        btn = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, ".btn.btn-default.btn-tab-avail")))
        btn.click()
        d.find_element_by_css_selector("[heading=Literature]").click()

        ele = wait.until(EC.text_to_be_present_in_element((By.CSS_SELECTOR, "#literature_div [ng-repeat]"), "PMID"))
        eles = d.find_elements_by_css_selector("#literature_div [ng-repeat]")

        d.find_element_by_css_selector('[ng-click="selectPage(page + 1, $event)"]').click()

        #d.quit()





        share|improve this answer













        Have you tried



        driver.find_element_by_css_selector('[ng-click="selectPage(page + 1, $event)"]').click()


        You can improve but this works for the click (Assuming I am targeting the right Next)



        from selenium import webdriver
        from selenium.webdriver.common.keys import Keys
        from selenium.webdriver.common.by import By
        from selenium.webdriver.support.ui import WebDriverWait
        from selenium.webdriver.support import expected_conditions as EC

        url = "https://ibl.mdanderson.org/fasmic/#!/"
        d = webdriver.Chrome()
        wait = WebDriverWait(d, 10)
        d.get(url)
        d.find_element_by_css_selector('[type=text]').send_keys('AKT1 (3 mutations)')
        d.find_element_by_css_selector("input[type='text']").send_keys(Keys.RETURN)
        btn = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, ".btn.btn-default.btn-tab-avail")))
        btn.click()
        d.find_element_by_css_selector("[heading=Literature]").click()

        ele = wait.until(EC.text_to_be_present_in_element((By.CSS_SELECTOR, "#literature_div [ng-repeat]"), "PMID"))
        eles = d.find_elements_by_css_selector("#literature_div [ng-repeat]")

        d.find_element_by_css_selector('[ng-click="selectPage(page + 1, $event)"]').click()

        #d.quit()






        share|improve this answer












        share|improve this answer



        share|improve this answer










        answered Nov 15 '18 at 17:08









        QHarrQHarr

        38.8k82245




        38.8k82245



























            draft saved

            draft discarded
















































            Thanks for contributing an answer to Stack Overflow!


            • Please be sure to answer the question. Provide details and share your research!

            But avoid


            • Asking for help, clarification, or responding to other answers.

            • Making statements based on opinion; back them up with references or personal experience.

            To learn more, see our tips on writing great answers.




            draft saved


            draft discarded














            StackExchange.ready(
            function ()
            StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53322893%2fclick-on-next-button-using-selenium-until-all-pages-are-parsed-python3%23new-answer', 'question_page');

            );

            Post as a guest















            Required, but never shown





















































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown

































            Required, but never shown














            Required, but never shown












            Required, but never shown







            Required, but never shown







            Popular posts from this blog

            Darth Vader #20

            How to how show current date and time by default on contact form 7 in WordPress without taking input from user in datetimepicker

            Ondo