Job has been attempted too many times or run too long









up vote
8
down vote

favorite














I have a job that works flawless locally, but in production I run into issues where it doesn't work. I've encompassed the entire handle() with a try/catch and am not seeing anything logged to Bugsnag, despite many other exceptions elsewhere from being deployed.



public function handle() 
try

// do stuff

catch (Exception $e)
Bugsnag::notifyException($e);

throw $e;




According to Laravel Horizon this queue job runs for 0.0026001930236816406 seconds and I never see it work and never see any other errors in the failed_jobs table as it relates to this job.



config/queue.php



 'redis' => [
'driver' => 'redis',
'connection' => 'default',
'queue' => 'default',
'retry_after' => (60 * 10), // 10 minutes
'block_for' => null,
],


config/horizon.php



'environments' => [
'production' => [
'supervisor' => [
'connection' => 'redis',
'queue' => [
'default',
],
'balance' => 'auto',
'processes' => 10,
'tries' => 3,

// 10 seconds under the queue's retry_after to avoid overlap
'timeout' => (60 * 10) - 10, // Just under 10 mins
],


If something is causing this job to retry over and over, how can I find out how? I'm at a loss.



Investigation thus far



  • My expectation is I should be able to run the query:

SELECT DISTINCT exception, COUNT(id) as errors
FROM failed_jobs
WHERE payload LIKE '%[TAG-JOB-HAS]%'
GROUP BY exception;


To see more than this error message:




Job has been attempted too many times or run too long




but that's all I see.




  • Laravel Horizon's dashboard shows the job in question is running for < 1 second, so I know it's not actually timing out.









share|improve this question



















  • 1




    Have you tried logging debug statements to prove that the job has actually been run? Can you include more information about what the job actually does, and maybe some code? It could be failing in a way that doesn't generate exceptions
    – Travis Britz
    Nov 4 at 1:52






  • 1




    "...but in production I run into issues where it doesn't work" - what doesnt work exactly? Do you see an error in the logs?
    – Laurence
    Nov 4 at 8:57










  • Just a thought, is it the Bugsnag::notifyException($e) line that is causing an exception, causing Laravel to requeue your job to retry?
    – Jamesking56
    Nov 5 at 17:27










  • What is "// do stuff"? Does it do anything weird like call exit() or die()?
    – Travis Britz
    Nov 6 at 11:28










  • Did you restart the queue after adding the try catch to the handle function?
    – Elie
    Nov 6 at 13:28














up vote
8
down vote

favorite














I have a job that works flawless locally, but in production I run into issues where it doesn't work. I've encompassed the entire handle() with a try/catch and am not seeing anything logged to Bugsnag, despite many other exceptions elsewhere from being deployed.



public function handle() 
try

// do stuff

catch (Exception $e)
Bugsnag::notifyException($e);

throw $e;




According to Laravel Horizon this queue job runs for 0.0026001930236816406 seconds and I never see it work and never see any other errors in the failed_jobs table as it relates to this job.



config/queue.php



 'redis' => [
'driver' => 'redis',
'connection' => 'default',
'queue' => 'default',
'retry_after' => (60 * 10), // 10 minutes
'block_for' => null,
],


config/horizon.php



'environments' => [
'production' => [
'supervisor' => [
'connection' => 'redis',
'queue' => [
'default',
],
'balance' => 'auto',
'processes' => 10,
'tries' => 3,

// 10 seconds under the queue's retry_after to avoid overlap
'timeout' => (60 * 10) - 10, // Just under 10 mins
],


If something is causing this job to retry over and over, how can I find out how? I'm at a loss.



Investigation thus far



  • My expectation is I should be able to run the query:

SELECT DISTINCT exception, COUNT(id) as errors
FROM failed_jobs
WHERE payload LIKE '%[TAG-JOB-HAS]%'
GROUP BY exception;


To see more than this error message:




Job has been attempted too many times or run too long




but that's all I see.




  • Laravel Horizon's dashboard shows the job in question is running for < 1 second, so I know it's not actually timing out.









share|improve this question



















  • 1




    Have you tried logging debug statements to prove that the job has actually been run? Can you include more information about what the job actually does, and maybe some code? It could be failing in a way that doesn't generate exceptions
    – Travis Britz
    Nov 4 at 1:52






  • 1




    "...but in production I run into issues where it doesn't work" - what doesnt work exactly? Do you see an error in the logs?
    – Laurence
    Nov 4 at 8:57










  • Just a thought, is it the Bugsnag::notifyException($e) line that is causing an exception, causing Laravel to requeue your job to retry?
    – Jamesking56
    Nov 5 at 17:27










  • What is "// do stuff"? Does it do anything weird like call exit() or die()?
    – Travis Britz
    Nov 6 at 11:28










  • Did you restart the queue after adding the try catch to the handle function?
    – Elie
    Nov 6 at 13:28












up vote
8
down vote

favorite









up vote
8
down vote

favorite













I have a job that works flawless locally, but in production I run into issues where it doesn't work. I've encompassed the entire handle() with a try/catch and am not seeing anything logged to Bugsnag, despite many other exceptions elsewhere from being deployed.



public function handle() 
try

// do stuff

catch (Exception $e)
Bugsnag::notifyException($e);

throw $e;




According to Laravel Horizon this queue job runs for 0.0026001930236816406 seconds and I never see it work and never see any other errors in the failed_jobs table as it relates to this job.



config/queue.php



 'redis' => [
'driver' => 'redis',
'connection' => 'default',
'queue' => 'default',
'retry_after' => (60 * 10), // 10 minutes
'block_for' => null,
],


config/horizon.php



'environments' => [
'production' => [
'supervisor' => [
'connection' => 'redis',
'queue' => [
'default',
],
'balance' => 'auto',
'processes' => 10,
'tries' => 3,

// 10 seconds under the queue's retry_after to avoid overlap
'timeout' => (60 * 10) - 10, // Just under 10 mins
],


If something is causing this job to retry over and over, how can I find out how? I'm at a loss.



Investigation thus far



  • My expectation is I should be able to run the query:

SELECT DISTINCT exception, COUNT(id) as errors
FROM failed_jobs
WHERE payload LIKE '%[TAG-JOB-HAS]%'
GROUP BY exception;


To see more than this error message:




Job has been attempted too many times or run too long




but that's all I see.




  • Laravel Horizon's dashboard shows the job in question is running for < 1 second, so I know it's not actually timing out.









share|improve this question

















I have a job that works flawless locally, but in production I run into issues where it doesn't work. I've encompassed the entire handle() with a try/catch and am not seeing anything logged to Bugsnag, despite many other exceptions elsewhere from being deployed.



public function handle() 
try

// do stuff

catch (Exception $e)
Bugsnag::notifyException($e);

throw $e;




According to Laravel Horizon this queue job runs for 0.0026001930236816406 seconds and I never see it work and never see any other errors in the failed_jobs table as it relates to this job.



config/queue.php



 'redis' => [
'driver' => 'redis',
'connection' => 'default',
'queue' => 'default',
'retry_after' => (60 * 10), // 10 minutes
'block_for' => null,
],


config/horizon.php



'environments' => [
'production' => [
'supervisor' => [
'connection' => 'redis',
'queue' => [
'default',
],
'balance' => 'auto',
'processes' => 10,
'tries' => 3,

// 10 seconds under the queue's retry_after to avoid overlap
'timeout' => (60 * 10) - 10, // Just under 10 mins
],


If something is causing this job to retry over and over, how can I find out how? I'm at a loss.



Investigation thus far



  • My expectation is I should be able to run the query:

SELECT DISTINCT exception, COUNT(id) as errors
FROM failed_jobs
WHERE payload LIKE '%[TAG-JOB-HAS]%'
GROUP BY exception;


To see more than this error message:




Job has been attempted too many times or run too long




but that's all I see.




  • Laravel Horizon's dashboard shows the job in question is running for < 1 second, so I know it's not actually timing out.






laravel queue horizon






share|improve this question















share|improve this question













share|improve this question




share|improve this question








edited Nov 9 at 14:37









HCK

2,8911829




2,8911829










asked Oct 31 at 2:11









Webnet

29.9k84245412




29.9k84245412







  • 1




    Have you tried logging debug statements to prove that the job has actually been run? Can you include more information about what the job actually does, and maybe some code? It could be failing in a way that doesn't generate exceptions
    – Travis Britz
    Nov 4 at 1:52






  • 1




    "...but in production I run into issues where it doesn't work" - what doesnt work exactly? Do you see an error in the logs?
    – Laurence
    Nov 4 at 8:57










  • Just a thought, is it the Bugsnag::notifyException($e) line that is causing an exception, causing Laravel to requeue your job to retry?
    – Jamesking56
    Nov 5 at 17:27










  • What is "// do stuff"? Does it do anything weird like call exit() or die()?
    – Travis Britz
    Nov 6 at 11:28










  • Did you restart the queue after adding the try catch to the handle function?
    – Elie
    Nov 6 at 13:28












  • 1




    Have you tried logging debug statements to prove that the job has actually been run? Can you include more information about what the job actually does, and maybe some code? It could be failing in a way that doesn't generate exceptions
    – Travis Britz
    Nov 4 at 1:52






  • 1




    "...but in production I run into issues where it doesn't work" - what doesnt work exactly? Do you see an error in the logs?
    – Laurence
    Nov 4 at 8:57










  • Just a thought, is it the Bugsnag::notifyException($e) line that is causing an exception, causing Laravel to requeue your job to retry?
    – Jamesking56
    Nov 5 at 17:27










  • What is "// do stuff"? Does it do anything weird like call exit() or die()?
    – Travis Britz
    Nov 6 at 11:28










  • Did you restart the queue after adding the try catch to the handle function?
    – Elie
    Nov 6 at 13:28







1




1




Have you tried logging debug statements to prove that the job has actually been run? Can you include more information about what the job actually does, and maybe some code? It could be failing in a way that doesn't generate exceptions
– Travis Britz
Nov 4 at 1:52




Have you tried logging debug statements to prove that the job has actually been run? Can you include more information about what the job actually does, and maybe some code? It could be failing in a way that doesn't generate exceptions
– Travis Britz
Nov 4 at 1:52




1




1




"...but in production I run into issues where it doesn't work" - what doesnt work exactly? Do you see an error in the logs?
– Laurence
Nov 4 at 8:57




"...but in production I run into issues where it doesn't work" - what doesnt work exactly? Do you see an error in the logs?
– Laurence
Nov 4 at 8:57












Just a thought, is it the Bugsnag::notifyException($e) line that is causing an exception, causing Laravel to requeue your job to retry?
– Jamesking56
Nov 5 at 17:27




Just a thought, is it the Bugsnag::notifyException($e) line that is causing an exception, causing Laravel to requeue your job to retry?
– Jamesking56
Nov 5 at 17:27












What is "// do stuff"? Does it do anything weird like call exit() or die()?
– Travis Britz
Nov 6 at 11:28




What is "// do stuff"? Does it do anything weird like call exit() or die()?
– Travis Britz
Nov 6 at 11:28












Did you restart the queue after adding the try catch to the handle function?
– Elie
Nov 6 at 13:28




Did you restart the queue after adding the try catch to the handle function?
– Elie
Nov 6 at 13:28












2 Answers
2






active

oldest

votes

















up vote
4
down vote



+75










Try to catch the exception in failed method given by laravel




/**
* The job failed to process.
*
* @param Exception $exception
* @return void
*/
public function failed(Exception $exception)

// Send user notification of failure, etc...



and check whether your default queue driver in local is sync then its expected behaviour






share|improve this answer



























    up vote
    3
    down vote













    According to documentation, you can handle job failing in two common ways:



    • using failed job events

    • using failed() method.

    In first case you can handle all jobs using Queue::failing method. You'll receive IlluminateQueueEventsJobFailed event as a parameter, and it contains exception.



    In other case you can use failed() method, it should be placed near your handle() method. You can receive Exception $exception as a parameter too.



    Example:



    public function failed(Throwable $exception)

    // Log failure



    Hope this helps.






    share|improve this answer




















      Your Answer






      StackExchange.ifUsing("editor", function ()
      StackExchange.using("externalEditor", function ()
      StackExchange.using("snippets", function ()
      StackExchange.snippets.init();
      );
      );
      , "code-snippets");

      StackExchange.ready(function()
      var channelOptions =
      tags: "".split(" "),
      id: "1"
      ;
      initTagRenderer("".split(" "), "".split(" "), channelOptions);

      StackExchange.using("externalEditor", function()
      // Have to fire editor after snippets, if snippets enabled
      if (StackExchange.settings.snippets.snippetsEnabled)
      StackExchange.using("snippets", function()
      createEditor();
      );

      else
      createEditor();

      );

      function createEditor()
      StackExchange.prepareEditor(
      heartbeatType: 'answer',
      convertImagesToLinks: true,
      noModals: true,
      showLowRepImageUploadWarning: true,
      reputationToPostImages: 10,
      bindNavPrevention: true,
      postfix: "",
      imageUploader:
      brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
      contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
      allowUrls: true
      ,
      onDemand: true,
      discardSelector: ".discard-answer"
      ,immediatelyShowMarkdownHelp:true
      );



      );













       

      draft saved


      draft discarded


















      StackExchange.ready(
      function ()
      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53075318%2fjob-has-been-attempted-too-many-times-or-run-too-long%23new-answer', 'question_page');

      );

      Post as a guest















      Required, but never shown

























      2 Answers
      2






      active

      oldest

      votes








      2 Answers
      2






      active

      oldest

      votes









      active

      oldest

      votes






      active

      oldest

      votes








      up vote
      4
      down vote



      +75










      Try to catch the exception in failed method given by laravel




      /**
      * The job failed to process.
      *
      * @param Exception $exception
      * @return void
      */
      public function failed(Exception $exception)

      // Send user notification of failure, etc...



      and check whether your default queue driver in local is sync then its expected behaviour






      share|improve this answer
























        up vote
        4
        down vote



        +75










        Try to catch the exception in failed method given by laravel




        /**
        * The job failed to process.
        *
        * @param Exception $exception
        * @return void
        */
        public function failed(Exception $exception)

        // Send user notification of failure, etc...



        and check whether your default queue driver in local is sync then its expected behaviour






        share|improve this answer






















          up vote
          4
          down vote



          +75







          up vote
          4
          down vote



          +75




          +75




          Try to catch the exception in failed method given by laravel




          /**
          * The job failed to process.
          *
          * @param Exception $exception
          * @return void
          */
          public function failed(Exception $exception)

          // Send user notification of failure, etc...



          and check whether your default queue driver in local is sync then its expected behaviour






          share|improve this answer












          Try to catch the exception in failed method given by laravel




          /**
          * The job failed to process.
          *
          * @param Exception $exception
          * @return void
          */
          public function failed(Exception $exception)

          // Send user notification of failure, etc...



          and check whether your default queue driver in local is sync then its expected behaviour







          share|improve this answer












          share|improve this answer



          share|improve this answer










          answered Nov 6 at 13:32









          Anil Kumar

          1564




          1564






















              up vote
              3
              down vote













              According to documentation, you can handle job failing in two common ways:



              • using failed job events

              • using failed() method.

              In first case you can handle all jobs using Queue::failing method. You'll receive IlluminateQueueEventsJobFailed event as a parameter, and it contains exception.



              In other case you can use failed() method, it should be placed near your handle() method. You can receive Exception $exception as a parameter too.



              Example:



              public function failed(Throwable $exception)

              // Log failure



              Hope this helps.






              share|improve this answer
























                up vote
                3
                down vote













                According to documentation, you can handle job failing in two common ways:



                • using failed job events

                • using failed() method.

                In first case you can handle all jobs using Queue::failing method. You'll receive IlluminateQueueEventsJobFailed event as a parameter, and it contains exception.



                In other case you can use failed() method, it should be placed near your handle() method. You can receive Exception $exception as a parameter too.



                Example:



                public function failed(Throwable $exception)

                // Log failure



                Hope this helps.






                share|improve this answer






















                  up vote
                  3
                  down vote










                  up vote
                  3
                  down vote









                  According to documentation, you can handle job failing in two common ways:



                  • using failed job events

                  • using failed() method.

                  In first case you can handle all jobs using Queue::failing method. You'll receive IlluminateQueueEventsJobFailed event as a parameter, and it contains exception.



                  In other case you can use failed() method, it should be placed near your handle() method. You can receive Exception $exception as a parameter too.



                  Example:



                  public function failed(Throwable $exception)

                  // Log failure



                  Hope this helps.






                  share|improve this answer












                  According to documentation, you can handle job failing in two common ways:



                  • using failed job events

                  • using failed() method.

                  In first case you can handle all jobs using Queue::failing method. You'll receive IlluminateQueueEventsJobFailed event as a parameter, and it contains exception.



                  In other case you can use failed() method, it should be placed near your handle() method. You can receive Exception $exception as a parameter too.



                  Example:



                  public function failed(Throwable $exception)

                  // Log failure



                  Hope this helps.







                  share|improve this answer












                  share|improve this answer



                  share|improve this answer










                  answered Nov 9 at 22:39









                  a_sarana

                  232112




                  232112



























                       

                      draft saved


                      draft discarded















































                       


                      draft saved


                      draft discarded














                      StackExchange.ready(
                      function ()
                      StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53075318%2fjob-has-been-attempted-too-many-times-or-run-too-long%23new-answer', 'question_page');

                      );

                      Post as a guest















                      Required, but never shown





















































                      Required, but never shown














                      Required, but never shown












                      Required, but never shown







                      Required, but never shown

































                      Required, but never shown














                      Required, but never shown












                      Required, but never shown







                      Required, but never shown







                      Popular posts from this blog

                      Ruanda

                      Makov (Slowakei)

                      Kleinkühnau