Fixing when queue workers keep popping the same job off a queue

My (Laravel) project uses a queue system for importing because these jobs can take a fair amount of time (up to an hour) and I want to have them run asynchronously to prevent my users from having to sit and watch a spinning ball.

I created a cron job which would run my Laravel queue work command every 5 minutes. PHP is not really the best language for long-running processes which is why I elected to rather run a task periodically instead of listening all the time. This introduced some latency (which I could cut down) but this is acceptable in my use case (imports happen once a month and are only run manually if an automated import fails).

The problem that I faced was that my queue listener kept popping the same job off the queue. I didn't try running multiple listeners but I'm pretty confident weirdness would have resulted in that case as well.

Fixing the problem turned out to be a matter of configuring the visibility time of my queue. I was using the SQS provided default of 30 seconds. Amazon defines the visibility timeout as:

The period of time that a message is invisible to the rest of your application after an application component gets it from the queue. During the visibility timeout, the component that received the message usually processes it, and then deletes it from the queue. This prevents multiple components from processing the same message. Source: Amazon documentation

This concept is common to queues and exists in various names in Beanstalk and others. In Beanstalk the setting is called time-to-run and IronMQ refers to it as timeout. So if your run time exceeds your queue availability timeout then workers will pop off a job that is currently being run in another process.

Comments

Separating business logic from persistence layer in Laravel

There are several reasons to separate business logic from your persistence layer. Perhaps the biggest advantage is that the parts of your application which are unique are not coupled to how data are persisted. This makes the code easier to port and maintain. I'm going to use Doctrine to replace the Eloquent ORM in Laravel. A thorough comparison of the patterns is available here . By using Doctrine I am also hoping to mitigate the risk of a major version upgrade on the underlying framework. It can be expected for the ORM to change between major versions of a framework and upgrading to a new release can be quite costly. Another advantage to this approach is to limit the access that objects have to the database. Unless a developer is aware of the business rules in place on an Eloquent model there is a chance they will mistakenly ignore them by calling the ActiveRecord save method directly. I'm not implementing the repository pattern in all its glory in this demo.

Fixing puppet "Exiting; no certificate found and waitforcert is disabled" error

While debugging and setting up Puppet I am still running the agent and master from CLI in --no-daemonize mode. I kept getting an error on my agent - ""Exiting; no certificate found and waitforcert is disabled". The fix was quite simple and a little embarrassing. Firstly I forgot to run my puppet master with root privileges which meant that it was unable to write incoming certificate requests to disk. That's the embarrassing part and after I looked at my shell prompt and noticed this issue fixing it was quite simple. Firstly I got the puppet ssl path by running the command puppet agent --configprint ssldir Then I removed that directory so that my agent no longer had any certificates or requests. On my master side I cleaned the old certificate by running puppet cert clean --all (this would remove all my agent certificates but for now I have just the one so its quicker than tagging it). I started my agent up with the command puppet agent --test whi

Preventing Directory Traversal attacks in PHP

Directory traversal attacks occur when your program reads or writes a file where the name is based on some sort of input that can be maliciously tampered with. When used in conjunction with log poisoning this can lead to an attacker gaining remote shell access to your server. At the most simple it could be to include a file like this: echo file_get_contents($_GET['sidebar']); The intention would be for you to be able to call your URL and send a parameter indicating which sidebar content you want to load... like this: http://foo.bar/myfile.php?sidebar=adverts.html Which is really terrible practice and would not be done by any experienced developer. Another common place where directory traversal attacks can occur is in displaying content based on a database call. If you are reading from or writing to a file based on some input (like GET, POST, COOKIE, etc) then make sure that you remove paths . The PHP function basename will strip out paths and make sure that y

Adventures in IT

Search This Blog