Fixing when queue workers keep popping the same job off a queue

My (Laravel) project uses a queue system for importing because these jobs can take a fair amount of time (up to an hour) and I want to have them run asynchronously to prevent my users from having to sit and watch a spinning ball.

I created a cron job which would run my Laravel queue work command every 5 minutes. PHP is not really the best language for long-running processes which is why I elected to rather run a task periodically instead of listening all the time. This introduced some latency (which I could cut down) but this is acceptable in my use case (imports happen once a month and are only run manually if an automated import fails).

The problem that I faced was that my queue listener kept popping the same job off the queue. I didn't try running multiple listeners but I'm pretty confident weirdness would have resulted in that case as well.

Fixing the problem turned out to be a matter of configuring the visibility time of my queue. I was using the SQS provided default of 30 seconds. Amazon defines the visibility timeout as:

The period of time that a message is invisible to the rest of your application after an application component gets it from the queue. During the visibility timeout, the component that received the message usually processes it, and then deletes it from the queue. This prevents multiple components from processing the same message. Source: Amazon documentation

This concept is common to queues and exists in various names in Beanstalk and others. In Beanstalk the setting is called time-to-run and IronMQ refers to it as timeout. So if your run time exceeds your queue availability timeout then workers will pop off a job that is currently being run in another process.

Comments

Separating business logic from persistence layer in Laravel

There are several reasons to separate business logic from your persistence layer. Perhaps the biggest advantage is that the parts of your application which are unique are not coupled to how data are persisted. This makes the code easier to port and maintain. I'm going to use Doctrine to replace the Eloquent ORM in Laravel. A thorough comparison of the patterns is available here . By using Doctrine I am also hoping to mitigate the risk of a major version upgrade on the underlying framework. It can be expected for the ORM to change between major versions of a framework and upgrading to a new release can be quite costly. Another advantage to this approach is to limit the access that objects have to the database. Unless a developer is aware of the business rules in place on an Eloquent model there is a chance they will mistakenly ignore them by calling the ActiveRecord save method directly. I'm not implementing the repository pattern in all its ...

Using Azure Active directory as an OAuth2 provider for Django

Azure Active Directory is a great product and is invaluable in the enterprise space. In this article we'll be setting it up to provide tokens for the OAuth2 client credentials grant. This authorization flow is useful when you want to authorize server-to-server communication that might not be on behalf of a user. This diagram, by Microsoft, shows the client credentials grant flow. From Microsoft documentation The flow goes like this: The client sends a request to Azure AD for a token Azure AD verifies the attached authentication information and issues an access token The client calls the API with the access token. The API server is able to verify the validity of the token and therefore the identity of the client. The API responds to the client Setting up Azure AD as an OAuth2 identity provider The first step is to create applications in your AD for both your API server and the client. You can find step-by-step instructions on how to register the applications o...

"Word of the Day" PHP script (with word list)

I was looking around for a way to generate a word of the day on the web and didn't find anything. So I coded a quick and dirty script to do it. Just in case anybody does a Google search and manages to find my blog: here is my Word of the Day PHP script : Copy this code snippet into a wordoftheday.php file: $file = fopen("interesting_words.txt","r"); $raw_string = fread($file,filesize("interesting_words.txt")); fclose($file); $words_array = explode("|",$raw_string); echo $words_array[array_rand($words_array)]; Of course the real issue I had was finding a list of interesting words in the right format. Here is the list of interesting words that I used: Copy this into a file called interesting_words.txt : ubiquitous : being or seeming to be everywhere at the same time; omnipresent| ecdysiast : a striptease artist| eleemosynary : of, relating to, or dependent on charity| gregious : c...

Adventures in IT

Search This Blog