Adventures in IT

I'm using Varnish to cache the frontend of a site that a client is busy promoting.  It does a great job of reducing requests to my backend but is prone to random crashes.  I normally get about two weeks of uptime on this particular server, which is significantly lower than other places that I've deployed Varnish. I just don't have enough information to work with to try and solve why the random crash is occurring.  The system log shows that a child process doesn't respond to CLI and so is killed.  The child never seems to be able to be brought up again. My /var/log/messages file looks like this: 08:31:45 varnishd[7888]: Child (16669) not responding to CLI, killing it. 08:31:45 varnishd[7888]: Child (16669) died signal=3 08:31:45 varnishd[7888]: child (25675) Started 08:31:45 varnishd[7888]: Child (25675) said Child starts 08:31:45 varnishd[7888]: Child (25675) said SMF.s0 mmap'ed 1073741824 bytes of 1073741824 08:32:19 varnishd[7888]: Child (2...

From Amazon Queue documentation My (Laravel) project uses a queue system for importing because these jobs can take a fair amount of time (up to an hour) and I want to have them run asynchronously to prevent my users from having to sit and watch a spinning ball. I created a cron job which would run my Laravel queue work command every 5 minutes.  PHP is not really the best language for long-running processes which is why I elected to rather run a task periodically instead of listening all the time.  This introduced some latency (which I could cut down) but this is acceptable in my use case (imports happen once a month and are only run manually if an automated import fails). The problem that I faced was that my queue listener kept popping the same job off the queue.  I didn't try running multiple listeners but I'm pretty confident weirdness would have resulted in that case as well. Fixing the problem turned out to be a matter of configuring the visibility time of m...

Hiphop has a built-in admin server that has a lot of useful functions.  I found out about it on an old post on the Hiphop blog . Since those times Hiphop has moved towards using an ini file instead of a config.hdf file. On a standard prebuilt HHVM on Ubuntu you should  find the ini file in /etc/hhvm/php.ini Facebook maintains a list of all the ini settings on Github . It is into this file that we add two lines to enable the admin server: hhvm.admin_server.password = SecretPassword hhvm.admin_server.port = 8888 I then added a server to Nginx by creating this file: /etc/nginx/conf.d/admin.conf (by default Nginx includes all conf files in that directory): server { # hhvm admin listen 8889; location ~ { fastcgi_pass 127.0.0.1:8888; include fastcgi_params; } } Now I can run curl 'http://localhost:8889' from my shell on the box to get a list of commands. Because I host this project with Amazon and have not set up a ...

Postgres has two very useful extensions - earthdistance and postgis .  PostGIS is much more accurate but I found earthdistance to be very easy to use and accurate enough for my purpose (finding UK postcodes within a radius of a point). To install it first find your Postgres version and then install the appropriate package.  On my Debian Mint dev box it looks like the below snippet. My production machine is an Amazon RDS and you can skip this step in that environment. psql -V sudo apt-get install postgresql-contrib postgresql-contrib-9.3 sudo service postgresql restart Having done that you should launch psql and run these two commands.  Make sure that you install cube first because it is a requirement of earthdistance. CREATE EXTENSION cube; CREATE EXTENSION earthdistance; Now that the extensions are installed you have access to all of the functions they provide. If you want to check that they're working you can run SELECT earth(); as a quick wa...

I'm busy on a project which involves importing fairly large datasets of about ~3.3GB at a time.  I have to read a CSV file, process each line, and generate a number of database records from the results of that process. Users are expected to be able to rerun batches and there is overlap between different datasets.  For example: the dataset of "last year" overlaps with the dataset of "all time".  This means that we need an elegant way to handle duplicate updates. Searching if a record exists (by PK) is fine until the row count in the table gets significant.  At just over 2 million records it was taking my development machine 30 seconds to process 10,000 records.  This number steadily increased as the row count increased. I had to find a better way to do this and happened across the option of using a database rule to ignore duplicates.  While using the rule there is a marked improvement in the performance as I no longer need to search the database fo...

I am coding a queue worker that is handling some pretty large (2gig+) datasets and so wanted some details in my logs that Vanilla laravel didn't offer. Reading the documentation at  http://laravel.com/docs/4.2/errors wasn't much help until I twigged that I could manipulate the log object returned by  Log :: getMonolog ( ) ; . Here is an example of adding memory usage to Laravel logs. In app/start/global.php make the following changes Log::useFiles(storage_path().'/logs/laravel.log'); $log = Log::getMonolog(); $log->pushProcessor(new Monolog\Processor\MemoryUsageProcessor); You'll find the Monolog documentation on the repo

On my desktop machine I run my package upgrades every day.  The other day my Hiphop version got updated to 3.6.0 and suddenly my Postgres support died. Running Hiphop gave a symbol not found error in the postgres.so file ( undefined symbol: _ZTIN4HPHP11PDOResourceE\n ) exactly like the issue reported on the driver repository ( here ). I tried to recompile the postgres driver against Hiphop 3.6.0 but hit a number of problems, mostly to do with  hhvm-pgsql-master/pdo_pgsql_statement.cpp it seems. The fix for the incompatibility was unfortunately rolling back to my previous version of Hiphop.  To do this on Mint/Ubuntu just do this: Run cat /etc/*-release to get your release information Download the appropriate package for your distro from http://dl.hhvm.com/ubuntu/pool/main/h/hhvm/ Remove your 3.6.0 installation of hhvm: sudo apt-get remove hhvm Install the package you downloaded : sudo dpkg -i <deb package> After that everything should be ...