I'm not multitasking - I'm distracted

Of Django Models...

Wed, 06 Feb 2013 17:27:20 +0000

Note to self, make sure app_label matches what you put in settings.py’s INSTALLED_APPS section, or syncdb won’t create tables for your model.

class CIJob(Loop):
    """Defines the base class for a CI Job."""
    class Meta:
        app_label = 'ci_job'

INSTALLED_APPS = (
...
    'frontend.ci_job',

My wife is just as silly as I am, so when I turned 33 a few days...

Thu, 26 Apr 2012 15:52:20 +0100

My wife is just as silly as I am, so when I turned 33 a few days ago I was ambushed by soot sprites! More pics on the click through.

laughingsquid: President Obama & Star Trek’s Lt. Uhura Pose...

Wed, 04 Apr 2012 17:16:46 +0100

laughingsquid: President Obama & Star Trek’s Lt. Uhura Pose...:

In the UK I would like Leonard Nimoy to nerve pinch David Cameron..

Font Smoothing - at last!

Fri, 23 Mar 2012 15:23:10 +0000

Recently I have been using Ubuntu 12.04’s development branch and while it sometimes exhibits comedy instability (yesterday the crash reporter kept crashing!) and sometimes is just plain frustrating (I still can’t get my Nvidia graphics card + two external monitors to work smoothly in Unity) I have finally got frustrated and fixed the broken font rendering when using XFCE.

Turns out that the XFCE font configuration utility sets up some, but not all font smoothing settings. This being Linux, there are several places to look for settings and several different types of font settings to look after. Yay. The key one that it seemed to be missing is the fonts.conf / fonts/conf.d setting. What needed to happen was to enable font smoothing for everyone using the machine was get the soft links in /etc/fonts/conf.d to be:

10-antialias.conf, 10-hinting.conf, 10-hinting-full.conf

And remove any other 10- files. I only had to change one on my system.

No PCI cards for me

Thu, 02 Feb 2012 15:48:13 +0000

Looks like my server is suffering from a bit of shoddy hardware. I had previously thought that I had hardware that had poor drivers, or just bad hardware, sitting in my PCI slots, but no, I had an ASM1083 PCIe <-> PCI bridge on my motherboard and it doesn’t play nice.

Arg!

So, my expandability is down to using the single PCIe 1x slot that I have left. If I plug anything into the PCI slots the machine grinds to a halt within a few minutes. Great.

Web site development using an overlay proxy

Fri, 20 Jan 2012 17:33:00 +0000

I have just written some updates to the web site at android-build.linaro.org. It is a build service that uses Jenkins as a back end (running at https://android-build.linaro.org/jenkins) and uses Django and Apache 2 to serve up the UI that you see. The UI uses JavaScript to dynamically create some page parts by querying Jenkins using JSON and querying an API that we have crafted using Django. A simple front page change is relatively simple since you can create a new instance of the entire server and play with it. My problem was that I was going to change the build pages and needed the build history from the live server to test against.

My fist thought was that since I was just modifying some JavaScript to use a Greasemonkey script to unload the JavaScript that I was modifying from the page supplied by android-build.linaro.org and replace it with my own. This worked at first, but seemed a bit unreliable because if the functions that I wanted to modify had already run by the time Greasemonkey had replaced it with my new script, nothing would actually change. I also realised very early on that I wanted to add a new API, so what I needed was to run the Django development server to serve up the front end pages, serve up my JavaScript using a static file server and proxy in the Jenkins instance running on android-build.linaro.org. Easy right?

Turns out it is. In node.js you can write a couple of lines of code to proxy a remote HTTPS server. I just needed to make it look like http://localhost:<port>/jenkins alongside the Django development server running on the same port on the server root and serve up static files next to it as well. The code is in launchpad and pasted below. I will just mention a couple of problems I found along the way and how they were solved.

The first problem I had was that some API calls to my local Django instance didn’t return the same value as those on the real server. This is because the local development server was not a full blown instance of everything on android-build.linaro.org. Wanting to keep using the Django development server so I could debug my API changes I changed the proxy to map my new/changed API calls to my local Django server and the rest to the remote server.

The second problem I ran into was that that some parts of android-build.linaro.org are only available over HTTP and others only over HTTPS. Some on both. I had started with just an HTTPS proxy, so I added an HTTP one and mapped the connections using the requested URL. Unfortunately this didn’t work. If you visit android-builds.linaro.org you will get a nice web UI. If you visit the server that it is running on (currently an EC2 instance) you will find the standard Apache 2 “It works” page. This is what I was seeing through my proxy - the server name wasn’t being used as I was expecting. Turns out I needed to add in the option “changeOrigin: true” to the proxy settings.

So, the proxy does the following:

If a file called manifest.txt is requested, return a local one. This doesn’t exist on any remote servers yet, but doing this allows me to emulate that it does.
If a new API call is seen, send it to the local Django development server.
If a static file is requested, serve it up from the local disk (allows me to test new JavaScript).
If a relative URL starts with /build serve it up using the HTTP proxy to android-build.linaro.org (this part of the server isn’t available over HTTPS).
Everything else is served up by proxying it to android-build.linaro.org. This accounts for all API calls I am not acting on locally and all calls to the Jenkins server.

Here is the whole thing. It should be easy to modify for any similar projects.

var http = require('http'),
    httpProxy = require('http-proxy'),
    util = require('util'),
    static = require('node-static');
// Create proxy servers for each remote server we are going to talk to.
var proxy_android_build_https = new httpProxy.HttpProxy({
  target: {
    host: 'android-build.linaro.org',
    port: 443,
    https: true
  }
});
var proxy_android_build_http = new httpProxy.HttpProxy({
  target: {
    host: 'android-build.linaro.org',
    port: 80
  },
  changeOrigin: true // Required because the host machine has many virtual hosts on it.
                     // Go to the ec2 instance - you will just the Apache "It worked!" page.
});
var proxy_django = new httpProxy.HttpProxy({
  target: {
    host: 'localhost',
    port: 9000
  }
});
var proxy_static = new httpProxy.HttpProxy({
  target: {
    host: 'localhost',
    port: 10101
  }
});
// Create a node-static server instance to serve static files.
var file = new(static.Server)('/home/dooferlad/dev/android-build/linaro-android-frontend/');
require('http').createServer(function (request, response)
{
  request.addListener('end', function ()
  {
    // Serve files!
    file.serve(request, response);
  });
}).listen(10101);
// Create an HTTP server - this is the one our browser will talk to. Each
// connection to it is routed based on its URL to one of the above servers.
http.createServer(function (req, res)
{
  util.puts(req.url);
  // Special case - when a manifest.txt is requested, return a static file (this isn't enabled on server yet)
  if(req.url.match(/.*manifest.txt$/))
  {
    util.puts(">static, remap");
    req.url = req.url.replace(/.*manifest.txt/, "/static/manifest.txt");
    proxy_static.proxyRequest(req, res);
  }
  else if(   req.url.match(/^\/api\/proxy-remote-file/)
          || req.url.match(/^\/api\/test-dir-exists/))
  {
    // In the case of proxy requests, make sure that localhost isn't used (not whitelisted)
    req.url = req.url.replace(/http:\/\/localhost\//, "http://android-build.linaro.org/")
    req.url = req.url.replace(/https:\/\/localhost\//, "https://android-build.linaro.org/")
    util.puts(">local");
    proxy_django.proxyRequest(req, res);
  }
  else if(req.url.match(/^\/static\//))
  {
    util.puts(">static");
    proxy_static.proxyRequest(req, res);
  }
  // Some parts of android-build.linaro.org aren't available over https...
  else if(req.url.match(/^\/builds\/.*\/\d+\//))
  {
    util.puts(">remote (*http*)");
    proxy_android_build_http.proxyRequest(req, res);
  }
  else
  {
    util.puts(">remote (httpS)");
    proxy_android_build_https.proxyRequest(req, res);
  }
  util.puts(req.url);
  util.puts("---");
}).listen(8001);

Thinking if contact cards for JoCo Cruise Crazy II.

Thu, 29 Dec 2011 00:03:11 +0000

Thinking if contact cards for JoCo Cruise Crazy II.

Backing up backups...

Sat, 01 Oct 2011 15:59:13 +0100

Since I am paranoid about disk death (I have suffered it several times) and I have machines backing up to, well, a disk, I like to make sure that those backups don’t vanish if a disk dies or if someone deletes a file by accident. I don’t need snapshots, I just need to keep as much history as I can. I also don’t care about keeping old files that have changed - my assumption is that accidental deletions may happen, but changing a backup archive by accident is unlikely.

For this reason I have hacked together a little script to do just that. You can find it here.

So, you using like this: sync_delete_oldest.py source dest

You need rsync and unix find, the script is written in python. If your source disk dies, rsync won’t delete files from the destination disk.

Well, makes me happy anyway.

Crashplan Vs Quassel: Port Fight!

Tue, 27 Sep 2011 16:08:07 +0100

I spend my days on IRC to talk to people I work with and I find that Quassel is a great IRC client and running the back end on my server means that I can always look back at what was discussed over night if I need to. I also always have all my IRC logs wherever I am, which since a lot of important stuff is discussed on IRC is invaluable.

I am also a data security nut. I have suffered the curse of hard disks going to the great spinning magnet in the sky. I haven’t found a decent solution to the problem of keeping large quantities of data in sync between multiple computers reliably without spending a fortune, but mostly I want everything backed up. Today I thought I would give CrashPlan a go. The problem was that the GUI wouldn’t connect to the back end service that does the back up work.

So, what was going on? Both Quassel and CrashPlan default to using port 4242. CrashPlan’s GUI just said that it couldn’t connect to the back end. The back end seemed convinced that it was running (trying to stop it didn’t throw any errors about it not running). A simple “port in use” message from CrashPlan would have been useful!

Since it is relatively easy to move Quassel to another port (quasselcore -p <port number>) this was easy to solve, but debugging took more than a couple of minutes.

Adding a new column to a table in django

Mon, 19 Sep 2011 18:28:00 +0100

Well, the Django book basically covers it, but because I am new to this stuff I was left asking myself what my appname was. Turns out this is just the name of the directory containing models.py, so that is easy. Since I am using PostgreSQL I used the rather great pgadmin3 to insert the new columns I found by running manage.py sqlall <appname> rather than resorting to actually writing any SQL.

Update: South looks like it manages DB changes automatically.

Adding a raw printer to CUPS

Mon, 22 Aug 2011 17:09:05 +0100

There seem to be two options:

lpadmin -p [new queue name] -E -v [URI of printer]

For example, adding a new queue for my HP LaserJet 1200:

lpadmin -p LaserJetRaw -E -v usb://HP/LaserJet%201200?serial=00CNCF274876

The other option, which I am not sure if it actually works (not tested), is to set an option in your printer drivers. For me on Ubuntu I tried under Printing [double click on printer], Job Options, scroll to the bottom, add a new option “format” and set the value to “l”. I have seen it said that this makes CUPS recognise the job as a raw job.

Both of these are ways of (potentially) getting another machine to send a job directly to a network attached printer without the machine they are being served from from doing any processing. Since my server isn’t very fast, this may help speed up printing because there are fewer machines processing the data.

Doing more with less

Thu, 14 Jul 2011 12:29:52 +0100

Something odd is going on with my server. I upgraded it to an ASUS E35M1-M PRO based system and I have been monitoring its power usage at the plug. First off that when my Kodak ESP 9250 is in its low power state it still draws 5W with the WiFi on. I really should try connecting it to Ethernet and disabling the wireless and see if that helps, but for now just turning it off gives the best power savings! What is really strange is that if I disable GDM on the server, it uses more power than if GDM is running. My guess is that with GDM loaded the graphics card drivers tell the GPU to scale its power usage to what is required. Slightly less odd is when I log in the power usage drops even more after a while, which I expect is because my user is set up to allow the hard disks to spin down. I could use hdparm to get the disks to spin down, but since logging in is fine (it is a home server) I can’t be bothered. So there you go, if you have an AMD E350 based server, try running something graphical and see if your power usage drops!

On the joys of migrating disks

Wed, 02 Mar 2011 20:20:00 +0000

My poor old server. It is running on a Via PC2500 motherboard with a lot of disks attached to it via a SATA expansion card. Its OS disk has started making unhappy sounding noises, probably because the disk has spent a lot of time spinning down and then back up again for the last two years, so I thought trying an SSD would be fun. No mechanical parts and the OS doesn’t do much writing to the disk (just backing up my gmail account using imapsync and writing logs and using swap), so hopefully it will last a good long time.

I first booted up Acronis True Image 2011, but it displayed a warning that if a partition was bootable I would have to do stuff to it to make it work, that it didn’t document for grub, as used by Ubuntu. Next fired up Acronis Disk Director Home 2011, which gave no such warning. It copies and resizes the partitions, but doesn’t set the MBR up correctly, so now I have to dive into Ubuntu to try and fix it. Why isn’t anything easy? Mind you, at least a GUI did most of the hard work for me!

Edit: HOWTO: Restore GRUB (if your MBR is messed up) has the answer:

Start using old boot disk
Hit ‘c’ when grub loads to enter command mode
Use find (hdx,y)/ then hit [Tab] to examine the contents of disks until you find the right one - note that you (probably?) booted from hd0,y. The partition you are looking for is the boot one, which contains lots of files called vmlinuz[bla] and System.map[bla] (among others).
Type root (hdx,y) (without the trailing /)
Type setup (hdx)

If that worked you should be able to reboot and boot from the new disk.

After this, SSD #2 is going to find its way into my DVR box (still running Windows XP, with SageTV for DVR duties) to avoid the unresponsive while disk spinning up problem. Acronis normally does a perfectly good job with Windows disks, so hopefully it will be a bit smoother.

Time for a slightly pointless iTunes update? Hopefully it will bring better iPods!

Tue, 22 Feb 2011 17:33:38 +0000

Time for a slightly pointless iTunes update? Hopefully it will bring better iPods!:

Apple may be upping the quality of tracks on iTunes to 24 bit / 96KHz. Dull. Better quality sound out of iPods is more interesting though. Don’t really care about sample rates, but better DACs and amplifiers are always welcome.

Of course, if we used the entire dynamic range of 16 bit / 44KHz tracks things would sound better anyway on decent kit!

Who is trampling my memory?

Tue, 22 Feb 2011 17:19:26 +0000

I think it must be a common problem for C++ programmers. Something is corrupting memory and valgrind <program> isn’t helping. I am currently trying the experimental Valgrind ptrcheck and the Microsoft Application Verifier. I am hoping one of them turns something up. It is times like this that I wish I was programming in a VM!