Posts

  • Companion Objects Make a Mockery of Testing Case Classes

    Checking scala outputs by comparing them to case class instances in the test can be tempting, but it’s too easy to accidentally turn them into mocks by introducing companion objects.

  • Elasticsearch Bulk updates with NOOP

    Sometimes, you want to throw a bunch of data at an existing Elasticsearch index and let the database tell you what has changed. The magic words are “update” and “doc_as_upsert”

  • Class Properties in Python

    During a code review, I spotted this interesting misunderstanding, which could have caused quite a lot of unexpected trouble miles away from the actual change.

  • Patching Methods in Python

    Today, a colleague was adding tests to an old and previously untested class. One problem was that the __init__ method was long and made several calls to set up connections with remote services etc. He asked me how to go about adding tests in this situation, so I made a quick example.

  • Animating Anaglyphs

    The trouble with 3D anaglyph images is that I don’t have a pair of blue/red spectacles. However, given a stereoscopic image, you can turn it into an animated gif using Wiggle Stereoscopy to give a bit of a 3D effect.

  • Wildcards in Elasticsearch Phrase Searches

    A few times now, I have been asked to add wildcards to an exact phrase (match_phrase) search in an application based on Elasticsearch. It is not a simple matter of enabling a feature, but requires a different kind of query, which can become quite complex.

  • A Generative Image Twitter bot with Glitch, Pillow and Tracery

    @RandomTartan is a Twitter bot that posts a randomly generated tartan image every hour. It is driven by CheapBotsDoneQuick and Glitch, and uses Pillow.

  • Asciifolding and unicode normalisation

    I was working on simplifying some Elasticsearch queries, and I discovered that asciifolding was not working on a particular field. However, it was not so simple. Not only could I not find records with accented characters using their folded version in a query, I could not find them with the accented version in the query either.

  • Gifitup - Falling Blossom

    I have entered this year’s Gifitup with an animation based on a Japanese woodblock print Shinobugaoka no tsuki-Gyokuensai, partly generated using Pillow and NumPy. Animating with Python allowed me to experiment quickly with different framerates and step sizes.

  • Motivators

    As the anniversary of my redundancy from my previous workplace approaches, I have been thinking about workplace motivation. A blog post entitled The four motivators influenced some of my thinking at the time.

  • Radon and the Jenkins warnings plugin

    The Warnings plugin for Jenkins is a very versatile tool for reporting on the health of a job. Radon is a useful tool for analysing a Python codebase. Here’s how they can play together.

  • profileit

    Timers, like the timeit module, or a timing decorator can help to find candidates for optimisation. Proper profiling is better, but it can sometimes be difficult to isolate the specific area in order to run the cProfile module. A decorator can help.

  • Jenkins connectivity check

    One aspect of ensuring that a deployment will go smoothly, is ensuring that it can happen at all. If you are not yet in a position to practice continuous deployment, the process around a failure can be unpleasantly heavy.

  • Using Python requests to verify proxy settings

    Using requests.utils, you can check whether a connection to a given URL will go through a proxy.

  • Python locals()

    I recently encountered some python code that made frequent use of locals(), in a way that I considered bad form. It offers no performance benefit, and makes the code less readable.

  • Ansible Smoke Testing

    An advantage of declarative code like (most of) Ansible, is that it is its own test suite.
    If the state of the host can’t be made to match your definitions, it’s a failure.
    But what if your definitions are wrong?

  • Ansible group_vars load order

    The order of precedence for Ansible vars is defined in the Ansible documentation under Variable precedence, but what if two vars have the same level of precedence according to the list?

  • Verifying Ansible inventories part two

    You can build up the pattern for verifying a naming convention, using fragments specified in group_vars.

  • Ensuring Ansible Tasks are Safe to Run

    An important factor in testing definitions before deploying, is that they do not require a connection to the hosts they would deploy to. However, there is no guarantee that that is the case. Someone could simply add a ‘check_vars’ tag to a task that modifies the host and the tests are worthless. Something is needed to guard against such an eventuality.

  • Verifying Ansible Inventories

    All your roles do exactly what they are supposed to, all your vars are correct. What could go wrong? Your inventory may contain incorrect entries.

  • Keep Deployment Keys out of the Codebase.

    This may sound like obvious advice, but with vaults, Ansible provides an excellent system to allow you to keep sensitive data encrypted in your repository, so that they can be versioned alongside the rest of your definitons, and you don’t have to find somewhere else to put them.

  • Checking vars in Ansible.

    Ansible is great, but as with anything to do with deployment configuration, there are some things you can only test when you actually use it to deploy to a given environment. You can reduce the number of surprises awaiting a production deploy, by checking vars without needing to connect to the target host.

subscribe via RSS