Ben Dodson

Freelance iOS, macOS, Apple Watch, and Apple TV Developer

Finding the next available driving test with Goutte PHP Scraper

Over the past week I’ve had a lot of fun working on some small scripts to aid my general day-to-day living. My wife is currently waiting to take her driving test but our local area only has one test instructor with a waiting list of over 6 months! Occassionally slots open up due to cancellations and so she has been logging into the Government website multiple times a day to check for a nearer date. Time for some automation!

To stop her having to constantly go to a website, I decided to knock up a quick PHP script that would basically pretend to be a browser, log in, go to the correct page, and rip out the next available appointment. It’s been a while since I wrote a script like this so I had a look around for something better than SimpleXML and found Goutte, a simple PHP Web Scraper. Within 10 minutes, my script was working. It looks something like this:

<?php

require 'vendor/autoload.php';

use Goutte\Client;

$licenseNumber = 'xxx'; // Driving License Number
$referenceNumber = 'xxx'; // Reference number for current test date as looking to rebook

$client = new Client();

$crawler = $client->request('GET', 'https://driverpracticaltest.direct.gov.uk/login');
$crawler->filter('h2')->each(function($node) {
	if ($node->text() == "Sorry, you can’t use this service right now") {
		echo 'Site is down.';exit; // Inexplicably, the site goes offline from 9pm-6am. It thinks it's an office!
	}
});

$form = $crawler->selectButton('Continue')->form();
$crawler = $client->submit($form, array('username' => $licenseNumber, 'password' => $referenceNumber));
$link = $crawler->selectLink('Change Date and time of test')->link();
$crawler = $client->click($link);
$form = $crawler->selectButton('Find available dates')->form();
$crawler = $client->submit($form, array('testChoice' => 'ASAP'));

$crawler->filter('.slotDateTime')->each(function ($node) {
	$date = $node->text();
	echo $date.' is the next available test slot.';
	// Here I check if it has changed since last time and send an email if it has and nearer than currently booked test. Left out for brevity.
    exit;
});

?>

As you can see, the code is pretty simple, readable, and powerful. Within minutes, I was able to jump through 3 pages without worrying about sessions, cookies, or any of the other nonsense that often occurred when dealing with this kind of scraping. I run this every 5 minutes on a FortRabbit server and we both get emailed if a closer test date appears. For just a few minutes work, this means better peace of mind as we don’t have to check manually and this checks far more frequently than we ever could. Now to wait and hope for a nearer test date…

TransferWise

I’ve been meaning to write about TransferWise for a little while but there new support for Apple Pay put it back in the front of my mind. TransferWise is a service that transfers money for you into multiple currencies at a fraction of the cost that your bank would usually charge. I’d seen it advertised a few times on Facebook but didn’t take too much notice as I didn’t really make any international transfers; that stopped when we hired Robbie at WallaBee last year. I quickly found that my bank was not only charging me with exhorbitant conversion rates and transfer charges but that the recipient was also being charged by their bank for accepting an international transfer!

The way TransferWise works is that it lets you choose a specific amount of money to transfer (either in the original or converted currency1) and then it tells you how much it thinks it will cost. You then pay that amount (plus a small buffer to account for fluctuating bank transfers) direct to TransferWise by card or bank transfer. They then do the conversion and pay the recipient from a local account in their own currency so they don’t get hit with international charges. Once it is all done, you get a refund for any of the amount that wasn’t used for the conversion. The first few times the transfer took around 5 working days but now my transfers are done within 24 hours.

I’ve saved over £500 in fees over the past 6 months and couldn’t be happier with the service. I tend to use the website but they have a great iOS app which has recently been updated to accept Apple Pay. If you are making any sort of payments overseas, I’d strongly recommend using TransferWise rather than your standard banking transfers. If you sign up with this link, you’ll get a fee-free transfer of up to £3000.

  1. This is an important distinction as when paying an invoice I need to pay in a specific amount of USD so the amount of GBP will change. There may be situations where you always want send a specific amount of GBP and the USD will then be the thing that varies based on currency conversion. Obviously they support more than just USD / GBP but that’s all I use currently. ↩︎

UK Apple Pay

I’ve been excited about Apple Pay ever since it was first announced for the US last year. It finally arrived in the UK today and so I’ve made a few purchases in a selfless act of testing! There has been quite a bit of confusion and misinformation about how it would work in the UK so hopefully I’ll clear up some issues there as well as giving my opinion on how Apple Pay works.

First of all, it is insanely easy to setup. You just open up the Passbook app on your iPhone1 and a new “Pay” option will appear above your passes. When you add a card, Apple will automatically default to the card you have on file for iTunes purchases or you can add a new one via the camera. Some banks have a security step and I was offered the choice of calling RBS or having them text me a security code; the code came through instantly and everything activated within seconds. Unfortunately neither my credit card company nor my business bank are using Apple Pay yet so I could only add my personal account card.

If you have an Apple Watch, you need to repeat this step in the “Passbook & Apple Pay” section of the Apple Watch app. The reason for this is that your card information is stored securely within the hardware of the device, not in software (so there is no iCloud syncing or anything like that).

When it comes to paying, you can use Apple Pay at every single shop in the UK that has a contactless card reader. When paying, you simply say you are paying by card and if the amount is under £202 the contactless section will light up and you can then place your iPhone near the reader. Your iPhone will automatically come out of standby mode, show your card, and ask you to hold the fingerprint scanner on the home button; within a second it’ll buzz and the payment is done. If paying with the Apple Watch you have to double-tap the side button to bring up your cards and then hold it against the scanner (for the simple reason that the NFC chip isn’t always on as it’d kill the battery). Once the payment is done, an unexpected nice touch is that you get a push notification telling you how much you paid and where - useful for making sure that you paid the correct amount. I tried this in W H Smiths, which is not an official “Apple Pay” partner, and it all went through fine. Your iPhone (or Apple Watch) is simply pretending to be a contactless card so if your card works, then Apple Pay will work.

The main piece of misinformation about Apple Pay in the UK has been this £20 limit; “Apple Pay limited to £20” is a frequent headline but it isn’t actually true. If a launch partner (like Marks & Spencer) supports full Apple Pay, then you can make a purchase of over £20 using the same system as above. The difference is that they have upgraded their hardware so that the contactless part will work specifically with Apple Pay rather than just being the standard contactless payment system. I tested this by buying £35 of stuff at M&S and it all worked flawlessly. The real issue is that “Apple Pay” encompasses both the “pretending to be regular contactless” and “being an upgraded contactless system” and it isn’t easy to tell which a retailer supports. I think if you see an Apple Pay logo on the reader (like I did at McDonalds), then they support the full no transaction limit Apple Pay.

After successfully paying with both my iPhone and Apple Watch at multiple stores, I decided to give the in app payment system a try. This is live for several apps (highlighted in the App Store currently) and basically allows you to make payments in app using Apple Pay. This is limited to physical goods and services3 so I bought some stuff from Etsy, supported a project on Kickstarter, and bought a new Apple Watch strap from the Apple Store app. In all cases, the process was insanely simple; during the checkout process, I was never asked for my address or card details but a simple “Apple Pay” button bought up a sheet which showed the address my card is issued to and let me pay by holding my fingerprint on the home button.

Overall, the whole process has been simple and fast. A lot of people wonder why bother using this when you can already use a contactless card? For me, it boils down to two reasons: simplicity and security. With my Apple Watch on, I didn’t have to take my wallet out of my pocket; whilst this isn’t a mindblowing change (only takes a second), it is an added level of convenience and means I can make purchases if I leave my wallet at home (or if I’m somewhere sketchy and don’t want to take it out). The real benefit though is in security. When you make a payment, the retailer never gets your card number; Apple Pay gives a randomly generated number every time you buy something which means retailers can’t track you across stores or link you to payments. I was able to check this by looking at my receipts and each time the last 4 digits they displayed of my card number were different. This may be a minor thing to a lot of people but this increase in privacy and the convenience of making payments (some of them over £20) from my wrist mean that this is a big deal to me.

The real test will be when I’m in London next and can try it out in place of an Oyster card4.

  1. Only iPhone 6 and 6 Plus are supported for Apple Pay via contactless payment. You can use iPad Air 2 and iPad mini 3 for in-app payments. ↩︎

  2. This is increasing to £30 for all contactless payments in September. ↩︎

  3. Obviously you can already pay for in-app purchases like extra game levels or subscriptions using TouchID which works essentially the same way but charges your card on your iTunes account. ↩︎

  4. I’ve seen a lot of comments about “the readers are on the right so how will I use it if I wear my watch on my left hand like 90% of the population”. I’ve got this clever trick called “moving your arm across your body”… worked like a charm at the McDonalds payment terminal. ↩︎

Forcing UIStatusBarStyle for UIImagePickerController

I stumbled upon an interesting problem today which I’ve experienced a couple of times before; if you are using white text for the status bar in your application (UIStatusBarStyle.LightContent) then it will be turned black once you present a UIImagePickerController. This can not only look ugly but will remain when you dismiss the controller. Usually I just change the appearance of the navigation bar within the picker to get around this (and set it back to the correct colour upon dismissing the controller) but on this particular app I wanted to force the UIImagePickerController to use the light style status bar. Fortunately it is quite easy.

When you use the UIImagePickerControllerDelegate, you are also required to adopt the UINavigationControllerDelegate. This has a method named navigationController:willShowViewController:animated: which will be called both when you present the controller and when you navigate between albums. Using this, it is trivial to force the status bar to remain the same1:

func navigationController(navigationController: UINavigationController, willShowViewController viewController: UIViewController, animated: Bool) {
	UIApplication.sharedApplication().statusBarStyle = .LightContent
}
  1. This is of course assuming you have View controller-based status bar appearance set to NO in your Info.plist file so that you can use UIApplication to change the status bar. ↩︎

Optical media and the pain of Remote Disc

I hate optical media. I avoid it as much as possible with streaming or digital versions of music and films but sometimes it is unavoidable. An album I wanted1 recently wasn’t available on iTunes or any of the other online stores so I had to give in and buy a physical version on Amazon. Unfortunately this didn’t give me a free MP3 version as it often does2 which left me with a slight issue as neither my iMac nor MacBook have a CD drive.

I didn’t think this would be a big problem as my wife has an older MacBook with an optical drive so I’d be able to use Remote Disc to access the drive. Remote Disc is a useful feature that lets you access the optical drive on another computer via your network and has been useful for me in the past if I’ve had some data lying around on an old disc. Unfortunately, it didn’t work at all for this as Remote Disc is locked to prevent the streaming of audio discs (or movie DVDs). To make matters worse, Remote Disc doesn’t straight up tell you this but instead gives you a cryptic message about the drive being unavailable. I can see no logical reason for this but that is still the state of play in 2015.

In the end, I had to rip the CD on my wife’s MacBook, then copy the files back to my computer. Crazy.

  1. The album, in case you were wondering, was the original Broadway version of “La Cage Aux Folles” ↩︎

  2. In ridiculous circumstances, it is often cheaper to buy a physical CD from Amazon than it is to buy the MP3 version! They sometimes give you the digital version free with the physical so I bought “Bright Idea” by Orson and just recycled the Amazon package that arrived without opening. ↩︎

Adding Exercise Data To Activity

I’ve been using the health features of the Apple watch a lot recently but today is the first time it has stopped me from doing a specific exercise. I was planning on going for a swim but obviously I don’t want to do that with my watch on. I assumed I’d be able to either enter the workout directly into the Health app (or use a 3rd party app to add the data to HealthKit) and that would then count as part of my 30 minutes of exercise on the Apple Watch; unfortunately this is not the case.

I’ve tried with manual data entry and 3rd party apps1 but there is no way for workout data not created on the Apple Watch to be used as part of your Activity… yet. On checking the WWDC keynote, Kevin Lynch mentions that this feature is coming as part of watchOS 2. Specifically he says that “workouts with these [native on the Apple Watch] apps will count” - I’m assuming that any app that uses HealthKit will actually count but we’ll see. I’m not going to install a potentially buggy watchOS 2 on my Apple Watch one week before Apple Pay comes to the UK!

  1. I even built my own basic app to add data into HealthKit with the “user entered” flag set to false in the hope that this was the piece that meant the Apple Watch ignored it. ↩︎

Since, an NSDate Extension in Swift

I’ve just committed a very simple NSDate extension that gives you a human readable version of the time difference between the date and now (i.e. “2 hours ago”, “just now”, or “last week”). I needed this functionality in a project I’m currently working on but existing solutions I found were all Objective-C and fairly complex. I try not to add any complex 3rd party dependencies when I don’t have to (especially as I don’t want an Objective-C bridging header) so I built this in a few minutes to give me the basic functionality I required. You can check it out on GitHub.

In many ways, this sums up how I feel about some development today. Many developers will either embed a 3rd party solution without thinking about it (giving you licensing and support issues in the future) or they’ll spend hours crafting the perfect solution that will give you to-the-second precision with unit tests and a whole host of other stuff you probably don’t need1. Sometimes you just need the simplest thing that works so you can move onto more important things.

  1. An oft-mentioned story about Steve Jobs regards how his father would spend as much time making the back of a cabinet (which you’d never see) look as good as the front as a sign of good craftsmanship. This has its place but its important to remember that “real artists ship”; I’d rather have a project that is live and performing than spend hours crafting the “perfect” piece of code. ↩︎

Using an SSD with the Xbox One

I love my Xbox One but the one big issue I have with it is loading times. This is mainly due to the spinning hard drive1 inside and the huge amount of data it is expected to push for some of the bigger open world games like GTA V or The Witcher 3. In an effort to speed this up (and because the 500GB hard drive is almost full), I’ve upgraded my Xbox One with an external SSD2.

Process

The process is actually incredibly simple. You’ll need the following two things:

Once you have these, it’s a simple case of plugging the SSD into the enclosure and attaching the USB cable. This leaves you with an incredibly small device which can then be plugged into any of the 3 USB 3.0 ports on the Xbox One:

As soon as you turn the Xbox One on, you’ll be alerted that a new media device has been attached and given the option to format it for games and give it a name. When you have done this, you’ll be able to copy games and apps by going to their individual storage settings and choosing the move option.

Results

I’ve only put 3 games on the SSD so far but they are all dramatically faster for loading and saving. I’ve put a few benchmarks below:

GTA V

This is by far my slowest game. Testing was from a cold startup (which automatically loads your previous save point) to the point at which the game was playable:

  • Internal HDD - 2 minutes 31 seconds
  • External SSD - 1 minute 9 seconds

SSD produces a 60% reduction in loading time.

The Witcher 3: Wild Hunt

The first test was from a cold start to the opening menu.

  • Internal HDD - 40s
  • External SSD - 28s

SSD produces a 30% reduction in loading time (although that seemed to be as quick as it could go due to the logos and publisher videos at startup).

The second test was from resuming a game from the main menu.

  • Internal HDD - 1 minute 21 seconds
  • External SSD - 46 seconds

SSD produces a 57% reduction in loading time.

LEGO Jurassic World

For this test, I timed from the loading of the last save point on the main menu.

  • Internal HDD - 37 seconds
  • External HDD - 37 seconds

It looks like LEGO Jurassic World’s loading sequence is limited to 37 seconds in order to show 3 pieces of dinosaur trivia at 12 seconds each. Playing through the game though, the time it takes to start a story and fast travel to new locations seems much faster.

Overall, it seems that the SSD offers around a 55-60% reduction in speed unless there is something specifically limiting the load time (i.e. videos or placeholder content). I’m planning on just using the external drive for single player games at the moment as there isn’t much benefit in loading the Titanfall or Halo online textures any quicker when the limiting factor is the broadband connection along with everyone else. For me, the SSD option is definitely worth the cost for the big open world games that have a heavy loading time penalty when you die (looking at you GTA V) - it has also increased my overall storage space by 50% which is useful when some games are in the 100s of GB.

  1. Gross! One day I’ll look forward to telling my kids we used to store data on discs spinning at 7200rpm - it’ll be baffling to them. ↩︎

  2. As far as I’m aware, doing it internally would be slightly faster but would definitely invalidate any warranty! ↩︎

  3. The first enclosure I got was the EC-UKMS from Sabrent which was advertised as having UASP. It doesn’t and led to a 70% decrease in speed over the internal HDD with a load time of over 4 minutes for GTA V! This is the first product I’ve actually left a review for on Amazon as the title and images are blatent lies. ↩︎

Glenfiddich Residence Changhi

An iPad app I worked on a little while ago has recently gone live at a popup store in Changhi Airport: Glenfiddich Residence. This was a little different to the work I usually do as it isn’t for distribution on the App Store or ad hoc for enterprise but for a single iPad locked in a display stand. The app is part of a large installation to showcase a limited edition of Glenfiddich which is available exclusively at Changhi Airport. The single cask1 that has been selected by Glenfiddich is on display within a specially constructed glass chamber that showcases the weather and atmospheric conditions of the distillery in Dufftown, Scotland.

The iPad app allows for customers to see the current conditions in Dufftown including temperature, wind speed and direction, weather conditions, and the local time. This is paired with multiple sound effects which match with the weather conditions2. The app also talks with a control PC via a direct socket connection over WiFi in order to control the weather effects within the chamber simulating morning fog, wind, and even rain!

There is a continuous slideshow of images from around the distillery and nearly everything is animated to entice customers to interact with it. All conditions are in realtime from a weather station near the distillery.

The final aspect is the “start movie” button which reduces the weather effects and begins playback of a video from the Glenfiddich Malt Master on a 1080p portrait screen inside the glass chamber. This video gives an overview of the distillery and details on the bottles from the limited edition cask that are available at the store.

The app is written entirely in Swift 1.2 and the iPad is locked down with Guided Access to prevent customers accessing anything else.

This was a really exciting project to work on as it is very different to the usual iOS apps I do. If you’re visiting Changhi Airport or passing through, be sure to visit the Glenfiddich Residence; I believe it will be there for the next year.

  1. Cask #8247 with a 1992 Glenfiddich to be precise. ↩︎

  2. For example, a thunder storm with a strong wind will have a very different array of sounds to a sunny day with no wind. ↩︎

Working out with Apple Watch

In an effort to get fitter, I’ve been using the two exercise apps available on the Apple Watch; Workout and Activity. It seemed strange to me that they were separate apps (which also feed into the Activity and Health apps on the iPhone) but after a bit of time with them I’m starting to understand the distinction.

Activity

The Apple Watch tries to get you active by introducing you to the three activity circles; Move, Exercise, and Stand.

  • Move: This is basically active calories and is a measurement of how much movement you’ve done. Your goal is based on height and weight but mine started off at 410 calories.
  • Exercise: I’m not 100% sure on how this is calculated but it seems to be doing anything that gets your heart rate up. The goal is 30 minutes per day.
  • Stand: This is designed to make you stand up for at least 1 minute every hour. If you get to 50 minutes past the hour and you haven’t stood up, you get a little reminder. The goal is 12 instances of standing for 1 minute per hour (so standing up for 12 minutes in a row won’t complete the circle - has to be at least 1 minute for 12 hours).

Along with the goals, there are also awards to motivate you to complete your circles each day. These look really nice1 and they use a decent curve so you start off getting quite a lot of achievements in order to make you persist with it. I’ve got most of them now so the next major ones are more tricky ones like “reach your move goal 100 times” or “complete your circles for every day of a month”.

Activity Complication

The nice thing about the circular design they’ve gone for is that it fits nicely as a complication on the watch face. I alternate between two watch faces — Simple in the week, Mickey at the weekends — and Activity is the only complication I have active (aside from the date). It highlights the best use of a complication in that I can quickly glance at my watch without touching it and see how far I am for each of my goals; if I want to see specifics, I can tap the complication to go into the Activity app proper. I used to use a FitBit2 but the problem with those is that you either had to use the iPhone app to see how far you’d got or tap multiple times to go through the different views (steps, distance, calories, etc). With the Apple Watch, I can see the 3 important things without getting my phone out and that is a big advantage in my opinion3. It is very motivating as you can see how close you are to completing a circle throughout the day.

Workout

This is the app to use when you are doing a proper piece of exercise. There are presets for things like Running, Walking4, Cycling, etc, but I’ve been using the Elliptical whilst on my Crosstrainer. I’m desperately unfit so I only do a 15-minute session each day (along with a brisk walk to get to my 30 minutes of Exercise) but the app is quite good at prompting me to extend my session on the next day. You get tapped when you’re at the halfway point and you can check your heart rate and calories burned during the workout. Speaking of heart rate, the Apple Watch does continuously track your heart rate throughout the day at intervals of around 10-20 minutes (it seems to check more the more movement you are doing). However, if you are doing a workout, it constantly checks your heart rate in order to more accurately see how many calories are being burned. I like to listen to music whilst I’m exercising so I have a pair of Bluetooth headphones which I pair to the watch (not my phone) and then I listen to a playlist I have synced across. This means I can exercise without my phone. I haven’t tried this on a run but my wife has and on doing a few runs with and without her phone the measurements are pretty accurate (on an 8 mile run, the distance was only off by 0.05 miles compared to GPS). When you finish a workout, you are given the option to “save” or “discard” it - this was slightly confusing as when you press “save” you are taken back to the app carousel and there is no way of seeing those stats again on the watch5. It turns out the workout is actually saved to the Activity app on the iPhone:

Activity iPhone App

The Activity app on the iPhone only appears once you have paired an Apple Watch. It works the same as the Activity app on the watch in that you have your 3 circles but the difference is that you can see historical data here, review your achievements, and see more numbers and graphs. The History window shows your circles at a glance but if you pull from the left you can see how your daily move goal has changed over time (so in the screenshots below you can see how this week the watch asked me if I wanted to increase my daily move goal from 410 calories to 450 calories):

You can also look at your day and your workouts in more detail such as this elliptical workout I did at lunchtime today:

I tweeted about this over the weekend but one of the most motivating things for me has been the “Avg Heart Rate” on the above screen. Each day over the past week, that number has decreased with the same 15 minute workout which shows (I think) an improvement in my overall fitness.

Health iPhone App

The final part of the fitness puzzle is the Health app on the iPhone. This came as part of iOS 8 and is a way for you to store all of your health data securely. There are loads of apps which can read and write to this store and the Apple Watch is no exception. By default, it will populate the Flights Climbed, Steps, Walking + Running Distance, Active Calories, Workouts, and Heart Rate sections. You can choose to display each of these as graphs within the app although I’m hoping for more control over this in future versions as you can’t view specific dates, only “Today”, “Week”, “Month”, and “Year”. This is one of those areas where the continuous heart rate monitoring is really nice as you can get an overall picture of how much you are fluctuating throughout the day - whilst there are plenty of heart rate monitors available on the market, one you don’t have to think about that is constantly monitoring will always win out (and it is incredibly accurate).

(A small aside, my Apple Watch has just told me to stand up. The nice thing about it doing it 10 minutes before the hour is up is that I often have things scheduled on the hour like phone calls. It’s turned into a nice reminder that I should go and get a drink6 and have a stretch before next hour of work. It is definitely one of those small things I didn’t think about when the Apple Watch was announced but it is the thing that has changed my behaviour more than anything else.)

In conclusion, I’ve found the Apple Watch to be a much better fitness companion than I expected. My assumption was that it would cater for those that like to do running and cycling but wouldn’t do much for the more casual exerciser. In actual fact, it has made me more aware of how little movement I was doing and also helped motivate me more to do things as simple as standing every hour. The majority of its features can be done on other devices, such as the FitBit, but the tight integration is key. The heart rate monitoring showing up in the Health app satisfies the stats nerd part of me whilst the simplicity of 3 goals per day has definitely made me more likely to succeed as I don’t want to break the streak I’m on. If you’ve got the Apple Watch and haven’t explored the exercise parts, I’d definitely recommend giving it a go; once you’ve done it for a day or two, you’ll find it very hard to stop!

  1. Apparently they were inspired by Olympic medals. They do look really nice especially when animating - the black background definitely helps make them stand out more than something like Game Center achievements. ↩︎

  2. I started with the FitBit One, then went up to the Flex, and eventually the Charge. ↩︎

  3. A summary of the Apple Watch could really be “something that makes interactions quicker” - it isn’t necessary but it feels so good to be able to get things done quickly. “But it only takes 5 seconds to get your phone out and check it” I hear you cry; true, but those instances add up. Like I say, it isn’t necessary but it’s definitely nice. ↩︎

  4. I did this a few times whilst walking the dogs but I don’t log them anymore - a walk isn’t really a workout in my eyes. Also, you still get credit towards your Exercise ring if you go for a walk but don’t activate a workout - it knows when you are walking briskly and so that does count as exercise. ↩︎

  5. It’ll show you how long your last workout was and when it was when you open the Workout app but you can’t see calories burned, average heart rate, etc. ↩︎

  6. I’ve also been trying to drink the recommended amount of water per day so this has been a good point to go and top up my water. I’ve been using the WaterMinder app to keep track of this as they have a nice Apple Watch app which follows a similar circular goal design. ↩︎

« Older Entries Newer Entries »