Ben Dodson

Freelance iOS, macOS, Apple Watch, and Apple TV Developer

A simple Audio Recorder controller in Swift (for iPhone)

iOS provides a number of useful modal views within it’s system frameworks with everything from sending email to taking photos or video. However, there is still not a standard way to perform what is a fairly basic task; recording a voice memo.

I had a need for such a feature on a project recently and so I put together AudioRecorderViewController. It’s a basic modal view controller that allows you to record, playback, re-record, and save audio locally in mp4 format.

Usage is insanely simple and modelled on such classes as UIImagePickerController:

@IBAction func presentAudioRecorder(sender: AnyObject) {
    let controller = AudioRecorderViewController()
    controller.audioRecorderDelegate = self
    presentViewController(controller, animated: true, completion: nil)
}

func audioRecorderViewControllerDismissed(withFileURL fileURL: NSURL?) {
    // do something with fileURL
    dismissViewControllerAnimated(true, completion: nil)
}

Check out AudioRecorderViewController-Swift on GitHub.

HKHealthStore Extension, deleteSamplesOfType

Following on from my post yesterday about the broken implementation of the HKHealthStore method deleteObjectsOfType:predicate:withCompletion:, I’ve put together a gist which provides a fixed method that works with HKSampleType objects; deleteSamplesOfType:predicate:withCompletion:

import Foundation
import HealthKit

extension HKHealthStore {
    
    func deleteSamplesOfType(sampleType: HKSampleType, predicate: NSPredicate, withCompletion completion: (success: Bool, count: Int, error: NSError?) -> Void) {
        
        let query = HKSampleQuery(sampleType: sampleType, predicate: predicate, limit: 0, sortDescriptors: nil) { (query, results, error) -> Void in
            
            if let _ = error {
                completion(success: false, count: 0, error: error)
                return
            }
            
            if let objects = results {
                if objects.count == 0 {
                    completion(success: true, count: 0, error: nil)
                } else {
                    self.deleteObjects(objects, withCompletion: { (success, error) -> Void in
                        completion(success: error == nil, count: objects.count, error: error)
                    })
                }
            } else {
                completion(success: true, count: 0, error: nil)
            }

        }
        self.executeQuery(query)
    }
    
}

It simply uses the NSPredicate within a HKSampleQuery in order to find any objects that match the HKSampleType and then deletes them with the HKHealthStore method deleteObjects:withCompletion: - this correctly ensures that you are only deleting objects that match both the predicate and type.

By way of example, this simply extends HKHealthStore so you can do something like this:

let date = NSDate()

let predicate = HKQuery.predicateForSamplesWithStartDate(date.beginningOfDay(), endDate: NSDate().endOfDay(), options: .None)

let stepType = HKQuantityType.quantityTypeForIdentifier(HKQuantityTypeIdentifierStepCount)!
healthStore.deleteSamplesOfType(stepType, predicate: predicate, withCompletion: { (success, count, error) -> Void in
    NSLog("deleted \(count) step objects: \(error)")
    let stepQuantity = HKQuantity(unit: HKUnit.countUnit(), doubleValue: 9800)
    let stepSample = HKQuantitySample(type: stepType, quantity: stepQuantity, startDate: date, endDate: date)
    self.healthStore.saveObject(stepSample) { (success, error) -> Void in
    }
})


let flightsType = HKQuantityType.quantityTypeForIdentifier(HKQuantityTypeIdentifierFlightsClimbed)!
healthStore.deleteSamplesOfType(flightsType, predicate: predicate, withCompletion: { (success, count, error) -> Void in
    NSLog("deleted \(count) flight objects: \(error)")
    let flightsQuantity = HKQuantity(unit: HKUnit.countUnit(), doubleValue: 15)
    let flightsSample = HKQuantitySample(type: flightsType, quantity: flightsQuantity, startDate: date, endDate: date)
    self.healthStore.saveObject(flightsSample) { (success, error) -> Void in
    }
})

As a reminder, you can file a duplicate of (rdar://22977320) if you’d like to see the original bug fixed but in the meantime the above should allow you to carry on with your HealthKit app without worrying about deleting other data.

HealthKit Radar 22977320

Working with any code, it is inevitable that you’ll stumble across bugs. When this happens with iOS development, Apple ask developers to submit a “radar” so that they can track and hopefully fix any issues that arise. The issue is that the Apple Bug Reporter is locked down and non-searchable so you have no idea if anyone else has submitted the same issue. Apple supposedly collate all of the bugs and mark any issues as duplicates and then use the number submitted to gauge how important an issue is and prioritise it’s fix. This has led to sites like Open Radar that let developers copy and paste their bug reports so other developers can search and then simply submit new bugs as a duplicate.

In any case, today I found an interesting bug in HealthKit. iOS 9 added a new method named deleteObjectsOfType:predicate:withCompletion which is supposed to let you delete any objects you have written to HealthKit nice and easily. For example, I might have an app which constantly checks your step count and flights climbed throughout the day and so it will want to delete any steps and flights from 00.00 to 23.59 before it adds the updated count. Unfortunately, the method doesn’t seem to respect the type parameter and so it will delete everything that matches your query regardless of what HKObjectType it is.

For example, consider the following code:

let date = NSDate()

let predicate = HKQuery.predicateForSamplesWithStartDate(date.beginningOfDay(), endDate: NSDate().endOfDay(), options: .None)

let stepType = HKQuantityType.quantityTypeForIdentifier(HKQuantityTypeIdentifierStepCount)!
healthStore.deleteObjectsOfType(stepType, predicate: predicate, withCompletion: { (success, count, error) -> Void in
    NSLog("deleted \(count) step objects: \(error)")
})
let stepQuantity = HKQuantity(unit: HKUnit.countUnit(), doubleValue: 9800)
let stepSample = HKQuantitySample(type: stepType, quantity: stepQuantity, startDate: date, endDate: date)
healthStore.saveObject(stepSample) { (success, error) -> Void in
}

let flightsType = HKQuantityType.quantityTypeForIdentifier(HKQuantityTypeIdentifierFlightsClimbed)!
healthStore.deleteObjectsOfType(flightsType, predicate: predicate, withCompletion: { (success, count, error) -> Void in
    NSLog("deleted \(count) flight objects: \(error)")
})
let flightsQuantity = HKQuantity(unit: HKUnit.countUnit(), doubleValue: 15)
let flightsSample = HKQuantitySample(type: flightsType, quantity: flightsQuantity, startDate: date, endDate: date)
healthStore.saveObject(flightsSample) { (success, error) -> Void in
}

The expected result is that you’ll end up with an entry for Step Count (9800 steps) and an entry for Flights Climbed (15). However, you’ll only get the Flights Climbed entry as the line healthStore.deleteObjectsOfType(flightsType, predicate: predicate, withCompletion:... will delete all entries that match the predicate, regardless of the HKObjectType. In addition, if you look at the headers for this method within the HealthKit framework, they don’t add up with what is expected:

This suggests that this method was refactored at some point prior to release and has this rather serious bug. I’ve filed this as a radar (rdar://22977320) but in the mean time you’ll need to do a manual search for objects with HKSampleQuery and delete them with a separate call if you want to avoid this.

Update: I’ve put together a fix for this issue with a new HKHealthStore method, deleteSamplesOfType:predicate:withCompletion:

Update: Whilst iOS 9.1 hasn’t fixed this issue, my bug report has today (23rd Oct 2015) been marked as a duplicate of issue 23097982 so looks like they may be getting around to fixing it. iOS 9.2 perhaps?

The Times They Are A-Changin'

Over three and a half years ago I announced the launch of WallaBee, “the ultimate collectibles game for iOS”. Whilst it has been a labour of love for me for over the past four years, it has also been incredibly stressful and draining at times. Today I was very happy to announce that I’d sold WallaBee to Munzee. I won’t go into the details of how that will work for players (you can find out all about it on the WallaBee Blog) but I did want to talk a bit about how a relatively simple game became a huge business.

I created WallaBee as it was a game I wanted to play1. I’d built several iOS apps in the past (including several that were featured by Apple) and my day job previously had been as a PHP developer so it was fairly trivial for me to throw the basic building blocks together. Over a few months of testing and iteration the game went live; it was free but monetised by using In-App Purchases for in game currency as well as a pro subscription for $5 a month which gave additional benefits to players. The feedback was great and I iterated quickly to a v1.1 product which launched at Nonick 012 where WallaBee was voted the ‘Best European Gaming Startup of 2012’. Over the coming years I’d add extra bits of complexity and hidden secrets2 and generally expand things out.

However, my big plan was to launch a version 2 product. I’d always considered v1.* to be a proof of concept and I had a ton of ideas for things I wanted to improve on. I pitched the idea to my co-founders in November 2012 and we showed off some screens and concepts to our players on April 1st 20133. It was around that time that the number of players was expanding rapidly and so I spent most of my time dealing with server issues and scalability. Over time, the fast iteration I’d started with slowed to a halt as I dealt with server scaling and that lead to issues when the innovation of iOS overtook us; for example, we didn’t implement AutoLayout and so weren’t equipped to deal with things like the iPhone 6 resolution change. This eventually led to nearly every issue needing the same solution; a complete rewrite of both the app and API. Unfortunately this was difficult to manage as all of my time as the sole developer was being taken up either trying to hold the server issues together enough for another release or dealing with player issues that crop up when you have thousands of people able to communicate with each other.

After a while, it just wasn’t fun for me anymore so I made the decision that I would shut it down. A few minutes after that announcement, Rob Vardeman from Munzee got in touch and asked if we could speak. I’d met him before when he was in London for a Munzee event and after a discussion, some emails, and a lot of thinking, I decided to let Munzee acquire the company so that they could continue to grow it in the way I had wanted to. I am confident that they will do an excellent job at maintaining the quirky side of WallaBee4 whilst also being able to add some amazing new features; I’m so sure of it that I have remained as a shareholder of the company.

Based on my experiences, I have a few pieces of wisdom to share:

  • It’s fine to get something launched but don’t forget to put in the groundwork for scalability. If you don’t know what you’re doing, find someone who does.

  • Monetise your game from the start. We never had any issues but I see startups every day that get a big of investment and think that if they build it they can monetise it later; don’t do that. We launched with 30-day and 90-day subscriptions and most players bought a 90-day one straight off. It is much easier to alter your existing payment structure (as we did when we went from subscriptions to single payments for goods) than to try and add one when everything was free previously.

  • Never announce something before it is built. This is a lesson that few people have mastered as it is insanely difficult trying to keep something awesome from the people who are going to love it most; it is also easy to announce something as then you feel like it forces you to launch it by that date but it doesn’t work. Take a leaf out of Apple’s book and only announce something when it is done and going to ship within the next 2 weeks (and when it comes to software, announcing something that is available “Today” should be the rule).

  • Do not underestimate how difficult it is to manage a community of players or how crucial it is that you do. I frequently had to deal with issues such as bullying, stalking, and just generally petty squabbles. We had births, deaths, celebrations, and everything in between5. There was also the inevitable bugs and server issues that required an apology. However, because of the way we handled those issues publicly and fairly, it cut us a lot of slack when things went wrong. A good community is the key thing for any social app and must be nurtured.

So what now? Without WallaBee to manage over the past few weeks, I’ve found myself at a bit of a loose end with so much more free time! I’ve written three apps so far including a content blocker, an app based on HealthKit, and an app to let me fly my Bebop Drone with an iOS MFI controller6; all of them will be available on the App Store at some point. I’m also collaborating on an Apple TV game with another developer which will hopefully be live in November. Beyond that I haven’t decided what the future has in store but I’m enjoying working on small, offline apps for the time being.

The final thing to mention is a slight change to the way I do my freelance work. Due to WallaBee becoming my main business for a while, I changed my company name from Ben Dodson Apps Ltd to WallaBee Ltd. As Munzee are based in the US, they have no need for a UK Limited company and so they have simply purchased the rights and assets of the WallaBee app and brand. I’ve therefore changed the company name to Dodo Apps Ltd7 and that is what I’ll be using for all of my freelance business from now on.

  1. And the nice thing now is that I get to play it and enjoy it! ↩︎

  2. One of our players, ‘SkipOkane’ put together a list of some of the secrets. My favourite is probably the ‘Mirage’ item which would always appear as a #1 when you purchased it (although the ‘Boomerang’ item that comes back if you try and recycle it is a close second as is the jousting minigame…) ↩︎

  3. My reasoning was that if they hated it, we could pretend it was a joke. ↩︎

  4. They’ve done well so far; the first set they created is made entirely of dancing cats. I think it’s in safe hands! ↩︎

  5. I would never have believed that somebody would want to speak with me in the final weeks leading up to their death as they wanted to make arrangements for how their game items could be distributed. It’s humbling and hard to deal with. ↩︎

  6. I would love to do a Steve Jobs “these are not 3 separate devices” but they are. ↩︎

  7. I first used the name “Dodo Apps” when I was doing my A-Level IT course. I built a choral robe management system in Microsoft Access and had to choose a name for the about page; I chose “Dodo Apps” as I was a big fan of the dodo from Alice in Wonderland and because it used letters from my surname. ↩︎

Brapp, the rap app

I’m happy to announce that an iPhone app I’ve been working on since March this year has gone live in the App Store; Brapp.

Brapp is a free and easy way to create and discover exciting musical collaborations all over the world. Producers can share their beats allowing anyone to make mini music videos right on their phone. When you choose a beat and hit record, the camera starts rolling and the custom studio technology balances your audio (be it singing, rapping, or playing an instrument) onto the beat ready for you to add video filters and share with the world.

I was the sole iOS developer on the project and worked closely with the app creator1 and his API developer to bring it all to life. I was able to write the app in Swift and use AutoLayout to make sure everything scales beautifully on all 4 sizes of iPhone from the 3.5” iPhone 4S upto the 5.5” iPhone 6S Plus. In addition, I made use of GPUImage for the custom video filters and a whole load of low-level audio APIs to get the audio balance just right as well as real time waveforms and switching between external microphones. The design went through a number of iterations, each better than the last, and I was able to use my extensive UX knowledge to help shape the end result.

This was a great project to work on with a load of interesting technologies plus I got to lay down some sick beats2. You can check out Brapp on the App Store (it’s free) or learn more on the official website.

  1. These were great guys to work with. During a Slack conversation, they made the mistake of saying “Yes @bendodson, that is right” to which I said “I want that on a t-shirt”. A couple weeks later, this arrived in the post… ↩︎

  2. Thankfully all now removed. ↩︎

AppIndex Interview

Earlier this month I was interviewed by AppIndex about my work as a freelance app developer. You can read the full article on their website but I’ve also put their questions and my answers below:

How long have you been involved in developing apps for?

I started developing apps professionally in August 2009, shortly after iOS 3.0 was released, but I tinkered around with the iOS SDK when it was first announced the year before.

What types of apps do you work on?

I have quite a varied portfolio but for a long time the apps I worked on were location based (i.e. travel guides, restaurant finders, interesting places nearby, etc). Nowadays I tend to work on a lot of social network type apps but that’s probably more indicative of what clients are looking for at the moment.

What platforms, tools and languages do you work with?

On the mobile side I only deal with iOS and all my new projects are written in Xcode using Swift 2.0 (with only a couple of legacy apps written in Objective-C). However, I also do backend development for APIs which I write exclusively in PHP. All my work is done on OS X.

What apps have you worked on that you are most proud of?

I think the app I was most proud of with was the “Horse Tracker” for the Grand National horse race. I worked with a company that attached trackers to the horses so we could see in realtime where they were on the field and display that to the user. It was really nice to use, especially on the iPad which had a full map view of the racecourse (which was never released in the end) and used lots of really cool technology such as audio indexing to perfectly match the time of the race on your TV screen to the data coming from the trackers, even if you were watching on a delayed stream or via a catch-up service hours later. Of course, nothing makes me more proud than the apps I write and publish myself so they will always beat client projects but that one was special.

I’m starting to see a lot more thoughtful design coming through at the moment. Previously everybody was still trying to work out how to get data onto a small screen and most designers I worked with via clients had experience in print or web which didn’t always translate across; nowadays the level of design work seems to be much improved and everybody wants a slicker experience with more animation. The transition to the iOS 7 style of design (flat and minimal) seems to have helped as well.

What is the best thing about working in apps and mobile?

I really enjoy the varied work. It can be difficult sometimes with deadlines and quite a big responsibility if your app is in millions of peoples hands but there is a huge amount of variety. I never really know what I’m going to be working on next which I find exciting.

What is the one thing you’d like to change about app development?

I have two big gripes which are more to do with people than app development itself; being told I should port my apps to Android (“you’d make so much more money”) and being pitched an idea which isn’t so much an app as a service that requires servers, infrastructure, and has large operating costs (“I want an app that can let people upload photos and videos – I’ll give you 25% of the profit if you build it for free”).

What mobile devices do you use?

I use an iPhone 6 Plus and an iPad Air at the moment. I’m particularly interested in the rumoured 12” iPad Pro although I have something similar in the form of the 12” Macbook which I take to client meetings.

What are your favourite apps / games?

Ironically I don’t actually use that many apps beyond the stock things like mail. I’d say my most used 3rd party apps are Dark Sky for their weather alerts and Philips Hue for turning the lights on and off in my house. With games, I tend to play something solidly for a week and then get bored of it; I’m currently playing Alto’s Adventure, Rollercoaster Tycoon 3, and Kindo.

What do you think of the Apple Watch?

I love my Apple Watch. It’s definitely a luxury rather than a necessity but I love getting notifications on my wrist, the fitness features, and Apple Pay. I’m looking forward to watchOS 2 especially as then I’ll be able to log a swim and have it count towards my exercise totals. It’ll be interesting to see how much better the 3rd party app experience is when the update arrives – I’ve dabbled with developing a few apps but the lag talking to the iPhone always limited the amount I could do.

Microsoft Bridge and the "App Gap"

I was recently asked my opinion on Windows Bridge and the “App Gap” between iOS, Android, and Windows Phone by FierceDeveloper. You can read the article on their website but I’ve also put the full questions that were put to me along with my answers below:

Would something like Windows Bridge make you any more likely to bring your apps to Windows Phone? Why or why not?

It wouldn’t convince me as there are two big problems: firstly, it only works with Objective-C whilst I have already made the switch over to Swift so I wouldn’t be able to make use of this (and the benefits of Swift massively outnumber the benefits of being on Windows Phone for me); secondly, whilst it may be easy to port an Objective-C app over, I would still need to build a custom Windows UI which would take a long time. In addition, this custom UI would need to be supported as and when new devices and screen sizes come out and I’d need to maintain feature parity between iOS and Windows making every release take significantly longer to bring to market.

What do you think are the biggest reasons for the “app gap” between Apple’s App Store and what’s available in Microsoft’s app store?

There are lots of reasons for this but I think the two bigs ones are developer usage and money. When the iPhone came out, it was revolutionary and most developers who work in mobile began due to using an iPhone and wanting to write apps on it (that is certainly true in my case). That meant the iPhone got a lot of exciting apps developed on it first as the people that made those apps used iPhones. Over time, this has compounded into giving an Apple a massive advantage and whilst Android has caught up in many regards they haven’t got anywhere close when it comes to money. Simply put, iOS users spend more than Android users. I don’t know if that is due to the security and convenience of iTunes and TouchID or whether it is a more social thing (you’ve spent $700 on a phone so what is $100 a year on apps) but in all of the research I’ve done it has been the case that iOS users are willing to pay where Android users are not. It is for that reason I don’t port any of my apps to Android. Back to Microsoft though, the reason there is an app gap is that all of the developers started on iOS (and now Android) leading to a shortage of apps - this led to less customers and thus less money so they are now in the situation where it is very hard to attract developers as there are neither usage numbers nor financial figures to entice us.

Is there more Microsoft could/should do to make developers more interested in creating apps and games for their platform?

To be honest, I don’t think there is much Microsoft can do at this point. Consumers are simply not buying Windows Phones and that is the real problem. Whilst their initiatives such as building one app that can run on Windows 10, Xbox One, and Windows Phone are interesting, there simply isn’t enough consumer interest nor money to get iOS and Android developers to create apps on those platforms. Ultimately, Microsoft would probably be better chasing the gaming market over the app market - produce a Windows Phone that is more akin to a Playstation Vita and is able to run games that can also run on Xbox One and Windows 10. These wouldn’t be massively powerful games but would fit in with the “write once, run everywhere” better than apps do (typically games are used to having to scale at different resolutions and have custom UIs whereas regular apps tend to be designed to the UI of the platform making “write once, run everywhere” unfeasible).

LEGO Scooby-Doo

I’m a huge fan of the Hanna-Barbera cartoons but Scooby-Doo has always been my favourite so I’ve been looking forward to the LEGO Scooby-Doo sets which officially launch tomorrow. Luckily, due to an error on the part of Argos1, I was able to get all five sets a week early so I’ve put together a very brief review. Make sure you scroll to the bottom to see the coolest feature which appears in every set!

Mummy Museum Mystery

This is the cheapest of the sets but it has a lot of detail for such a small set. Scooby and Shaggy come in every set but are joined by a mummy complete with headpiece. The museum itself has a hidden compartment with jewels and a liftable sarcophagus in which to hide the mummy.

Mystery Plane Adventures

This is my least favourite set but you do get a fair amount for the low price. Scooby, Shaggy, and a headless horseman complete with a bi-plane with revolving propellor and (for some reason) burger bombs2 that you can detach. The Scooby model in this one is slightly different to the others in that he is wearing flying goggles.

The Mystery Machine

Now we’re getting to the good stuff. The Mystery Machine is iconic and I’ve no doubt that, at the low price, this will be the most purchased set in this series. For under £30, you get Scooby, Shaggy, Fred, a zombie, a scary tree, and the Mystery Machine itself. The model is much smaller than I’d originally anticipated but it is very detailed and opens up allowing access to the inside which contains a kitchen and some props like a camera and torch. Overall, this is a fantastic set although it does use a huge amount of stickers which are prominently visible.

Haunted Lighthouse

This may be my favourite set as it captures everything great about Scooby-Doo. You get a lighthouse keeper (it must have been him), swamp creature, Scooby, Shaggy, Daphne, a speedboat (with skis), and an awesome rock formation with lighthouse and secret pirate cave. There are nice little touches such as scratches inside the prison cell to show that somebody had been counting days, a wonky “keep out” sign, and a face on the hidden cave complete with gold tooth.

Mystery Mansion

The biggest and most expensive of the sets, the Mystery Mansion shares a lot in common with the retired “Monster Fighters” sets that LEGO released in 2012 (of which I have the Haunted House, my all time favourite LEGO set). The mansion is broken into 3 core areas; a greenhouse (with man eating plant), kitchen, and the tower. I personally think the greenhouse is the most interesting as there is good use of transparent panes and wedges to give the impression of smashed glass. The tower also has a nice feature in that when you turn a knob at the back, the clock hand advances and a column in the middle rotates to switch from a bat to a place where you can stand the vampire minifigure. There are a lot of stickers although the majority are on transparent panes which I find transfer better than on bricks where a slight misplacement can look terrible. In terms of minifigures, you get Scooby, Shaggy, Daphne, Velma3, a suit of armour, a ghost, and a vampire. Both the vampire and ghost glow in the dark if you shine light on them first. Overall, this is a great set for both adult collectors and for kids to play with.

Minifigures

The main draw for most adult collectors will be the minifigures. Whilst Scooby4 and Shaggy come in every set, Daphne appears in two, and Fred and Velma only appear in one each so you’ll need practically every set if you want all 5 of them. In terms of the figures themselves, the designs are perfect and I believe this is the first time that LEGO have done custom arms on licensed models (note how Shaggy’s arms show a t-shirt style rather than extending to the hand). However, the best thing of all in these sets are the villains…

Every villain5 has a double-printed head so you can unmask them and mimic the whole “I would have gotten away with it too if it weren’t for you meddling kids” speech! This for me shows how much thought LEGO have put into these sets and makes every minifigure one worth having (whereas on face value6 they seem a little bland being mainly reprints from other sets).

Overall I highly recommend the Scooby-Doo LEGO sets. They offer great value for money, especially for licensed sets, and will look good as display models for adult collectors whilst also being high in playability for kids. Don’t forget that the LEGO Big Bang Theory set also launches tomorrow though!

  1. They accidentally broke the embargoe and were selling them instore - I managed to rush to my local branch and get all 5 sets about an hour before the mistake was fixed and they were pulled. ↩︎

  2. Every set has Scooby and Shaggy but they also always have either a sandwich or a burger. On the cheaper sets they’re a bit oddly placed (i.e. in the Mummy Museum they are used as the lever for the sarcophagus) but they are quite nice as mini models. ↩︎

  3. Nice read, Velma” ↩︎

  4. It’s worth noting that there are 4 Scooby variants; one paw up, one paw up and scared, sat down and scared, and sat down with flying goggles. ↩︎

  5. OK, almost every villain. The zombie doesn’t have anything to cover the back so instead they have put a crease and some buttons to show it’s a mask. Genius. ↩︎

  6. *chortle* ↩︎

The mobile web

There has been a huge amount of talk over the past couple of weeks about mobile browsers mainly due to two announcements at WWDC; Content Blockers in Safari and Apple News. This has led to discussion of mobile ads as it has been discovered there are huge speed boosts to be made when certain elements are blocked (for example, the iMore site weighs 14MB for something that is essentially plaintext). The thing that got me was when Niley Patel of The Verge created an article called “The mobile web sucks” whilst failing to notice the irony that the article he created takes 30s to load, and fetches 9.5MB across 263 HTTP requests. The bit that irritated me though was this passage:

Similarly, Apple News lets you publish directly onto iPhones, bypassing Apple’s own Safari browser, which feels mildly crazy. Yet there’s some excitement over this: I just read Ben Thompson’s excellent piece explaining how the ad market for smaller publishers leads inexorably to bloated web pages that load poorly. That piece was itself a response to John Gruber saying that Apple’s new Safari Content Blocker system would cause a “reckoning” for publishers by letting users block performance-degrading Javascript. And yes, most commercial web pages are overstuffed with extremely complex ad tech, but it’s a two-sided argument: we should expect browser vendors to look at the state of the web and push their browsers to perform better, just as we should expect web developers to look at browser performance and trim the fat. But right now, the conversation appears to be going in just one direction.

And that’s troubling. Taken together, Apple News and Facebook Instant Articles are the saddest refutation of the open web revolution possible: they are incompatible proprietary publishing systems entirely under the control of huge corporations, neither of which particularly understands publishing or media. Earlier this year, I called Facebook the new AOL; Instant Articles comes from the same instinct as AOL trying to bring Time Warner’s media content into its app just before the web totally kicked its ass. Apple and Facebook are turning their back on the web to build replacements for the web, and with them replacements for HTML and CSS and every bit of web innovation it’s taken 20 years of competitive development to achieve.

I’m not 100% sure on Facebook Instant Articles but Apple News is basically just an RSS reader. Sure you can use some custom syntax to make things look slightly nicer but they have “over a million topics” because they’ve basically scraped all the big RSS feeds and packaged it nicely. This brings me to the one thing I probably agree with Niley on1, the mobile web does suck. The difference is that I don’t ever use it…

On mobile, apps have completely replaced the web for me. Safari isn’t on my first home screen, let alone my dock; it’s buried in a folder with Stocks, Tips, and other undeletable2 Apple apps. The majority of browsing I do on my phone is either my RSS feeds or Twitter; if I want to read a link, they usually open in their own browser and I usually regret it. Mobile web sites are generally horrible things that are slow to load and have ridiculous interfaces. There are some sites that take longer to load on 4G than the equivelant WAP3 website used to on GPRS! Even the option to choose to render the desktop version often doesn’t work due to the horrible way in which most companies detect you are using a mobile browser. Due to this, I typically save websites I want to read to my Pinboard and read them later on my iMac or use the “readability” feature in Safari.

I love RSS feeds. I subscribe to around 130 websites4 and I get every article fast, cached for offline usage, rendered in the same font, and without any JavaScript crap. I simply don’t understand people that don’t use RSS. Twitter is no match; you have to scroll through a ton of stuff you aren’t interested in to find links which you then have to click and wait for your browser to render. RSS keeps everything uniform and consistent and ensures I don’t miss anything. That is essentially what Apple News is trying to do (albeit with a discovery angle and a bit more design) and it seems like a perfectly good idea to me but I doubt it’ll make me switch from Reeder and FeedBin, my current setup.

The only minor problem I have with RSS is sites (like The Verge5) who only give you a paragraph of the text before making you click through to the site so they get your eyeballs on their ads and which always leads to clickbait headlines and paragraphs ending in a manufactured cliffhanger6. Fortunately there aren’t many sites that do that but Apple News is not going to fix it; they have provision for this with a “Read Full Story” link they embed that opens up the rest of the page within an internal browser. There are also sites which have ads as content (i.e. Daring Fireball) but those work well as they can be skimmed over quickly if they are of no interest and they don’t track you, take over your screen, or do any of the other things that browser ads do.

So whilst everybody is up in arms about mobile browsers using up their battery and CPU, tracking their usage across the web, and having the unwinnable argument over whether to block ads or not7, I’ll be sat reading that debate in my JavaScript-free, fast, RSS reader. I’ll likely end up installing some form of content blocker when iOS 9 is released for the few times that I do end up using the browser, but those times are thankfully becoming fewer and fewer on mobile.

  1. I have an instant dislike to him after his Apple Watch review (which has a horrible, horrible website) in which he said the notifications made him ruder in conversation (as he constantly looked at his watch rather than ignoring it - that just makes him a jerk, not the watch) and claimed he felt “ridiculous wearing the Milanese Loop” yet is happy to wear a massive chunky punk bracelet↩︎

  2. I’m still surprised this hasn’t been remedied. In iOS 3 you could enable the Nike Running app by a toggle in settings (as the 3GS had the Bluetooth chip to talk to the shoe tracker) so I don’t see an issue why the built-in apps couldn’t be turned off in a similar way from settings. ↩︎

  3. Who remembers WAP? Fun times. ↩︎

  4. A mixture of tech, politics, development, LEGO, video game, and Disney blogs. Everything published in the areas I care about all in one place. ↩︎

  5. Turns out that The Verge does offer a full RSS feed after all! A shame that 9to5mac.com have started truncating theirs… ↩︎

  6. After clicking a few of them, you realise the site is terrible and unsubscribe from it. Problem solved. ↩︎

  7. The argument is either “if you block ads you are getting something for free and going to put the publisher out of business” or “you shouldn’t be using up my battery and CPU with this privacy invading advertising”. You can’t win either way. I block ads personally but I’ll do a post soon about how I do it on my iMac. ↩︎

Finding the next available driving test with Goutte PHP Scraper

Over the past week I’ve had a lot of fun working on some small scripts to aid my general day-to-day living. My wife is currently waiting to take her driving test but our local area only has one test instructor with a waiting list of over 6 months! Occassionally slots open up due to cancellations and so she has been logging into the Government website multiple times a day to check for a nearer date. Time for some automation!

To stop her having to constantly go to a website, I decided to knock up a quick PHP script that would basically pretend to be a browser, log in, go to the correct page, and rip out the next available appointment. It’s been a while since I wrote a script like this so I had a look around for something better than SimpleXML and found Goutte, a simple PHP Web Scraper. Within 10 minutes, my script was working. It looks something like this:

<?php

require 'vendor/autoload.php';

use Goutte\Client;

$licenseNumber = 'xxx'; // Driving License Number
$referenceNumber = 'xxx'; // Reference number for current test date as looking to rebook

$client = new Client();

$crawler = $client->request('GET', 'https://driverpracticaltest.direct.gov.uk/login');
$crawler->filter('h2')->each(function($node) {
	if ($node->text() == "Sorry, you can’t use this service right now") {
		echo 'Site is down.';exit; // Inexplicably, the site goes offline from 9pm-6am. It thinks it's an office!
	}
});

$form = $crawler->selectButton('Continue')->form();
$crawler = $client->submit($form, array('username' => $licenseNumber, 'password' => $referenceNumber));
$link = $crawler->selectLink('Change Date and time of test')->link();
$crawler = $client->click($link);
$form = $crawler->selectButton('Find available dates')->form();
$crawler = $client->submit($form, array('testChoice' => 'ASAP'));

$crawler->filter('.slotDateTime')->each(function ($node) {
	$date = $node->text();
	echo $date.' is the next available test slot.';
	// Here I check if it has changed since last time and send an email if it has and nearer than currently booked test. Left out for brevity.
    exit;
});

?>

As you can see, the code is pretty simple, readable, and powerful. Within minutes, I was able to jump through 3 pages without worrying about sessions, cookies, or any of the other nonsense that often occurred when dealing with this kind of scraping. I run this every 5 minutes on a FortRabbit server and we both get emailed if a closer test date appears. For just a few minutes work, this means better peace of mind as we don’t have to check manually and this checks far more frequently than we ever could. Now to wait and hope for a nearer test date…

« Older Entries Newer Entries »