After a short summer break we’re back with episode 15 of The Divide podcast which is all about virtual reality. John has a fancy new HTC Vive, I’m trying to convince myself I don’t need one, and Chris just wants to be on the bridge with Captain Janeway…
As I’ve mentioned many times before, iOS had a pretty terrible privacy flaw in that apps didn’t need any permissions in order to read through your media library. This was an issue as it meant you could be fingerprinted easily and tracked across various apps1. Thankfully, this has now been fixed in iOS 102.
In this article, I’ll explain how to update your apps to support this new privacy requirement. Before I do that, I’m going to show you what happens if you run an app built against the iOS 9.3 SDK (or earlier) on iOS 10:
You’ll be prompted3 to allow permissions as soon as any media library code is encountered be that on app startup or in a background process such as that used by Music Tracker. If you decline to give permission then the app will quit to the home screen and you will not be able to use it. This is a big change from previous permission switches whereby apps built against the old SDK would be exempt from new permissions (i.e. if your iOS 7 SDK compiled app ran on iOS 8, it wouldn’t crash because it wasn’t using the new location privacy options). Personally I like this change as it allows you to see clearly which apps were abusing these APIs such as Canary and Google Calendar.
Bottom line: If you are using MPMediaQuery or similar in your app, you’d better update it with the iOS 10 SDK as soon as possible as otherwise you are going to get a lot of crashes if your users don’t allow the permission (or a lot of awkward questions if you shouldn’t be using this API).
How do you update your app to request permission for these APIs? First of all, lets whip up a basic example with Swift 3 that will pull the title of the first track in your music library:
In keeping with other permissions based APIs such as photos, contacts, and calendars, iOS 10 requires that you add a new key to your Info.plist file to explain why you want to use this permission; for music library access, this key is NSAppleMusicUsageDescription. If you don’t add this key, your app will crash as soon as you try and access an MPMediaQuery with the following message:
This app has crashed because it attempted to access privacy-sensitive data without a usage description. The app’s Info.plist must contain an NSAppleMusicUsageDescription key with a string value explaining to the user how the app uses this data.
With the NSAppleMusicUsageDescription key in place, you will now be given a standard permission dialogue when you first try and access the users media library. If the user chooses “Don’t Allow”, then the media query will fail and any subsequent calls will request in the query.items property being nil. If they choose “OK”, then nothing happens (as execution of the code is not suspended and there is no callback). To fix this, we need to use the MPMediaLibrary.authorizationStatus() and requestAuthorization((MPMediaLibraryAuthorizationStatus) -> Void) APIs that were added in iOS 9.3 to do something like this:
You first need to request authorization and then run your code only if the status is authorized. If not, then you should display an error specific to whether or not the request was denied or restricted (usually by corporate or parental controls).
I’m extremely grateful to everyone that has helped share my posts about this issue and to the engineers at Apple for fixing this privacy flaw. I’ll update this post should anything change between now and the expected public release of iOS 10 in September.
I don’t want to take all the credit for this but I honestly don’t think this would have been fixed if I hadn’t raised the issue repeatedly over the last 7 months. ↩︎
In earlier builds of iOS 10 you’d be asked “[App Name] Would Like to Access Apple Music” (as shown in screenshots above) but as of iOS 10 beta 4 this has been changed to the more appropriate “[App Name] Would Like to Access Apple Music and Your Media Library”. ↩︎
I started off strictly as a web developer, moving from agency to agency, not really enjoying what I was doing. At the same time, Apple had just released the iPhone and, shortly afterwards, the App Store. I thought iOS development looked fun, so I quit my job and started teaching myself how to do it.
Where did your first clients come from?
I had been freelancing as a web developer on the side for a number of years, so I had a few clients through that. Coincidentally, when I decided to freelance full-time, a friend from Twitter reached out and asked if I would be interested in appearing on The Gadget Show. For the segment the show wanted a freelance developer and a small company to build two separate apps, then see which was more popular. At the time, I had never built an app, so naturally, I said yes. It forced me to learn the ropes pretty quickly! I cobbled this app together and it ended up doing really well, scoring a feature from the App Store. From there, clients just kept coming.
What type of projects do you work on now?
Now there’s also the iPad, Apple Watch, Apple TV, so I work on a variety of projects — everything from little apps to find the best steak restaurant nearby to bigger ones for clients like L’Oreal and Glenfiddich.
What does your day-to-day look like?
At this point, I try to exclusively work from home. In the beginning, clients would insist on coming to their office. Those days would always be spent in meetings, talking about nothing, and not getting much done. Nowadays, I push for a video call in lieu of all that. I have clients I’ve never even spoken to, that I work with via email or Slack. They’re the best.
What has been the greatest challenge of kind of being a freelancer?
For me, it’s been time management. It’s so easy to procrastinate. Then you hit a deadline and realize that you’ve done nothing, you have to tell the client, but you don’t want to tell the client, so things get worse quickly. The biggest challenge for me was getting to a point where I can switch my work mode on and off.
What’s next for you?
A few years ago, I had my own app, an iPhone game called WallaBee. For about three and a half years, I focused on that and was extremely adamant about not hiring anyone else. In retrospect, some help might have made sense, but I was worried about being able to pay someone reliably month to month. It was a lot of fun, but eventually, I burned out and sold it.
Ultimately, I’d like to do something like that again, but at the moment, I enjoy freelancing.
It’s a constant battle between doing things that you want to do, but not earning any money, and doing the things that are less enjoyable, but make money.
I’m sure that statement resonates with a lot of freelancers.
When I was young and single, it was easy to spend twenty hours a day working on my pet project for no pay. The money earned mattered less because at the end of the day, it was just me. But when you have a family with mouths to feed and bills to pay, you find yourself taking less of those risks and opting for projects that provide a bit more security.
What advice would you give to someone just starting out?
Keep it simple. I spent a long time researching and reading up on various how-to-get-things-done philosophies, buying every to-do list manager there was. Now I just have a basic list, make my own invoices, and track everything else in Cushion. If you spend a fortune on loads of tools and to-do lists and all that, you end up spending more time tweaking your setup for productivity than actually being productive.
If you’re a freelancer, you should definitely check out Cushion. It’s an awesome online tool for keeping track of your clients and scheduling - I’ve been using it for nearly a year and it has helped me avoid overbooking myself. I’m super excited for their invoicing tools which they are launching soon. ↩︎
I’m currently in the process of writing an app which will store some information for each user on a server; the stored data should be available to the user on all of their devices. Usually there are two ways of dealing with this:
Build a user access system such that the user has to register to use the service. They’ll need to log in to the app (usually with email address and password) on each of their devices.
Use CloudKit to store all of the information in an Apple provided database.
The first option is overkill for this fairly simple app1 and the second is too restrictive as it means I need to use Apple’s specific data modelling system. However, there is a third way which uses a bit of each of them…
When a user is logged in to an Apple account on their device2, they are automatically signed into the iCloud system. With CloudKit, every app gets its own identifier based on both the CloudKit container and user account which doesn’t change. By retrieving this identifier, you can be 100% sure of the user that is using your app without knowing anything personal about them and without requiring them to manually log into your app.
To get the identifier, you first need to enable CloudKit within the iCloud section of the Capabilities panel in Xcode (which will require updating your provisioning profile). Then, it is a simple case of importing the CloudKit framework and using the following code:
The identifier returned will be 34 characters long and look something like this:
_e990774f93dd6625b11af6d40fceb310
Once you have that, you can then send it to your server to match it against whatever content you want to link to this particular user. Now whenever they use your app on any device with the same Apple account, they’ll have access to everything without the need for a manual account creation process and without having to hand over any personal information. The entire process is completely silent and is very secure as the identifier is generated from both the Apple account and your CloudKit container - it can’t be used by other apps nor can it be reverse-engineered to give you personal details of the user.
I find the code above a bit messy to be used multiple times throughout an application (not to mention the import CloudKit requirement) and so I’ve wrapped this up in a very basic Swift class that can be called as such:
UPDATE [3rd May 2019]: I needed use of this in a recent project and so I’ve updated the code to make use of Swift 5 and the new Result type.
It’ll also lead to a sharp drop-off in users as people don’t like having to create multiple accounts all over the place. You could implement something like Facebook Login but then you still have an issue in that you are requiring a fair amount of personal information upfront which you probably don’t need. ↩︎
I’ll hazard a guess that 99.9% of iOS devices are logged into an Apple account. They need to be in order to download apps, send iMessages, create backups, etc, so very unlikely you wouldn’t be. ↩︎
The 14th episode of The Divide podcast is now available in which we take look at the new announcements about watchOS, tvOS, macOS, and iOS in a special episode almost as long as the WWDC keynote itself. We also have a brief chat about the latest news from E3 and Chris gets upset about the Stay Puft Marshmallow Man.
I was recently interviewed by Fast Company for an article they wrote about iOS features they’d like to see at WWDC this year. A couple of my suggestions were featured in the article but I thought I’d publish my full iOS 10 wishlist.
System Defaults
One of the things that I feel has been long missing from iOS is the ability to choose a default app for certain tasks such as email, web browser, phone, or notes. I would love to see a way for developers to make their app conform to some set of standards that would allow them to be selected as a default app. For example, Skype could conform to the “phone app” standard so when you tap a phone number in your mail app it’ll launch Skype and dial the number rather than using the Phone app. Similarly, email links could go to Polymail and web links to Chrome.
Siri API
I’ve long wanted a full API for Siri but so far there hasn’t even been a stepping stone. I imagine the first phase would go hand in hand with the System Defaults plan I outlined and that apps that adopt a certain standard would be able to show up in Siri. For example, if Spotify conformed to being a “music app” then saying “Hey Siri, Play C’est La Via by B*Witched” would start that track playing in Spotify rather than Music.
Control Center Buttons
When you swipe up from the bottom of the screen on iOS, you get Control Center which gives you quick access to certain features (like WiFi), controls (for music / video), and apps (Torch, Timer, etc). I’d like the ability to a) use 3D touch to get more options when using the features section (so I can choose a new WiFi network) and b) be able to choose the apps and features that show up in that area. I never need quick access to the calculator.
CarPlay
I’ve been using CarPlay for around a year now (via an after market Alpine iLX-700) and I absolutely love it. However, 3rd party apps are limited to a few selected audio apps like Spotify and Overcast which use a simple template that doesn’t work very well. I’d like to see support for any 3rd party app to use CarPlay and for it to use UIKit so that any interface can be displayed. I noticed in March that iOS 9.3 added a UIUserInterfaceIdiom for CarPlay so hopefully this is coming.
AirPlay
I have a number of AirPlay devices around the house and one of the things that frustrates me most is that I can’t use Siri to control them. I’d like to be able to say “Hey Siri, play my cooking playlist in the kitchen” and have it just work. I’ve been able to hack this functionality together via some Applescript and a fake HomeKit setup on my Mac called Homebridge but it’d be nice to have it built in natively. Also, iTunes on the Mac can AirPlay to multiple speakers at once whereas iOS can’t.
The 13th episode of The Divide podcast is now available in which we discuss video game consoles from our first tentative steps in the 8-bit era to the almost HD consoles of the modern day. We also discuss the Flic button, Day of the Tentacle, the Nvidia GTX 1080, Stellaris, and our predictions for WWDC.
A few days ago I wrote an article about getting my Flic button to work with HomeKit. Since then, I’ve had a few requests from people wondering how I was getting iTunes to work with HomeKit such that it could start a playlist on some AirPlay speakers. Today I’ve released a couple of HomeBridge plugins to NPM and I’ll detail how I’ve got my system working.
First of all, my entire setup is powered by the awesome Homebridge system which I’ve written about at great length. I found a plugin from Dan Budiac called homebridge-applescript which allowed you to run a line of applescript via a system of fake switches (so a different script could be called when the switch is turned on and off). This is great for simple one-liners (like “tell application iTunes to pause”) but I needed something a bit longer for dealing with AirPlay. I forked Dan’s project to create my own Homebridge plugin that would work with a path to an AppleScript file rather than a single line of AppleScript:
This allowed me to then use my existing alarm clock script in order to make iTunes connect to my kitchen speakers and play a specific playlist1 when my “kitchen speakers” switch is turned on and disconnect from AirPlay and pause iTunes when it is turned off.
This worked well but it has a few problems as the music is coming from a Mac in a different room; there is no way to change the volume, skip a track, or do a basic play / pause toggle2 without going upstairs to do it manually on iTunes. After thinking about it for a little while, I was able to get the track skipping and play /pause done fairly easily. First of all, I use the following config:
I won’t detail the AppleScript for each of these as they are fairly basic.
Next, I used the ‘Scenes’ feature of HomeKit to set up named scenes that related to each toggle. For example, “iTunes next track” is a scene that ensures the “Music Track Control” accessory is turned on whereas “iTunes previous track” ensures that accessory is off. The play / pause controls work in a similar way. Once activated, this meant I could say things like “Hey Siri, iTunes next track” or “Hey Siri, pause iTunes”.
Whilst scenes can be incredibly useful due to the ability to use any text you want, they can’t be used for something like volume where you want to use a specific amount3. Unfortunately HomeKit doesn’t have the concept of speakers so there isn’t a way to naturally create an accessory that deals with volume so I improvised and created a fake lightbulb that controlled an AppleScript via the brightness property; homebridge-applescript-file-lightbulb4.
The volume.applescript file looks like this:
With this in place, I can say “Hey Siri, set iTunes volume to 50%” and it will work. This is because lightbulbs have a brightness setting and Siri interprets “set [name] to [percentage]” as being “change the brightness of this device” just in this case we are hitting an AppleScript with our brightness value.
This has been a fun little exercise in trying to hack HomeKit into what I want to use it for. I’m hopeful that iOS 10 will add native support for HomeKit speakers as then this could be done without reverting to AppleScript and I’d be able to stream from my iPhone via Siri but it’s pretty cool for the time being.
You can find both of my Homebridge plugins on NPM and GitHub.
Actually my own setup has two playlists. I have a playlist for Monday through Saturday with my music but on Sunday my wife gets up first so I have a different playlist for her. The code for that is if weekday of (current date) is Sunday then↩︎
At first I’d get around this by just using “turn off kitchen music” followed by “turn on kitchen music” as the playlist was shuffled but hardly an ideal solution. ↩︎
I could have done the same as the track control AppleScript and had a “iTunes volume up” and “iTunes volume down” scene to increase or decrease by 10% each time but that would be a pain if you wanted to change the volume by a large amount. ↩︎
Last year I worked on rebuilding the Great British Bee Count iPhone app for Friends of the Earth. Today, an updated version of the app has gone live to support this years count which runs from 19th May until 30th June 2016:
There are several updates including a redesigned bee picker, more details of the bees, and a list of plants suitable for attracting bees sponsored by Waitrose. The app is also now universal with support for both the iPhone and iPad1.
This came about due to a restriction in the App Store. If you search for an iPhone-only app on the iPad, you have to manually select “iPhone-only” when searching which nobody in their right mind does (as iPhone apps run on iPad just fine in a scaled mode). To make it so that searching “Great British Bee Count” on the iPad App Store would show the app, it had to be a universal app but the client didn’t want to have a custom design for the iPad but continue to use the iPhone scaling. There isn’t an elegant way to do that (as turning on Universal mode will cause AutoLayout to treat the iPad like an iPad) so I solved this by writing a custom view controller that would take the iPhone 4s XIB files and then simply scale them upwards with an affine transform. I also added an extra border on the sides that matches the yellow background to avoid the black boxing that the previous version had. This works great on all iPads (including the iPad Pro) but the text on some items is quite blurry on the non-retina iPad 2 due to a bug within the Apple Frameworks; it’s intelligent enough to use the retina imagery so that images look smooth but not on text apparently. ↩︎
When I started out as a web developer I was fascinated by APIs; the ability to hook into other data sources or update other applications from my own had huge potential that is finally being realised with the Internet of Things1. It is now possible to write apps that interact with physical objects via such things as IFTTT or dedicated APIs for devices like the Philips Hue. I’ve personally been very interested in the HomeKit initiative from Apple although it hasn’t really taken off yet.
Whilst I have a lot of internet connected devices, I currently own just a single HomeKit-compatible device; an Elgato eve door & window that lets me ask Siri if my back door is currently open2. Luckily there is the Homebridge project (which I’ve written about previously) that allows me to connect all of those together and use them as if they were native HomeKit devices.
Over the past few months, I’ve been dabbling with HomeKit scenes so that I can control groups of devices all at once like so:
Good Morning
Turns on downstairs lights
Starts playing my “morning” playlist in kitchen via AirPlay
Disables security cameras
Good Night
Turns off all lights (including bedside table lamp)
Stops any playing music
Enables security cameras
This has been working well but I frequently forget to use scenes as many of these things can be controlled in other ways. For example, I might turn off the downstairs lights with my Hue Dimmer Switch and then turn off the bedside table lamp via the Hue today widget which means I forget to activate the security cameras. The eventual solution I’ve come up with is a simple push button (from Flic) stuck under my bedside table that toggles between my Good Morning and Good Night scenes. In this article, I’ll tell you how I did it!
First of all, you are going to need a Flic. I did a lot of research on “smart buttons” and this one was far and away the best3. It’s small, has a changeable battery (that lasts 18 months), looks good, and it has an iOS SDK complete with background Bluetooth LE support so you can wake your app up even if it is inactive. The amount of code necessary to support Flic is ridiculously small; you can take a look at their iOS tutorial to see it in more detail but essentially you will be kicking your user to the Flic app in order to assign a button to the app. Once you’re back, it’s a simple case of activating a singleton and listening for a manager:didReceiveButtonDown request.
With the button tested and working, I set to work on adding HomeKit support to my app. This is relatively straightforward although you do need to create a dedicated provisioning profile in order to add the necessary entitlements to your project. Within a few minutes, I had a toggle in my app that would toggle between my Good Morning and Good Night scenes. That was when disaster struck; the HomeKit APIs do not work when your app is not in the foreground4.
This threw me for a little while until I decided that my app would simply have to connect directly to my mac and interface with the Homebridge software that was basically powering everything5. After a quick search, I found Homebridge-websocket, a plugin that basically adds a custom websocket server as a platform to HomeKit. You can create accessories (like a switch) within the websocket server and then you’ll get callbacks when they are turned on and off.
I planned to make use of this via a HomeKit property known as “triggers”. A trigger is basically a way for HomeKit to monitor a specific device for a specific value (i.e. when a switch is “on”). As soon as it sees that value, it will activate a chosen scene. I set this up by creating two new switches within Homebridge-websocket; ws_morning and ws_evening. I then set up two triggers within HomeKit; WebSocket Morning (which triggers the Good Morning scene when the ws_morning switch turns on) and WebSocket Evening (which triggers the Good Night scene when the ws_evening switch turns on). The only change I needed to make to my scenes was to ensure that both ws_morning and ws_evening are set to “off” when either is activated; this ensures that I can always turn them from “off” to “on” and thus cause the trigger.
The final step was to update my iOS app to talk to the websocket server instead of to Homebridge. I found a simple drop-in framework named Starscream which allowed me to connect to the server and send commands which led me to this code for the entire project:
Note: this is just a personal project that I built as quickly as possible. By nature, it is a fairly dirty hack. You shouldn’t write professional apps like this.
With this code in place, I can now press my Flic button (stuck to the underside of my bedside table) and it will toggle between my Good Morning and Good Night scenes. This is the full process of what is going on:
Press the Flic button
Wakes up the iOS app (if terminated or backgrounded) and hits the toggle function
App sends a websocket command to the Homebridge-websocket server
Server turns on a fake switch
HomeKit notices that the fake switch has been turned on so fires trigger to run appropriate scene
Scene activates or deactivates various accessories and then switches the fake switch back off ensuring it is ready for next button press
This is a long way away from the process I’d envisioned (press button, wake up app, toggle scene with HomeKit API) but it has been a fun challenge finding a way to get it work. I now have a physical button that controls multiple aspects of my home with a single press - it’s pretty cool!
“Why not just build your own with a Raspberry Pi or similar?”. Because I can’t be bothered; the cost of my time to do that is more than the cost of buying an incredibly well put together button that already does exactly what I want. ↩︎
Probably for a very good reason, you likely don’t want an app reading information about or controlling your HomeKit devices when you aren’t literally using the app. It’s a bit of a pain though for a simple app like this. I did manage to find a com.apple.developer.homekit.background-mode entitlement but there is no way to generate a provisioning profile that uses it. Maybe Apple will add this in a future version of iOS but I wouldn’t count on it. ↩︎
Technically, as all of my devices have their own APIs I could have just built my app to directly talk to Hue, Canary, iTunes, etc and rebuild my scenes that way. That idea didn’t sit well with me though as it’d be a lot more work and effectively duplicating what I already have. Similarly, I could have just used the native IFTTT support with the Flic to turn on or off some of my devices but that would add more latency and I don’t want to trust my connected home to something that has to be connected online; the beauty of HomeKit is that it works locally. ↩︎