Want to keep up to date? Sign up to my free newsletter which will give you exclusive updates on all of my projects along with early access to future apps.

As we get to the final month before WWDC 2023, I’m reminded of all the new APIs that were released at WWDC 2022 that I haven’t made use of yet. One of those new APIs was the DeviceDiscoveryUI framework which allows an Apple TV app to connect and communicate with an iPhone, iPad, or Apple Watch.

A good example of this would be how the Apple Watch communicates with the Apple Fitness app:

Image © 2022 Apple

It’s not necessarily a fair comparison as whilst you might expect them to be the same, the DeviceDiscoveryUI framework has a number of restrictions:

• It only works on tvOS (so you can’t communicate between an Apple Watch and an iPad like Apple Fitness can)
• It only works on Apple TV 4K (Apple Fitness can work with Apple TV HD)
• The tvOS app can only connect to one device at a time (i.e. you couldn’t make a game with this that used two iPhones as controllers)
• The tvOS app can only connect to other versions of your app that share the same bundle identifier (and are thus sold with Universal Purchase)
• This will not work on either the tvOS or iOS simulators. You must use physical devices.

The UI for the connection setup is also different to Apple Fitness as we will see shortly.

My use case for this technology is a bit convoluted as I was really looking for an excuse to use it rather than the best fit. I have a personal app named Stoutness that I use on my Apple TV every morning to give me a briefing on my day whilst I do my chiropractic stretches. Using shortcuts and various apps on my iPhone, I send a ton of data to my server which the Apple TV app then fetches and uses. The app also communicates directly with some 3rd party APIs such as YouTube and Pocket.

One of the main reasons for the app is to get me to work through my backlogs of games, books, videos, and articles by having the app randomly pick from my various lists and presenting them to me; I then know “out of the 4 books I’m currently reading, I should read x today”. The problem is that later in the day I often forget what the app had decided I should use, a particular problem when it suggests 5 articles for me to read from a backlog of about 200 😬. Whilst I cache this information daily in the Apple TV app, it’s a bit of a pain to fire it up just to skip through a few screens and remember what I should be reading. Surely this information would be better on my phone?

The obvious way to do this would be for the server to make the calls to Pocket and YouTube and then store the daily cache in my database along with the random choices of games and books. An iOS app could then download that in the same way the tvOS app does. This is true, but it’s not as fun as learning a new framework and having my phone connect to the Apple TV to a) send all the data that my shortcuts used to do directly and b) have the cache be sent back in response ready to be used on iOS.

After a brief look at the docs, I naively assumed this would be done in an hour as it looked vaguely similar to the way in which an iPhone app can talk to an embedded Apple Watch app or a Safari extension via two way messaging. After 4 hours, I finally got something working but it does not feel as solid as I would like…

Apple provide a developer article titled “Connecting a tvOS app to other devices over the local network” that sounds like it should be exactly what we need. It details how we present the connection UI (in both SwiftUI and UIKit), how to listen for the connection on iOS / iPadOS / watchOS, and how to initiate the connection. However, there are two issues with this article.

First of all, most of the code in it doesn’t actually compile or is being used incorrectly. The SwiftUI code references a “device name” variable which isn’t present¹, fails to include the required “fallback” view block (for displaying on unsupported devices like the Apple TV HD), and presents the device picker behind a connect button failing to notice that the picker itself has it’s own connect button which sits transparently above the one you just pressed.

For the UIKit code, it references an NWEndpointPickerViewController which doesn’t exist. The correct name is DDDevicePickerViewController.

Once the actual picker is presented, things start to look very promising. You get a fullscreen view that shows your app icon with a privacy string that you define within Info.plist on the left hand side whilst any applicable devices are listed on the right hand side:

An important thing to note here is that the devices do not necessarily have your app installed, they are merely devices potentially capable of running your app.

When we initiate a connection to an iPhone, a notification is displayed. The wording can’t be controlled and will be different depending on whether the corresponding app is installed or not:

Connection notification request for iOS from tvOS both with and without the app installed. If the app is installed, the notification uses the Apple TV name for the title (“Office” in this case).

You seem to have around 30 seconds to accept the connection otherwise the tvOS interface goes back a step and you need to send a new request. If you do not have the app installed, tapping the notification will take you to the App Store page.

We now come to the second problem in Apple’s documentation:

As soon as the user selects a device, the system passes you an NWEndpoint. Use this endpoint to connect to the selected device. Create an NWConnection, passing it both the endpoint and the parameters that you used to create the device picker view. You can then use this connection to send or receive messages to the connected device.

The emphasis above is mine. This is the extent of the documentation on how to actually use the connection to send and receive messages. It turns out that the connection uses classes from the In-Provider Networking that was introduced in iOS 9 specifically for network extensions. In fact, this is still the case according to the documentation:

These APIs have the following key characteristics:

• They aren’t general-purpose APIs; they can only be used in the context of a NetworkExtension provider or hotspot helper.

There is zero documentation on how to use these APIs in the context of Apple TV to iOS / iPadOS / WatchOS communication 🤦🏻‍♂.

In terms of sending messages, there is only one method aptly named send(content:contentContext:isComplete:completion:). This allows us to send any arbitrary Data such as a JSON-encoded string.

The real problem is how to receive those messages. There is a method named receiveMessage(completion:) which, based on my work with watchOS and iOS extensions, sounds promising. Apple describes it as “schedules a single receive completion handler for a complete message, as opposed to a range of bytes”. Perfect!

Except it isn’t called, at least not when a message is sent. In a somewhat frustrating act, the messages only appear once the connection is terminated either because the tvOS app stops or because I cancel the connection. I tried for multiple hours but could not get that endpoint to fire unless the entire connection was dropped (at which point any messages that were sent during that time would come through as one single piece of data). I can only assume the messages are being cached locally without being delivered yet when the connection drops it suddenly decides to unload them 🤷🏻‍♂.

It turns out you need to use the more complex receive(minimumIncompleteLength:maximumLength:completion:) which requires you to say how big you want batches of data to be. You also need to resubscribe to this handler every time data appears on it. The problem here is that whilst there is a “completion” flag to tell you if the whole message has arrived this is never true when sending from tvOS, even if you use the corresponding flag on the send method. In the end, I limited the app to 1MB of data at a time as everything I send is well below that. I’ve never run into a problem with only partial data being sent but it is a potential risk to be aware of.

If you were using this for critical data, I’d probably suggest only sending encoded text and providing your own delimiter to look for i.e. for each string that comes in batch them together until one ends in a “|||” at which point you will know that was the end of a message from tvOS.

On the positive side, the connection setup and data sending are near instantaneous and the user facing UI works well. However, as there were already low-level network solutions to send data between devices (including non-Apple devices) it’s incredibly odd to me that Apple went to the effort of creating a beautiful device pairing API and UI for both SwiftUI and UIKit but didn’t extend that to the basics of sending data. Local networking is hard. I have no interest in diving into the minutia of handling UDP packets; I just want to send some basic strings between devices!

In order to get this all working for my own app, I created a class named LocalDeviceManager that handles this all for you along with a SwiftUI demo project for both tvOS and iOS that demonstrates how it works. The call site on tvOS is very simple:

@ObservedObject private var deviceManager = LocalDeviceManager(applicationService: "remote", didReceiveMessage: { data in
    guard let string = String(data: data, encoding: .utf8) else { return }
    NSLog("Message: \(string)")
}, errorHandler: { error in
    NSLog("ERROR: \(error)")
})

@State private var showDevicePicker = false

var body: some View {
    VStack {
        if deviceManager.isConnected {
            Button("Send") {
                deviceManager.send("Hello from tvOS!")
            }
            
            Button("Disconnect") {
                deviceManager.disconnect()
            }
        } else {
            DevicePicker(.applicationService(name: "remote")) { endpoint in
                deviceManager.connect(to: endpoint)
            } label: {
                Text("Connect to a local device.")
            } fallback: {
                Text("Device browsing is not supported on this device")
            } parameters: {
                .applicationService
            }
        }
    }
    .padding()
    
}

Similarly, it’s trivial to set up an iOS app to communicate with the tvOS app:

@ObservedObject private var deviceManager = LocalDeviceManager(applicationService: "remote", didReceiveMessage: { data in
    guard let string = String(data: data, encoding: .utf8) else { return }
    NSLog("Message: \(string)")
}, errorHandler: { error in
    NSLog("ERROR: \(error)")
})

var body: some View {
    VStack {
        if deviceManager.isConnected {
            Text("Connected!")
            Button {
                deviceManager.send("Hello from iOS!")
            } label: {
                Text("Send")
            }
            Button {
                deviceManager.disconnect()
            } label: {
                Text("Disconnect")
            }
        } else {
            Text("Not Connected")
        }
    }
    .padding()
    .onAppear {
        try? deviceManager.createListener()
    }
}

There are more details on how this works on GitHub.

Judging by the complete lack of 3rd party apps using this feature or articles detailing how to use this API I’m going to go out on a limb and say it’s unlikely we’ll see any improvements to this system in tvOS 17. Regardless, I’ve filed a few bug reports in the hopes that the documentation can be tidied up a bit. Just be aware that this is not the robust solution I was hoping it would be!