Ben Dodson

Freelance iOS, macOS, Apple Watch, and Apple TV Developer

Why I built an Item Finder for Gowalla

Last Thursday, I expanded my Gowalla Tools website (a companion site for the popular geo-location game Gowalla) to include a new feature known as the Item Finder. It works by displaying a list of spots in any area you choose, and them removing those that don't have items you are missing from your vault. I always knew that this would be a controversial feature and that I'd need to write a piece explaining why I had built it - now that a few people have complained that it is "changing the core nature of the game", it is probably an apt time to publish my opinion on why this is not the case.

Ever since I began work on my Gowalla Tools project, people have consistently been in contact to ask me to make an item finder. They were frustrated that it seemed so difficult to find existing items in their areas when they were so abundant in other areas. For example, in the UK it's very difficult to find a "Cowboy Hat" as these are only available in spots marked as "Saloons" (of which we have very few). However, a few of these rare items have made it to our shores thanks to random items being seeded into new packs (meaning new players have a chance of having a rare item such as this available to them from the start) which are then dropped at locations to make them founded spots.

In the first version of Gowalla Tools, I added a feature which allowed you to see what items were available at any spot you chose. This was not exactly game changing as you could work this out yourself on the Gowalla website by seeing what items had been dropped and picked up until you ended up knowing what was there - my site simply did it faster. When I was initially asked to add a search feature, I was hesitant as I felt it took a lot away from the game and made it simply an item collection game similar to Packrat (which is how the vast majority of players came to find Gowalla) rather than a game about going to places. I actually felt so strongly about changing the game into item collection that I edited the first draft so that items appeared in a popup window rather than inline - this meant it took a good 3 seconds or so before you could view the items at a spot and thus made it so that you couldn't just click around and see what was there. Aside from my issues around gameplay, there were technical constraints as searching that many spots would place heavy demand on both mine and Gowalla's servers so I ruled it out.

It was a month or so later that I suddenly started to hit a brick wall with items. I was missing around 10 items all of which were proving to be incredibly elusive as they were created mainly for the Texas area (e.g. Longhorn). I realised I was spending a lot of time on the Gowalla Tools map clicking around trying to find out what items were available in my area and found quite a few items I had been missing that way. Gowalla also stepped up their game and introduced version 1.3 of their app which allowed you to vault items from within the application - they were starting to take items more seriously so my site needed to do similarly.

I started to get underway with a brand new site and introduced the 13 days of Gowalla Tools Christmas - a daring attempt to build 13 new tools for the site within the 25 days leading up to Christmas. The first tool was obvious to me; the map needed to be radically overhauled and easier to search and view spots. I improved the item offering quickly by showing you what items were at a spot and highlighted those items that were missing from your vault. On day 2, I added the item search. It didn't work the same way as many people had hoped (e.g. type in "Longhorn" and it'll show you all the spots that exists) due to the technical constraints. However, it did allow you to load all the spots in your area and then remove all but those that had items you needed.

Initial feedback was excellent and I had felt that my compromise between ease of use and item searching had been well balanced. However, I was recently alerted to a post on GetSatisfaction (and several tweets) that felt that the new feature was game changing for Gowalla and was tantamount to cheating.

I disagree.

My main reason for playing Gowalla is nothing to do with items - it's about finding new places and visiting those places. It's about finding interesting spots in your neighbourhood that you may never have known were there and then going out and visiting them. In my opinion, items were added in order to lure more people from Packrat and to make it seem more like a game but the key thing is locations. Yes it could be perceived that the Gowalla Tools Item Finder makes it unfair to those that knew where items were previously who are now unable to get them as somebody else has now found them, but that's all part of the game. You can't pick up items unless you are checked in at a location so if you don't go there, you can't have it. If someone else gets there first, then the item is theirs!

The one thing that has annoyed me from the start about Gowalla is the rights that people have given themselves in terms of "that item is mine" or "I'm going to create that spot so you can't". There are no "rules" in Gowalla apart from "first come, first served". The claim that this is game changing is ruled out completely by the fact that Gowalla are fully aware of the Gowalla Tools website and support it (with many of their developers retweeting and using the functionality). I speak with the developers frequently to let them know when I'm releasing new tools that I feel conflict with the core gameplay or could interfere with their servers (e.g. by placing too much demand). I would of course remove any tool they felt violated these rules and as yet no request has been made - only encouragement to do more!

On the positive side, there are a lot of people using the Item Finder in the way I intended. I received a lot of tweets yesterday from people thanking me as they had found an item at a spot and were now going their specially to get it. Some people were travelling up to an hour out of their way to get an item and I think that's great! They were playing the game, going outside to new places rather than waiting and hoping that nobody would know an item they would quite like was there.

The item Finder does not change the core gameplay of Gowalla in any way - it adds to it by showing you spots that are of interest to you. There are people who are affected (namely those on the Android platform) but unfortunately that is a shortcoming on Gowalla's part in not implementing the same level of gameplay for all users. They shouldn't have launched the Android version without all the core game components but this does not mean that Gowalla Tools should be limited. There are several people on devices that can't play Gowalla who would like to - should we horde items away for them? Of course not! Items can still be obtained in the traditional ways and as yet they aren't limited to batches in the same way Packrat items are so I don't see the problem.

Over the coming weeks I'll be releasing more tools that push forward this key feature of getting out there and discovering new places and todays new tool (Flickr Integration) hopefully expands this more by giving you more visual information about a place before you get there. I stick firmly by my decision to release the item finder and I hope that the majority of you will agree with me.

Now get out there and start Gowalla'ing!

The Apple Magic Mouse: Necessary upgrade or expensive luxury?

Apple recently announced the introduction of their latest peripheral, the Magic Mouse (so called due to trademark problems with the existing "Mighty Mouse" name). The new mouse offers the same multitouch features as found in the trackpads of recent MacBooks and of course the iPhone and iPod Touch. But is it any good?

Previous Apple Mice

The previous model (the "Mighty Mouse") was a complete disaster in my opinion and the only Apple product apart from the Tiger Xserve that I have never been able to defend (even as a devout fanboy). The main problem was that the scroll button (or 'nipple') would get clogged with dirt so easily that it would invariably stop working after a couple of weeks heavy usage. The official Apple solution to this (which has since been pulled from the KnowledgeBase) was to turn it upside down and bang it in your palm! Whilst it had a couple of benefits in terms of OS X integration (such as activating Exposé by squeezing) these features quite often generated more problems than they solved (such as activating Exposé by accidental squeezing). So how does the Magic Mouse differ?

Design

The Magic Mouse has no buttons, no squeezing, and no cable. Like the unibody MacBooks, it has gone for a simple look using the minimum number of parts; one piece of aluminium and one piece of glass. It is incredibly short (in terms of height) compared to other mice whilst being slightly longer than others. With no distinguishing buttons and a symmetrical shape, the only visual aid to placing it the right way round is the standard Apple logo displayed on the top. In my opinion, the design is beautiful yet simplistic (as you would expect from any recent Apple product) but would look at home in a museum of modern art. Not many people care about the aesthetics of a mouse (which is strange considering it's prominent location in most homes) but now I've had this on my desk, I'd find it very difficult to go back to something uglier.

Before you've even got it to your desk you are struck by the beauty of the device. The packaging is incredibly similar to the new iPods (which makes a lot of sense) but it still amazed me to see just how little packaging was used.

Apple Magic Mouse packaging

It is difficult not to pick up the Magic Mouse in the Apple Store and not be impressed with the way it is packaged. This is how all packaging should be in the modern world - I'm not talking just about technology (although there is large scope for improvement) but in everything from food to clothing. Smaller packaging means not only that you can put more product in a store but also that you save fuel from transportation (as you can move more units in the same lorries / boats leading to less trips) and minimise wasteful plastic packaging. I was impressed when I found out that the mouse came with batteries (very rare these days) but even more impressed when I discovered they were already in the device which was simply turned off. This saves a huge amount of space and more plastic.

Functionality

As other commentators have noted, there is no "wow" moment with this mouse. By this, they mean that the majority of Apple products have that initial euphoria when you use it for the first time (e.g. when you first unlock the iPhone or when you lift the MacBook out of it's stunning packaging) but this mouse doesn't have that. You just use it. However, what it does have is a strange sensation roughly after an hours use when you try to go back to using a normal mouse. You find the scrolling isn't as intuitive and that you've been using the multitouch features without even thinking about it. It is a very easy mouse to get used to and it is that which provides the "wow" as you realise you have been completely taken in.

In terms of setup, it's a standard bluetooth mouse so a quick trip to the "mouse" page in Settings will have it set up in know time. As of the time of writing, you need to download an update via "Software Update" to take advantage of the multitouch gestures but when OS X 10.6.2 is released it will be built into the OS by default. The only negative here is that the update is 62MB which is a little excessive - this is most likely due to the videos that show you how to use the gestures in a similar way to the videos found on the MacBooks to show you how to use the trackpad. In any case, setup is quick and painless.

At present, there are very few actual uses of the multitouch. Clicking is performed by a physical click (not just tapping as per the trackpads) but there is only one button. The multitouch comes into play by detecting where your finger is on the surface and then linking that up to whether it is a primary or secondary click - this is very useful if you are left-handed or ambidextrous (as I am) as you can quickly switch the mouse to your other hand and it still feels comfortable thanks to the symmetrical design. The only other two uses are scrolling (a simple case of moving your finger around on the surface - you can go up, down, left, right, and diagonal which is useful for zooming in photos, etc) and the two finger swipe which lets you go backwards and forwards through browser history and photos. Apple also point out you can zoom into a page by holding the control button on the keyboard and swiping your finger up and down but this has always been possible by holding control and using a scroll wheel so I wouldn't describe it as a unique function.

Criticism

There appear to be two main criticisms of this device if you read the forums or the reviews pages on the Apple Store; size and functionality.

A lot of people have found the size uncomfortable as it's so much shorter than previous mice. However, I note that most of these reviews were from people going to their local Apple Store and trying it rather than at home. I had read these reviews and so when I got to the store I made a point of squatting down to try it as the benches in the store are very low (meaning you're not holding the mouse as you normally would). This made a huge difference to how it sat in my hand and so I think a lot of these reviews just stem from this problem. I would never buy a mouse that cost this much money without trying it out first but it is important that you try and recreate how you would use it as much as possible.

The second problem of functionality is much more valid in my mind. For a multitouch device, this mouse uses a woeful amount of multitouch features. When it was first announced I was expecting all the features of my trackpad such as pinch, rotating, zooming, 3-4 fingered swiping, and maybe a few extras as well. Without this option (and the customisation that goes with it) there are just too few buttons for a lot of consumers. The squeeze buttons on the Mighty Mouse may have been annoying to some but at least it gave you a way to control Exposé from your mouse - something that can save a lot of time over the course of a day. Likewise with no middle button control you can't get to your dashboard or spaces easily (or use it like a PC mouse for opening new tabs in browsers).

My theory on this lack of functionality is that Apple know they are going to get negative reviews that focus intently on this issue. They will therefore release an update in a month or so which fixes it and adds full multitouch support which will then write off all of the negative reviews. In this way, they are able to choose the negative issue that people will focus on safe in the knowledge they can resolve it later on and make it seem that everybody loves the device. People who had an issue with the device can't then say "x is bad about it" as their original review will have focused on the multitouch issue as the only negative. From what I've seen of Apple's marketing machine (one of the best in the world to my mind) and my studies of politics, this is exactly the kind of controlled negativity that I would expect. I guess we'll see if I'm right in a month or so!

Conclusion

So is the mouse worth £55 (or $69 - strange currency conversion there...) - at the moment I would say no simply due to the lack of features supported by the multitouch. If these are corrected in a future update (and I'm confident they will be) then this will be one of the finest mice on the market, but at the moment it feels as if it's full potential is being restricted. There is no question that this is a beautiful device, but the functionality needs to match the aesthetics before I can fully endorse it.

Update - 11th Nov 2009: I was recommended BetterTouchTool by @ricklecoat which is a free app that enables some extra features of the mouse. For instance, my Magic Mouse now reveals my desktop when I slide two fingers up / down and does Exposé when I tap (yes tap, not click) with three fingers. Has improved my productivity no end!

Gallery

I've put together a few shots of the unpacking of the Magic Mouse as well as comparing it with a few of my older mice. Check it out on Flickr.

How to pitch an app idea to an iPhone developer

Thanks to my appearance on The Gadget Show earlier this week, I've been inundated with people emailing me with ideas for iPhone applications. The majority of them have no understanding of the development aspects (which is fair enough) but have ideas for apps they want me to build, usually with payment via the profit made. I've had quite a few interesting ones come through, but some have been suggested with very little thought or realism applied. I decided to create this article to show prospective idea senders how an idea should be presented along with a few answers to common questions I've received

Research your idea

There is nothing worse than replying to 2 or 3 emails in which the sender is being cagey about their "brilliant new idea" only to discover (often after spending time signing, scanning, and sending an NDA) that it is actually an idea for which there are over 250 apps available on the store. The key thing before contacting a developer is to make sure that your idea has not been done already. If it has, then either think of something else or find a unique selling point in your app that will make it stand out from the crowd.

Make sure your app is relevant

So your idea hasn't been done before? The App Store has been open for over a year with over 65,000 applications so the next question should be "why hasn't it been done". There are three answers to this question: it's impossible, it's not a good idea, or it's unique.

Impossible: When I say "impossible", I mean that you've come up with something that the iPhone SDK won't allow (it may still be a good idea). For example, you may have an idea to download soundclips off the internet and store them in the iPod library. A good idea in theory but the iPod library only has read access so you are limited to what you can do. There are several rules within the iPhone SDK which limit what can be done so be prepared that your idea may not be possible within those confines - there may, however, be workarounds (e.g. in the example above you could build a custom application to play the soundclips you've downloaded).

Bad Idea: The most common bad ideas I've had sent tend to be duplicating the functionality of an existing website into an iPhone app. You might have a killer idea on your website and be the market leader in a certain arena. This does not mean that converting it directly to the iPhone is going to be a good idea. If your idea is working on a website already, then it is already accessible from an iPhone (provided it's not made in flash) so there is little benefit to making a custom application apart from it'll look slightly better. It may be that there are a huge number of potential iPhone users who are put off by your website running on the device. In that case, you should consider making a web app. This is a way of using standard HTML and CSS to make a website look like an iPhone application but takes no extra knowledge than that of building your regular site. I've built a few web apps myself and found it to be incredibly easy if you are using something like UiUiKit.

Unique: You've checked the App Store and found no trace of your idea, you are certain it's possible to do with the features of the device, and it's not a simple port of an existing website into an application. In that case your app is probably a unique endeavour and should definitely be pursued!

Know your audience

When I've been pitching for web development work in the past, I've often had clients say to me "we should definitely make an iPhone app for this as well" to which I usually reply "why". There is an unhealthy obsession at the moment with making applications simply for the fact that it shows you are modern. However, you can easily spend a large sum of money building an application for a small bit of street cred only to find out that none of your target audience actually use iPhones (or wouldn't use your application anyway).

If your idea is simply duplicating functionality of your website, then you shouldn't make it an iPhone application (or even a web app) unless more than 20% of your visitors are using an iPhone or you've had a lot of people ask for an app. You wouldn't suddenly start supporting Internet Explorer 5.5 on the mac again but quite often you'll find more people visiting your website with that than with an iPhone yet people will often overlook simple statistics to try and seem more modern.

Don't make custom apps or websites for devices or browsers that your users aren't using!

Understanding development costs

An iPhone app is not an easy thing to build and so, like a house or a website, you are paying for the expertise of the developer you commission to create your application. If you believe your idea is unique and you are going to make a large profit from it, then you need to pay your developer for the application in a one-off fee the same way that you would pay a builder or any other tradesman. However, if you don't have the finance to pay to have your idea built, then some developers will be open to the idea of building the idea for you for free but then taking a percentage of the profit that comes in from selling the app.

Negotiating for development costs is often very similar to the Dragon's Den program in that you will negotiate in terms of equity (how much of the profit you are willing to share) in order to get an expert to build the application. However, where this often falls down is with prospective clients offering around 20% equity. If you come to the table with nothing but an idea (and aren't planning on doing any designing, building, or marketing) then you can't expect to find a developer that will build your app and agree to taking a small percentage.

In my own case, I prefer to be paid for the application but I will occasionally deal in terms of equity if I think the idea is good enough. Having said that, I refuse to take less than 50% if I'm expected to build an application from the ground up with no other form of payment - most other developers are the same.

Update, December 22nd 2015: Whilst this was the case in 2009, it certainly isn't the case in 2015 and the vast majority of developers will not work for any form of equity. Please ready my new article iOS developers don't work for free.

It is important to note that it is not possible to give a straight up cost for an application before you've heard the idea. I've received several enquiries of the sort "can you tell me how much it'll cost to get an app built". Pricing is generally based on the amount of time required so if you want a basic utility then it will cost a great deal less than a complex 3D game. Bear in mind that you may be asking your developer to build a website or server software to run your applications (if you plan on using push notifications for example) and so these should be factored into your financial calculations.

Note: with all applications, Apple takes approximately 30% of the sale of each application to cover the costs of the App Store. This means that for every 59p sale, you keep approximately 42p. You are paid at the end of each month by Apple but you are paid by territory and if you haven't earned over $150 in that territory then the amount rolls over to the next month that you do. For example, if you made $100 in one month in the USA, then you are under the $150 threshold. That rolls over to the next month. If you then made another $100 you would be over the threshold so at the end of that month you'd be sent $200. When you are negotiating with an iPhone developer, be sure to clarify if they are talking of their percentage in terms of sales price (before Apple takes it's 30% cut) or profit (after Apple has taken it's cut).

Building an app on your own

Apple provides all of the tools you need to make an iPhone application via it's developer website at http://developer.apple.com/iphone/ although you will need to be using a mac with an intel processor. You can't build iPhone applications on windows.

With the free SDK that Apple provides, you can use all of the features of the iPhone and test them in the iPhone Simulator that is also provided. You can't, however, run the code on your own device or submit it to the App Store. To do that, you'll need to get a developer license which costs $99 per year and can be purchased through the developer website above. Once you have the license, you'll be able to generate provisioning profiles for your apps which will enable it to run on your own device or up to 100 other devices (e.g. friends, colleagues, testers).

All apps are written in Objective-C so I'd highly recommend you buy a book on the subject. If you are coming from a web-based background (e.g. PHP, Ruby, JavaScript, .net) then I'd also recommend you start by learning C before moving onto Objective-C. You may be raring to jump into the SDK and start using things like the accelerometer and location services, but if you don't know how an array or a dictionary works, then you'll never be able to build an app that works well.

Getting your apps into the App Store

The only things you need are your completed application code (that should have been tested extensively in both the simulator and on actual devices) and a developer license. Once you have these, you are able to generate the correct certificates to publish your application to the App Store. You will be asked to supply not only text such as descriptions, app title, and keywords, but also screenshots and a 512x512px image of your application icon for use in Apple's promotional materials so make sure you have these available.

The actual process for submission has been under a lot of criticism but basically you submit your app and then wait for a while (usually around 14 days - Apple have placed an indicator of queue length on the iTunes submission portal now) to find out if your app has been rejected or accepted. If it's been rejected then they should supply you with information about what is wrong and how to fix it. If it's been accepted, then congratulations, your app is ready for prime time!

Note: a common question seems to be "can I submit an app with your developer license". If we were to assume that I built you an iPhone app, then yes I could submit it to the app store using my license saving you $99. However, this would mean the app appeared under my company name rather than your own and all payments for the app would go into my bank account. Whilst this is possible, most clients would prefer that their company name is displayed and that all finance goes through them.

Copyright and other legalities

When Apple checks the application in it's approval process, it does not take into account copyright or any other legalities of that type. This means that technically you could steal an idea such as "Super Mario Bros", make a duplicate app, and then submit it. However, you are still liable for breaches of copyright and so could be sued by the correct copyright holders and have to repay damages. The basic rule of thumb is don't use copyright images, text, or music, and don't mimic other peoples ideas or intellectual property.

Contacting a developer

If after reading all of the above you are pleased with your idea and want to get an iPhone developer on board, you will need to contact them with the following pieces of information:

  • A detailed explanation of your idea - if you are not comfortable with giving up your idea, then get the developer to sign a Non-Disclosure Agreement (or NDA) which will prevent them from stealing the idea. It is worth pointing out to the developer that you have already done your research and know that your idea is unique before asking them to sign anything as you will be more likely to get a favourable response.
  • What you are looking for - you'll need to detail what you need the developer to do (e.g. build the app, suggest changes, recommend a designer, etc) and also if you are looking to pay outright for the code or enter into a profit-share agreement (and the potential terms of such an agreement).

I reply to all emails that I receive but many developers will not get back to you if the two points above are not fulfilled. The more detail you supply, the more likely it is that a developer will want to work with you and form a professional relationship as it shows that you are serious and have researched your idea rather than being someone interested in simply making a quick bit of cash by copying an existing flash game.

Summary

I hope that this article has given you a quick insight into how you can make your ideas more appealing to an iPhone developer and answers some of the more common questions about the process. If you have further questions, please feel free to leave them in the comments below or contact me. I'll be updating this article as and when other common questions come in.

Social Beacon: Developing An iPhone App for "The Gadget Show"

You can watch the Gadget Show episode online and download Social Beacon from the App Store.

A couple of months ago, I was asked if I'd like to appear on Channel Five's "The Gadget Show" to take part in a challenge about iPhone applications with Jason Bradbury and Suzi Perry. I had only been developing iPhone apps for a short while but decided to enter into the spirit of the challenge by jumping head first into some of the trickier aspects of the iPhone SDK. The show and results of the challenge aired last night so I thought it was time for me to do a write up of the application and the process of building it.

The Early Stages

My day-to-day job involves me working as a freelance PHP developer but I'm a huge Apple fan and have owned every iteration of the iPhone. It wasn't long before I started toying with the idea of building iPhone apps, but the programming language was different to anything I'd ever done before. As a PHP developer, I found it very hard initially learning how to code in Objective-C (the coding language for iPhone apps). However, I purchased a couple of books including the excellent "iPhone SDK Development" and "Learn Objective-C on the Mac" and it wasn't too long before I became an iPhone developer.

After making it on to the App Store with Magnetic Flux & Metal Detector [iTunes Link], I was asked if I'd be interested in making the iPhone Application for "The Gadget Show". The initial concept was fairly simple: an application that allowed you to quickly choose from a list of questions to build a sentence which could be sent to your social networks. I agreed immediately and began working with my friend Liza Hayes on the design but hadn't realised exactly what I'd let myself in for.

In order to work as expected, the application would need to access an SQLite database on the device, be able to store custom sentences input by the user to that database, be able to build a sentence based on building an SQL query, and use the networking features of the iPhone to post to both Facebook and Twitter. I had done a lot of work with Facebook Connect and the Twitter API in my web development work and I knew that Facebook had a custom integration of its API for the iPhone. Even so, the networking problems were going to be tricky to overcome. To add to all of this, Jason suggested that he'd like a way to show off the accelerometer in the iPhone, and I was interested in the settings panes and how we could make the app easily customisable.

Developing the app

The iPhone 3.0 OS had just been released, yet as a registered iPhone developer I was working on iPhone 3.1 so there were enumerable problems in switching my Xcode environment and devices back and forth between the two versions. This was compounded with problems with Facebook Connect in that it had been written for the 2.1 OS and so was not 100% compatible with 3.0 leading to some caffeine fuelled Googling!

We had decided that the accelerometer could be used in what Jason called "Fingerless Functionality" whereby you could shake the phone left and right to navigate around the Social Beacon wheel, and then tip the phone up and down to progress backwards and forwards through the option. A simple shake of the phone would be enough to regenerate a sentence. These parts were all very straightforward (especially the shake, which can be detected with one line of code in OS 3.0) yet we came to a usability problem once we thought about sending the message. We couldn't use the shake or the down gesture, as these were used for regenerating the sentence on the final screen, so we needed to find another fingerless input method to post the message. The answer eventually came in the guise of the microphone: we could detect sound so that blowing into the mic would blow your message to the internet! Of course, this meant learning yet another aspect of the SDK that I had previously left untouched -- Core Audio -- but I'm always up for a challenge!

Distribution and the App Store

With various bugs overcome and a final test version working on my device, we were ready to distribute the app via the App Store. You may have heard some of the horror stories from the App Store -- it can take a very long time and you end up being rejected without reason -- but I have to say that I found the process to work incredibly well. It took 14 days from submission before Apple got back to us to say that the Application had been rejected due to a bug in the networking code (if you weren't connected to the internet the app would hang, or worse, say that the message had been sent). They were incredibly helpful and gave a detailed explanation of the problem as well as including some sample code to show how the networking portion could be handled better. After a couple of hours fixing it up, we resubmitted and the app was available around the world a few days later.

We were of course competing against Suzi's "Biker Blast-Off! [iTunes Link]" which was distributed at the same time, but crucially neither app was publicised on either Twitter or any other medium without mentioning the other with the same weight (so we'd invite people to try both apps). Within 3 days, Social Beacon was in the top 20 free social media applications whilst Biker Blast-Off was climbing up the overall free application charts worldwide and racking up a huge number of downloads. Interestingly, as both of our apps climbed higher, the reviews became more negative and I realised that this was true of nearly every app in the store. It seems that even if you are willing to give away a free product which could easily be charged for, people will still rip the idea to pieces (some comments on the game even saying "you'd need to pay me to play this game" - no pleasing some people!).

The Aftermath

Jason Bradbury and Ben Dodson

The show was aired last night (and is available to watch online) and has the full story as well as the final figures and the winner of the challenge, but the story of the apps continues. Since the show, both applications have now shot back up into the charts with both apps appearing in the top 10 free applications in the UK App Store. Social Beacon is currently sitting pretty at spot number 4 and I couldn't be happier with it.

A big thank you to the Gadget Show team (especially Colin Byrne, Chris Payne, and Jason Bradbury) and to Liza Hayes for her fantastic design work. It was a great experience and it looks like I'm going to be developing iPhone applications for a long time to come!

Gowalla Tools Web App: Find your missing Gowalla items!

For those of you who play the excellent iPhone GPS game Gowalla, I've built a handy web app that will allow you to see all of your missing items and where you can find them. In addition to telling you what type of spot a particular item is likely to appear at, it will also list specific spots if applicable (such as states - some items only appear in Texas for instance) and allow you to use the built-in location awareness of Safari in iPhone 3.0 to show you where the nearest spots of that type are. It is available today at http://gowallatools.com/ and is the first in a series of small utilities I'll be creating to help players.

Gowalla Tools - Login

The first thing you will see when you open the web app is a prompt for your Gowalla username. You don't need to login to the service, but it does need to know your username so it can search through your pack effectively (although you can also see your friends' missing items if you want to help them out with any of your own spare items).

Gowalla Tools - Missing Icons

You will then be shown all of your items in a list. Tapping any of these (e.g. "Conference Badge") will then show you which spots are likely to randomly give you the item in question when you check in.

Gowalla Tools - Icons Appear At...

The list will sometimes be broken into two sections: spot categories where you have a chance of finding the item on check in, and internal categories or spots. These internal categories are usually things such as states (e.g. "Texas" has quite a few items) or freebies (which I believe are items randomly dropped by devs) but are often actual spots (e.g. "Alamofire" - the offices of the developers).

With the standard categories, you can tap them to find spots in your local area (but you'll be prompted to allow the site to use your location - your location information isn't stored, it's just used to determine your nearest spots).

Gowalla Tools - Location Confirmation Gowalla Tools - Nearest Spots

If there are spots in your area, you can tap on them to view the spots information on the Gowalla website.

If you save the web app to your homescreen (press the '+' symbol in Safari and choose "Save to Homescreen") then you'll get a pretty custom icon so it looks just like a normal iPhone app (which it may well become one day). I'll be expanding the service offering as and when I find new things to include :)

Feedback

The site is in beta mode so there may be a few bugs and tweaks but if you have any feedback, please let me know. Happy Gowalla'ing!

Legal

The web app is not owned, maintained, or developed by Alamofire, Inc. - Gowalla and all other trademarks and imagery are copyright of Alamofire.

iPhone 3GS: Review and Speed Test (vs. iPhone 3G)

I was up at 5:30am this morning in order to start queueing for the iPhone 3GS outside my local O2 store - I'm happy to say that I was the first person in the queue and although I had problems in getting a second contract (eventually deciding to buy a PAYG version from the Apple Store) I am now the proud owner of the iPhone 3GS. In this short post, I hope to review a few of the key features as well as giving you some real world stats from tests I've run to show the differences between the iPhone 3G and the iPhone 3GS. Please use the comments section if you have any questions!

iPhone 3GS vs iPhone 3G

Initial Thoughts

My first thought was that the iPhone 3GS somehow felt nicer than the previous models of the iPhone. I remember upgrading from the original to the 3G and thinking that the new plastic back made it feel more comfy and it seems that again something has been done to the texture to make it seem more solid and comfortable. Although it looks exactly the same as the 3G, there are a few minor aesthetic details such as the lettering on the back now being the same colour as the Apple logo which makes it stand out a bit more.

The "oleophobic screen coating" (that's oil resistant to you and me) really does work incredibly well. With previous iPhones, greasy finger marks on the screen wouldn't go away even if you rubbed them with your t-shirt or trouser leg; they just smeared. With the new screen coating, one wipe with a t-shirt makes the screen look just like new. This is a very useful addition to my mind!

One other detail I noticed straight away is that the screen is a lot brighter. I had thought it was slightly better (in the same way that Snow Leopard is much clearer than Leopard although this is to do with a switch to the 2.2 gamma standard) but wasn't aware of how much better until I placed it next to my old iPhone 3G - you can see the difference in the photo above. The key thing here is that both phones were set to the same brightness level so there really is an improvement in the hardware somewhere.

New Features

Compass - the new digital compass (or magnetometer if you prefer?) was one of the big talking points of the iPhone 3GS as it allows for far more accurate turn-by-turn navigation. It also added a sexy new app appropriately named "compass". The compass app itself is fairly basic and I felt that the actual readings were quite slow to adjust. Additionally, making a very small change to the orientation of the phone doesn't always reflect in the compass which is a little frustrating when you are trying to get it to point exactly North. Having said that, it's good enough to get a basic idea of which way is which. The real area the compass shines in is in the updated Maps application where pressing the location button re-orientates the map to the direction you are facing. This is absolutely invaluable when navigating and is a feature I will be using heavily.

Voice Control - When it was announced at the WWDC Keynote, I felt that Phil Schiller sounded a bit stupid going on about how this great phone was now able to do voice commands seeing as it was something my Nokia could do 8 years ago. However, I now realise why he was quite as smug as he was. It really does work exactly as they demoed it. After I synced my contacts and music, I tried a few of the commands such as "phone Ben Dodson" (to which it replied "work, home, or mobile?"), "play panic at the disco", "play more songs like this", and "what song is this". Every name and command I tried worked flawlessly so I was incredibly impressed. The real power is that with other phones you'd need to add a voice tag for each contact whereas with the iPhone, it just reads the text and interprets your voice accordingly so there is no need for you to record a voice command prior to using it. The app looks awesome as well!

Camera - The new camera app is fantastic. I can't believe that it's only a 3MP camera as the quality of the images is as good as some phones I've seen with 5 or even 7MP. The video app is simple to use (as you'd expect) and again the quality is very very good. It's a shame it doesn't film 720p but the colour balancing and overall quality make up for the relatively small resolution. The only negative I can find is that video at nighttime is fairly grainy (whereas in daylight it's beautifully smooth) and the camera would really have benefited from having a flash. I was really hoping the rumours that the Apple logo would act as a flash light were true but it appears that it's not the case... for this model at least!

I've got the need, the need for speed!

My main reason for buying the iPhone 3GS is that I wanted faster app loading times and generally quicker responses within the apps. Playing Sonic the Hedgehog on my 3G nearly bought me to tears as it was actually unplayable (I'm sure they only tested it on a 2nd generation iPod Touch...) and I'd always get frustrated playing Tap Tap Revenge 2 when the app would skip a little due to memory running out. So, speed was a big thing I was interested in.

I did not imagine it would be as good as it actually is.

The speed increases I've noticed so far have been nothing short of phenomenal for something that got a 50% speed boost and a doubling of RAM. Quite often, load times have been reduced by up to 4x and overall app reliability is nothing short of flawless. Here are a few stats based on some of my most commonly used and intensive apps:

Bejeweled 2 - app launch to menu screen
3G = 12.1s
3GS = 3.7s

iDracula - selecting "grave park - survival" on menu to actual gameplay
3G = 21.6s
3GS = 6.5s

Peggle - app launch to "touch to play" message
3G = 25.4s
3GS = 10.2s

Sonic the Hedgehog - app launch to "SEEEGGAAAAA" message
3G = 5.9s
3GS = 2.7s

Tap Tap Revenge 2 - app launch to main menu
3G = 6.4s
3GS = 3.3s

Tap Tap Revenge 2 - selecting "The Sound of Settling" on "Hard" to start of track
3G = 8.9s
3GS = 3.5s

All apps were tested on the 3.0 OS after an iPhone restart. They were timed using a stopwatch and each test was run 3 times and then averaged in order to minimise discrepancies.

Conclusions

The speed boost was definitely the biggest thing for me and I have to say that it has exceeded my expectations. The small range of stats above don't accurately display how snappy everything has become. Previously, navigating the menus of Tap Tap Revenge 2 always a pause of around a second between each screen whereas now it's instant. Also, actually playing the game could be incredibly frustrating as I knew I was in time but a memory glitch along the way would cause the tappers to move erratically causing you to miss them even though you hit the area at the right time. This was verified to me when playing on the 3GS as I got a 100% streak straight away without really trying too hard. Another game that suffered horribly on the 3G was Sonic the Hedgehog which really shouldn't have been allowed to go on the App Store. It was probably ok on the 2nd Gen iPod Touch as that had a slightly faster CPU, but on the 3G it was just dismal with stuttering sound, obvious slow down and speed up, and a whole host of other glitches such as unresponsive controls. On the 3GS, it plays exactly as it always should have done - exactly the same as it did on the Mega Drive.

I haven't even touched upon areas such as the speed increases in Safari rendering (pages are near instant - truly amazing mobile web browsing), the noticeably smoother animations between apps, or any of the other minor tweaks that make sure that the 3GS not only outperforms the 3G, but actually completely exceeds the speeds that were previously attainable.

However, there are one or two problems in all of this. For me, the biggest question mark hangs over how the App Store is going to be managed. The iPhone 3GS has much better hardware and allows for much better graphics which means that theoretically we should get into a situation where apps are available only for 3GS. However, it looks as if Apple is going to resist this route and that the 3GS upgrade is purely for across the board speed increases rather than in making more powerful apps. I can't predict what is going to happen but I fear that there will be a lot of apps made that will only work on the 3GS but they won't be labelled as such in the App Store (in the same way that Sonic the Hedgehog should have been labelled 2nd Gen iPod Touch only). This leads to a lot of frustration when you are paying £5.99 or so for a game which then won't work on the existing hardware.

So, do I think the 3GS is worth the upgrade?

Yes. Yes I do.

The tale of the "O2 Fail" (starring the iPhone 3GS)

Ever since the iPhone 3GS was announced in the Apple WWDC Keynote last week, the internet has been ablaze with criticism about both AT&T and O2 with regards to their upgrade policies to the new phone for existing iPhone 3G customers. I have battled long and hard on Twitter and on the O2 Forums to try and put across that they have in fact done nothing wrong, it is the public that are misguided, yet this has fallen on relatively deaf ears. I decided it would be easier to put one whole post together about the "O2 Fail" (or #o2fail for you Tweeters) and the rational response to it so I could just point people in this direction rather than re-explaining myself over and over!

Disclaimer: I don't work for O2 and never have done. I used to be on the 3 mobile network in the UK as it was the cheapest before switching to O2 in order to get the iPhone. I bought the original iPhone about 4 months before the 3G came out (and so paid full price for it which was around £330 plus a £35 contract) and then upgraded to the iPhone 3G by paying £59 and upgrading to the £45 18-month contract. I will be buying the iPhone 3GS next Friday by taking out a second contract and letting the remaining 6 months on my current contract run out. Therefore, I have no reason to be supporting O2 as I would benefit greatly if they caved in and let people upgrade for free - the point I'm putting across is that there is good reason why they aren't doing that and people need to understand why that is.

O2 Fail Twitter

The iPhone 3GS

During the WWDC Keynote, Apple announced that the new model of the iPhone (the iPhone 3GS - the 'S' stands for 'Speed') would be released on the 19th of June in 6 countries at a price of $199 for the 16GB model and $299 for the 32GB model. Crucially, these were prices on a specific AT&T 24-month tariff for new customers. Here in the UK, the prices vary from 'free' to £280 or so (or even £550 if you choose Pay As You Go). Again these prices are for new customers only. So, as was bound to be the case, there are a large number of iPhone 3G customers who want to upgrade to the iPhone 3GS who are being told they have to finish their existing contracts before they can get the new model. This can be done in two ways; wait until your contract ends or pay to get out of it now. This very simple issue is the basis for all of "O2 Fail" comments over the past few days. Let's look at why it has annoyed people so much...

"But last time..."

When the iPhone 3G as announced, O2 allowed original iPhone customers to terminate their contracts early at no cost and start a brand new contract. The only cost was that of the phone and was the same cost that applied to new customers. In my own case, this was around £59 as I moved up to the £45 a month contract (18 months). The reason O2 did this (when they and other carriers have never done this for a phone before) is because they didn't subsidise the original iPhone. When you went to the Apple Store or to an O2 store, you paid the full retail price (which was around £330 for the 16GB model). Therefore, the contract you were on was purely making money based on calltime - that was a pretty sweet deal for O2 and so it was surprising they allowed people to upgrade.

In any case, most people seem to think that as it happened last time it should happen this time. They have failed to grasp the crucial word 'subsidy'. The original iPhone wasn't subsidised, the iPhone 3G was (as is the iPhone 3GS). This means that a large portion of the money you now pay to O2 from your contract goes directly to Apple for the cost of the phone. It's worth pointing out that this is the same with every other phone on the market with every other carrier (unless you're using Pay As You Go in which case you pay the full price for the phone up front). So, if O2 were to say at this point "of course you can upgrade under the same conditions as last time" they would lose a HUGE amount of money as not only would they not have made any money on airtime, they would have lost the money on the subsidised phone.

What about paying off the remaining subsidy on my phone?

There have been a fair number of people asking this question - "why can't I just buy out my subsidy and get a new contract". The key problem here is that O2 would still not have made any money as they would have basically given you free airtime for the last 12 months. If you only pay your subsidised part, then they aren't making money. This is where the crucial word 'contract' comes into use as you signed an agreement to pay a certain amount of money per month to cover the cost of the phone and the airtime.

But what about customer loyalty!?!

The follow up argument to the above is "but they'd make money on the airtime in my new contact". So, you've just screwed O2 for the past 12 months by using their network without effectively paying for it and now you want a new contract where they will be able to make their money back? I don't follow that for one instant as what happens when the next iPhone comes out in 12 months ? We'll go back to square one with the argument being "they did it the last 2 times, why can't I upgrade now" which means that again O2 will go without being paid for their network usage.

Ok, how about they just add my remaining contract to my new contract?

This is a slightly harder argument as in a way it makes some sense. Rather than waiting 6 months for my contract to expire and then upgrading to a new 18 month contract, why can't O2 just let me upgrade now and add an additional 6 months to my new contract? This would work in theory but the problem is that you'll start getting people on contracts from 18-36 months and I'll guarantee that come next year they'll be the first ones to ask for another extension putting them on 24-42 month contracts and so on. This is a similar problem to people extending loan terms forever and ever in that there comes a point where they simply aren't going to pay it back. I don't know the full details but I imagine there are some laws on how many times a contract can be extended as well in order to ensure fair fiscal management but don't quote me on it!

Don't they realise we're in the middle of a recession!

This is by far the best argument I've seen used - people actually pointing out that due to the "credit crunch", O2 should be lowering prices so they can get the phone cheaper. If the recession is really that bad, don't you think it's a little stupid of you to be arguing about getting a slightly faster iPhone? Surely you should be worrying about the price of basic essentials like food, clothes, and petrol rather than the ability to get a slightly better camera in your smart phone? And how do you think mobile phone carriers are doing in the economic crisis? They need to make money as well!

The Facebook Group

One of my other favourite things during this whole saga has been the Facebook group I DONT WANT TO PAY TO UPGRADE MY IPHONE! - I don't think I even need to provide an argument against this.

My Point

I've spoken about the reasons why these arguments are flawed but my real point is that people don't "need" the iPhone 3GS so I don't understand why so many people are desperate to upgrade. I'm an iPhone developer and therefore want the new hardware so that I can write better apps that utilize it's hardware, but to everybody else there is very little gain. Sure you get a slightly better camera, the ability to film video, and a digital compass (as well as a processor and RAM upgrade) but you get far more from the 3.0 upgrade than you do from the hardware upgrade.

I suppose my real point is that there are things called "contracts" that people have entered into - they are now throwing their toys out of the pram because they can't get the latest shiny object from Apple without paying for it but unfortunately that's just how it is and O2 are not going to change their minds. Why not? Because if they did they would lose huge amounts of money. At this point people decry the greedy networks and the fact that they are going to take their business elsewhere. The crucial point is that O2 is the only network that can cope with the iPhone properly and they are the only supplier so you either stay with them or you get a Palm Pre, a Google Android, or one of the new Windows Mobile phones from another supplier. But, I can guarantee that in 12 months when a new model of those phones comes out, the networks won't let you upgrade early for free so you'll be back where you started.... just with a rubbish phone instead!

The key point is that the iPhone 3GS has not been designed for existing iPhone 3G customers - it has been designed specifically to address the concerns of people who were deciding whether to get one of the other smartphones I mentioned or an iPhone. They have added MMS, Video, and a better camera - the three major flaws with the iPhone that any undecided customer would count as negatives. If you're an iPhone 3G user at present, I highly doubt that you are going to stop becoming an iPhone user over this issue as you have already been sold on the idea of the phone and you know that no other phone currently suits your needs.

Besides, as a network O2 has proved itself to be far better than many other iPhone carriers (*cough* AT&T *cough*). They are supporting MMS on launch day (including video for those with the 3GS), the push notification system has worked flawlessly so far (I've been beta testing it), and they have internet tethering working as well (which also works very well - I'll be coming back to the price of this later).

What can I do?

So if you're read this and been swayed over to rational thinking rather than getting involved in the anti-O2 hype, what are your options for upgrading to the iPhone 3GS?

Wait. I know it's difficult but you don't need the iPhone 3GS. If you had a great need for video then you would have bought a different phone in the first place. If you bought your iPhone 3G on launch day, you should have 6 months of you contract left and this may be reduced to 0 months, 3 months, or 5 months depending on how much you've spent recently - take a look at the O2 Priority List for details.

Buy a Pay As You Go iPhone 3GS and sell your old iPhone 3G on eBay (or to a friend, etc). You get the full functionality, get to keep your number, and it'll all work on launch day just by putting your existing SIM card into the new phone. Visual Voicemail, etc, will work as you'll still be on a contract plan (the hardware between the contract and PAYG phones is the same). Plus you might even make money this way as the iPhone 3G's are selling for a high price - best to get on their quickly though!

Pay off your existing contract. This route is the most expensive but you could pay off your existing contract and take out a new one - the price is generally the number of months remaining times your monthly rate so in my case it was £270 (£45 x 6). This is the easiest way but the most expensive.

Take out a second contract. This is what I'm going to be doing next Friday. Rather than paying off my contract in one lump sum, I'll keep that contract and my iPhone 3G (I use it for app testing) but I'll just get a new contract with the iPhone 3GS. This means I'll be paying for 2 contracts for around 6 months but that spreads the cost a bit rather than having one big hit all at once. Also, you can downgrade your iPhone contract one level after 9 months so I'll be able to move my iPhone 3G down to the £35 plan this month (and the £30 the month after) which will save some money - it also means I have a spare, live phone in case I need it for anything. This is again an expensive method but it spreads the cost rather than having it all in one hit - however, it may be subject to a second credit check to make sure you can afford two contracts at once.

What should O2 do?

To my mind, this whole saga could have been avoided if O2 still offered 12 month contracts. Of course, the main problem is that phones are getting more and more expensive and operators want to keep overall contract prices as low as possible so they look competitive. I believe it was 3 mobile who were the first to bring in 18 month contracts as they could do so incredibly cheaply - they got a huge market share simply because people didn't realise what they were signing up for.

So, on the one hand you could have 12 month contracts but charge people a large amount a month so you cover both subsidy and airtime, or you can have 18-24 month contracts which are cheaper but then the customer has to pay out if they want to upgrade to the latest and greatest phone. Both have their issues and overall the cost is probably the same; the real difference is when you pay it. Personally, I would like to see mobile operators offer all three services so people can choose straight up if they want 12 months (high price, low lock-in), 18 months (medium price, medium lock-in), or 24 months (low price, high lock-in). That would avert this whole discussion on upgrading to new phone models!

Conclusion

So there you have it - O2 haven't done anything wrong, they are simply doing the same as every other provider has done since they were created. We can shout all we like but they won't back down on this issue as it doesn't make any financial sense for them to do so. If you are desperate for the iPhone 3GS, then you'll just have to pay for it or wait it out a bit longer.

Now, if I've convinced you to stop petitioning O2 about iPhone 3GS upgrades, perhaps I can convince you instead to spend your time petitioning them about their ridiculous Internet Tethering charges as that is a fight I believe can be won - £15 for 3GB of transfer is far, far too much and I can't see any justification for it! So, give up the 3GS battle and instead shout about the tethering charges :)

Mastering phpMyAdmin 3.1 for Effective MySQL Management

I was recently asked by Packt Publishing to review a copy of their latest book, Mastering phpMyAdmin 3.1 which promises to "increase your MySQL productivity and control by discovering the real power of phpMyAdmin 3.1". I was a little skeptical at first of a book on phpMyAdmin, the most widely used MySQL admin tool, especially when it arrived at 325 pages! However, there is a huge amount of information that really is very useful to every PHP developer out there whether you are a beginner or an advanced user.

[123/365] Mastering phpMyAdmin 3.1

Now, most people I've mentioned the book to have scoffed and said something along the lines of "I already know how to use phpMyAdmin". Like them, I thought I knew what phpMyAdmin was and what it could do but it turns out there are huge amounts of functionality I never knew existed in MySQL let alone in phpMyAdmin!

For the Beginner

The book starts off with a very gradual introduction to phpMyAdmin covering everything from basic installation and setup to a detailed explanation of the overall interface. I was particularly pleased to see an in-depth chapter on security configuration at the beginning of the book which would help any newcomer make sure that their setup is completely secure - usually such chapters are found at the back in the appendices! The first six chapters follow in a similar vein with very basic information about how to run SQL queries, edit data, change structures, and so on but chapters seven and eight deal with exporting and importing data which is one of the many areas that I have seen developers struggle with in the past. There is a good explanation of the different methods for importing / exporting including the benefits of certain types over others. Crucially, there is a section on CSV using LOAD DATA which is something that has always seemed to lack proper explanation to me in the past.

There then follows a few more chapters which more advanced users can probably skip such as searching, an overview of relational databases, and table / database operations.

Advanced Topics

I would say that the real meat of the book for experienced PHP developers begins at chapter thirteen with each further chapter adding useful knowledge. I've listed the key highlights of these chapters below:

  • The Multi-Table Query Generator - A powerful tool which enables you to fine tune complex queries via a series of forms thus allowing you to specify multiple criteria. It contains features such as automatic joins which allow you to very easily build up complex queries.
  • Bookmarks - A feature I was completely unaware of that allows you to save queries for future use. This is particularly useful if you happen to be a database administrator that administers purely on a table by table basis within phpMyAdmin and has a number of queries to run. I always used to have popular queries I'd use stored in a notepad on my OS X Dashboard but no need anymore!
  • System Documentation - I recently had a need to produce some MySQL documentation so was very happy to read this chapter and find out about the excellent documentation tools available within phpMyAdmin. This includes not only a basic print view, but also a data dictionary and a relational schema which are all exported as PDFs.
  • MIME-Based Transformations - If you're the kind of developer that likes to store images, etc, as BLOB fields. With transformations, you can make images appear as images within the phpMyAdmin results rather than as indecipherable encoded text. Very useful!
  • MySQL 5.0 and 5.1 - a quick look at the enhancements that MySQL 5 added with things such as views, routines, stored procedures, and very interestingly, triggers (a way to run other MySQL commands when a certain thing happens - e.g. when a table gets updated). You'd probably want a separate book to cover MySQL 5 if you were planning on doing any development with it, but this chapter gives you a good overview of some of the things you can expect.
  • MySQL Server Administration - The final chapter deals with some of the more fundamental maintenance tasks related to the actual server and improvements that can be made with caching etc as well as a good comparison of the different types of storage engine you can choose.

Summary

All in all, I would highly recommend this book to any PHP developer or anybody that is using phpMyAdmin on a regular basis. It could really have been broken into two books - a beginners and an advanced - but it works well by acting as a reference for those developers that have grown up using phpMyAdmin. The main thing though is that it taught me a great deal about phpMyAdmin that I didn't realise was even there - just goes to show that even a basic sounding book can have a great deal to offer.

Mastering phpMyAdmin 3.1 is available online from Packt Publishing

TwitterFollowers PHP Class - A Better Way To Track Followers, Quitters, and Returning Followers on Twitter

Over the past few months, there have been a number of web apps that have popped up with the task of feeding your ego (or indeed deflating it) by telling you exactly who is following you on Twitter and giving you pretty graphs to show you how your followers are increasing - some of them even go so far as to estimate how many followers you are likely to have in a weeks time! However, the key thing for me that is missing from Twitter is the ability to see who has stopped following you and also those people that stopped but are now following you again as you don't get email alerts from Twitter for these two things. This is a useful piece of information to have as it will let you know when people drop off and whether they are important (e.g. friends who don't care what you are talking about thus suggesting you should stop talking crap) or not (e.g. spam bots).

I did a bit of research and the only real web app to fulfill this need that I could find was the beautifully designed Qwitter. However, the problem with Qwitter is that it only gives you the details for one person with the idea being that you say "tell me when this username stops following me" - it's a little bit stalkerish in my opinion! Like any PHP developer, I decided that I could build my own little system to give me my Twitter ego boost and so have come up with the class below which you can all now take and use as you see fit.

Update: Turns out I was wrong about Qwitter as the username you put in to follow is supposed to be yours, not the person you want to watch when they leave you. They need to do better copywriting! In any case, the class below serves as a good demo of the public Twitter data and allows you to extend it the way you want.

Note: This won't work straight out of the box - I've put in a few comments which say "SQL Required". This is because you may well have your own schema (although I do provide one) and you may have your own framework or DB connection functions (I know I do). Once you've done those, just substitute the constants for your own details and it should all work

TwitterFollowers.class.php

<?php

/**
 * Crawls Twitter Followers and sends an email alert to show you who has started following, stopped following, and started re-following
 *
 * @author Ben Dodson
 **/
class TwitterFollowers
{
	// define constants

	const username	= 'bendodson';
	const email 	= 'ben@bendodson.com';
	const subject	= 'Twitter Updates';
	const from	= 'TwitterBot <twitter@bendodson.com>';
	
	// define internal variables

	protected $actualFollowers = array();
	protected $internalFollowers = array();
	protected $followerChanges = array();
	protected $now = '';
	
	function __construct()
	{
		$this->now = date('Y-m-d H:i:s');
		
		$json = file_get_contents('http://twitter.com/followers/ids/'.self::username.'.json');
		$this->actualFollowers = json_decode($json);
		$this->internalFollowers = $this->getInternalFollowers();
		
		foreach ($this->actualFollowers as $actualFollower) {
			if (!in_array($actualFollower, $this->internalFollowers)) {
				if ($this->getTweeter($actualFollower)) {
					$this->followerChanges['returning follower'][] = $actualFollower;
					/*
					--SQL REQUIRED--
					UPDATE TwitterFollowers SET start = $this->now, end = NULL WHERE id = $actualFollower
					*/
				} else {
					$this->followerChanges['new follower'][] = $actualFollower;
					/*
					--SQL REQUIRED--
					INSERT INTO TwitterFollowers (id) VALUES ($actualFollower)
					*/
				}
			}
		}
		
		foreach ($this->internalFollowers as $internalFollower) {
			if (!in_array($internalFollower, $this->actualFollowers)) {
				$this->followerChanges['stopped following'][] = $internalFollower;
				/*
				--SQL REQUIRED--
				UPDATE TwitterFollowers SET end = $this->now WHERE id = $internalFollower
				*/
			}
		}
		
		$this->sendEmail();	
	}
	
	protected function getInternalFollowers()
	{
		$data = array();
		$raw = 
		/*
		--SQL REQUIRED--
		SELECT id FROM TwitterFollowers WHERE end IS NULL
		*/
		foreach ($raw as $r) {
			$data[] = $r['id'];
		}
		return $data;
	}

	protected function getTweeter($id)
	{
		return
		/*
		--SQL REQUIRED--
		SELET * FROM TwitterFollowers WHERE id = $id LIMIT 1
		*/		
	}
	
	protected function getTweeterDetails($id)
	{
		$json = file_get_contents('http://twitter.com/users/show/'.$id.'.json');
		$tweeter = json_decode($json);
		return $tweeter->name . ' ('.$tweeter->screen_name . ')';
	}
	
	protected function sendEmail()
	{
		$to      = self::email;
		$subject = self::subject;
		$headers = 'From: ' . self::from . "\r\n" . 'Reply-To: ' . self::from;

		$message = 'Hi,' . "\r\n\r\n";
		$message .= 'Here are your Twitter Updates:' . "\r\n";
		
		if (count($this->followerChanges) > 0) {
			foreach ($this->followerChanges as $changeType => $change) {
				$message .= "\r\n" . '--'.strtoupper($changeType).'--' . "\r\n\r\n";
				foreach ($change as $tweeter) {
					$message .= '*' . $this->getTweeterDetails($tweeter) . "\r\n";
				}
			}
		} else {
			$message .= "\r\n" . '--NO UPDATES FOUND--' . "\r\n";
		}
		
		mail($to, $subject, $message, $headers);
	}

}

?>

MySQL Database Schema

CREATE TABLE IF NOT EXISTS `TwitterFollowers` (
  `id` int(20) NOT NULL,
  `start` timestamp NOT NULL default CURRENT_TIMESTAMP,
  `end` timestamp NULL default NULL,
  PRIMARY KEY  (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;

How does it work?

First of all, you will need to substitute the SQL sections for your own particular schema and database functions. Once you've done that, alter the class constants so that they are your own username and the email address you want to send your updates to. Finally, set up a CRON job so that it runs at a certain point every day. I currently have mine set to run at 9am every morning but I may well change it to run every time I post a tweet as then I'd be able to see which tweet had made people start or stop following me.

The script works by checking the publicly accessible JSON feed of your followers and getting all of their IDs. I say it's publicly accessible, but I don't think it is if you have protected your updates which will of course cause problems! Once it has all of the IDs, it checks this against the IDs stored in your database - if there aren't any then everyone will show up as following you on the first run. If it finds an ID in your database that isn't in your JSON string then you've been dumped! Conversely, if it finds an ID in the JSON string but not in the DB then, congratulations, you've got a new follower. The final instance is if it finds an ID in the JSON string that is in the DB but has an end datetime assigned to it. This means the person was following you, stopped, and has now decided to re-follow you.

The whole lot then gets packaged up and emailed to you with each section broken down so you can read it clearly. In order to do this, it looks up each ID that goes into the email against that persons publicly available Twitter information to give you both their "real name" and "username".

Known Problems

  • I don't think it will work if you have your updates set to hidden.
  • If one of your followers gets banned from Twitter, then their name won't show up in the email (it will just be blank)
  • This script won't work if you have more than 5000 followers - this is because that is the maximum result set from the JSON string. You'd need to add paging information to get more than 5000 although this would be fairly easily. Alas I don't have that many followers to be able to test that out!

Conclusion

So now you can easily (if you know PHP) get updates on all your Twitter followers and non-followers. Feel free to use all the above code and modify to your hearts content - if you found it to be useful, then please leave a comment below. Oh, and I couldn't possibly write a post about Twitter without reminding you that you can follow me @bendodson ;)

Getting Xbox Live Achievements Data: Part 2 - The AppleScript Solution

Following on from the first of this series of tutorials on how to extract Xbox Live achievement data using PHP and AppleScript, I thought I would use this second part to look at the AppleScript that powers one side of the system I've put together. In the next part, I'll be explaining the PHP class I've built, and in the fourth part (the last of the series) I'll be showing you how the two talk together and how you can use the collected data with other APIs such as Facebook Connect.

So, let's have a look at some code!

XboxLive AppleScript

set urlFilePath to "http://externals.bendodson.com/weblog/XboxLive/urls.txt"
set dataFilePath to "server:XboxLive:data.txt"
set toCrawl to ""

set dataFile to open for access file dataFilePath with write permission
set eof of dataFile to 0
close access dataFile

tell application "Safari"
	open location urlFilePath
	delay 1
	do JavaScript "window.location.reload()" in the front
	delay 1
	try
		set toCrawl to (the text of the front document)
	end try
end tell

if length of toCrawl > 0 then
	tell application "Safari" to open location toCrawl
	delay 15
	tell application "Safari"
		set sourceCode to the source of front document as string
		set dataFile to open for access file dataFilePath with write permission
		write sourceCode to dataFile starting at eof
		close access dataFile
	end tell
end if

tell application "Safari" to close every window

This is the first time I've used AppleScript for anything other than just playing around and I have to say that as a language it's incredibly good. Just by reading through the above, you'll probably be able to work out what's going on even if you've never seen any type of programming code in the past. Even so, I'll go through each section and explain what it does along with the reasons why I decided to do it in this particular way rather than some of the other ways I could have chosen.

Note: All of this AppleScript is completely self-taught from searching around on the internet. I was going to buy the book AppleScript: The Missing Manual but I was able to read all the sections I needed on Google Books which was convenient - I'll probably buy the book anyway to brush up on a few other areas. If you are an AppleScript guru and you know a way to optimize my code, please use the comments section below so others can learn and so I can update it.

Before we get on to looking at the code, it might be worth briefly recapping how everything will work. My server will check the XboxLive API in order to see if my gamerscore has increased. If it has, then it saves the URL of the achievements page for my most recently played game (which it can't itself read due you needing to login to Xbox Live with javascript enabled - something cURL can't do!) in a text file on the server. My mac mini at home then runs the above AppleScript in order to retrieve the saved URL, open it in Safari, and save the HTML in it's own text file that is available via the internet. My server will then check this text file, parse the HTML, and save the achievements to a database.

How does it all work?

Now we've got that out of the way, let's look at that AppleScript in more detail:

set urlFilePath to "http://externals.bendodson.com/weblog/XboxLive/urls.txt"
set dataFilePath to "server:XboxLive:data.txt"
set toCrawl to ""

These three lines of code are used to define variables which we will use later on in the code. The first one, urlFilePath, is the URL of the text file on my server that will tell our script what URL we need to retrieve the HTML from. You'll see how we populate that text file with my XboxLive php class which will be discussed in part three of this four part series. The second variable, dataFilePath, is interesting as it contains the path to the file we are going to save the HTML to on the local machine. So why the strange syntax? This is referred to as Finder syntax and is a way for AppleScript to reference a particular section within Finder, in this case a text file. It is essentially the same as "/server/XboxLive/data.txt" which we could have used - the difference is that we would have had to have converted that into the Finder syntax in order to use some of the file editing commands we'll use later so I thought it best just to save it correctly at the beginning.

set dataFile to open for access file dataFilePath with write permission
set eof of dataFile to 0
close access dataFile

These three lines are again fairly easy to follow. We set a variable of dataFile to be the handler of the file declared in the path of dataFilePath. Note that we have specifically mentioned we want to use write permissions as the default is just to use read permissions. The next line sets the eof or "end of file" within the handler to 0 whilst the following line tidies up by closing the file handler (which isn't strictly necessary but good practice). The reason for setting eof to 0 is that we want to delete the contents of the file before we put anything else in later. This is practical for the simple reason that we don't want our PHP script on the server to parse a load of data in the text file (or even download it) if it's something it has already read as that would be a waste of processing power and bandwidth.

tell application "Safari"
	open location urlFilePath
	delay 1
	do JavaScript "window.location.reload()" in the front
	delay 1
	try
		set toCrawl to (the text of the front document)
	end try
end tell

Now we get to the first real section of the code that deals with our problem. In these lines, the application Safari is made to open the text file on the server that may contain the URLs, refresh that page using JavaScript, and then attempt to set the variable toCrawl to the text of the file. Before we even examine this in depth, you may be wondering something along the lines of "why don't you download the file or read it with FTP rather than opening it in Safari" and this would be sensible. My initial thoughts on how to get the text on the server into a variable in my AppleScript was to access the file via FTP. OS X has very basic FTP support (read only unfortunately) that can be mounted through Finder and then accessed using the Finder syntax we used earlier on. I had originally written some AppleScript that would run in the startup process of the mac mini which would mount the drive. Then, this AppleScript read the file in using open for access file urlFilePath and set the variable that way. It all worked perfectly until the server changed the contents of the URL text file. No matter what I did, the text file came back the same as it had when first fetched and it was that that I realised that the FTP built into Finder is fundamentally flawed as everything is cached. If you don't edit the file through Finder (e.g. by using a mac application that saves it again through Finder) then it will never know it's updated and thus can't be used in this scenario.

With that out of the way, let's look at my workaround. The first and last lines are the opening and closing of a tell; a way to get an application to do what we want. In this instance we are going to tell Safari to open the URL saved in the variable urlFilePath and then delay for one second. This delay is needed as Safari may take this long to open the URL. Without the delay, we may be in danger of running code on a page that hasn't loaded. In the next line, we tell JavaScript to reload the document before waiting another second for this to complete. This refresh is necessary to clear any caching of the document. The final section is used to set the variable toCrawl to the contents of the browser window. You may wonder why there is a try statement wrapped around it? This is because if the text file was empty and you tried to get the text of the front document, the script would error. To get around that, we initially set the variable (in the very first block of code if you remember?) and then use try to reset the variable only if no error would be caused in doing so. Very useful!

By the end of this block of code, we will have a variable which will either contain a URL if the text file on the server had one, or it will be empty, meaning that the server is not requesting any HTML. Let's move on to the next section:

if length of toCrawl > 0 then
	tell application "Safari" to open location toCrawl
	delay 15
	tell application "Safari"
		set sourceCode to the source of front document as string
		set dataFile to open for access file dataFilePath with write permission
		write sourceCode to dataFile starting at eof
		close access dataFile
	end tell
end if

This is the core piece of functionality that I was trying to achieve all in this block of 11 lines. This script will open a URL in Safari, and then save the source code of the loaded page in a text file. You'll notice that the first and last lines are an if statement relating to the length of the toCrawl variable. I don't unlock achievements every 5 minutes of the day and so, more often than not, the toCrawl variable will be empty. If it is, then we want to completely ignore the next section of code as there is no reason whatsoever to run it!

The next line is a one line tell to make Safari open the URL we saved which is then followed by a 15 second delay. You'll notice this is a lot longer than the 1 second delays earlier. The reason for this is that, in the first case, it was a simple text file with around 100 characters in it and so loaded incredibly quickly. This URL, conversely, will be a very large page (around the 100kb mark) that may go through a series of 5 redirects depending on how recently the page was accessed. This is because after 15 minutes of inactivity, the site forces you back to the login page but I have a cookie stored that will then automatically log me back in. It just takes a few seconds to go through the process of all the redirects to get to the actual page hence the long time delay.

The last section is a simple expansion of the code we used at the beginning. We tell Safari to set a variable of sourceCode to be the source of the page that's open - we also tell it to be forced as a string in case there are any casting issues. Next, we open the dataFilePath and set a handler of dataFile so that we can then write the sourceCode variable into the file starting at the end of the file (which we all know is masquerading as the beginning of the file also as we set it earlier on... keep up!) before tidying up after ourselves and closing access to the handler. Easy!

tell application "Safari" to close every window

In the very final line of code, we tell Safari to close all of it's windows in preparation for the next iteration. This may not seem terribly important, but trust me, if you neglect to put it in and then unlock 10 achievements, you'll find your mac now has 20 open Safari windows!

Conclusion

So that's all there is to this section - a large chunk of AppleScript and a rationale as to why I had to open Safari to get at a text document rather than doing it a slightly more simple way via FTP (due to massive caching issues). I hope this post has introduced some of you to AppleScript which I have found to be a rather powerful tool when it comes to mac development. It's very easy to understand and is a great way of transitioning from a web-based language to a desktop-based one especially as you can save AppleScript as a standard mac application.

Join me again for part three of this four part series when I'll be looking at this from the other angle; the PHP server that needs to parse the HTML we have gathered using this AppleScript. To make sure you don't miss a section, you can subscribe to the RSS Feed or follow me on Twitter. Please feel free to leave any comments or suggestions on this page.

« Older Entries Newer Entries »