Ben Dodson

Freelance iOS, Apple Watch, and Apple TV Developer

Fetching RSS Feeds for Steam game updates

I’m currently in the process of switching from my Xbox One to a gaming PC1 and have been spending a lot of time curating my Steam collection on my Mac. One thing I’ve noticed recently is that I often see games being updated but then have to go to their individual pages in order to view the release notes of what has changed (and that is if I even notice a game has been updated). What I really want is a single page that shows me all the patch notes in date order and sends me a notification when something updates.

A quick search found SteamNews, a website that aims to turn Steam Community pages into RSS feeds. However, a lot of games are missing and it doesn’t solve the problem of fetching the feeds for my account; I’d still need to go through and add them manually. I also don’t like relying on 3rd party services which I can’t install on my own server.

After a bit of exploring on the Steam pages, I found that most games have an RSS feed of their news page; this, combined with my publicly available profile page listing my owned games, has led me to write a basic PHP script to loop through my games, find the matching RSS feed, and then add it to my Feedbin account:

<?php

define('FEEDBIN_USERNAME', 'your username here');
define('FEEDBIN_PASSWORD', 'your password here');
define('STEAM_ID', 'your steam id here i.e. bendodson');

require 'vendor/autoload.php';
use Goutte\Client;

$html = file_get_contents('http://steamcommunity.com/id/'.STEAM_ID.'/games/?tab=all');

$match = "rgGames = ";
$start = strpos($html, $match) + strlen($match);
$json = substr(trim(strtok(substr($html, $start), "\n")), 0, -1);
$array = json_decode($json);

$games = explode("\n", file_get_contents('games.txt'));
if ($games[0] == "") {
	$games = [];
}
foreach ($array as $game) {
	$unsubscribed = $game->appid.'-0';
	$subscribed = $game->appid.'-1';
	$failed = $game->appid.'-x';
	if (!in_array($unsubscribed, $games) && !in_array($subscribed, $games) && !in_array($failed, $games)) {
		$games[] = $unsubscribed;
	}
}
file_put_contents('games.txt', implode("\n", $games));

$games = explode("\n", file_get_contents('games.txt'));
$index = 0;
foreach ($games as $game) {
	list($id, $subscribed) = explode("-", $game);
	if ($subscribed == '0') {
		subscribeWithSteamIDAtIndex($id, $index);
	}
	$index += 1;
}

echo 'Done';


function subscribeWithSteamIDAtIndex($id, $index, $url='') {

	$key = $url == '' ? $id : $url;
	$feedURL = $url == '' ? 'http://steamcommunity.com/games/'.$id.'/rss/' : $url;
	echo 'Subscribing to '.$feedURL.'<br>';
	$post = json_encode(["feed_url" => $feedURL]);

	$ch = curl_init();
	curl_setopt($ch, CURLOPT_HTTPHEADER, ['Content-Type: application/json; charset=utf-8']);
	curl_setopt($ch, CURLOPT_URL, 'https://api.feedbin.com/v2/subscriptions.json');    	
	curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
	curl_setopt($ch, CURLOPT_TIMEOUT, 15); 
	curl_setopt($ch, CURLOPT_USERPWD, FEEDBIN_USERNAME . ":" . FEEDBIN_PASSWORD);
	curl_setopt($ch, CURLOPT_POST, 1);
	curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
	$output = curl_exec($ch);
	$statusCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
	curl_close($ch);

	if ($statusCode == 404 && $url == '') {

		$success = false;

		$client = new Client();
		$crawler = $client->request('GET', 'http://steamcommunity.com/app/'.$id.'/allnews/');
		$elements = $crawler->filter('#apphub_InitialContent .Announcement_Card:first-child');
		if ($elements->count() > 0) {
			$url = $elements->attr('data-modal-content-url');
			if ($url) {
				$client = new Client();
				$crawler = $client->request('GET', $url);
				$url = $crawler->selectLink('Subscribe to RSS Feed')->attr('href');;
				if ($url) {
					$success = subscribeWithSteamIDAtIndex($id, $index, $url);
				}
			}
		}

		if (!$success) {
			$games = explode("\n", file_get_contents('games.txt'));
			$games[$index] = $id.'-x';
			file_put_contents('games.txt', implode("\n", $games));			
			echo 'Failed<br><br>';
		}


	} else if ($statusCode == 201) {

		$subscription = json_decode($output);
		$post = json_encode(["feed_id" => $subscription->feed_id, "name" => "Steam Games"]);
		
		$ch = curl_init();
		curl_setopt($ch, CURLOPT_HTTPHEADER, ['Content-Type: application/json; charset=utf-8']);
		curl_setopt($ch, CURLOPT_URL, 'https://api.feedbin.com/v2/taggings.json');    	
		curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
		curl_setopt($ch, CURLOPT_TIMEOUT, 15); 
		curl_setopt($ch, CURLOPT_USERPWD, FEEDBIN_USERNAME . ":" . FEEDBIN_PASSWORD);
		curl_setopt($ch, CURLOPT_POST, 1);
		curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
		curl_exec($ch);
		curl_close($ch);

	}

	if ($statusCode == 201 || $statusCode == 302) {
		$games = explode("\n", file_get_contents('games.txt'));
		$games[$index] = $id.'-1';
		file_put_contents('games.txt', implode("\n", $games));
		echo 'Subscribed!<br><br>';
		return true;
	}

	return false;
}

There is a fair amount going on in this script2 but the basic overview is:

  1. Fetch a list of owned games from the public profile of the entered Steam ID. Most of the Steam pages are built with Javascript but this page fortunately has the entire library as JSON within the page. After a simple bit of extraction, I have a full list of every game owned.

  2. Update a local text file3 – games.txt – with a list of games. I store each game as an ID followed by a hyphen and a flag to denote whether the game is unsubscribed, subscribed, or failed. For example, if I’m subscribed to the Elite: Dangerous game, the text file will contain 359320-1 as the game ID is 359320.

  3. Just to be a pain, Steam doesn’t have a standard format for it’s RSS feeds. Whilst most of them contain the game ID, some of them don’t. The first step is to go with the usual URL and then if that fails we’ll use Goutte to manually scrape the news page and then the first news item in order to grab the correct RSS feed. For example, the game ID of Counter-Strike: Source is 240 so the script will first try http://steamcommunity.com/games/240/rss/ but when that fails it’ll crawl through some pages to discover the correct URL is http://steamcommunity.com/games/CSS/rss/ - there seems to be no logic as to which games have a shorthand name and which don’t and there isn’t an easy way to work out what the name would be.

  4. Once I have a valid RSS feed, it is subscribed to via the Feedbin API. As an added bonus, I also tag the subscription “Steam Games” so they show up all together in my RSS reader of choice, Reeder: Steam RSS feeds in Reeder via Feedbin

  5. Sometimes, the process might fail as in the case of Titan Quest which has no updates and thus no RSS feed. If that happens, the game is marked in the text file so that it can be safely ignored. In my own version of the script, I added Slack integration so I get a notification when a game fails so I check to see if it is a simple case of no feed or if my script has broken: Steam RSS feed failure in Slack

I have the script set up on a cron so if I purchase a new game the RSS feed for it should appear in Feedbin within the hour4.

One idea I dabbled with was fetching the feeds myself and building a single large RSS feed that I could subscribe to. There are a few issues there though:

  • It would require me to constantly fetch each feed which could be huge if I own hundreds of games. I already pay for my RSS subscription service (Feedbin) so why add more complexity when they can do that for me?

  • I’d be sacrificing some customisation. If I’m not interested in a specific game, I can simply remove the feed from Feedbin. My script doesn’t check subscriptions so it won’t try and re-add it.

  • There is no benefit to one large RSS feed. If I was doing this as a commercial service (i.e. a hosted page where you enter your username and I give you a single feed) then it may make sense but I’m not interested in doing that5.

I’ve put all of the code on GitHub in case it is of interest to anyone.

  1. The main reasons being far higher resolution and quality, cheaper games, and virtual reality. I’ll miss Xbox achievements but the boost in visual quality more than makes up for that (and with controller support and a SteamLink, I can still play most games on my sofa with an Xbox One Elite Controller). ↩︎

  2. Please do not look at this and assume this is any way good practice. This is a very fast and loose piece of coding in order to achieve a specific goal as quickly as possible; most of it will break if Steam update their HTML pages (very likely). ↩︎

  3. “Why not use a database” - I couldn’t be bothered to set one up. A basic text file does the job. ↩︎

  4. “Within the hour” - can you tell I’ve been rewatching 24 lately? ↩︎

  5. When I was younger, that is exactly what I would have done but as I’ve grown older I’ve realised that hosting stuff for free is a massive headache. Much better to stick the code on GitHub and let people use it on their own if they want. ↩︎

Using a physical button (Flic) with HomeKit scenes and triggers » « AirPlay Alarm Clock with iTunes 12