third

Parsing Hashtags and Users with Python Twitter

« Using JSFL to Verify Flash Art ||  blog  ||  StarCraft II(r) : Wings of Liberty(TM) Beta »

February 18, 2010

Python Twitter is a python wrapper around the Twitter API. I use this on my main time line on GeoTheThird which includes tweets as well as blogs, images, and videos that are posted.

This article will just review the twitter python portion and it is not a intro on how to use the api, but an advanced look at using python regular expressions to replace @user and #hashtags from your twitter comments, with HTML links. Example usage would be using this in your own applications or web pages.

If you don't have the Python Twitter wrapper, download it, and review the how to to get started with the api.

First we need to get an api instance, and then load it up with our user/password for Twitter.

api      = twitter.Api(username='username', password='password')
statuses = api.GetUserTimeline()

We loop over the statuses to generate the selected tweet output

for s in statuses:

Next we create the regular expressions to find our hash tags and users from their respective @ and # locations.

hash_regex = re.compile(r'#[0-9a-zA-Z+_]*',re.IGNORECASE)
user_regex = re.compile(r'@[0-9a-zA-Z+_]*',re.IGNORECASE)

If you don't understand the regular expressions here is a primer on regex in Python.

We then loop through the matches and replace them with our HTML counterparts.

for tt in user_regex.finditer(tweet):
	url_tweet = tt.group(0).replace('@','')
	tweet = tweet.replace(tt.group(0),
		'<a href="http://twitter.com/'+
		url_tweet+'" title="'+
		tt.group(0)+'">'+
		tt.group(0)+'</a>')

The finished code looks something like this, you would need to modify this to fit your environment etc, but its a start.

import twitter
import re

api      = twitter.Api(username='usernamehere', password='userpassword')
statuses = api.GetUserTimeline()
	
for s in statuses:
	tweet = s.text;
	hash_regex = re.compile(r'#[0-9a-zA-Z+_]*',re.IGNORECASE) 
	user_regex = re.compile(r'@[0-9a-zA-Z+_]*',re.IGNORECASE)
	
	for tt in user_regex.finditer(tweet):
		url_tweet = tt.group(0).replace('@','')
		tweet = tweet.replace(tt.group(0),
			'<a href="http://twitter.com/'+
			url_tweet+'" title="'+
			tt.group(0)+'">'+
			tt.group(0)+'</a>')

	for th in hash_regex.finditer(tweet):
		url_hash = th.group(0).replace('#','%23')
		if len ( th.group(0) ) > 2:
			tweet = tweet.replace(th.group(0),
				'<a href="http://search.twitter.com/search?q='+
				url_hash+'" title="'+
				th.group(0)+'">'+
				th.group(0)+'</a>');

	posts.append({'summary': tweet})

Download twitter-timeline.py

Related tags: hashtags, python, twitter

exavolt

You should care about word boundary. Twitter will parse @example as username but not with foo@example.com.

Leave a comment