I considered using a cloud solution, but I don’t think that kind of scaling will be necessary at this point.  It is also fairly expensive and may fluctuate depending on usage.  So I opted to use a fixed price VPS host.  There are many, many offering available but the most economical which still provide a full feature set, and the one I ultimately choose was  VPSLAND.com.  Like I mentioned in my original shared hosting post, I wouldn’t post about a hosting company unless they really impressed me.  That is still true here.  I believe they have the best VPS hosting out there for any Windows-based projects.  I think a lot of it has to do with the fact that their main focus is VPS.

Their starter package costs a very reasonable $17.99 a month for a Windows 2008 Server and includes full root, remote desktop access, support, backups etc.  They also have promotions from time to time and I believe they currently have 50% off the first month.  Check them out:  http://www.vpsland.com.


DECLARE @SEARCHSTRING VARCHAR(255)
SELECT @SEARCHSTRING = 'some string'
SELECT DISTINCT sysobjects.name AS [Object Name] ,
case when sysobjects.xtype = 'P' then 'Stored Proc'
when sysobjects.xtype = 'TF' then 'Function'
when sysobjects.xtype = 'TR' then 'Trigger'
end as [Object Type]
FROM sysobjects,syscomments
WHERE sysobjects.id = syscomments.id
AND sysobjects.type in ('P','TF','TR')
AND sysobjects.category = 0
AND CHARINDEX(@SEARCHSTRING,syscomments.text)>0


The Tele-Window

If you have the bandwidth for it, I highly recommend setting up a high quality webcam conferencing solution to provide a window into each location’s department. This should provide ample face-time with your team members and provide a great venue for impromptu meetings. You can glance over at the “window” and see if team members are chatting. You can chime in or they can call out to you when they see you back at your desk. This window also comes in handy for the next tip, the roll-away scrum board.

Roll-away Scrum Boards

Depending on how many scrum teams you have, you can roll scrum boards within the viewport of your tele-window just before the daily standup for that project. At the end of the daily standup, you can take a screenshot of the scrum board and post it on a team wiki or other web-based tool.

Virtual Pair-Programming

As a big advocate of XP practices, I cannot overstate how important it is to have at least some pair-programming sessions with other team members.  Although difficult, it is possible to have a decent pair-programming session with geographically distributed team members.  If you’re on a Mac, you can use iChat to share the screen with your partner.  Make sure you’ve got some voice communication so you can decide who is driving and when.  Two people moving the mouse in all different directions simultaneouly is rather annoying.  On Windows, you can use the Windows Remote Assistance, Microsoft SharedView or download a third-party tool.

The most important factor to ensure success of your agile development projects in communication with your team members. When your team is geographically distributed you’ll find challenges in your execution. Make sure to be innovative and use the technology available to you in order to foster high levels of collaboration.

So, what groundbreaking web service will we be exposing to the masses?  We’ll be streaming the web server’s current datetime!  Ingenious, I know!  Hopefully, you’ll be able to adapt this to a slightly more engaging experience.

To start, let’s just take an ASP.NET MVC project and code the default controller action.  We clear all the headers of the response and add the “application/json” header.  Since we won’t be returning a view in the traditional sense of ASP.NET MVC, we don’t have to create an actual view.  We’ll just pause the thread for 1 second and then return the current timestamp for as long as the service is running.

On the consumer side, we’ll just refactor the code we had in the previous post to connect to the URL of the project we just created.  I’m running in on the local development machine so your port will likely be different.

You’ll notice we added some exception handling logic in case we lose our connection to our streaming API.  In this case, we’ll just keep trying to re-connect to the stream.  I’ll show you a sample of this in the last part of the post.

Once we have all the bits in place, we run the ASP.NET MVC application first so it is ready to start handling requests.  Then, we create multiple instances of our consumer console application to simulate multiple clients connecting to the streaming API.  You’ll end up seeing something like this:

To simulate what would happen on the consumer side if there was a broken connection or other exception on the server side, I killed the local development server for a few seconds and then restarted it.  Below you can see that we recovered after a few exception handling loops.

Of course, this post was just intended to provide a possible option for implementing a streaming API with ASP.NET MVC.  For a more realistic scenario you would likely have a queue listener around our Thread.Sleep() code.  As messages are picked up from the queue, you could stream it out and immediately start listening for the next message on the queue.

In a follow-up post, I will demo this same concept using Node.js.

Hope this helps.

Which Twitter users are mentioned most often when a given keyword is included in the tweet?

How useful is this information?  One idea could be to identify influencers in real time when you’re expecting a sudden increase in a keyword or hashtag for a sponsored event or TV spot.  If people are mentioning a particular user over and over again with your brand in it, you could connect with that user and help spread your message.

Let’s Get Started

For demonstration purposes, the components I’ll be building include:

1. twitter_stream_db (SQL Server Database) – This will store the mention count for individual users
2. MSMQ – I’ll be creating a queue that will be sent messages from the Twitter Streaming API. I’m going to leave this for a future post since I’m just doing a POC at this point. If I were to even consider using this in production I would definitely develop a queuing system but at this time it’s a bit of overkill.
3. TwitterReader (console) – A small application that will read the Twitter API and drop messages into an MSMQ channel.
4. TwitterWriter (console) – A small application that will read messages from MSMQ and update the database No MSMQ as explained above so no need to read off the queue. Again, I’ll write these components so we can scale in a later post. For now, consider this just a POC

TwitterReader

This will be a slightly modified version of the console application we created in the previous post. Instead of writing to a console window, we’re going to parse the JSON objects using Json.net and then insert users mentioned in the tweet into a table using the SQL MERGE command.

Looking at the code, you’ll see we modified the stream URL slightly to include the keyword we want to track and sending that JSON result to a new method ParseJson.   In this case, let’s see who are the most popular users mentioned every time someone tweets something with the keyword “love” in it.   (I know it sounds corny but I needed something popular so that I can show off the results.)

We’re using Json.net’s Linq to Json feature to navigate to the user_mentions array. Once we have it, we just loop through all the users in the array and MERGE them into the database table through the stored procedure (see below).

twitter_stream_db

For simplicity, I’m going to create a single table to store the data as it comes in.  The primary key is the Twitter user’s id since there should be only one record per user at any given time.

I used the new MERGE command to perform an “upsert” of the data.  If its the first time the user has been mentioned, it will perform an insert and set the mention_count to 1.  Otherwise, we’ll update the record by setting mention_count to mention_count + 1.

Since I’m using the MERGE command, I encapsulated it into a stored procedure as oppose to writing LINQ queries. The stored procedure receives a user id and twitter name and performs the insert/update logic.

The MERGE command is an incredibly useful feature introduced in SQL Server 2008.

Parsing the JSON Result

As mentioned above,  we’re just going to parse the JSON object and iterate through the users mentioned in the tweet.  As we do that, we’ll pass the users into the stored procedure above and MERGE the data into the SQL table.

Important: Since we’re not using a queuing system, the rate at which we can process tweets will depend largely on the speed of our SQL stored procedure. If you’re considering something similar in a production environment, please implement a queuing system to handle the load

Results

Once you start running the application, you’ll start to see Twitter ids and screen names appearing on the console window.  Let it run for a few minutes and, depending on the popularity of your search term, you should start to see some results.  You can then go to SQL Server Management Studio and run a simple query to get a view of the activity on Twitter for that keyword / user mention combination.

Hope you enjoyed this post and have some ideas for implementing something similar with your next social media and Twitter campaigns!

Most of these server implementations use some sort of queuing system and a long-lived HTTP connection which clients use to have data delivered in near-realtime. In this post, I’m going to demonstrate just how easy it is to hook into the Twitter Streaming API.  Specifically, we’ll be using the streaming/filter method to consume any tweets which match a filter condition in realtime. I’m sure you can think of some neat ideas that can leverage this concept!

To demonstrate, let’s just create a console application in Visual Studio. We’re going to create a simple WebRequest to connect and just start reading the stream. Then, we’ll just print out the json result to the console window. I’ll leave it up to you to write some parsing logic to actually do something with the data.

That’s all there is to it!   Your output on the console window will look like this:

What you do with the data is where the real magic happens. The last I heard Twitter was producing somewhere in the range of 1000 tweets per second and I’m sure its much higher than that now.  You will probably want to implement message queuing system where you hand of the response as soon as possible for some other process to handle without blocking the incoming stream.

In a future post I’ll describe some neat ideas for what you can do with the data from the stream.

Hope this helps.

I knew Three20 handled this scenario but I couldn’t find a good example that walked through each component that was required to implement it.  It was a challenge to get all the bits to work together so here is my breakdown in case others are facing a similar challenge. I’m going to divide the post into two parts. In part 1 we’ll just hook up the classes with each other then in part 2 we’ll implement the actual request / response logic.

If you’re not familiar with the Three20 library, I highly recommend you take a look at it for your iPhone / iPad projects. For this example, I’ll assume you have integrated Three20 into your project already and are somewhat familiar with the library.

To demonstrate the load more feature we’ll be doing a simple integration to display my public Twitter timeline with paging. Since my timeline doesn’t require authentication, we won’t have to get bogged down with implementing the OAuth protocol. Now, let’s look at the players:

TTTableViewController

To start, let’s add a subclass of the TTTableViewController. This will just provide us with a base controller with a table view to utilize for demonstration purposes. In addition, we’re going to stub in a class which will serve as the datasource for this controller.

MBTwitterController

Subclass of TTTableViewController

That’s pretty much all you have to do within the tableviewcontroller thanks to Three20 encapsulating a lot of the delegate logic into the datasource object. Now, let’s create the datasource for the table.

TTListDataSource

If you’ve worked with Three20 and table implementations, you’re probably familiar with this class. Again, we’ll subclass it to create a new datasource object which will be consumed by our table view controller.  We’ll also add some code which will work with the TTURLRequestModel.

Subclass of TTListDataSource

A few things are going on in this class. First, we allocate the TTURLRequestModel which we’ll add in just a minute. We also implement the model method which we’ll set to the TTURLRequestModel above. This is very important if you want to keep your sanity trying to debug later on. Below is the header file for the class.

MBTwitterDataSource.h

@interface MBTwitterDataSource : TTListDataSource {
	  MBTwitterRequestModel* _twitterFeedModel;
}
@end

TTURLRequestModel

The last class we’re going to create will be a subclass of the TTURLRequestModel. This class will encapsulate all the network operations to interact with the Twitter API.

MBTwitterRequestModel

@interface MBTwitterRequestModel : TTURLRequestModel {
}
@end

These are all the classes involved in getting this to work. In the next part, we’ll look at writing the code to interact with the Twitter API and how to actually add the “Load More” button to the table cells.

As a [role], I want to [feature] so that [goal]

Many organizations translate this into a spreadsheet where each column represents a different field the story writer can fill in:

I have used this template on almost every agile project I have participated in.  For the most part, it works very well. It typically provides a great launch point for additional discussions before it makes it into a sprint.

One of the most common mistakes I see when introducing the template (and agile stories in general) to a new team is a tendency to define every role as just a generic “user”. For example, I’ve seen user stories as generic as:

“As a user, I want to upload photos to a library so that I can use them on a post”

Although its syntactically correct, it is vital that we define a role and not a “seat” on the system. A more valuable user story would look like this:

“As a user designer content author, I want to upload photos to a library so that I can use them on a post”

Notice how this story now has a better context of who will be using the feature and how. This could provide valuable insight into the intended audience and how best to implement the feature — a designer may imply more advanced features while a content author would use the image as is.

Always avoid the generic “user” role in your user stories.

Hope this helps.

http://api.twitter.com/version/statuses/public_timeline.format

So while this public timeline call will return XML:
http://api.twitter.com/1/statuses/public_timeline.xml

This one would return JSON:
http://api.twitter.com/1/statuses/public_timeline.json

The proper REST way to do this is by changing the headers in your request to ask for a particular file format.  Indeed, an implementation utilizing request headers has already been written.  However, I prefer Twitter’s implementation which coincidentally seems to be a feature of the Scala / Lift framework.  To accomplish this with ASP.NET MVC, we are going to build an attribute that checks the incoming request and then attempts to render the resulting model in the selected format.

First, some words of caution.  I don’t think this method will scale if your API / web application takes off.  Although it is a great example of the DRY (don’t repeat yourself), code-once software development practice, it does somewhat tightly couple your API to your front end.  Twitter faced a similar situation when their API was hosted on the same domain as the web application.  Eventually, Twitter moved the API to api.twitter.com.  This practice allows the API to evolve independently of the front-end of your application.  With that warning in mind, let’s dive into the code.

They way we’ll implement this functionaly is to develop a custom action attribute which will check for a “format” parameter on each request to decide whether to return the model as a JSON, XML or the default view that the controller will return. This is slightly different from Twitter’s implementation but it simplifies some of the routing issues we would have with ASP.NET MVC.  Using an API-driven development approach, I’ll start with an existing controller and add a (non-existent).  This gives me an idea of how it would work.

I could add some additional parameters at this point to restrict which formats to accept, but we’ll leave that out for now (KISS – Keep It Simple Stupid!).  Next we’ll just generate the class using the smart tag in Visual Studio which will create the stub for it.  We want this action filter to execute before the response is returned to the client but after the data model has been populated.  This implies we’ll be overriding the OnActionExecuted method:

At this point, everything should build.  Our next step is to create two custom ActionResult subclasses which will return either Xml or Json representations of the model you pass in.

ASP.NET MVC includes a JsonResult which we’ll use for the JSON format.  Although there is no XML equivalent, the MVCContrib library does have an XmlResult class.  You can download the bits here to provide the XmlResult class (along with some other very useful features).  Add the assembly as a reference to your project.

Now we have the two ActionResult’s we need to implement the solution.  Below is a screenshot of the finished method.  Essentially, we’re looking into the filter context to find the file extension.  Depending on if it’s json or xml, we use a different action result serialization technique.

Notice that we need to set the JsonRequestBehavior in order for this to work.  There are several security vulnerabilities which you should explore before defaulting this behavior at such a broad level. Phil Haack details some of the issues with JSON Hijacking here.

That’s all there is to it.  Hope this helps!