Amir Razmjou bio photo

Amir Razmjou

Researcher

Email Twitter Facebook Google+ LinkedIn Instagram Github Stackoverflow Last.fm Pinterest Foursquare Youtube

Collecting Data using Windows Azure Part 1

Overview

Overview

I decided to develop an Azure WebJob to collect my twitter data into MongoDB, using LinqToTwitter library along with MongoDB made this job pretty easy.

var auth = new SingleUserAuthorizer
{
    CredentialStore = new SingleUserInMemoryCredentialStore
    {
        ConsumerKey = ConfigurationManager.AppSettings["twitterConsumerKey"],
        ConsumerSecret = ConfigurationManager.AppSettings["twitterConsumerSecret"],
        AccessToken = ConfigurationManager.AppSettings["twitterAccessToken"],
        AccessTokenSecret = ConfigurationManager.AppSettings["twitterAccessTokenSecret"]
    }
};

Once CredentialStore object constructed it is time to extract tweets by a LINQ statement and to store them in the MongoDB. 

using (var twitterCtx = new TwitterContext(auth))
{
    twitterCtx.Log = Console.Out;

    var searchResponse =
        (from search in twitterCtx.Search
         where search.Type == SearchType.Search &&
         search.Query == "\"LINQ to Twitter\"" 
         select search)
            .SingleOrDefault();

    MongoServer mongo = new MongoClient(_connectionString).GetServer();
    mongo.Connect();

    var db = mongo.GetDatabase("CiteTrack");
    var collection = db.GetCollection("Tweets");

    if (searchResponse != null && searchResponse.Statuses != null)
        foreach (var status in searchResponse.Statuses)
        {
            collection.Insert(status.ToBsonDocument());
          
            Console.WriteLine(
                "User: {0}, Tweet: {1}, Date: {2}",
                status.User.ScreenName,
                status.Text, status.CreatedAt);

        }
    
    mongo.Disconnect();

}