Setting up a daily task on Heroku

When I set out to create the Skills App site, one of the main things I wanted to figure out was how to scrape the data on a daily basis and automatically insert into the database, preferably at the same time each day. I started out with some quick googling while I was developing the app, but nothing obvious stood out. There were a couple of ways that seemed to be repeated on how to get data into a rails app. One was having a rake task, and the other one was via a seed file. I wasn't sure what to do so I tabled it, and decided that it would be one of the first things that I figure out after the initial deploy.

After the deploy, my good friend Scott Parker, who makes awesome dragon games, gave me some feedback on the site after reading my initial post.  He suggested that I take a look into using rake and the scheduler add-on for Heroku. I took a look at rake and played around with it a bit. I also found a useful railcast about it which helped me figure out what I needed to do.

So I took the majority of my scraper code and put it into the rake task and tested it locally to make sure it was working, then I set up the scheduler to run around midday, uploaded all of the data I have been collecting every day and anxiously waited to see if it would automatically add today's data.

Success!  Data from today!

Enjoy the site and please feel free to reach out to me with feedback, feature requests, or anything else.  For those that have given me feedback, I really appreciate it and hopefully I will implement your features soon!