Random:

Here's a look at my top 5 applications from 2008 and how much time I spent in them by week.
(from an alpha version of an upcoming version of RescueTime)



TextMate is the app I use to do all my coding. It was a rough summer for writing code, it seems...

Today I enabled threaded (mpm worker) mod_python to try to boost performance of the RescueTime back end and started noticing the following error showing up in the logs:

File "/opt/lib/python2.5/site-packages/django/db/models/query.py", line 497, in _filter_or_exclude
clone = self._clone()
File "/opt/lib/python2.5/site-packages/django/db/models/query.py", line 595, in _clone
query = self.query.clone()
File "/opt/lib/python2.5/site-packages/django/db/models/sql/query.py", line 195, in clone
obj.__dict__.update(kwargs)
TypeError: descriptor '__dict__' for 'Empty' objects doesn't apply to 'Query' object'


The problem appears to be due to the magic that function does of
changing the runtime class of an object (which is explicitly listed as a known bug with using psyco on their site).

To fix it, I figured I'd just prevent that particular method from getting optimized by psyco, so I added this to the top of my models.py file:

from django.db import models
import psyco
psyco.cannotcompile(models.sql.query.Query.clone)


So far so good...

RescueTime Proving Useful For The Enterprise, Raises $900k

Y Combinator startup RescueTime lets users monitor which applications and websites they use/visit the most, and then lets them use that information to try to cut down on inefficient uses of time. It’s useful on an individual basis, and it helps businesses (who pay $4-$8/user/month) monitor what applications are being used, and where time is being wasted.

Not even a year old, our work is just getting started - but we have a lot more time and resources to help us now.

Tony talks to Scoble about RescueTime, and uses my computer to show it off. Oh he mentions my work habits too (ha!). It was filmed at the old Jobster offices in Pioneer Square.



http://www.fastcompany.tv/video/see-how-much-tme-you-are-wasting-with-rescuetime

Listen here (we're mentioned at 01:10)

Day to Day, July 8, 2008 · Outsource your daily tasks, stop making to-do lists in favor of to-don't lists. Timothy Ferriss, author of The 4-Hour Workweek: Escape 9-5, Live Anywhere, and Join the New Rich, offers guidelines for a plusher life, without a Blackberry. Hard-working Madeleine Brand considers how she could adapt them to her life.

This article interviewing Tim Ferriss, productivity guru and author of The 4-Hour Work Week, is on the front page of U.S. News and World Report's website today.

This is my favorite part:

That leads me to the importance of becoming aware of how you are spending or misspending time while you're on a computer. Do a time audit. Rescuetime.com will tell you how much time you spent on different websites and pages. It creates graphs and charts and tells you how you're wasting your time.

- Tim Ferriss


Thanks, Tim! That's a great plug!

Here's a snippet from our google analytics for the web traffic of rescuetime.com.





The spike in the beginning is TechCrunch/LifeHacker coverage. The rest is word of mouth.

Our user signup graph is starting to look exponential.

Awesome.

We're going through the guts of RescueTime this week doing our monthly optimization of the database and all of the queries we use to generate and visualize our time management data.

One of the things we turned up that happens whenever a user client pushes data to our api servers is that this is the first thing that happens:


User Load (0.044165) SELECT * FROM users WHERE (email = 'blah@blah.com') LIMIT 1
User Columns (0.023302) SHOW FIELDS FROM users
User Load (0.612006) SELECT * FROM users WHERE (LOWER(users.email) = 'blah@blah.com' AND users.id <> 323224212) LIMIT 1


Notice the 0.6 second select call. That happens when we update the last_data_sent field on the user object and save it using ActiveRecord's save! method. The reason is because it's validating uniqueness of the user's email address (in case it happened to have been changed) - even though it clearly hasn't changed. Why isn't there a dirty flag there? Anyway, that's beside the point. This is going to continue to get worse for us as we grow our user table size.

The solution is to do this:

User.update_all(["last_data_sent = ?", Time.now], ["id = ?", 323224212])


which doesn't perform any validations (which we don't need here) and is appropriately fast. This is an example of how Rails is super great for getting things up and running quickly, but forgetting to go back and optimize can really hurt you. In our case we're shaving off more than 1/2 a second from every api call.

We've also already made changes to our database which have cut it in size from 20GB to 2GB... but I won't go into those here.

I've gone back and forth through my career from jobs where I was writing code behind the safety net of a QA department to jobs where I was the only gateway between my code and the wild world of users. At the end of last year I quit my job and started working on RescueTime full-time, once again switching from the world of QA-ed code to what amounts to the seat of my pants. While the details of this jump are fresh in my head, I figured I'd share some of the things that I've learned that should help when you are the only one you're relying on to protect your users from bugs, crashes, and downtime.

1. When you think you're ready to release a significant batch of new features, wait a couple of days before you deploy to production. The temptation for me is always to release immediately (OMG our users are going to love this! They need it now!), but I've found that by waiting a day or two I always come up with some scenarios, use cases, boundary cases, and bugs that I hadn't thought of. It isn't until you start to get bored with the new features you've written that you really understand what their limits are. It doesn't cost much to take the extra time to let the dust settle in your mind before pushing something out that's not quite ready. Your users who will almost always immediately find what's wrong.



2. Deploy to a staging environment first. This seems obvious but when it's just you, it's very tempting to feel like your code is golden and just push it right from your laptop to production. Make sure your staging environment runs on its own database (preferably a reasonably recent copy of production) and test out any database migrations on the staging database. A lot of times I make last minute changes to my code on my machine and think they look good enough to not have to test them again. Staging is your sanity check. Use it to your advantage, because nothing drives you more insane than struggling to fix a botched deploy on the live site.



3. Write tests. It's difficult to justify a huge number of tests in the early phases of a new project (are people even going to use this?) but for features that have been especially tricky to nail down or that just are essential to your product working correctly, it really helps to have tests that you can use as a baseline when you need to refactor. RescueTime just went through a significant architecture change and I could've used a few more tests around to make sure we covered our bases.



4. Have someone else verify that everything works. Get your other co-founders to use it for a day or, if you have to, contact one of your power users and invite them to use the staging server. You need to have someone use it other than you before launching. There will inevitably be something that breaks caused by doing something you just hadn't thought of doing.



5. Branch, tag, then deploy. Don't deploy from trunk. Srsly. You're going to have issues if you do. Take this scenario: You spend a week coding up features for a new release. The day before the deploy there's a bug in the existing code that needs to be fixed immediately. In a rush, you submit a fix to a source file from your trunk enlistment on your machine and deploy it to production. Whoops! Now you have added code from features that haven't been released yet. Not only that, but you've only been using trunk, so you have to go back and back-merge all of your changes. I branch every release that will be deployed to production, and only deploy tags from that release. This makes it easy to roll back, and forces you to have separate code enlistments for each release of the code you're working on. It also forces you to be disciplined with your checkins since each production deploy requires a new tag. Here's a link to a discussion on how to add this functionality to your capistrano deploy configuration.



6. Don't push new bits at peak usage. Find a time, usually late at night (7-9PM PST is usually a good bet) when there aren't a huge number of users on your site. That way, if you need an outage (or worse - you have a big problem), it won't affect too many users.



7. If you do have a problem - don't panic! Users are grinding away on a gnarly bug on your production server? Oh well, you tried as hard as you could (presuming you followed the previous advice) to release smoothly. Your code is still "beta" after all. Put up an outage page and calmly debug and fix the problem. If you can't in a reasonable amount of time, roll back and fix it tomorrow. The worst thing to do is to panic and turn a small issue into a larger one (like accidentally dropping your DB for example).

That's all the advice I have for now. This week we'll be launching a new release of RescueTime with some nifty (and complex) new features. I'll let you know how it goes, and will amend this list accordingly. :)

On Saturday I finally accomplished the goal I set out to complete in October of 1996. In reality, it was a goal I set for myself when I was somewhere around 7 years old. I completed my BS in Computer Science. I'm glad I attended the commencement ceremony, it really put a period at the end of the 11 year trail I took to get here. I was even a minor celebrity at the event. So now I'm officially a scientist! I even get to include it on my corporate profile. Not having my degree was never something that got in my way, since I had taken almost all of my courses before I left school the first time. It really does feel like my career is more grounded now, though. It's somewhat hard to describe.

At the end of the year I'll be leaving my favorite company to work for (up until this point) - Jobster. I'll be cutting the strings to go full time and full speed ahead on RescueTime. I feel like I've learned so much in the last <2 years - enough that I'm ready to go at it for real this time. With Jobby, I was in a little over my head. Now, I'm extremely confident that I can help to start and run a company at this point in my career. RescueTime is already building up a lot of good traction and interest without any real market prodding. It speaks to the fact that people really seem to have the problems we're trying to help solve, and are very much in need of a good product to help them.

So, next year should be exciting! I'll be focused on only one thing (effort-wise) for the first time in years. While the road ahead will be very energy-consuming, I'll actually be less burdened than I have in a while, and it should go quite smoothly. 2007 was a year of learning, branching out, refactoring, and accomplishment. 2008 will hopefully be the beginning of a whole new phase in my life and career. Wow, does that sound sappy.