Weekly Training Site

I blogged last month about my goals and where I wanted them to be. In response to that I built a site that helps me track where I am for the 7 days trailing. That site can be found at http://training.wraithan.net/.

Users log into my site via Dailymile using OAuth2 since I need to get their API token in order to collect their workouts and display them. You can see my profile at http://training.wraithan.net/profile/Wraithan. At time of writing I am nearing my goal for biking but my running and hiking have suffered.

I built this site using Django, I did all the OAuth2 stuff myself because when I last surveyed the existing work with OAuth2 and Django, I found I would have to write my own. I turns out it is pretty simple, and because of the many drafts that exist, it would be pretty trying to have a more generic app for this. This will hopefully change when OAuth2 is finalized.

Dailymile’s API has some warts but it is usable and they were rather responsive when I had some requests for features and the one or two bugs I ran into. Plus their terms of service for their API are really reasonable. I can’t say the same about other workout tracking sites I looked into, either they had a horrible ToS or they plain didn’t have an API.

ZenIRCBot 2.0

Today marks the day when the bot’s API should only expand, without removing or changing the currently available messages that the core bot sends.

admin.js now uses forever a Node library for process management akin to runit, god, or supervisord. The big win with forever over those others is that it has both a command line interface as well as the ability to use it as a library. This lets admin.js no longer have to shell out to fabric which shelled out to tmux (you can see why I replaced it).

The types of messages that the bot emits is still more limited than I’d like, but I can add to the API without making backwards incompatible changes. Also the bot is still just a neat bot not a useful one for most people. This will hopefully be changing in the near future.

If you are following the development or using the bot yourself, these are some things you should know. First off, master should remain stable now. All development will take place on develop and in branches. Once I am done with a feature I’ll merge it into master and tag it with the version. Versioning will work as follows: x.y.z

  • x will change if there is a backwards incompatible change in the bot. This includes config, protocol, and services API changes that are backwards incompatible.
  • y will change if there is a change to the core bot and its protocols that is backwards compatible. This means if I add any new message types or options that default to how behavior worked before you’ll see a y version change.
  • z will change when it is just a service changing. For now these wont ever trigger a y or x change unless it is pretty drastic. This may change in the future.

Also, if you use the bot, or would if there were certain features/services available, please post them as issues on GitHub. If you like it or want to discuss it, let’s hear about it in the comments or in #pdxbots on Freenode.

Read the Docs

Over the last few months I’ve really started to love Read the Docs. Several of my favorite projects are hosted on there. Projects such as Fabric, Requests, and Celery are all hosted on there. The list keeps growing, which is great. Read the Docs is a fantastic project that is promoting and unifying documentation even across language boundaries, both human and programming.

I got involved with Read the Docs during the sprints of DjangoCon 2011 as I have mentioned before on my blog. So my opinion isn’t entirely unbiased, but I wouldn’t have gotten involved if I didn’t believe in it and what goals it is pressing towards. I strongly urge others to also contribute, which can come in many forms, such as spreading the word on using Read the Docs, writing docs and posting them on there, filing bug reports on things you find wrong or feature requests for things you think could be done better, or finally by contributing your coding or design skills.

Something that is pretty interesting is there have been a number of projects that aren’t software libraries that have found their way on to Read the Docs. There have been a few books written using Sphinx and then posted on Read the Docs. Notes from various talks and conferences. And recently I stumbled upon a resume that is hosted on there and it got me thinking that I should do the same.

My resume is now hosted on there, though a warning, it is a work in progress. It looks really nice though, and I have access to multiple file types of it that sphinx and Read the Docs have generated, including a PDF. I highly recommend it to anyone that wants to write their resume, have it stand out a bit, but still look really nice.

Basically, this post is just to talk about how much I have come to depend on Read the Docs and to also to encourage people to use and help out in various ways. I love this project and if have ever come across horrible documentation, so should you.

Django Generic Class Based View Tip

Those reading this via a feed reader will have to view the page on my blog as I am using embedded gists. I’ll find a solution for that in the future.

So say you have a base template that looks something like:

And a template that looks like either of these:


It used to be that you could write something like this:

But generic function based views are deprecated and the world is being strongly urged to move to generic class based views. If you would like to get extra_content working with the direct_to_template replacement TemplateView, you can use a view like the following:

And a urls.py like the following using it:

The code used in this blog post can be found in this gist.

Being a Contributor

When I attended DjangoCon I decided I needed to work a bit less on my own projects and start putting more time into contributing to other projects. I use so many free and open libraries both in my work and my personal code, and it is time to start contributing again.

The first project I decided to try to contribute to was Read the Docs. The choice was pretty easy since I know Eric Holscher and it is a project I benefit from on a regular basis. So at the DjangoCon sprints I sat down near him and asked him if there were any particularly valuable tickets I could take and start hacking on. He gave me a list of issues and I started in.

By the end of the day I had a patch for RTD, granted it didn’t fully work but it was a start. Added some unit tests and fixed up the code and soon I had a complete contribution that was added and it made me smile. And smile even more to be added to the AUTHORS file. Couple more patches later and Eric asked if I wanted to help out with the servers and get access to them. I said yes and now I am part of the RTD ops team as well as a contributor.

I used to contribute to a couple projects a few years ago. In recent history I’ve spent my time scratching my own itches and working on proprietary code for work. Going through the process of contributing has made me excited to do it some more on various projects. And GitHub makes it so easy to just send a pull request, comment a couple times back and forth and get patches accepted.

I am far from the first to talk about how easy it is to contribute to projects compared to the old days of sending patches to mailing lists. I just want to be another voice in the crowd, if even one person decides to contribute because of this, or it makes a person happy and continues contributing this was a success.

Django, Redmine both on the same vhost in apache

I’ve spent a good portion of my day messing with this, asking questions in various channels on irc.freenode.net, and generally trying to not let my head explode.

What ‘this’ is exactly is this:

I wanted my django app to run on / of my domain. Since I don’t own my domain it would be easiest(or so I thought) to setup redmine on /redmine of my domain. This way it is only one domain and I can run my bug/project tracker on the same place as my website.

http://synput.thruhere.net
http://synput.thruhere.net/redmine

That shows that I have it all working, now how did I do it… It will be simple for you but was not a simple process for me due to a couple settings I had tweaked without realizing the effect.

First off lets get the django app setup, I use mod_wsgi for my django apps, and run them in daemon mode so I can easily update the app without taking down the server at all.

My wsgi file (should look pretty typical for those who have seen one before):

import os
import sys

root_dir = os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))
sys.path.append(root_dir)

os.environ['DJANGO_SETTINGS_MODULE'] = 'recipes.settings'

import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()

You’ll need to toss

LoadModule wsgi_module modules/mod_wsgi.so

into your httpd.conf

Here is my vhost stuff without the Redmine stuff in it:

<virtualhost 80="">
ServerName synput.thruhere.net
ServerAdmin xwraithanx@gmail.com

Alias /media /srv/wsgi/recipes/media

<directory srv="" wsgi="" recipes="" media="">
Order allow,deny
Options Indexes FollowSymlinks
Allow from all
</directory>

WSGIDaemonProcess recipes python-path=/srv/wsgi/recipes/lib/python2.6/site-packages display-name=%{GROUP}
WSGIProcessGroup recipes
WSGIScriptAlias / /srv/wsgi/recipes/recipes.wsgi

<directory srv="" wsgi="" recipes="">
Order deny,allow
Allow from all
</directory>
</virtualhost>

So far this should look pretty typical for anyone who has ever setup a django vhost with apache.

Now for Redmine I chose to use Phusion, why you might ask? Well sycopomp uses it and when I asked him why, because he found a guide somewhere that made it easy and his memory usage isn’t very high. So basically, not for any good reason.

Go ahead and symlink your redmine/public directory into your DocumentRoot for me that was /srv/http.

You’ll have to install Phusion:

gem install passenger
passenger-install-apache2-module

That will toss up a helpful wizard to do the installation for you with a couple of prompts. You will need to copy the lines from the wizard into your httpd.conf, they should look something like:

LoadModule passenger_module /usr/lib/ruby/gems/1.9.1/gems/passenger-2.2.9/ext/apache2/mod_passenger.so
PassengerRoot /usr/lib/ruby/gems/1.9.1/gems/passenger-2.2.9
PassengerRuby /usr/bin/ruby

Next we need to add a line to our vhost and change another one, first line we’ll add tells the server where to find and put redmine:

RailsBaseURI /redmine

And secondly we need to change out:

WSGIScriptAlias / /srv/wsgi/recipes/recipes.wsgi

Add put in the following instead since that will make django not catch /redmine:

WSGIScriptAliasMatch ^\/(?!redmine(.*)) /srv/wsgi/recipes/recipes.wsgi

Restart the server and we should be good.

I didn’t make this page/code as generic as I could have but hopefully you can get the meaning from it, if not the kind folks in #httpd on irc.freenode.net can help you. They’ll understand this if you don’t. Also, do what some like to call frobbing, play with it, break it, whatever it takes so you understand it.

bpython and django

There are a few posts out there to get basic support for django inside of bpython but I as far as I have seen they ignore the virtualenv stuff that most of us django users do.

virtualenv is a must when doing development on django projects, especially when you have several projects going on the same machine or you want to involve other developers. It allows for a clean environment with known versions of every package that is installed inside of the virtualenv.

The first part of this is a known hack or method (take your pick) to make bpython recognize the django project.

First you need to modify your ~/.bashrc or ~/.bash_profile (system dependent as to which is used) and add the following line to it:

export PYTHONSTARTUP=~/.pythonrc

What this does is it tells python to run ‘~/.pythonrc’ before it does anything else in the interpreter, this is a python specific thing not bpython specific thing.

Next you need to create the ~/.pythonrc file and put the following into it:

try:
   from django.core.management import setup_environ
   import settings
   setup_environ(settings)
except:
   pass

Now if we didn’t care about virtualenv we’d be completely done. If you go the less strict route and don’t use –no-site-packages (the switch means that system wide packages are to be ignored in the virtualenv) then you will need to edit your bpython binary, most commonly located at /usr/bin/bpython or /usr/local/bin/bpython and change the shebang from:

#!/usr/bin/python

(it may be /usr/local/bin/python)
to:

#!/usr/bin/env python

This makes bpython use the value of the environment variable python instead of using a specific binary, the environment variable gets set by virtualenv so you are good to go after this.

If you use –no-site-packages then you will have to install bpython in each virtualenv that you want to use it in.

If you have any questions toss them in the comments or find me on freenode as Wraithan.

Habits, past jobs to present.

Something that has been a big part of each job I have had (excluding Ross and Taco Bell) was that I would always find something I could fix with software (well actually I could’ve fixed the Taco Bell stuff but they wouldn’t let me) so I begin coding away on the problem.

At first just taking a small portion of each day working on it, then progressively ramping up the amount of development time I take at work, along with spending more time on the project at home. I’ve always chocked this up to me wanting to be a professional programmer, along with my drive to fix problems I see, especially problems that inconvenience me.

Each of these times people asked me why I would work on work related stuff at home when I wasn’t getting paid for it and I would tell them it fun or it will make work easier, or in the case of working at Retail Imaging it was to get off the phones and into the IT department.

It gave me experience, made work easier, though I was working at home so there was no net lessening of work, in fact there was probably an increase. Be that as it may, it was a fun increase in work, it let me relax and gave me a sense of accomplishment.

Now I am programming for a living, I love it, I get to come in and work my way through problems and challenge myself. I have already begun fixing things and am getting that sense of accomplishment, along with the experience and with each thing work seems to get a bit easier.

But I am still doing my old habit. We have these data files at work (YAML) that are unweildly and not very fun to edit, add to, or create. There is no validation that the type of information put in for a value will actually go into the database until it blows up when trying to shove it all in there.

I decided I wanted to write a tool for this, now it isn’t a bid deal for everyone else it seems, even the QA folks don’t seem to care too much about this. For me though this is a big deal, so I spoke to my boss to see if he knew if there were any existing tools, and he gave me a detailed explanation about why not.

So I have now started development on a tool to do this task, that I am working on at home, and now I have no excuses, it is just fun for me and if my friends and family insist on questioning it and warning me I am going to burn out, I am just going to ignore them.

Programming is a passion for some of us, sure there is too much of a good thing, but that is why there are many programming languages, design models, domains, and environments to program in.

Exciting Introduction to Newsqueak, for me at least. (NL10)

Sykopomp and I have been speaking on concurrency a lot lately.

I’ve used concurrency in several applications, the largest of which was a bot for EQ that used an Actor based concurrency system. Other that have used concurrency haven’t used any of the larger patterns, and honestly were just off the cuff and likely would not have worked well at all when scaled up.

Sykopomp pointed out Communicating Sequential Processes. I read about it and it seemed like a really good idea, but implementing it was still a bit beyond what I could imagine. He then clued me in on a Google Talk done by Rob Pike, Advanced Topics in Programming Languages: Concurrency/message passing Newsqueak.

After watching this, I am really excited for this style of concurrency. I am looking to use it with Python but I have yet to find a library that implements it. I have considered using newsqueak directly but the problem there is the small community and even smaller amount of libraries available.

If I can not find such a library, I may take it upon myself to implement this, possibly as a wrapper on python. Sykopomp would just as soon I learned lisp and used the lib he found for lisp using this, but I am going to stick with my python for now.

EDIT: Found a library for pyhton: PyCSP. Going to install and play with it today, I am really excited. I’ll report back my experience as soon as I have enough code written to decide on if I really like it or not