Install Graphite on OSX 10.7 Lion

Installing Graphite locally on my machine was a bit annoying. Hope this helps the next person.

1. Virtualenv

Because we are clean and tidy people, let’s start by first creating a new virtualenv

mkvirtualenv graphite

2. Pip

This installs graphite into the default location at /opt/graphite

pip install whisper
pip install carbon
pip install graphite-web

3. Install py2cairo

Follow the instructions here

    1. Download, unpack, and cd into the the py2cairo directory
    2. Activate your virtual environment
    3. Follow the standard build procedure
./waf configure --prefix=$VIRTUAL_ENV
./waf build
./waf install

4. Configure conf files

cp /opt/graphite/carbon.conf.example /opt/graphite/carbon.conf
cp /opt/graphite/storage-schemas.conf.example /opt/graphite/storage-schemas.conf

5. Configure graphite

cp /opt/graphite/webapp/graphite/local_settings.py.example /opt/graphite/webapp/graphite/local_settings.py

Then edit settings.py

STORAGE_DIR = os.environ.get('GRAPHITE_STORAGE_DIR', join(GRAPHITE_ROOT, 'storage'))

DATABASES = {
    'default': {
        'NAME': os.path.join(STORAGE_DIR, 'graphite.db'),
        'ENGINE': 'django.db.backends.sqlite3',
        'USER': '',
        'PASSWORD': '',
        'HOST': '',
        'PORT': ''
        }
}

# Default sqlite db file
# This is set here so that a user-set STORAGE_DIR is available
#if 'sqlite3' in DATABASE_ENGINE \
    #and not DATABASE_NAME:
  #DATABASE_NAME = join(STORAGE_DIR, 'graphite.db')

SECRET_KEY = '<insert your secrey key>'

ALLOWED_HOSTS = '*'

This makes the settings file compatible with Django 1.5. You can get a secret key generator here.

6. Syncdb

cd /opt/graphite/webapp/graphite
sudo python manage.py syncdb

You should see a file called graphite.db in /opt/graphite/storage

7. Run

sudo python /opt/graphite/bin/carbon-cache.py start
sudo python /opt/graphite/bin/run-graphite-devel-server.py /opt/graphite

This is the expected output

graphite hello world

Here’s a test script to see if your graphite is working locally. https://gist.github.com/nychng/5883607. Got it off http://coreygoldberg.blogspot.sg/2012/04/python-getting-data-into-graphite-code.html

Then under tree, graphite, you should see foo/bar/.

Advertisements

A Review of PyCon SG 2013

Pycon SG 2013

PyCon SG 2013

As an Organizer

I was helping out with this year’s PyCon Singapore 2013. It’s my first time helping out in an event and I enjoyed it greatly. Here are some of my takeaways:

  1. It is better to present something simple very well than to present something hard/challenging poorly
  2. The conversations between the talks are the best parts of any conference
  3. Coffee is a must have
  4. The common mindset is to assume all the speakers to be domain experts in their fields. But it’s actually ok and acceptable for ‘newbies’ to take the stage and give it a go. It is only in ‘failing’ and feedback from the community does the person learn and grow. This ultimately also benefits the community in the long run
  5. Impostor syndrome is very real
  6. It’s the people and newly formed relationships that determines whether a conference is successful or not. As an organizer, your job is to provide “points of contact” such that there’s a good balance of talks and actual interaction.
  7. Juggling between beginner vs. expert level talks and managing the resulting expectations is probably the hardest aspect of the conference. An idea could be to run two tracks: one focused on beginner stuff an the other on more obscure/advanced stuff.

As An Attendee

I can only mention the talks I went to, so here they are:

  1. @_nishad‘s create_awesome_api(time_limit=’1 week’), talked about the tech schema at Plivo. They essentially have a proxy server using flask and gevent which in turn pings another server running Django. The Django server just serves as a provider of API endpoints via Django-tastypie. On the side, I found this cool monitoring app that shows the uptime of your API.
  2. @tsudot‘s talk on Graph everything! kinda confirmed my belief that going Graphite is the best solution for monitoring analytics. He also introduced Graphene which is a wrapper around d3.js for prettier visualizations.
  3. @rohit01 covered building a dashboard using Shinken. Shinken is essentially a Python rewrite or Nagios which is written in C. I suppose once we start hitting more servers, setting up our own service monitoring system will be the way to go.
  4. @vanzaj‘s talk was on data management for scientists and he  introduced the problem of keeping track of binary data and its resulting output for a scientist running experiments. Essentially, he is unable to reproduce his thesis results based on the experiments he ran years ago which was a WTF!? moment for me. It’s an interesting problem in academia which no one seems to have solved properly yet. His open source tool, dacets, is trying to solve this problem.
  5. @sajnikanth‘s talk on blackbox testing was very well presented. Talking from a QA perspective, he covered Selenium in a very entertaining manner — a word not frequently associated with testing. He’s also involved with Holmium.
  6. Vikram’s talk on Music beat aware interactive physics simulation. This was interesting to me as I’ve not really been exposed to the musical aspects of Python before. He demonstrated what a sine wave sounded like and went through basic modules such as the pyo library which is used to do audio stuff with python. He had a funny moment when he was demo-ing a high frequency sine wave and quipped: “No one has ever died from a Python interpreter”. lol!!
  7. @mbrochh spoke on writing reusable django apps. Surprising, this was the only django talk at PyCon SG. There’s also a django tutorial too. He has a nice template for making apps. Lessons learnt the hard way from making over 65 django apps. Generally speaking, a well written app will simply work out of the box by:
    1. pip install
    2. add to installed apps
    3. add to urls.py
    4. load template tag
  8. @wesmckinn‘s talk was obviously on data. It was a high level talk about Python and Data and where stuff is heading. It seems to be moving towards a cloud based collaboration context with pretty visualizations. Also, interesting to know that  “data warehousing as a service” is on the rise. Case in point is Amazon’s Redshift.

Also, thanks to all the startups that attended this year’s Startup Booth. They are:

It was interesting to hear how different startups were using Python in their stack. From NLP at Infotrie, to gevent-socketio at Beautiplan and even the story of a language migration from PHP to Python by Flocations. Also, this could also be the first public announcement of Billpin pivoting into the payments space. @ruiwen gave a Steve Jobs-esque vision of what Paypal might look like if they were to start today. A frighteningly ambitious idea but they certainly possess the tech and business smarts to execute it.

Thanks to all the organizers and sponsors that have made this conference possible. Also, a big thank you to Plug-In Blk 71 for letting us use their rooms for our meetings.

Lastly, a shoutout to our friends from Tokyo. Thanks for visiting! PyCon APAC will be held in Tokyo from 13 – 16 September. Ping @ryokamiya for more information.

PyCon SG 2013

LiveReload. Best $12 Bucks Spent Ever.

nuff said

nuff said

I’ve been working on angular.js using yeoman for the last couple of weeks. One thing I really liked about it is the live reload functionality. Essentially, any changes made in the DOM or CSS gets updated immediately. It’s a really really powerful feature for web developers. It’s also one of those features you never knew you needed until you’ve used it.

I got so addicted to it, I went hunting around for an app that is able to do the same for my django projects. Enter LiveReload which is quite possibly the best invention since sliced bread. The setup is super trivial. Add my django project to the folder list, install a chrome extension and I’m done. Now, all changes in the DOM or CSS gets updated immediately when I’m working on Django. Best SGD $12 bucks spent on an app ever.