Andi Albrecht

Running flake8 in Emacs

2011-09-26T00:00:00+02:00

Running flake8 in Emacs

Some time ago I've discovered python-pylint.el, a minor mode for Emacs to run pylint in compilation mode. Running pylint in compilation mode is really nice since you can jump between errors with the usual Emacs keyboard shortcuts M-g n / M-g p.

But when focussing more on style then I want to run flake8 instead. Here's a thin wrapper (gist.github.com) around python-pylint that runs flake8 in compilation mode and parses the output. Nothing special, just the right commands and a regex to parse the output - but helpful :)

For the context switchers - git branch I/O

2011-08-11T00:00:00+02:00

For the context switchers - git branch I/O

At work I'm switching contexts quite often. While working on a long-term feature, I'll have to switch to do a bug fix, then I'll have to add a feature request that's done in a few hours and so on…

Usually I end up with several git branches in my working copy. For example, currently I have three branches:

$ git branch
  issue3521
  master
* work

One goes upstream ("master"), one for instant fixes ("work"), one feature branch ("issue3521").

To get a quick overview of the current state of all branches I've added a very simple git extension to my setup that shows incoming and outgoing commits for each branch. I found the script (besides the issues it has) quite useful. It gives me just as little information as needed to orient myself after a couple of changes here and there. It just visualizes incoming and outgoing commits relative to a branch.

The following screenshot illustrates it's usage. The colors have the following meanings:


green	everything's fine
blue	almost ok
yellow	take care
red	do something

(Note, git bio is just an alias to git branch-io.)

So in that case I should keep an eye on issue3521 to keep it in sync with master, should merge back one commit from work to master and definitely should to something about the branch called outofdate.

If you want to give it a try, here's the extension (gist.github.com).

sqlparse 0.1.3

2011-07-29T00:00:00+02:00

sqlparse 0.1.3

I've just released a new bug fix release for sqlparse.

Changes since 0.1.2

Bug Fixes

Improve parsing of floats (thanks to Kris).
When formatting a statement a space before LIMIT was removed (issue35).
Fix strip_comments flag (issue38, reported by ooberm…@gmail.com).
Avoid parsing names as keywords (issue39, reported by djo…@taket.org).
Make sure identifier lists in subselects are grouped (issue40, reported by djo…@taket.org).
Split statements with IF as functions correctly (issue33 and issue29, reported by charles….@unige.ch).
Relax detection of keywords, esp. when used as function names (issue36, nyuhu…@gmail.com).
Don't treat single characters as keywords (issue32).
Improve parsing of stand-alone comments (issue26).
Detection of placeholders in paramterized queries (issue22, reported by Glyph Lefkowitz).
Add parsing of MS Access column names with braces (issue27, reported by frankz…@gmail.com).

Other

Replace Django by Flask in App Engine frontend (issue11).
Documentation moved to Read the Docs. Thanks for this great service!

Special thanks to Andriy Senkovych for packaging sqlparse for Debian/Ubuntu and giving some insights in all that packaging stuff. Thanks to all other package maintainers too!

Please file bug reports and feature request on the issue tracker.

A Selenium and Python appetizer

2011-07-12T00:00:00+02:00

A Selenium and Python appetizer

Selenium is a browser automation framework and was released in version 2.0.0 just this week. In this post I'll show you how easy it is to get started with using this fine framework in Python.

Peparing the kitchen

Let's create and jump into a virtualenv. You don't really need it, but at least we don't need no sudo for anything :)

$ virtualenv demo
New python executable in demo/bin/python
Installing distribute..................................................done.
$ source demo/bin/activate
(demo) $

Installing the Selenium Python package is as easy as possible. Just use your favorite package manage and install it from PyPI:

(demo) $ pip install selenium

This installs the selenium Python package which includes all necessary drivers to run your browser automation using Firefox, Chrome or IE locally or anything that the Selenium Remote Control provides. We come back to that later.

If you want to automate Google's Chrome on your local machine some little setup is needed. If you want to follow the examples in this post and if you want to use an installed Firefox feel free to skip this section. To automate Chrome make sure Chrome is installed and then download and unzip the chromedriver for your system from the Selenium download page. Place the chromedriver binary somewhere in your $PATH. That's it.

Is it Google?

Now that Selenium is up and running, let's write a simple script that verifies that the title of the Google page actually contains the word "Google". That's simple and obvious, but the script illustrates how to use Selenium in UnitTests.

And you can do all this fancy stuff interactive in the Python shell:

#!/usr/bin/env python

import unittest

from selenium import webdriver


class TestTitle(unittest.TestCase):

    def setUp(self):
        # Toggle comments to test with a different browser
        self.driver = webdriver.Chrome()
        #self.driver = webdriver.Firefox()
        #self.driver = wevdriver.Ie()

    def tearDown(self):
        self.driver.close()

    def test_title_tag(self):
        self.driver.get('http://google.com')
        title_tag = self.driver.find_element_by_tag_name('title')
        self.assertEqual(title_tag.text, 'Google')


if __name__ == '__main__':
    unittest.main()

This script has one little drawback. It initializes a new webdriver (aka browser) for each tests. That's very accurate to start with a fresh browser for each test, but very time consuming too. As an alternative you can modify this script to use a global webdriver instance that is initialized before unittest.main() and closed after that line.

Since webdriver uses a network protocol to communicate with the browser, it's painless to use it in an interactive shell:

(demo) $ python
Python 2.7.1+ (r271:86832, Apr 11 2011, 18:13:53)
>>> from selenium import webdriver
>>> driver = webdriver.Chrome()
>>> driver.get('http://google.com')
>>> driver.find_element_by_tag_name('title')
<selenium.webdriver.remote.webelement.WebElement object at 0x29c2510>

That's a nice and easy way to figure out what webdriver commands are available and how they work.

Going headless

When using Selenium as part of your test suite, you don't have always an X environment around to start Firefox or Chrome. As you can see in the sample script above, it's easy to switch between different browsers without the need to change anything in your actual tests.

As the name suggests the remote webdriver talks to a Remote Control Server that let's you choose a browser. The nice thing is that the standalone server comes with a headless browser that even supports JavaScript execution much like Firefox or Chrome.

Download the standalone server JAR on the Selenium download page and simply run java -jar selenium-server-standalone-2.0.0.jar to start it. When the server is started you can initialize a headless browser in your scripts like this:

from selenium import webdriver
driver = webdriver.Remote(
    command_executor= 'http://127.0.0.1:4444/wd/hub',  # that's the default
    desired_capabilities={'browserName': 'htmlunit',
                          'version': '2',
                          'javascriptEnabled': True})

That's all to run your tests using a headless browser. Ready for your continuous integration system running on a server somewhere.

Testing web pages is fun, isn't it?

Edit 2011-07-12: Corey Goldberg posted a very similar article on his blog a few hours before I published this post.

Back from PythonCamp

2011-04-18T00:00:00+02:00

Back from PythonCamp

I'm back from a phantastic weekend at the Python BarCamp in Cologne. We've had two great days with lots of very interesting sessions (I'm still wondering how it's possible to stuff that many topics with such depth in two days, but it worked!), lightning talks, discussions, pizza, tweets, coffee, and and and … We're still working on the wiki pages to merge all notes taken during the sessions on various Etherpads.

To get an impression search for #pycamp on flickr.

Last but not least, big, big thanks to everyone hanging around at the PythonCamp this year for making it a very interesting and relaxing weekend!

Regex to parse pylint's parseable output

2011-04-13T00:00:00+02:00

Regex to parse pylint's parseable output

Just in case someone (including myself) is looking for that: Here's the regex I'm using to parse the output from Pylint to extract error messages. Pylint needs to be called with the -f parseable flag.

Try to match each line of Pylint's output with this regex. If it matches call groupdict() to access to information bits easily.