Problems Stopping Matplotlib FuncAnimation

I am writing a variant of the matplotlib strip chart example. I am running matplotlib version 1.1 on Ubuntu 12.04 from Aptana Studio. I stop the animation using the close button on the plot window.

When I do that, I get the following error message:

invalid command name "182058716callit"
    while executing
"182058716callit"
    ("after" script)
Traceback (most recent call last):
  File "/home/xxxxx/Documents/Aptana Studio 3 Workspace/foot_force/plot_boxing.py", line 179, in 
    plt.show()
  File "/usr/lib/pymodules/python2.7/matplotlib/pyplot.py", line 139, in show
    _show(*args, **kw)
  File "/usr/lib/pymodules/python2.7/matplotlib/backend_bases.py", line 109, in __call__
    self.mainloop()
  File "/usr/lib/pymodules/python2.7/matplotlib/backends/backend_tkagg.py", line 69, in mainloop
    Tk.mainloop()
  File "/usr/lib/python2.7/lib-tk/Tkinter.py", line 328, in mainloop
    _default_root.tk.mainloop(n)
AttributeError: 'NoneType' object has no attribute 'tk'

Its not the end of the world. The code still runs as needed. But the error message is ugly, especially if the code is being used to demo something to a customer.

Turns out the solution is simple. FuncAnimation has a keyword call interval, which is the time between calls to your update function in milliseconds. Increase the interval and the problem goes away.

Using Python Logging Across Multiple Modules

For some reason, I do not use the builtin logging module in Python as often as I think I should. I am not sure why. I have used logging several times before, yet it has not become part of my standard tool kit. My first suspicion is that I am just being lazy and print statements are so easy. But now as I wade back into the waters, I am starting to remember that logging uses some form of magic that I never took the time to get comfortable with.

The magic appears when you use logging across multiple modules. Suppose you have an app called app.py that loads a function from a module as in the following code:

# app.py
from my_module import my_func
my_func()

# my_module.py
import logging
logger = logging.getLogger(__name__)

def my_func():
    logger.error('message from mod 1')
    logger.debug('a debugging message')

If you run app.py you will get the error:

No handlers could be found for logger "my_module"

But add an import in app.py and suddenly the error in my_module.py goes away:

# app.py
from my_module import my_func
import logging

logging.error('A message from app.py')
my_func()

I lied, you also have to have line 5.

I cannot think of another case where an import in one module effects another module. That's why I did not like using logger. I never got comfortable with how it worked across modules. But magic is also powerful. Time to learn it. Before we go any further, you should read the basic tutorial in the Python docs.

The questions we are going to address are:

• how does logging come to life?
• how does it stay alive when execution passes to a new module?

The answer is that once the logger is configured, it becomes part of the Python interpreter process that is executing your code. It effectively becomes global.

Part of the confusion also comes from how easy it is to configure logging. As shown in the snippet above, simply importing logging and logging a message auto-magically configures a logger for your entire project. Convenient, but un-nerving if you do not know its happening.

Here is a simple example of a more explicitly configured logger:

# app.py
from my_module import my_func
import logging
logging.basicConfig(filename='example.log',level=logging.DEBUG)

logging.error('A message from app.py')
my_func()

The logger in the previous example wrote to the console. This logger writes to a file and changes the default level. The amazing thing here is that the log messages in mod1.py also show up in this file.

This also explains why the line:

logger = logging.getLogger(__name__)

shows up in so many modules. This line causes the logging messages to be prepended with the module name. Pretty helpful when you have 47 different modules writing to the same log file.

I think that does it for me. With these mysteries cleared up, the rest of the Python docs are pretty straight forward.

But since we have gotten this far, I want to share a debugging trick. Often when I am debugging a module, I run it as __main__ along with some setup code. Here is how to make the logger work with __main__ and when it is called as part of a larger app:

# mod1.py
import logging
logger = logging.getLogger(__name__)

def mod1():
    logger.error('message from mod 1')
    logger.debug('a debugging message')
    
if __name__=='__main__':
    logging.basicConfig(level=logging.DEBUG)
    mod1() 

This sets up a different logger than the one when the module is not __main__. In the app, you may have the level set at ERROR. But when running the module as main, you can have the level be DEBUG. Also, this debugger writes to the console, which is much more convenient than writing to a file. It essentially creates print statements that can be turned on and off.

Python Video Player Part Duex

After too many hours I gave up on MLT. Their documentation is pretty bad. I could not get "multi" working. Back to gstreamer.

Initially I was reluctant to use the gstreamer command line; get-launch. But I was not making headway in gstreamer in python, so I thought it would be good to get python out of the way for now.

My ultimate goal is to figure out how to use "tee". But I could not find good documentation on the, so switched to getting mux to work, since it has one input and 2 outputs, like tee.

I still could not get things to work. I could run the gst-launcher but it would PAUSE and I could not figure out how to get it to START. It seemed like it had something to do with putting queues on the mux output. Then all was revealed in this post. For the mpegdemux, the queues need to go after the decoders... frustrating. Here is code that works (just put your path to your mpg file):

#!/bin/bash
gst-launch-0.10 -v --gst-debug-level=1 \
mpegdemux name=demuxer \
demuxer. ! mpeg2dec ! queue ! ffmpegcolorspace ! xvimagesink \
demuxer. ! mad ! queue ! audioconvert ! pulsesink \
filesrc location="PATH TO .mpg FILE" ! demuxer.

Now to tee the video. OMG - this worked! Two identical video windows (note: they are positioned on top of each other).

#!/bin/bash
gst-launch-0.10 -v --gst-debug-level=1 \
mpegdemux name=demuxer \
demuxer. ! mpeg2dec ! queue ! ffmpegcolorspace ! tee name=my_tee \
my_tee. ! queue ! xvimagesink \
my_tee. ! queue ! xvimagesink \
demuxer. ! mad ! queue ! audioconvert ! pulsesink \
filesrc location="/home/chuck/Desktop/test.mpg" ! demuxer.

Life is good. Here it's working in python:

import sys, os
import pygtk, gtk, gobject
import pygst
pygst.require("0.10")
import gst

FILEPATH='MY FILE PATH.mpg'

class GTK_Main:
   def __init__(self):
       window = gtk.Window(gtk.WINDOW_TOPLEVEL)
       window.set_title("Mpeg2-Player")
       #window.set_default_size(200, 100)
       window.connect("destroy", gtk.main_quit, "WM destroy")
       vbox = gtk.VBox()
       window.add(vbox)
       hbox = gtk.HBox()
       vbox.pack_start(hbox, False)
       self.button = gtk.Button("Start")
       hbox.pack_start(self.button, False)
       self.button.connect("clicked", self.start_stop)
       window.show_all()
      
       self.player = gst.Pipeline("player")
       source = gst.element_factory_make("filesrc", "file-source")
       source.set_property("location", FILEPATH)
      
       demuxer = gst.element_factory_make("mpegdemux", "demuxer")
       demuxer.connect("pad-added", self.demuxer_callback) # evidently demux needs to be setup dynamically
      
       self.video_decoder = gst.element_factory_make("mpeg2dec", "video-decoder")
       self.colorspace = gst.element_factory_make("ffmpegcolorspace", "colorspace")
       self.audio_decoder = gst.element_factory_make("mad", "audio-decoder")
       audioconv = gst.element_factory_make("audioconvert", "converter")
       audiosink = gst.element_factory_make("pulsesink", "audio-output")
       videosink1 = gst.element_factory_make("xvimagesink", "video-output1")
       videosink1.set_property("name", "vid1")
       videosink2 = gst.element_factory_make("xvimagesink", "video-output2")
       videosink2.set_property("name", "vid2")
       self.tee=gst.element_factory_make("tee","my_tee")
      
       self.queuea = gst.element_factory_make("queue", "queuea")
       self.queuev = gst.element_factory_make("queue", "queuev")
       self.queuet1 = gst.element_factory_make("queue", "queuet")
       self.queuet2 = gst.element_factory_make("queue", "queuet2")
      
       self.player.add(source, demuxer, self.video_decoder, self.audio_decoder, audioconv,
           audiosink, videosink1, videosink2, self.queuea, self.queuev,self.queuet1,
           self.queuet2, self.colorspace,self.tee)
       gst.element_link_many(source, demuxer)
       gst.element_link_many(self.video_decoder,self.queuev, self.colorspace, self.tee )
       gst.element_link_many(self.audio_decoder,self.queuea, audioconv, audiosink)
       gst.element_link_many(self.tee, self.queuet1, videosink1)
       gst.element_link_many(self.tee, self.queuet2, videosink2)
      
       bus = self.player.get_bus()
       bus.add_signal_watch()
       bus.enable_sync_message_emission()
       bus.connect("message", self.on_message)
      
   def start_stop(self, w):
       if self.button.get_label() == "Start":
           self.button.set_label("Stop")
           self.player.set_state(gst.STATE_PLAYING) # tells mux it needs to connect to something, then starts
       else:
           self.player.set_state(gst.STATE_NULL)
           self.button.set_label("Start")
                      
   def on_message(self, bus, message):
       t = message.type
       if t == gst.MESSAGE_EOS:
           self.player.set_state(gst.STATE_NULL)
           self.button.set_label("Start")
       elif t == gst.MESSAGE_ERROR:
           err, debug = message.parse_error()
           print "Error: %s" % err, debug
           self.player.set_state(gst.STATE_NULL)
           self.button.set_label("Start")
  
   def demuxer_callback(self, demuxer, pad):
       if pad.get_property("template").name_template == "video_%02d":
           qv_pad = self.video_decoder.get_pad("sink")
           pad.link(qv_pad)
       elif pad.get_property("template").name_template == "audio_%02d":
           qa_pad = self.audio_decoder.get_pad("sink")
           pad.link(qa_pad)
      
GTK_Main()
gtk.gdk.threads_init()
gtk.main()

Python Video Player

For a research project, I am trying to create a python based video player that will allow me to create multiple windows that play a single video, all synchronized.

I started out with gstreamer. It looked promising. It even had a element called "Tee" which seemed like a good thing for splitting the source video. Alas, the documentation was lacking. Especially regarding "Tee".

It seems I am not the first person to run into this problem. This article made me decide to switch to MLT. Openshot seems like a pretty developed, live project. If this guy was willing to put this much work into an MLT based project, then I thought I would give it a try.

Unfortunately things did not go much better in MLT. First, I had trouble installing it (Ubuntu 11.10). I solved that problem by installing Openshot. I installed the python bindings using the Ubuntu Software Center.

Next I found the docs to be pretty cryptic - given that I wanted to work in python. Eventually I found the mlt-0.7.8.tar.gz file. In ./src/swig/python was some demo code that was enough to get me started.

Below is some code that will open a media file and play it. Just assign the full path FILE and run.

import mlt
import os.path
import time
import sys

FILE='/home/chuck/Desktop/test-mpeg.mpg'
if os.path.exists(FILE):
# Start the mlt system
mlt.Factory().init( )

# Establish a profile
profile = mlt.Profile( )

# Create the producer
p = mlt.Producer( profile, FILE  )

# Create the consumer
c = mlt.Consumer( profile, "sdl" )

# Turn off the default rescaling
c.set( "rescale", "none" )

# Connect the producer to the consumer
c.connect( p )

# Start the consumer
c.start( )

# Wait until the user stops the consumer - ie closes the window
while c.is_stopped( ) == 0:
    print 'look, we can do other stuff while the video runs'
    time.sleep( 1 )
print 'The window was closed.'
else:
# Diagnostics
print "Unable to open ", FILE

Not sure how I am going to solve my original problem. But this is a start.

Webfaction, virtualenv and --no-site-packages

So I thought I had finally figured out how I wanted to configure my code on webfaction using env. My plan involved using the flag --no-site-packages.

Once I created the virtualenv, I fired up python and ran the code:

import sys

for x in sys.path:

print x

Much to my surprise, all "my site packages" (/username/lib/python2.5 ) paths were in the system path. I was hoping these packages would not be included.

This post on the webfaction forum explained what was happening. Basically the --no-site-packages flag removes the ones installed by webfaction for all users.

Next, I tried to setup the paths using a sitecustomize.py file. When python starts, it looks for that file. If it finds it, it imports it. Unfortunately that did not work either because webfaction already has one in /usr/local/lib/python2.5.

I tried adding an import statement to a .pth file. That solution did not work because there is no guarantee of when that file will be executed relative to the other .pth files. In my case, it ran before the paths I wanted to remove from sys.path.

For now, here's my soln. The only project I have running in virtualenv is django. I am just going to add an import statement in manage.py. It will configure the site path. On my development computer, I am not having all these problems. So I will put the import statement in a try block. If the config program is not found, it will just move on.

virtualenv and --no-site-packages

I originally created my virtualenv without the flag --no-site-packages. I figured that the virtualenv paths would over-ride (come before) the global paths. That turned out to be wrong. The global paths come first.

One way to get around this is the create the env with the --no-site-packages flag and then manually add the global packages that I want using the virtualenvwrapper command add2virtualenv.

Since the env was already setup, I looked for ways to add that flag after the repo was created. Here is the answer. To have virtualenv not use global libs, just add the file "no-global-site-packages.txt" to the path: virtualenv_name/lib/python2.5 - Note this path is different from the one given on stackoverflow. I figured out the path by creating a test virtualenv and seeing where that file turned up.

But that's not really what I want to do. I want to have my env access the global paths because there are a lot of useful libs in there. It would be a lot of work manually adding each to the virtualenv.

Part of the answer seems to be in the pth files. It turns out that when you install a package in the virtual env using easy_install, easy_install creates a .pth file that removes versions of that package that are already on the path.

A quick peak in easy-install.pth reveals lines of code in addition to the paths! Here it is:

import sys; sys.__plen = len(sys.path)

./setuptools-0.6c11-py2.5.egg

./pip-0.7.2-py2.5.egg

import sys; new=sys.path[sys.__plen:]; del sys.path[sys.__plen:]; p=getattr(sys,'__egginsert',0); sys.path[p:p]=new; sys.__egginsert = p+len(new)

I found more info on this magic here. Apparently if you start a line with import and put all the code after it on the same line using ";" as a delimiter, then you can hack the path at startup.

Setting up Eclipse for Python and Django

After spending more hours than I would like to admit, I finally have Eclipse installed, with the Pydev plugin and I have debugging working for django. It was that last part that caused all the misery.

These notes are small additions to the excellent online tutorial by Nick Vlku. You will want to watch that first. I strongly recommend watching the whole thing once without pause. If you try to do things as he does them you will encounter errors - ones that he will give the solution to a little further into the screencast.

Here is my setup:

Windows XP

Django 1.1.1

Python 2.5

Eclipse 3.5.2

PyDev 1.5.5

This version of Pydev includes the extensions. Although the menus might be in different locations, it's not hard to guess what the new locations are. Enter the same values, etc... as Nick does.

After I got eclipse and pydev installed, I played around with it for a while with an ordinary python app (e.g non-django). I got used to the auto complete and debugging. Being familiar with these things definitely made it easier to setup debugging for django.

Here are some tips for setting up eclipse to work with the django development server. The goal is to have the development server work the same way it does outside of eclipse, but with breakpoints.

The first tip is to watch the windows process monitor for python.exe processes. It is very easy to end up with some orphaned processes which can cause things to stop working right. When in doubt, restart eclipse. After you start eclipse, you should see one python.exe process. Starting the remote debug server does not instantly start another python process. Running your code in debug mode adds two processes.

Next, Nick gives some code for your manage.py module. I could not get things to work using that code. I do not know if the problem was the code or something else. I eventually got things working using this code.

Next, make sure to start the Debug Server before you run your django code. When you run your django code in debug mode, you will see a message "pydev debugger: starting" - this is not the Debug Server. If you do everything right, you see three python.exe processes.

Here is what my debug window looks like when everything is working and the code is stopped at a breakpoint at the function homepage in the views.py module:

I have not figured out how to stop the debug server - outside of killing it from the process monitor. If you read the comments on Nick's post, you will see that others are wondering the same thing.

As for other setup tips, an other blogger recommended setting the environment variables DJANGO_SETTINGS_MODULE and PYTHONPATH in the Python Run configuration. I found that made things worse. If you do not know what I am talking about, then don't worry about it.

Others have suggested adding the --noreload flag after the "runserver" command in the Run Python configuration. Evidently setting that flag causes eclipse to print more informative messages in the console (like the django development server run outside of eclipse) with the downside being that you must stop and start the server if you change any of your code. For me the most important thing to know was that django debugging can work with or without that flag.