Showing posts with label django. Show all posts
Showing posts with label django. Show all posts

Saturday, March 9, 2013

Problems with Django Testing and Selenium

As mentioned in previous posts, I am thrilled with the Django testing infrastructure. Although TestCases are a great start, I have a situation where I really would like to automatically push some buttons on my page. This seemed like a job for Selenium. The Django docs give a nice example, so I thought I would give it a try. Things did not go as expected.

This was related to the way Django (version 1.4) runs tests and the way the staticfiles app works. When tests are run, DEBUG is set by the test runner to False. No problem when running TestCases because the pages are not actually loaded. No static files are needed.

Setup a test using LiveServerTestCase, and now static files are needed. But when DEBUG=False, the staticfiles app leaves serving static files to you. If you have never run DEBUG=False in your development environment, you probably are not setup to serve the static files. Hence you get a flood of HTTP 500 errors.

The solution is on stackoverflow. Here is my code (IS_DEV is a setting indicating the code is running on the development server):

if settings.IS_DEV and ('test' in sys.argv):
    urlpatterns += patterns('',
                            url(r'^%s/(?P.*)$'%settings.STATIC_URL[1:], 'django.views.static.serve', {
                                'document_root': settings.STATIC_ROOT,
                                }),
                            )

urlpatterns += staticfiles_urlpatterns()

When I first implemented this solution, it appeared to not be working because I still got one HTTP 500 error. This was a little difficult to debug because instead of displaying a debug error page, it just generated a server error (because DEBUG=False). I was able to track down the source of that error by turning on the logger. This showed the error was due to a missing favicon.ico file.

Monday, March 4, 2013

Django Testing Examples

Django's testing functionality is awesome. But the docs are somewhat lacking. Here's a great post on stackoverflow that will get you started. Here are a few more examples:

Fixtures

These are databases that are used only for testing. They have known, fixed content that the tests can test against. The easiest way to make a fixture is to run:

python manage.py dumpdata

The problem with that approach is it will include stuff that will break the fixture. What you need are some excludes. And to make things more readable, you might want to add some indents. Here is the command I used:

python manage.py dumpdata --indent 4 -e contenttypes -e auth.Permission -e sessions -e admin.logentry > simple_auth_fixture.json

Note, this will not remove those tables from your testing database. It will just not provide initial data for those tables.

Also, I use south for database migrations. South makes database entries and .py files to track migrations. By not excluding south there was the potential for a conflict with my migration .py files. This did not happen in the tests I did.


Testing an Object Edit View

The usual flow is for the user to load the form using initial data, then edit the data and post it. Here is one way to do that:

response = self.client.get('/app/edit_obj/3/', follow=True)
# Make sure the form loads
self.assertTemplateUsed(response, 'app/edit_obj.html')
# Get the initial form data 
form_data = response.context['form'].initial 

# Now you can change some data and post it
form_data['afield']='xyz'
response = self.client.post('/app/edit_obj/3/', form_data, follow=True)
# Use asserts to handle whatever responses you expect.

Figuring Out Unexpected Errors

Lets say you are testing a view that includes a form. You post some data and get an unexpected error. How do you figure out what the error is? Here's one way:

response = self.client.post('/app/edit_obj/3/', form_data, follow=True)

# These print statements should help you figure out what went wrong 
print 'Errors:', response.context['form'].errors
print 'Non Field Errors:',response.context['form'].non_field_errors
print 'Is Valid',response.context['form'].is_valid()

A Confusing Error from assertFormError

Here is a somewhat confusing error I got when using assertFormError. The error was:

Failure
Traceback (most recent call last):
  File "/home/junk/.virtualenvs/qdb_django14/lib/python2.7/site-packages/my_proj/an_app/tests.py", line 52, in test_call_view_fails_dup
    self.assertFormError(response, 'form', 'email', u"Email address already exists.")
  File "/home/junk/.virtualenvs/qdb_django14/lib/python2.7/site-packages/django/test/testcases.py", line 718, in assertFormError
    " response" % form)
AssertionError: The form 'form' was not used to render the response

The problem was the data I was sending to the form was supposed to be invalid. But there was an error in my form validation code. The method form.is_valid() returned true which lead to an unexpected redirect. The new page did not use the context variable 'form'. It would have been really confusing if the new page also had a context variable named 'form'.

One way to prevent this is to use assertTemplateUsed to make sure you have the correct template before you look for errors.

Testing by User Type

Often it is important to restrict certain activities to certain types of users. Since I made my fixture from a working website, it already contains the various types of users I might want to run a test on. The problem with this is I did not want to put passwords in my tests.

One way to handle that is just to reset each user's password before login. I made a method for that:

class TestEditClient(TestCase):
    def login(self,username):
        # Keep real password out of tests. Sets password to test.
        self.password='test'
        self.user = User.objects.get(username__exact=username)
        self.user.set_password(self.password)
        self.user.save()        
        response = self.client.login(username=username, password=self.password)
        if not response:
            raise RuntimeError('Could not login') 

An alternative would have been to create each user type using the standard commands for creating a user. For my project, users had a bunch of other records associated with them. I did not feel like creating those, so I just used the users we already had in the db.

Monday, February 18, 2013

Django Collectstatic Gotchas

Overall the Django staticfiles app is great. But I ran into one behavior I did not expect that caused some problems.

As noted in the documentation:
 The default is to look in all locations defined in STATICFILES_DIRS and in the 'static' directory of apps specified by the INSTALLED_APPS setting
and:
Duplicate file names are by default resolved in a similar way to how template resolution works: the file that is first found in one of the specified locations will be used.
To make my apps more modular, I put a static directory in each. Each containing among other things a css folder. Unfortunately for me, I used the same name for some css files in different apps. Thus the first one found made it into collectedstatic and the other one did not. This was not what I was expecting.

To solve the problem, I set

STATICFILES_FINDERS = ( "django.contrib.staticfiles.finders.FileSystemFinder", )

And I added my apps to STATICFILES_DIRS like this:

STATICFILES_DIRS=[('my_django',os.path.join(MY_DJANGO_PATH,'static')),
('app1',os.path.join(MY_DJANGO_PATH,'app1','static')),
('app2',os.path.join(MY_DJANGO_PATH,'app2','static')))

Now collected static made a folder for each app.


Sunday, January 23, 2011

Django Social-registration and the Facebook API

I am creating a Pinax based website that will require users to have an account and to login. I wanted users with Facebook accounts to be able to use OAuth and Facebook to create their account on my site. I also wanted to use data from Facebook to fill in as much of the user profile as possible. To accomplish this, I installed the app social-registration.

Facebook provides basic info such as first name and last name without extended permissions. Email address requires extended access. Extended access can be requested using the scope parameter during facebook login.

Social-registration has some javascript (templates/socialregistration/facebook_js.html ) that you can put in your templates to allow users to login with facebook. In that js file, the code that adds the scope parameter is commented out:

FB.login(handleResponse/*,{perms:'publish_stream,sms,offline_access,email,read_stream,status_update,etc'}*/);

Uncommenting that code solves part of the problem. However, the perms list is really just a placeholder. status_update and etc are not valid perms, but the rest are. Remove those and it should work.

If you put the social-registration middleware in your settings, then you can access all the users facebook info in your views using:

data_dict=request.facebook.graph.get_object('me')

Tuesday, March 23, 2010

More setting up Eclipse for Django debugging

I have been using eclipse with break points to debug a django app for a few days now. For a while the process was pretty flaky.

At first, I thought it was because I installed Aptana Web Development Tools after I had things working. The reason I installed that plugin was to get decent editors for HTML, CSS and javascript. After that install, I get the message: "Aptana JavaScript Scripting Console Started" whenever I start eclipse. It seems that message is benign.

I was able to get the debugger to work reliably with django by:

1) starting the remote debugger (as before)
2) using the Run icon to start manage.py instead of the Debug icon.

Tuesday, March 16, 2010

Setting up Eclipse for Python and Django

After spending more hours than I would like to admit, I finally have Eclipse installed, with the Pydev plugin and I have debugging working for django. It was that last part that caused all the misery.

These notes are small additions to the excellent online tutorial by Nick Vlku. You will want to watch that first. I strongly recommend watching the whole thing once without pause. If you try to do things as he does them you will encounter errors - ones that he will give the solution to a little further into the screencast.

Here is my setup:
Windows XP
Django 1.1.1
Python 2.5
Eclipse 3.5.2
PyDev 1.5.5

This version of Pydev includes the extensions. Although the menus might be in different locations, it's not hard to guess what the new locations are. Enter the same values, etc... as Nick does.

After I got eclipse and pydev installed, I played around with it for a while with an ordinary python app (e.g non-django). I got used to the auto complete and debugging. Being familiar with these things definitely made it easier to setup debugging for django.

Here are some tips for setting up eclipse to work with the django development server. The goal is to have the development server work the same way it does outside of eclipse, but with breakpoints.

The first tip is to watch the windows process monitor for python.exe processes. It is very easy to end up with some orphaned processes which can cause things to stop working right. When in doubt, restart eclipse. After you start eclipse, you should see one python.exe process. Starting the remote debug server does not instantly start another python process. Running your code in debug mode adds two processes.

Next, Nick gives some code for your manage.py module. I could not get things to work using that code. I do not know if the problem was the code or something else. I eventually got things working using this code.

Next, make sure to start the Debug Server before you run your django code. When you run your django code in debug mode, you will see a message "pydev debugger: starting" - this is not the Debug Server. If you do everything right, you see three python.exe processes.

Here is what my debug window looks like when everything is working and the code is stopped at a breakpoint at the function homepage in the views.py module:


I have not figured out how to stop the debug server - outside of killing it from the process monitor. If you read the comments on Nick's post, you will see that others are wondering the same thing.

As for other setup tips, an other blogger recommended setting the environment variables DJANGO_SETTINGS_MODULE and PYTHONPATH in the Python Run configuration. I found that made things worse. If you do not know what I am talking about, then don't worry about it.

Others have suggested adding the --noreload flag after the "runserver" command in the Run Python configuration. Evidently setting that flag causes eclipse to print more informative messages in the console (like the django development server run outside of eclipse) with the downside being that you must stop and start the server if you change any of your code. For me the most important thing to know was that django debugging can work with or without that flag.