PHP application with Mysql database to a Django app with Postgres Database

Recently I had a chance to work on a project, where a PHP app had to be converted to a Django one.

The work seemed trivial, but it brought a few difficult technicalities. It had a Mysql DB, with few tables. Overall, I had to  -

• Convert them into Django models
• Convert the app into Django
• Setup the app on Heroku using Postgres

1. Converting the models to Django is easy. Django provides an inspectdb command. It looks at the current model structures, inspects it and provides django model accordingly.
2. This is a nice feature. It gave me a push forward to begin the app development. However, this feature lacked some points that I would like to point out - 
1. In Mysql, we have a set type . This is basically similar to choices in Django models. I had to manually do that. 
2. The foreign key constraint is set to id by default. The command could check this and set the field in model declaration using to_field parameter.
3. The DB views had to created manually. 
3. For changing Mysql Data to Heroku, https://devcenter.heroku.com/articles/heroku-mysql is a good resource.

As always, heroku has to be provided with a S3 access to uploading of static and media files. But thats a different matter.

Python? , But why Python?

Well I am using python since last 3 years . It has been my main development language, other than javascript.

Often, someone comes around and asks , Why Python? I am usually left with my personal programming tasks where python comes in very handy when compared to other languages I have encountered, like  PHP or Java.

Well , here is a post for answering this question exactly. Check it here.

Here are the points -

• Efficient - Has generators 
• Fast
• Broad use
• Not just a language, has a number of implementations
• Easy

Cheers!!

Yet another post on Redis

While working for a project , we used Redis as queue, using python-rq.  Running a redis-cli , I used the following commands -

• keys *
• type <key name>
• and then according to the type , hash,list I would query the data

Some things were quite easy to understand

• rq:workers
• rq:queue:failed
• rq:queue:default
• and a success one as well

But apart from these, there were several entries - with name rq:job:<job_id>. After much reading, I found the internal working at http://python-rq.org/contrib/.

It says whenever a function call gets enqueued - 

• Pushes the job's ids into queue , in my case the default
• adds a hash objects of the job instance

So, when dequeue happens - 

• Pops jobid from queue
• Fetches Job data 
• Executes the function and saves result has a hash key if success
• else saves in failed queue with stack trace

All of this is given on Python-rq site.

There are two kinds of error I saw -

• RuntimeError: maximum recursion depth exceeded while calling a Python object - This happened at queue.py of python-rq module, where I think, it was caused when control crossed max recursive limit, when it didnt find the jobs hashes, as discussed above in dequeue
• Socket closed on remote end - The server closes client connection after 300s, in my case I didnt want to do them, so. let it be on forever by changing in /etc/redis/redis.conf , timeout value to 0

Go Redis!!

Python Decorators - The correct way to do it

Was going through Graham Dumpleton's blog post - how you implemented your python decorator is wrong. Simple points that were discussed were -

• Decorators can be functions as well as Classes.
  

  As a class -  

  
  

  class function_wrapper(object):
      def __init__(self, wrapped):
          self.wrapped = wrapped
      def __call__(self, *args, **kwargs):
          return self.wrapped(*args, **kwargs) 

  @function_wrapper
  def function():
      pass 

  
  

  As a function 

  
  

  def function_wrapper(wrapped):
      def _wrapper(*args, **kwargs):
          return wrapped(*args, **kwargs)
      return _wrapper 

  @function_wrapper
  def function():
      pass 

  
  

• Use the functools.wraps decorator , it only preserves original functions __name__ and __class__
  

  In functions 

  
  

  import functools 

  def function_wrapper(wrapped):
      @functools.wraps(wrapped)
      def _wrapper(*args, **kwargs):
          return wrapped(*args, **kwargs)
      return _wrapper 

  @function_wrapper
  def function():
      pass 

  
  

  In classes, use the update_wrapper method- 

  
  

  import functools 

  class function_wrapper(object):
      def __init__(self, wrapped):
          self.wrapped = wrapped
          functools.update_wrapper(self, wrapped)
      def __call__(self, *args, **kwargs):
          return self.wrapped(*args, **kwargs)

• Python 2.7 preserves Argument specification ( inspect.getargspec ) only in functional decorators, not in class based ones.
• Doesnt preserve function source code for inspection ( inspect.getsource ) 
• Cannot apply decorators on top of other decorators that are implemented as descriptors.

Timezone and Confusion in Django

https://docs.djangoproject.com/en/dev/topics/i18n/timezones/

1) We use USE_TZ = True in our settings. It means, that Django will use Time zone aware datetime objects.
so, instead of doing this,

import datetimecur = datetime.datetime.now() # which doesnt give us tzinfo

we should use this -

import datetimefrom django.utils.timezone import utc

cur = datetime.datetime.now().replace(tzinfo=utc) # Note: This is for timezone aware not for local time

This will remove the warnings we usually see, "Datetimefield received a naive datetime."

2) For converting local time into current time zone,  according to one specified in settings file. We should use this

from django.utils import timezonetimezone.localtime(timezone.now())

How to Test Facebook App using selenium

I had given a lot of thought for building test cases for the FB app .
Major areas for testing were -
a) New user registration
b) Permission acceptance flow
c) Other features of the app. but most of which should be manually tested.

Enter, Selenium . It simple automate browsers.  Its really simple . I did the following

1) Fired firefox. Added Selenium IDE Plugin . It basically records user actions , initially in HTML which later can be exported to python or other supporting laguage.

2) After step 1, half work is already done   :)

3)  Now on to FB part. Here I was trying to automate New User Registration Flow. Here I needed a new test user everytime I run the script.

Enter , Facebook's Test User Api.
a) First get App Access Token
b) Then get a new test user credentials

4) Now include this flow in your test case, Call the script to get test user credentials, and login and go to app, and viola, Test Case Completed.

5) There is one small thing though, :) , the python exported test case gave error on the action when it tries to select frame of the app. Here we can use switch_to_frame method.

Rest all was a piece of Cake.
//Edit - Not exactly piece of cake, cause the test cases are performing a bit weird.  :)

Parsing Excel sheets in python

Recently, I have used the XLRD library , to parse excel data and save it as a list of dictionaries.

The library can be found here.

To learn its use , easiest reference is here .