Be careful what you import
I learned a valuable lesson recently about being careful what you import into a Python application. The majority of the time I just find the module I need, import, use, and then move on. However, sometimes an unnecessary import can lead to a big waste of memory.
Searching imports dynamically
I learned this lesson while recently writing a tool to search a Python module and/or package for a given object. The script is interesting and worth another post all on it's own. The gist is you provide the script with two arguments, the module/package to search and a term to search for. The script should then give you a listing of all the places it found an object containing your search term.
For example, the documentation for PyQt4 can be pretty tough to search. I tend to use the standard Qt documentation and then translate any examples I find into code for the Python PyQt4 wrapper.
I was testing my script with various terms I tend to use in PyQt4 applications
python import_searcher.py -s MinimumExpanding PyQt4.Qt.QSizePolicy.MinimumExpanding = 3 PyQt4.QtGui.QSizePolicy.MinimumExpanding = 3 PyQt4.Qwt5.qplt.QSizePolicy.MinimumExpanding = 3
Notice anything odd about the above output? Looks like
shows up in two almost identical locations,
Naturally, I thought there was a bug in my script, but after some debugging and
reading I found the following jewel on the
PyQt4 wikipedia page:
The Qt module consolidates the classes contained in all of the modules described above into a single module. This has the advantage that you don't have to worry about which underlying module contains a particular class. It has the disadvantage that it loads the whole of the Qt framework, thereby increasing the memory footprint of an application. Whether you use this consolidated module, or the individual component modules is down to personal taste.
So, using the
PyQt4.Qt module is actually redundant and only for convenience.
The noticeable downside, besides redundancy, is it can actually import a lot of large modules that aren't typically needed such as
For example, assume your application is only using the
PyQt4.QtGui module and
then you mistakenly decide you need something from
PyQt4.Qt. This single import
could essentially double your memory usage :
Line # Mem usage Increment Line Contents ================================================ 1 @profile 2 def import_qt_module_by_module(): 3 #from PyQt4 import QtCore 4 #from PyQt4 import QtDBus 5 6.953 MB 0.000 MB #from PyQt4 import QtDeclarative 6 12.844 MB 5.891 MB from PyQt4 import QtGui 7 #from PyQt4 import QtHelp 8 #from PyQt4 import QtMultimedia 9 #from PyQt4 import QtNetwork 10 #from PyQt4 import QtOpenGL 11 #from PyQt4 import QtScript 12 #from PyQt4 import QtScriptTools 13 #from PyQt4 import QtSql 14 #from PyQt4 import QtSvg 15 #from PyQt4 import QtTest 16 #from PyQt4 import QtWebKit 17 #from PyQt4 import QtXml 18 #from PyQt4 import QtXmlPatterns 19 #from PyQt4 import phonon 20 #from PyQt4 import QtAssitant 21 #from PyQt4 import QtDesigner 22 #from PyQt4 import QtAxContainer 23 19.387 MB 6.543 MB from PyQt4 import Qt
Note that just importing 'PyQt4.Qt' increased the application memory usage by 6.543 MB. This could be costly depending on how much memory your application was already using and how much your system has available.
Be careful of what you import. It could be redundant or costly, possibly both.
Carefully read documentation. The main PyQt4 documentation alludes to this redundant
PyQt4.Qtmodule by describing it with the following, "Consolidates all other modules into a single module for ease of use at the expense of memory."
Don't forget to try stuff, play, and have some fun coding. A one-off script can lead to a nice, useful discovery.
 You can profile this for yourself with this gist.
Published: 02-15-2013 15:21:02