Make paging output from your python script

pydoc is a python library to make your console output paginates for easier reading. to use it follow the simple example below:

import pydoc

text = "...paginate me..."
pager = pydoc.ttypager(text)
print pager

you will see the output is gonna be truncated at the beginning when you start to see like when you open up a man page, then you have to scroll down with keyboard to read the rest of the text line by line. pretty cool!


Django table is marked as crashed and should be repaired

My django app is showing 500 because one of the tables in database marked crashed and should be repaired.

InternalError at /

(145, "Table './dbname/django_session' is marked as crashed and should be repaired")

I thought it was just an usual error from mysql and needed to restart but it wasn’t.

Solutions:

 Login to mysql, select your dbname, and use this command:

USE user_base;
REPAIR TABLE TABLE;

Get json results from mysql with python

This is just simple trick to retrieve data from mysql rows and turn into json.

#!/usr/bin/env python
import MySQLdb as mdb
import sys

con = mdb.connect('localhost', 'root', 'pass123', 'mydb1', charset='utf8')
cur = con.cursor(mdb.cursors.DictCursor)

def get_post():
    list = []
    gw = {}
    cur.execute("select * from tbl1_core")
    rows = cur.fetchall()
    for i in rows:
        tes = []
        # we want to fetch row to store in results
        gw["content"] = i["content"]
        tes.append(gw)
        list.append(tes)
        tes = []
        gw = {}
    return list
tes = get_post()
print tes

Install and Create Virtualenv Python in ubuntu machine

Let’s say I want to have a project called webproject, and this project has own user with the same name to access the project directory. And this is supposed to be a python project and it needs virtual environment. also I want the project has specific python version 2.7 with virtualenv. So let’s get started.

First you may have to check your python version if it is already installed python 2.7 otherwise you’re gonna have to download and install it, but if you prefer to not use python 2.7, let’s say version 2.6, or whatever. That’s fine as well.

Something related:

Install different version of Python in linux

Let’s start with install the virtualenv via apt in ubuntu

$ apt-get install python-virtualenv virtualenvwrapper

Create user and setup home directory

$ useradd -m webproject -d /srv/webproject

Change user to webproject, until you’re inside the home directory project

$ sudo su - webproject

Create virtual environment with virtualenv with python version 2.7

$ virtualenv ~/venv --python=2.7

Or, if you prefer not to use spesific python version, just run the command like this:

$ virtualenv ~/venv

Now, inside your home directory, there is a new directory named venv that contained your python binary.

You might want to put this line of code inside your .bash_history or .profile inside your home directory, so whenever you try to change user, it is automatically activate your python virtual environment.

$ vim ~/.profile
....
....
source ~/venv/bin/activate

and save the file.

Now, try to change user to webproject:

$ sudo su - webproject
(venv)webproject@localhost:~$

Create simple arg parse in python

argparse is a module to make user-friendly command-line interfaces. It’s probably the one of the most frequently used module when I create a script in python that needs to parse some arguments. Check this out.

$ vim test.py
#!/usr/bin/python
import argparse

def init_args():
  parser = argparse. ArgumentParser(description="This is the description")
  parser.add_argument("--arg1", required=True, type=str, help="This is arg1")
  parser.add_argument("--allow", required=False, action="store_true", help="Allow mode")
  return parser.parse_args()

def main(arg1):
  return "arg1: %s"%(arg1)

if __name__ == "__main__":
  args = init_args()
  main(args.arg1)

See what happens when we run it

$ ./test.py --arg1 "showme"
arg1: showme

Sometimes we want to create an argumen but only store it as a True variable. we can just create simple test like this.

$ vim test.py
#!/usr/bin/python
import argparse

def init_args():
  parser = argparse. ArgumentParser(description="This is the description")
  parser.add_argument("--arg1", required=False, action="store_true", help="Enable arg1")
  return parser.parse_args()

def main():
  return "arg1 is enabled"

if __name__ == "__main__":
  args = init_args()
  if args.arg1:
    main()
  else:
    print "arg1 parse is not enabled"

Try to run it

$ ./test.py --arg1
arg1 is enabled

Dump entire photos in Facebook album with Python

Simple dumping photos from albums in facebook by python. in this case, we must got an authorization before dump the albums. first of all, we have to login to the facebook required user and password. afterwards we have to fetching a couples of line called token key which that it’s an authorization key for grab the data.

#!/usr/bin/python
# -*- coding: utf-8 -*-
#   This library is free software; you can redistribute it and/or
#   modify it under the terms of the GNU Lesser General Public
#   License as published by the Free Software Foundation; either
#   version 2.1 of the License, or (at your option) any later version.
#
#   This library is distributed in the hope that it will be useful,
#   but WITHOUT ANY WARRANTY; without even the implied warranty of
#   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
#   Lesser General Public License for more details.
#
#   You should have received a copy of the GNU Lesser General Public
#   License along with this library; if not, write to the 
#      Free Software Foundation, Inc., 
#      59 Temple Place, Suite 330, 
#      Boston, MA  02111-1307  USA
#
#   by pulpn.com

import json, random, re
import urllib, urllib2
import sys, os
import cookielib

# Random user-agents
user_agents = [
    'Mozilla/5.0 (Windows; U; Windows NT 5.1; it; rv:1.8.1.11) Gecko/20071127 Firefox/2.0.0.11',
    'Opera/9.25 (Windows NT 5.1; U; en)',
    'Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50727)',
    'Mozilla/5.0 (compatible; Konqueror/3.5; Linux) KHTML/3.5.5 (like Gecko) (Kubuntu)',
    'Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.0.12) Gecko/20070731 Ubuntu/dapper-security Firefox/1.5.0.12',
    'Lynx/2.8.5rel.1 libwww-FM/2.14 SSL-MM/1.4.1 GNUTLS/1.2.9'
]

# Facebook photo dumper
class _fbphotosdumper_(object):
	def __init__(self, user, pswd, album_name):
    		self.dictid 	= {}
    		self.count 	= {}
    		self.user 	= user
    		self.pswd 	= pswd
    		self.album_name = album_name
    		self.value 	= {'post_form_id' : '42d7f2c5ff3976d42ead29cbda1561ec',
                  	      	   'email':user,
                  	           'pass' :pswd}
    		self.home 	= 'http://m.facebook.com/home.php'
    		self.url 	= 'https://www.facebook.com/login.php?m=m&refsrc=http%3A%2F%2Fm.facebook.com%2F&refid=8'
    		self.cj 	= cookielib.CookieJar()
    		self.opener 	= urllib2.build_opener(urllib2.HTTPCookieProcessor(self.cj))
  	def login(self):
    		login_data 		= urllib.urlencode(self.value)
    		self.opener.addheaders 	= [('User-agent', random.choice(user_agents))]
    		self.opener.open(self.url,login_data)

	# Doesn't need profile id, this is token rules. Get token key from developers.facebook.com
  	def get_token(self):
    		url_api = self.opener.open('https://developers.facebook.com/docs/reference/api/')
    		rurl_api = url_api.read()
    		get = re.search(r'access_token=(.*)">',rurl_api)
    		if get:
      			return get.group(1)
  	def get_albums(self):
    		url_album 	= self.opener.open('https://graph.facebook.com/me/albums?access_token=%s&limit=99'%(self.get_token()))
    		rurl_album 	= url_album.read()
    		_albumfilter_ 	= json.loads(rurl_album)
    		for i in _albumfilter_['data']:
      			try:
        			name 	= i['name']
        			albumid = i['id']
        			psum 	= i['count']
        			self.dictid[name] = albumid
        			self.count[name] = psum
        			#return name,psum
      			except:
       	 			pass
  	def get_photos(self):
    		self.get_albums()
    		sys.stdout.write('Downloading %s photos from "%s" album\n'%(self.count[self.album_name],self.album_name))
 	   	url_photos = self.opener.open('https://graph.facebook.com/%s/photos?access_token=%s&limit=99'%(self.dictid[self.album_name],
												   	       self.get_token()))
    		rurl_photos = url_photos.read()
    		_photosfilter_ = json.loads(rurl_photos)
    		image = urllib.URLopener()
    		for x in _photosfilter_['data']:
			try:
	      			link = x['source']
      				os.chdir(current)
      				image.retrieve(link,'%s'%link.split('/')[-1])
      				print link.split('/')[-1],'Saved in "%s"'%(current)
			except KeyboardInterrupt:
				sys.stdout.write('Download interrupted\n')
def main():
	global current
	if len(sys.argv) == 4:
		# Create a directory for fetched images from facebook album
		folder= 'Facebook Album - %s'%(sys.argv[3])
		current = os.getcwd()+'/%s'%folder
		os.system('mkdir "%s"'%folder)#"%s"'%folder
		facebook = _fbphotosdumper_(sys.argv[1],sys.argv[2],sys.argv[3])
		try:
			facebook.login()
		except:
			print 'Login Error'
		try:
			facebook.get_photos()
		except:
			print 'No album named "%s"'%(sys.argv[3])
	else:
		print 'Facebook Photos dumper'
		print '----------------------------------------'
		print 'Usage\t: %s user pswd "Album name"'%sys.argv[0]
		print 'Example\t: %s foo@bar.com fubar "Foo Bar"'%sys.argv[0]
		sys.exit(1)
if __name__ == '__main__':
	main()