Saturday, November 21, 2015

Quick Langauge Reference: Python

interactive prompt, for quickly testing code

    run (with no args):

        python


    # this starts an interactive shell  
    print "hi"

    use:
        ctrl-c to stop execution
        ctrl-d to exit


introspection/reflection

    # will print out object methods in the shell
    dir()

    #example

    import struct
    dir()
    dir(struct)  


line comment

    #

mulitline comment

    """
    you can use pydoc style quotes,
    though it's kinda a hack
    """

continue long lines

    this is a long line \
    that will be continued

combine two lines

    print "hi" ; print "there"


tab/spaces:

    best practice is (apparently) to use spaces instead of tabs

Note: This seems counter intuitive to me. No one seems to agree on how many spaces to use.   PEP 8 recommends 4 spaces, and Google recommends 2 spaces.  Personally, I think it makes more sense to use \t and let everyone set their own tabstop.


print

    print "hello world"


    concatinate string

    # with space
    a = "hello",  "world"
    print a
    # hello world


    # without space
    print "hello" + "world"
    helloworld


    WARNING: python3 has apparently changed this basic function syntax.  :-)
    You'll need to use parens with python3: print(string)


variables, no declaration needed

    # simple
    a = 1

    # can assign a list to a list
    a,b,c = 1,2,3

    #can chain
    a = b = c = 1


python assumes all function variables are local.  to access the global scope use

    global varname


basic strings

    s = 'A string constant'
    s = "another constant"



There is no difference between ' and " (unlike in Perl and PHP).  Typically is seems like python code I see tends to prefer ' (easier to type?)

multi line strings use a triple-quote syntax:

    s = """
    This is
    your string
    """


analogue of php require_once (include)


    EXAMPLE

        #### (file: config.py)
        # can be nested data structure

        constant1 = "value1"
        constant2 = "value2"


        ####  (consumer.py)
        # note, no .py extension.

        import config

        # note, by default, everything is under the namespace:
        print config.constant1
        print config.constant2


    better - wrap with try/except

        # create file named modulename.py
        # then in second script run  
        try:
            import modulename
        except ImportError:
            print 'importing modulename failed'



command line args

    import sys

    # 1 is first arg
    a = sys.argv[1]



function

    def concat(a, b):
        return a + b;


    functions can be called with named arguments

        concat('1', '2')

    or

        concat( a = '1', b = '2' )


    functions can also set default values

    def example( a, b = 0 ):
        return a + b;



nested functions

    you can define function in a function and return it:


    def example():
        def ret():
            print "hi"
        return ret

    t = example()
    t()
    #hi


more on strings

    a = 'string'
    a = "string"
    a = """string"""
    a = '''string'''

    # unicode
    a = u'strng'

    # binary
    a = b'strng'

    # regular expression
    regex = r"(\w)"


string formatting

    WARN: python3 changes this syntax:

    a = 5
    b = "hello"

    buf = "A = %d\n , B = %s\n" % (a, b)
    print buf

    # prints:
    # A = 5
    #  , B = hello

    c = 10
    buf = "C = %d\n" % c
    print buf
    C = 10



    You could as well use format:

    >>> print "This is the {}th tome of {}".format(5, "knowledge")
    This is the 5th tome of knowledge


    note: % will be deprecated in python 3.1


# string manipulation

    # to lower  case
    a.lower()

    # replace
    a.replace( find, start, [num_of_times] )

    # starts with string
    a.startswith( "/" )

    # ends with string
    a.startswith( "/" )


# regular expressions

    import re

    # substitute
    re.sub(r"(\w)(\w+)(\w)", yourreplacement, yourstr)

    # match
    if re.search('http', yourstr) != None:

int

    b = 2

float

    b = 2.0

booleans

    True, False

null value

    None


conditional

    if a < b:
        do_something();
    elif a < c:
        do_something_else()
    else:
        default_case()


logical operators (these do what you'd expect)

    and
    or
    not

    # don't forget and type &&  ||  !  :-)


main hook

    if __name__ == "__main__":
        main()


exit program:


    import sys
    sys.exit(2)


    #another example
    if sys.argv[1] == '':
        print "usage is ./yourprogram.py site url "
        sys.exit(2)



array

    empty list
    a = []

    with values
    a = ['spam', 'eggs', 100, 1234]

    # length

    len(a)

    #note: multidimensional arrays work as you'd expect

    # whitespace between elements is ignored:
    # valid syntax:
    a = [1,
            2,
            3,
            4
                ]


test if value is in array

    has_spam = 'spam' in s


loops

    for n in [1,2,3]:
        print n


list/array methods:

    list.append(item)
    list.extend(List)
    list.insert(i, item)
    list.remove(item)
    list.pop(i)
    list.index(item)
    list.count(item)
    list.sort()  # in place
    list.reverse()


hash/dictionary


    stock = {
        "key": "value",
        "key2": 2
    }

    stock['key3'] = 2

    # remove item

    del stock['key3']


nested data structure

    # nest arrays and
    a = [ {'a': 1}, {'a': 2} ]

    print(a[1]['a'])
    #prints: 2

loop over dictionary/hash

    d = {'x': 1, 'y': 2, 'z': 3}
    # loop over keys, read into var 'key'
    for key in d:
        print key, 'corresponds to', d[key]

    # loop over key and value
    for key, value in d.iteritems():
        print key, 'corresponds to', value


    breaking out of a loop:

        break        exists loop
        continue    skips iteration


cast

    int()
    float()
    str()
    chr()
    unichr()
    ord()
    hex()
    oct()

tuple (invariant data, like a constant struct)

    coord = (2,3)


set (like a dictionary with no data)

    a = { 'a', 'b' }

while loop

    while link != '':
        # do something

files

    f = open('doo.txt')
    while line = f.readLine()
        print line
    f.close()

user input

    name = raw_input("enter your name:")

array slice

    select all items j <= k < j
    a[i:j]

    -1 counts starting from end of list

    replace list subset with another

    a[1:2] = [1,2,3,4]

    concatenate lists
    a = a + b


tuples

    pack data together as a single unit
    a = ('GOOG', 100, 635.2)

    access like array

set

    like array keys, no duplicates

    s = set([3,4,5])


    operations
    |    union
    &    intersection
    -    difference (subtract)
    ^    nand


place holder

    if you need to stick something in a code block for syntax check
    does nothing but fill space

        if 1:
            pass

test if string contains substring

    x = 'hello'
    y = 'll'
    y in x
    # True

index of string

    >>> print "hello".find('h')
    0
    >>> print "hello".find('ss')
    -1


exceptions

    try:
        do_somthing()

    except:
        print "error: your error "



    full syntax
    except NameError, IndexError as msg:
        print "error", msg      


    raising execptions (throw)

        raise ValueError('invalid number')

    print error # older syntax

        try:
            doX()
        catch Exception, e:
            print e

generator

    use instead of return
        yield

    the function maintains persistent state and pauses at the yeild line

    to grok: it's a static function with static global variables.


 
lamba

    for short anon function, no (), no return statement, one line
 
        square = lambda x: x ** 2

        print square(2)
 


Object Oriented Programming in Python

For me, OOP in Python is difficult to use compared to Java/C#.  To me it just feels like OOP was bolted onto Python as an after-thought.  Or probably more accurately, it copied from a language which bolted OOP on as an after-thought.  OOP was not present in the pre 1.0 releases of Python.

Overall, I really wish Python had (a) explicit variable declarations like `var x` and (b) implicit self.  The scoping rules also feel screwy (why do we need `global`?  If can see a variable in a parent scope, why can't Python?)

Otherwise, you might initially think if you write a set of functions that access global variables, all you need to do to convert them into a class, is to indent them all and add a class name to the top.  For example, this syntax would make sense to me:


    # i'm a function accessing a global variable
    a = 1
    def get_a():
        return a

    # now i'm a class
    class your_class:
        a = 1 # now an instance variable
        def get_a():
            return a

This is simple and makes sense to me.  Unfortunately it won't work at all. :-)  Python forces you to rewrite every single function signature and variable reference.

After struggling with Python's approach, IMO the easiest way to understand understand OOP in Python is to realize that: Python is NOT really an OOP language, any more than Perl is.  It's a multi-paradigm language which happens to provide a little OOP "syntax sugar."  But it's really bitter tasting sugar. :-)


As a result, the OOP "object.function" syntax is largely redundant.  It's passing an object reference around as the first argument, and has a very lightweight syntax manipulation that just converts this syntax:

 
    obj.f(a)  ->  f(obj, a)

IOW, it's just moving the "obj" to inside the function.  That's not OOP; that's just a regular expression.


Anyway, accepting the screwy OOP syntax, here's an object example:

    class Test:

        # instance variable  
        your_variable = ''

        # static variable
        your_static = ''

        # constructor
        def __init__(self, a):
            # no class scope of variables.  must use explicit self.
            self.your_variable = a  
            Test.your_static = a  

        # like perl class
        def your_method( self, a ):
            self.your_variable = a  
            # ...


initialize class

    # NOTE: no 'new' keyword
    # this is less painful

    t = Test()


extending another class

    class Test( BaseClass ):

        #...

execute shell command

    import subprocess

    p = subprocess.Popen('ls', shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
    for line in p.stdout.readlines():
        print line,
    retval = p.wait()



    older method (prints output)

        os.system(cmd_string)



modules:

    save as yourfile.py


        import yourfile

    call as:

        yourfile.function()


    import indivdual functions into script namespace


        from yourfile import *
        from yourfile import f1, f2


urllib vs urllib2


    generally you should use the newest lib 2
    urlib (old) has the encode function

     urllib.urlencode({'abc':'d f', 'def': '-!2'})
    'abc=d+f&def=-%212'


    in python 3, everything has been merged into urllib



easy install for modules

    # shell level installer

    wget http://peak.telecommunity.com/dist/ez_setup.py
    python ez_setup.py


    # cache downloade files with
    easy_install  -b  /path/to/build/dir/

    # install from cache
    easy_install  /path/to/build/dir/


install pip to install new modules. like PCAN for Pearl


    new versions of python (>3.4) already include pip

    for old versions download

        https://bootstrap.pypa.io/get-pip.py

    run

        python    get-pip.py

    add python Scripts/ to PATH, if it's not already


    #verbose output
    pip -vvv install flask


better parsing of command line args

    # allow a hook for cli
    if __name__ == "__main__":
        opt = OptionParser()
        opt.add_option("-d", "--database", help="EZCONNECT string", action="store",
                     type="string", dest="db")
        opt.add_option("-n", "--requests", help="number of requests", action="store",
                     type="int", dest="requests", default=10)
        opt.add_option("-c", "--concurrency", help="number of concurrent connections",
                     action="store", type="int", dest="concurrency", default=1)
        opt.add_option("-s", "--sql", help="SQL query or PL/SQL block",
                     action="store", type="string", dest="sql")
        opt.add_option("-b", "--bind", help="dictionary of bind parameters",
                     action="store", type="string", dest="bind")
        (options, args) = opt.parse_args()
        bench = Orabench(options)
        bench.run()


python debugger

    within a python file:

        import pdb
        ...
        pdb.set_trace() begin the debugger at this line when the file is run normally

    start debug mode

        python -m pdb  <name>.py  [args]    begin the debugger
        help [command] view a list of commands, or view help for a specific command

    control execution

        l(ist)       list 11 lines surrounding the current line
        w(here)      display the file and line number of the current line
        n(ext)       execute the current line
        s(tep)       step into functions called at the current line
        r(eturn)     execute until the current functionṡs return is encountered
        b[#]         create a breakpoint at line [#]
        b            list breakpoints and their indices
        c(ontinue)   execute until a breakpoint is encountered
        clear[#]     clear breakpoint of index [#]
        q(uit)       exit the debugger

    changing variables / interacting with code

        p <name>      print value of the variable <name>
        ! <expr>      execute the expression <expr>



operators

    Operator    Description    Example

    +    Addition - Adds values on either side of the operator     a + b will give 30
    -    Subtraction - Subtracts right hand operand from left hand operand     a - b will give -10
    *    Multiplication - Multiplies values on either side of the operator     a * b will give 200
    /    Division - Divides left hand operand by right hand operand     b / a will give 2
    %    Modulus - Divides left hand operand by right hand operand and returns remainder     b % a will give 0
    **    Exponent - Performs exponential (power) calculation on operators     a**b will give 10 to the power 20
    //    Floor Division - The division of operands where the result is the quotient in which the digits after the decimal point are removed.     9//2 is equal to 4 and 9.0//2.0 is equal to 4.0



    ==     Checks if the value of two operands are equal or not, if yes then condition becomes true.     (a == b) is not true.
    !=     Checks if the value of two operands are equal or not, if values are not equal then condition becomes true.     (a != b) is true.
    <>     Checks if the value of two operands are equal or not, if values are not equal then condition becomes true.     (a <> b) is true. This is similar to != operator.
    >     Checks if the value of left operand is greater than the value of right operand, if yes then condition becomes true.     (a > b) is not true.
    <     Checks if the value of left operand is less than the value of right operand, if yes then condition becomes true.     (a < b) is true.
    >=     Checks if the value of left operand is greater than or equal to the value of right operand, if yes then condition becomes true.     (a >= b) is not true.
    <=     Checks if the value of left operand is less than or equal to the value of right operand, if yes then condition becomes true.     (a <= b) is true.


    =    Simple assignment operator, Assigns values from right side operands to left side operand     c = a + b will assigne value of a + b into c
    +=    Add AND assignment operator, It adds right operand to the left operand and assign the result to left operand     c += a is equivalent to c = c + a
    -=    Subtract AND assignment operator, It subtracts right operand from the left operand and assign the result to left operand     c -= a is equivalent to c = c - a
    *=    Multiply AND assignment operator, It multiplies right operand with the left operand and assign the result to left operand     c *= a is equivalent to c = c * a
    /=    Divide AND assignment operator, It divides left operand with the right operand and assign the result to left operand     c /= a is equivalent to c = c / a
    %=    Modulus AND assignment operator, It takes modulus using two operands and assign the result to left operand     c %= a is equivalent to c = c % a
    **=    Exponent AND assignment operator, Performs exponential (power) calculation on operators and assign value to the left operand     c **= a is equivalent to c = c ** a
    //=    Floor Dividion and assigns a value, Performs floor division on operators and assign value to the left operand     c //= a is equivalent to c = c // a


    &     Binary AND Operator copies a bit to the result if it exists in both operands.     (a & b) will give 12 which is 0000 1100
    |     Binary OR Operator copies a bit if it exists in eather operand.     (a | b) will give 61 which is 0011 1101
    ^     Binary XOR Operator copies the bit if it is set in one operand but not both.     (a ^ b) will give 49 which is 0011 0001
    ~     Binary Ones Complement Operator is unary and has the efect of 'flipping' bits.     (~a ) will give -61 which is 1100 0011 in 2's complement form due to a signed binary number.
    <<     Binary Left Shift Operator. The left operands value is moved left by the number of bits specified by the right operand.     a << 2 will give 240 which is 1111 0000
    >>     Binary Right Shift Operator. The left operands value is moved right by the number of bits specified by the right operand.     a >> 2 will give 15 which is 0000 1111


    and     Called Logical AND operator. If both the operands are true then then condition becomes true.     (a and b) is true.
    or    Called Logical OR Operator. If any of the two operands are non zero then then condition becomes true.     (a or b) is true.
    not    Called Logical NOT Operator. Use to reverses the logical state of its operand. If a condition is true then Logical NOT operator will make false.     not(a and b) is false.



    in    Evaluates to true if it finds a variable in the specified sequence and false otherwise.     x in y, here in results in a 1 if x is a member of sequence y.
    not in    Evaluates to true if it does not finds a variable in the specified sequence and false otherwise.    x not in y, here not in results in a 1 if x is not a member of sequence y.


    is    Evaluates to true if the variables on either side of the operator point to the same object and false otherwise.     x is y, here is results in 1 if id(x) equals id(y).
    is not    Evaluates to false if the variables on either side of the operator point to the same object and true otherwise.     x is not y, here is not results in 1 if id(x) is not equal to id(y).



Python Operators Precedence. The following lists all operators from highest precedence to lowest.


    Operator    Description
    **    Exponentiation (raise to the power)
    ~ + -    Ccomplement, unary plus and minus (method names for the last two are +@ and -@)
    * / % //    Multiply, divide, modulo and floor division
    + -    Addition and subtraction
    >> <<    Right and left bitwise shift
    &    Bitwise 'AND'
    ^ |    Bitwise exclusive `OR' and regular `OR'
    <= < > >=    Comparison operators
    <> == !=    Equality operators
    = %= /= //= -= += *= **=    Assignment operators
    is is not    Identity operators
    in not in    Membership operators
    not or and    Logical operators

No comments: