Python

Table Of contents

Cpython

A type of implementation of python programming language. It’s written in

Declaring and initializing variable

name = 'nitin';
name, age = 'nitin', 21;

>>> name, age = 'nitin', 21, 23
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: too many values to unpack (expected 2)
>>> name, age, status = 'nitin', 21
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: not enough values to unpack (expected 3, got 2)
>>>

Operators

1. // : floor division
2. ** : power

Data types

1. int
2. float
3. decimal
4. fraction
5. complex number
6. strings

print()

r for printing raw string

print('hello world\nbye world'); # prints with new line
print(r'hello world\nbye world'); # prints in single line with '\n'

end keyword for specifying what to insert at the end

print('hello world', end=','); #prints 'hello world,' without new line

String

greeting = 'good morning';
greeting = "good morning";

Multi-line string

greetings = '''
	Good morning
	Welcome everyone
''';
 
greetings = """
	Good morning
	Welcome everyone
""";
 
greetings = """\
	Good morning
	Welcome everyone
"""; # '\' prevents leading new line

Two or more string literals are concated automatically

>>> 'good' ' morning';
'good morning'

Strings can be indexed

project[0];   # returns 'y'
project[-1];  # returns 'a'

• Indices can be negative to start from the right.
• Strings are immutable here also

Slicing

''' Sytax :
	string[startIndex:endIndex];
'''
 
project[0:1]; # returns 'y'
project[0:4]; # returns 'yask'
 
project[:4]; # returns 'yask'. defaults to 0
project[1:]; # returns 'aska'. defaults to string length

String length

len(project); # returns 5

String formatting

Method 1

name = 'Nitin'
surname = 'Sharma'
print("Hello %s" % name)
print("Hello %s %s" % (name, surname))

This method is used in old versions of pythons.

Method 2

print(f"Hello {nitin}")

Method 3

print("Hello {}".format(name))

List

List is a collection of elements of same type or different types. It’s more likely arrays in javascript.

preferences = ['bengalore', 'pune', 'gurgaon'];
preferences[0]; # returns 'bengalore'
 
len(preferences); #returns list length i.e. 3
 
# elements can be changed using indices
preferences[0] = 'pune';
preferences[1] = 'bengalore';
 
# Assignment to slices is also possible
 
preferences[0:1] = 'Pune';
preferences; # returns ['Pune', 'bengalore', 'gurgaon']
 
preferences[:] = []; # clears the list

List in a list

interests = ['typing', 'reading'];
person = ['nitin', 21, interests];
 
interests.append('traveling');
person[2];  # it also gets 'traveling', i.e. list reference is stored

Note : for equal to operator, the list items are checked. It’s not javascript where references are compared

interests = ['typing', 'reading'];
_interests = ['typing', 'reading'];
 
interests == _interests; # True
 
_interests.append('travel');
interests == _interests; # False

List comprehension

List comprehension provides concise way to create lists

nums = [1, 2, 3, 4]
evens = [x for x in nums if x % 2 == 0]
 
print(evens) # [2, 4]

Look, how concise it is.

It starts with an expression followed by for statement. That expression is compulsory.

With if-else

def _filter(items: list) -> list:
    return [item if type(item) is int else 0 for item in items]

So, for if-else, the position of the components has to change

Tuples

Tuples are similar to lists but these are immutable.

names = 'nitin', 'hemant'
print(names) #('nitin', 'hemant')
 
names = () # for empty tuple
 
names = 'nitin', # for single value in tuple. Trailing , is needed
 
# unpacking the tuple
name1, = names # name1 contains 'nitin'

Numbers of variables on the left hand side should be equal to size of tuple.

names = 'nitin', 'hemant'
my_name, = names # won't work
 
name1, name2 = names; # will work

This is called sequence unpacking.

Note that multiple assignment is really just a combination of tuple packing and sequence unpacking

Set

It’s an unordered collection of items.

names = {'nitin', 'hemant'}
# Or
names = set(['nitin', 'hemant'])
 
names = set(['nitin', 'hemant', 'nitin'])
print(names) # {'nitin', 'hemant'}

Set operations

It also supports set operations

set1 = {1, 2, 3, 4}
set2 = {1, 4, 6, 9}
 
set1 - set2 # {2, 3}, in set1, not in set2
set1 & set2 # {1, 4} in both
 

Function

def greet():
	print('Good morning!')

Keyword Arguments

def greet(age, name):
	print('Good morning', name)
 
greet(12, name='nitin');

• Keyword argument should come after positional argument. If positional argument comes after keyword argument, interepreter throws error.

greet(name='nitin', 12) # won't work
greet(12, name='nitin')

*args and **keywordArgs

def function(arg, *args, **keywordArgs):
	print(arg)
	print(args)
	print(keywordArgs)

• arg is a positional argument,
• args is a tuple having other dynamic arguments,
• keywordArgs is a dict having keyword and value

function('hello', 'world','bye world', name='nitin', surname='sharma');

# output
hello
('world', 'bye world')
{'name': 'nitin', 'surname': 'sharma'}

• *args should be before **keywordArgs .

Special parameters

def f(pos1, pos2, /, pos_or_kwd, *, kwd1, kwd2):
      -----------    ----------     ----------
        |             |                  |
        |        Positional or keyword   |
        |                                - Keyword only
         -- Positional only

Names of positional only parameter can be used in keyword only parameters.

def foo(name, /, **kwds):
	return 'name' in kwds
 
foo(1, name='nitin') # return True
foo(1, **{name: 'niitn'}) # return True

Unpacking Argument list

# unpacks list
args = [1, 2]
list(range(*args))
 
# unpacks dictionary
greet(**{'name':'nitin', 'age': 21});

Lambda Expressions

is_even = lambda number: number % 2 == 0
 
def filter(array: list, predicate):
  resArray = []
  for item in array:
    predicate(item) and resArray.append(item)
  
  return resArray
 
print(filter([1, 2, 3, 4], is_even))

Module

function.py

def greet():
	print('Good morning')

import function
 
function.greet() # Good morning

function module is in global space where it is imported.

It’s like require in node. The module file is evaluated and all the functions and variables can be imported.

def fun():
	import function
	function.greet()
 
fun() # import function in it's space and calls function.greet
 
function.greet() # won't work as it's not in global space of the file

# different types of import
 
from function import greet
 
from function import *

All of these imports get the imported function or variables from the module into the global scope of the file.

import function as fun
 
# or 
from function import greet as printGreet
 
fun.greet()
printGreet()

Packages

A package is a directory having modules or python files.

app
├── app.py
└── src
    ├── components
    │   ├── __init__.py
    │   └── header.py
    └── layout.py

import app.app
 
app.App()

This is one way to import module from package.

from app.app import App
 
App()

This is another way to import module. It loads the module and puts the function in the namespace.

Importing * from package

from app import *

This doesn’t work until we put __init__.py in the package.

app
├── __init__.py
├── app.py
└── src
    ├── components
    │   ├── __init__.py
    │   └── header.py
    └── layout.py

app/__init__.py

__all__ = ["app"]

__all__ is a list having all the modules to make avaiable for import * So, app module gets imported the following works.

from app import *
 
app.App()
"""
This is app layout
This is a header
This is App
"""

Absolute imports and relative imports

In file layout.py

1. Absolute import

from app.src.components.header import Header;
 
def Layout():
  print('This is app layout')
  Header()

1. Relative import

from .components.header import Header;
 
def Layout():
  print('This is app layout')
  Header()

Python Namespaces and Scopes

Namespace

A namespace is a collection of unique names. Name in a namespace can be same as in another namespace.

Attributes of a dictionary are also names in dictionary namespace.

Note : The important thing to know about namespaces is that there is absolutely no relation between names in different namespaces

Types of Namespaces

1. Built-in : python built-in
2. Global : in a file or module
3. Local: in function or method

Classes

class SomeClass:
	def __init__(self, arg1, arg2):
		self.arg1 = arg1;
		self.arg2 = arg2;
 
	def do_something(self):
		pass
		//do something
 
instance = SomeClass(1, 2)
instance.arg1
instance.arg2
instance.do_something()

self has to pass as first argument to class function.

Class function and Instance method

There is a difference between class function and instance method. An instance method is created using class function.

When instance.do_something() is called, the attribute do_something is searched in the class that appears to be a function. instance is bounded with that class function and called with argument list.

Private variables

There is way to define private methods and variables in python. It’s all convention here. Variable starting with _ are considered as private.

class S:
	def __init__(self):
		self._private_attribute = 2

Callable class instances

A class instances can be called if class implements __call__ method.

class Counter:
  def __init__(self, start=0):
    self.count = start
  
  def __call__(self):
    self.count += 1
    return self.count
 
counter = Counter()
 
counter() # returns 1
counter() # returns 2

Iterator

An iterator has __next__() object method that gives the next value. for does the same thing, it gets iterator of the list and calls this __next__() method.

To create iterator, the class should have __next__() and __iter__() functions. for loop calls this __iter__() method to get iterator.

class Reverse:
  def __init__(self, items: list):
    self.items = items
    self.index = len(items)
  
  def __iter__(self):
    return self
  
  def __next__(self):
    if self.index == 0:
      raise StopIteration
    
    self.index -= 1
    return self.items[self.index]

File IO

f = open('file', 'r', encoding='utf-8')
f.read()

JSON in Python

import json
 
nums = [1, 2, 3, 4]
 
json.dumps(nums) # returns json of nums
 
json.dump(nums, fileObject) # writes the nums json to file

Loading Json files

json.load(fileObject)

Errors and Exceptions

Basic Syntax

try:
	# some code that might raise an exception
except Exception:
	# code that handles exception
else:
	# code that executed when try block gets executed successfully

else should follow all except clauses.

except can take multiple type of Exceptions together

except (ExceptionType1, ExceptionType2...):

There can be multiple except blocks.

Raising Exceptions

Exceptions are raised using raise statement.

try:
	raise Exception('demo')
except Exception as ex:
	print(ex)

Exception Chaining

An exception can cause another exception to raised.

try:
	open('some-bad-file', 'r')
except OSError:
	raise RuntimeError('unable to process')

This flows like, due to OSError another exception RuntimeError occured.

To make an exception direct consquence of another, from clause is used

try:
	open('some-bad-file', 'r')
except OSError as file_error:
	raise RuntimeError('unable to process') from file_error

Now, file_error is the direct cause of RuntimeError .

This chaining can be disabled by using from None

try:
	open('some-bad-file', 'r')
except OSError:
	raise RuntimeError('unable to process') from None

So, only RuntimeError will be raised.

finally block

try:
	# some code that might raise an exception
except:
	# exception handling code
finally:
	# code that will run always

Some points to remember:

1. Return value will be taken from the finally even though try returns some value.
2. If try raises some exception, except block handles that. If it doesn’t handle, the finally block will get exectued and the exception will be re-raised.
3. If finally has break, continue or return, exception won’t be re-raised.

Context Managers

Context managers allow allocate and release of resources. A common use case of context managers is to lock and unlock resources.

A context manager basically has two functions

1. __enter__
2. __exit__

with statement calls this __enter__ method and assigns to as parameter.

On any exception, with calls __exit__ method and passes, type, value, and traceback.

class FileManager:
  def __init__(self, filename):
    self.filename = filename
  
  # context manager implementation
  def __enter__(self):
    self.fileobject = open(self.filename, encoding='utf-8')
    return self.fileobject
  
  def __exit__(self, type, value, traceback):
    print('Error:', type, value, traceback)
    self.fileobject.close()
 
with FileManager('some-file') as file:
	file.read()
 
with FileManager('some-file') as file:
	file.some_undefined_method() # with will call __exit__ method
 

If __exit__ method returns True then the exception is handled by __exit__ otherwise with raises exception

For more information, read this.

Decorators

Decorators are function that wraps a function.

def decorator(function):
  def wrapper():
    print("doing something before calling actual function")
    function()
    print("doing something after calling actual function")
  
  return wrapper
 
def greet():
	print("Good morning peeps!")
 
greet = decorator(greet)
 
greet()

The same thing can be done using decorator

@decorator
def greet():
	print("Good morning peeps!")
 
greet()

Decorating function accepting argument

def decorator_with_args(function):
  def wrapper(*args, **kwargs):
    print("before calling actual function")
    function(*args, **kwargs)
    print("after calling actual function")
  return wrapper

It is not always necessary that decorate should have wrapper function inside it. The main focus should be on that the decorator takes a function as an argument and returns a function.

Note : With decorator, we are losing information about the actual function. We can’t inspect that function being decorated. For example,

@decorator
def greet():
	print("hi")
 
greet.name # would give wrapper
great.__doc__ # would give doc of wrapper

To solve this issue, we use python utility functools package.

import functools
 
def decorator(function):
	@functools.wraps(function)
  def wrapper():
    print("doing something before calling actual function")
    function()
    print("doing something after calling actual function")
  
  return wrapper
 
def greet():
	print("Good morning peeps!")
 
greet.__name__ # would give greet

functools.wraps is a decorator that takes the function being decorated. What it does is, copies the __doc__ and __name__ information of decorated function given to the wrapper function.

Decorators with arguments

A decorator is a function that takes function as a argument. To pass argument to the decorator, we need to make a function that returns decorator.

import functools
 
def repeat(times):
  def decorator_repeat(fn):
    @functools.wraps(fn)
    def repeat_wrapper(*args, **kwargs):
      for i in range(times):
        fn(*args, **kwargs)
    
    return repeat_wrapper
  return decorator_repeat
 
@repeat(2)
def greet():
	print("Good morning")

Class decorators

Class can be used to make stateful decorators. If we know how decorator works, it would be easy to understand how can be a class used to make decorators.

@decorator
def fn():
	pass

So, decorator is actually a function which takes fn as argument, wraps this function in a wrapper and returns that wrapper and assigns to same variable fn.

Scenerio with class

@SomeClassDecorator
def fn():
	pass

So, with this, an instance of the class is created. That instance is assigned to the variable fn. Ofcourse that instance should be callable. And we know how to make class callable.

class Countable:
  def __init__(self, fun):
    self.fun = fun
    self.count = 0
    functools.update_wrapper(self, fun)
  
  def __call__(self, *args, **kwargs):
    self.count += 1
    self.fun(*args, **kwargs)
    print(f"Called {self.fun.__name__} {self.count} times")

@Countable
def greet():
	print("hi")
 
# is equivalent to
def greet():
	print("hi")
 
greet = Countable(greet)
 

functools.update_wrapper is used to update the introspetion details of the instance.

Why do we need Decorators

Read more

To look

• Decorators
• Documentation Strings. Conventions to document.
• sorted() or .sort() method
• Inheritance
• “Compiled” Python files
• Make interfaces.
  • Try to pass classes to function.
  • Try passing drived class to function accepting base class and access drive class methods
• Decoratos
  • Caching values using decorators
  • Adding Information About Units
  • Validating JSON
• dags and airflow (jobs and pipelines)

Points Noted

1. Python code is synchronous in nature. If there is any blocking code in the infinite loop, the loop will wait for that blocking code to get executed.

   For example

   while True:
           data = connection.recv(1024)
           print("From client: ", data)
           if not data:
             print("Client closed the connection")
             break
           
           connection.sendall(data)

References

1. Standard types
2. Python 3 contents
3. Variables
4. Naming Convenstions
5. Walrus operator
6. Glob
7. Pathlib
8. Socket Real Python
9. PEP

multi-threading

Async IO

References

Pandas

opencv-python

Asyc Server

poetry