Python Mutable vs Immutable

Here is a basic introduction to immutable and immutable types in python.

In python there are two types of data… mutable and immutable. Numbers, strings, boolean, tuples, and other simple types are immutable. Dicts, lists, sets, objects, classes, and other complex types are mutable.

When you say:

a = [1,2,3]
b = a

You’ve created a single mutable list in memory, assigned a to point to it, and then assigned b to point to it. It’s the same thing in memory.

Therefore when you mutate it (modify it):

b[0] = 3

It is a modification (mutation) of the index [0] of the value which b points to at that same memory location.

However, when you replace it:

b = [0,0,0]

It is creating a new mutable list in memory and assigning b to point at it.


Check out the id() function. It will tell you the “address” of any variable. You can see which names are pointing to the same memory location with id(varname).


Bonus: Every value in python is passed by reference… meaning that when you assign it to a variable it simply causes that variable to point to that value where it was in memory. Having immutable types allows python to “reuse” the same memory location for common immutable types.

Consider some common values when the interpreter starts up.  You can see here there are a lot of variables pointing at the memory location held by abc.  cpython, at least, is smart enough to realize that the value `abc` is already stored in memory and because it is immutable, just returns that same memory address.

>>> import sys
>>> sys.getrefcount('abc')
68
>>> sys.getrefcount(100)
110
>>> sys.getrefcount(2)
6471

However, a value that is definitely not present would return 2. This has to do with the fact that a couple of references to that value were in-use during the call to sys.getrefcount

>>> sys.getrefcount('nope not me.  I am definitely not here already.')
2

Notice that an empty tuple has a lot of references:

>>> sys.getrefcount(tuple())
34571

But an empty list has no extra references:

>>> sys.getrefcount(list())
1

Why is this? Because tuple is immutable so it is fine to share that value across any number of variables. However, lists are mutable so they MUST NOT be shared across arbitrary variables or changes to one would affect the others.

Incidentally, this is also why you must NEVER use mutable types as default argument values to functions. Consider this innocent little function:

>>> def foo(value=[]):
...     value.append(1)
...     print(value)
...
...

When you call it you might expect to get [1] printed…

>>> foo()
[1]

However, when you call it again, you prob. won’t expect to get [1,1] out… ???

>>> foo()
[1, 1]

And on and on…

>>> foo()
[1, 1, 1]

>>> foo()
[1, 1, 1, 1]

WHY IS THIS? Because default arguments to functions are evaluated once during function definition, and not at function run time. That way if you use a mutable value as a default argument value, then you will be stuck with that one value, mutating in unexpected ways as the function is called multiple times.

The proper way to do it is this:

>>> def foo(value=None):
...     if value is None:
...         value = []
...     value.append(1)
...     print(value)
...
...
>>>
>>> foo()
[1]
>>> foo()
[1]
>>> foo()
[1]

Example of Python Generator Function.

One of the great features of Python is Generator Functions.  Generator functions allow you to convert any function to a generator function by simply including the yield keyword somewhere in the function body.  When a generator function is called, the response is a generator object, which can be iterated over among other things.

To boil this down to basics, generator functions allow you the programmer to create a function which yields one value at a time (and pauses until the next value is requested) until you decide it is done.  This opens endless possibilities for converting sequences, creating sequences, filtering, and more.

The following example is a generator function which will take a simple list or iterator and return pairs of (element, previous_element).  This is a great use for generators and the yield statement.

Generator Function:

def lineandlast(listish, first=True, last=True):
    iterator = iter(listish)
    lastline = next(iterator)

    if first:
      yield lastline, None

    for line in iterator:
        yield line, lastline
        lastline = line

    if last:
        yield None, lastline

Example:

for line, last in lineandlast([1,2,3,4,5]):
    print(line, last)

Output:

1 None
2 1
3 2
4 3
5 4
None 5

Notes:
There are two keyword arguments, first and last, which can be used to control the output of the first and last items on the output example above.

Python Word Find Program

Ezra and I took 30 minutes this evening and wrote a word find puzzle solver in Python.  This uses a little bit of pre-processing, sets, dictionaries, tuples, and other (less efficient) techniques to fairly quickly find the answers.

board = '''
m a i n v f
a n a f d d
s m i t h a
i r o n r v
e m o z a i
a e e t a d
'''
words = '''
ezra
david
sam
smith
iron
fire
'''
# use this to get a dictionary involved.
#words = open('dictionary.txt', 'rt').read()
BOARD = {}
x = 0
y = 0
for line in board.strip().split('\n'):
y += 1
x = 0
for char in line:
char = char.lower()
if char in 'abcdefghijklmnopqrstuvwxyz':
x += 1
BOARD[x,y] = char
WORDS = set()
for line in words.strip().split('\n'):
word = line.strip().lower()
if len(word) >= 3:
WORDS.add(word)
DIREC = (
(0,1),
(1,1),
(1,0),
(1,1),
(0,1),
(1,1),
(1,0),
(1,1),
)
PREFX = set()
FOUND = set()
for word in WORDS:
prefix = ''
for c in word[:1]:
prefix += c
PREFX.add(prefix)
# For each position on the board
for x,y in BOARD:
# For each direction of 8
for xx,yy in DIREC:
x1 = x #cur pos
y1 = y #cur pos
word = '' #cur word
while True:
# Check if off board
if (x1,y1) not in BOARD:
break
# Get character at current pos
word += BOARD[x1,y1]
# If it is a word, add it
if word in WORDS:
FOUND.add(word)
# If this is not a prefix, then bail out
if word not in PREFX:
break
# Increment current position based on direction
x1 += xx
y1 += yy
print(FOUND)

view raw
wordfind.py
hosted with ❤ by GitHub

VisPy Sinewave Demo

VisPy is quite an interesting tool.  They say it is for “scientific visualization”.  Here is an example of that.  I took a demo from their github page and added a little code to generate a sinewave.  The cool thing is that the graph, scaling, panning, zooming, and redrawing all come out of the box.

I previously wrote about installing VisPy.  Thanks to the great efforts of some unnamed people, Python on Windows is really working nicely now.

Here is a screenshot:

vis

Here is the code:

# -*- coding: utf-8 -*-
# Copyright (c) Vispy Development Team. All Rights Reserved.
# Distributed under the (new) BSD License. See LICENSE.txt for more info.
"""
Demonstration of InfiniteLine visual.
"""
import sys
import numpy as np
from vispy import app, scene
# vertex positions of data to draw
N = 200
pos = np.zeros((N, 2), dtype=np.float32)
x_lim = [50., 1750.]
y_lim = [2., 2.]
pos[:, 0] = np.linspace(x_lim[0], x_lim[1], N)
pos[:, 1] = np.random.normal(size=N)
pos1 = np.zeros((20000,2), dtype=np.float32)
color1 = np.ones((20000,4), dtype=np.float32)
pos1[0,0] = 0.0
pos1[0,1] = 15.0
pos1[1,0] = 100.0
pos1[1,1] = 15.0
import math
for x in range(20000):
pos1[x,0] = x*10
pos1[x,1] = math.sin(x/20.) * 40.
# color array
color = np.ones((N, 4), dtype=np.float32)
color[:, 0] = np.linspace(0, 1, N)
color[:, 1] = color[::1, 0]
canvas = scene.SceneCanvas(keys='interactive', show=True)
grid = canvas.central_widget.add_grid(spacing=0)
viewbox = grid.add_view(row=0, col=1, camera='panzoom')
# add some axes
x_axis = scene.AxisWidget(orientation='bottom')
x_axis.stretch = (1, 0.1)
grid.add_widget(x_axis, row=1, col=1)
x_axis.link_view(viewbox)
y_axis = scene.AxisWidget(orientation='left')
y_axis.stretch = (0.1, 1)
grid.add_widget(y_axis, row=0, col=0)
y_axis.link_view(viewbox)
# add a line plot inside the viewbox
#line = scene.Line(pos, color, parent=viewbox.scene)
line1 = scene.Line(pos1, color1, parent=viewbox.scene)
# add vertical lines
vert_line1 = scene.InfiniteLine(100, [1.0, 0.0, 0.0, 1.0],
parent=viewbox.scene)
vert_line2 = scene.InfiniteLine(549.2, [0.0, 1.0, 0.0, 1.0], vertical=True,
parent=viewbox.scene)
# add horizontal lines
hor_line1 = scene.InfiniteLine(0.3, [1.0, 0.0, 1.0, 1.0], vertical=False,
parent=viewbox.scene)
hor_line2 = scene.InfiniteLine(5.1, [1.0, 1.0, 0.0, 1.0], vertical=False,
parent=viewbox.scene)
# auto-scale to see the whole line.
viewbox.camera.set_range((0,1000), (100,100))
if __name__ == '__main__' and sys.flags.interactive == 0:
app.run()

view raw
vis.py
hosted with ❤ by GitHub

Installing VisPy on Windows 10

VisPy is a Python library for interactive scientific visualization that is designed to be fast, scalable, and easy to use.

Here is how I installed it:

First I installed the latest Python 3.6 on Windows 10 by following the directions on http://www.python.org.  Once this was installed, I opened up Windows PowerShell and ran this command:

py -m pip upgrade vispy PyQt5 --user

I found some sample code here, and using NotePad++, copied and pasted it, saving it to Desktop\Code\v1.py

https://github.com/vispy/vispy/blob/master/examples/basics/gloo/animate_shape.py

In PowerShell, I changed to the directory that I saved the python code to and ran it:

cd Desktop\Code
py .\v1.py

Here is the output:

zzzz

It’s a pretty smooth and clean looking UI.  It seems extremely powerful, but I’ll need to dig in and see what makes it tick…

2D Sine Wave Example Using PyOpenGL

Here is an example of a moving 2D sine wave using Python 3, PyGame, and PyOpenGL.  See a https://blog.gahooa.com/2018/02/11/pygame-and-opengl-on-windows-10/  for how to install them.

sine

This sample program is designed to have a 100×60 unit working area with a 10 unit buffer around the edges.  You can see the axis in the lower-left (0,0) where Y+ is up, and X+ is to the right.

The structure of the program was created to make it super easy to work on the “guts” of the graphics without getting it confused with the “bookkeeping” end of OpenGL or PyGame.

Note: the glOrtho() command is how 2D “parallel perspective” is setup.  It defines the left, right, bottom, top, near plane, and far plane.  Because it is parallel, there is not the notion of a “camera” per-se, but rather section of the plane that should be viewed.  Documented here:

https://www.khronos.org/registry/OpenGL-Refpages/gl2.1/xhtml/glOrtho.xml

Here is the code!

#!/usr/bin/env python
###############################################################################
# Action Happens Here 50 times per second
def tick(i):
#glRotatef(1, 0, 0, 1)
#glTranslatef(0, 0, 1)
# Draw Axis
axis(i)
# Draw sinewave
for x in range(200):
x = x/2.0
y = math.sin(math.radians(x+i) * 10) * 30 + 30
cquad((x,y,0), 1, (y/60.0,0,x/100.0)) #(center, diameter, color)
###############################################################################
# The rest of this is the bones that make it work
import time
import pygame
from pygame.locals import *
from OpenGL.GL import *
from OpenGL.GLU import *
from OpenGL.arrays import vbo
import math
FPS_TARGET = 50
def axis(i):
glBegin(GL_LINES)
#x = red
#y = green
#z = blue
glColor3f(1, 0, 0)
glVertex3fv((0, 0, 0))
glVertex3fv((1, 0, 0))
glColor3f(0, 1, 0)
glVertex3fv((0, 0, 0))
glVertex3fv((0, 1, 0))
glColor3f(0, 0, 1)
glVertex3fv((0, 0, 0))
glVertex3fv((0, 0, 1))
glEnd()
def quad(points, color):
glBegin(GL_QUADS)
glColor3f(*color)
for p in points:
glVertex3fv(p)
glEnd()
def cquad(point, size, color):
glBegin(GL_QUADS)
glColor3f(*color)
x,y,z = point
s = size/2.0
glVertex3fv((xs,ys,z))
glVertex3fv((x+s,ys,z))
glVertex3fv((x+s,y+s,z))
glVertex3fv((xs,y+s,z))
glEnd()
def main():
#initialize pygame and setup an opengl display
pygame.init()
pygame.display.set_mode((1200,800), OPENGL|DOUBLEBUF)
glEnable(GL_DEPTH_TEST) #use our zbuffer
glEnable(GL_BLEND);
glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);
#setup the camera
glMatrixMode(GL_PROJECTION)
#gluPerspective(45.0,1000/1000,0.1,1000.0) #setup lens
#glOrtho(-10,10,-10,10,1,20)
glOrtho(10,110,10,70,1,1)
#glTranslatef(0, 0, -100) #move back
#glRotatef(-20, 1, 0, 0) #orbit higher
nt = int(time.time() * 1000)
for i in range(2**63):
nt += 1000//FPS_TARGET
#check for quit'n events
event = pygame.event.poll()
if event.type == QUIT or (event.type == KEYDOWN and event.key == K_ESCAPE):
break
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
tick(i)
pygame.display.flip()
ct = int(time.time() * 1000)
pygame.time.wait(max(1,nt ct))
if i % FPS_TARGET == 0:
print(ntct)
if __name__ == '__main__': main()

view raw
sinewave.py
hosted with ❤ by GitHub

PyGame and OpenGL on Windows 10

First I went to http://www.python.org and downloaded the latest version of Python for windows.  I made sure to select the option to add it to the system path.

Then I opened windows PowerShell (just a nicer term)

py -m pip install pygame --user
py -m pip install numpy --user
py -m pip install pyopengl --user

(I also installed NoteTab++, a nice text editor)

From there, I went to github and copied one of the examples from https://github.com/pygame/pygame/blob/master/examples/glcube.py and saved it on my desktop as myglcube.py.

In PowerShell, you just type:

cd Desktop
py -m myglcube

Here is my example:

zzz

(here is the code)

#!/usr/bin/env python
"""Draw a cube on the screen. every frame we orbit
the camera around by a small amount and it appears
the object is spinning. note i've setup some simple
data structures here to represent a multicolored cube,
we then go through a semi-unopimized loop to draw
the cube points onto the screen. opengl does all the
hard work for us. :]
"""
import time
import pygame
from pygame.locals import *
try:
from OpenGL.GL import *
from OpenGL.GLU import *
except ImportError:
print ('The GLCUBE example requires PyOpenGL')
raise SystemExit
#some simple data for a colored cube
#here we have the 3D point position and color
#for each corner. then we have a list of indices
#that describe each face, and a list of indieces
#that describes each edge
CUBE_POINTS = (
(0.5, 0.5, 0.5), (0.5, 0.5, 0.5),
(0.5, 0.5, 0.5), (0.5, 0.5, 0.5),
(0.5, 0.5, 0.5), (0.5, 0.5, 0.5),
(0.5, 0.5, 0.5), (0.5, 0.5, 0.5)
)
#colors are 0-1 floating values
CUBE_COLORS = (
(1, 0, 0), (1, 1, 0), (0, 1, 0), (0, 0, 0),
(1, 0, 1), (1, 1, 1), (0, 0, 1), (0, 1, 1)
)
CUBE_QUAD_VERTS = (
(0, 1, 2, 3), (3, 2, 7, 6), (6, 7, 5, 4),
(4, 5, 1, 0), (1, 5, 7, 2), (4, 0, 3, 6)
)
CUBE_EDGES = (
(0,1), (0,3), (0,4), (2,1), (2,3), (2,7),
(6,3), (6,4), (6,7), (5,1), (5,4), (5,7),
)
def drawcube():
"draw the cube"
allpoints = list(zip(CUBE_POINTS, CUBE_COLORS))
glBegin(GL_QUADS)
for face in CUBE_QUAD_VERTS:
for vert in face:
pos, color = allpoints[vert]
glColor3fv(color)
glVertex3fv(pos)
glEnd()
glColor3f(1.0, 1.0, 1.0)
glBegin(GL_LINES)
"""
for line in CUBE_EDGES:
for vert in line:
pos, color = allpoints[vert]
glVertex3fv(pos)
"""
for x in range(50,50):
glColor3f(x*.03%1.0, x*.04%1.0, x*.05%1.0)
glVertex3fv((x, 0, 100))
glVertex3fv((x, 0, 100))
glVertex3fv((100, 0, x))
glVertex3fv((100, 0, x))
glEnd()
def main():
"run the demo"
#initialize pygame and setup an opengl display
pygame.init()
pygame.display.set_mode((1024,768), OPENGL|DOUBLEBUF)
glEnable(GL_DEPTH_TEST) #use our zbuffer
#setup the camera
glMatrixMode(GL_PROJECTION)
gluPerspective(45.0,1024/768.0,0.1,100.0) #setup lens
glTranslatef(0.0, 0.0, 20.0) #move back
glRotatef(60, 1, 0, 0) #orbit higher
nt = int(time.time() * 1000)
for i in range(2**63):
nt += 20
glTranslatef(0.0, 0.0, .1)
#check for quit'n events
event = pygame.event.poll()
if event.type == QUIT or (event.type == KEYDOWN and event.key == K_ESCAPE):
break
#clear screen and move camera
glClear(GL_COLOR_BUFFER_BIT|GL_DEPTH_BUFFER_BIT)
#orbit camera around by 1 degree
glRotatef(1, 0, 1, 0)
drawcube()
pygame.display.flip()
ct = int(time.time() * 1000)
pygame.time.wait(max(1,nt ct))
if i % 50 == 0:
print(ntct)
if __name__ == '__main__': main()

view raw
myglcube.py
hosted with ❤ by GitHub

 

 

Playing with Turtle XYZ

3D Rendering has always interested me, but I’ve never taken the time to mess around with it much.

When I was a kid I used to do stuff like this in Basic.  But I didn’t know trig or other similar functions so I was left to basic math and the random functions.

Here is a little turtle program (python) that will make a wrinkled fabric type display.

2015-12-18 - B163539.png

Programming

Hi, this is Eli.  One of our school assignments is to completely learn our math tables by heart.   We wrote a python program to help us practice with this.  This is a program for multiplication tables 1-12.

9-10-2014 9-31-13 PM

Ezra and I are learning how to program in python.  Dad helped us get the structure of this program right, but we entered into IDLE and debugged it ourselves.

Here is an example of the program in action:

9-10-2014 9-36-40 PM

When you activate the program it  asks you how many problems you want – you can go from 1 to 144 problems.

When you are done with your problems it tells you how long you took and how many problems you got right and how many problems you got right and how long it took you to answer each problem.

That’s all for now!

Python: ‘tuple’ object is not callable

This can be a bit of an obscure error, if you run into it…  It looks like this:

File ".../CCRM/Content.py", line 202, in Page_Update
    ('Nav1'       , Data.Nav1),
TypeError: 'tuple' object is not callable

In reality, it’s typically caused by accidentally forgetting a comma from the line before:

    Page_MNID = App.DB.Value('''
      UPDATE
        "Dashboard"."Page"
      SET
        [Field=Value]
      WHERE True
        AND "Page_MNID" = $Page_MNID
      ''',
      ('ScriptPath' , Data.ScriptPath)
      ('Nav1'       , Data.Nav1),
      ('Nav2_Icon'  , Data.Nav2_Icon),
      ('Nav2_Label' , Data.Nav2_Label),
      ('Title'      , Data.Title),
      ('Active'     , Data.Active),
      Page_MNID     = Data.Page_MNID,
      )

Notice that line 9 is missing a comma at the end?  That causes python to see this:

tuple_object = ("ScriptPath", Data.ScriptPath")
tuple_object("Nav1" , Data.Nav1)  #eg, next tuple looks like params

Solution?  Just add the comma :)