July 11, 2018 by Jason Garber

Example of Python Generator Function.

One of the great features of Python is Generator Functions. Generator functions allow you to convert any function to a generator function by simply including the yield keyword somewhere in the function body. When a generator function is called, the response is a generator object, which can be iterated over among other things.

To boil this down to basics, generator functions allow you the programmer to create a function which yields one value at a time (and pauses until the next value is requested) until you decide it is done. This opens endless possibilities for converting sequences, creating sequences, filtering, and more.

The following example is a generator function which will take a simple list or iterator and return pairs of (element, previous_element). This is a great use for generators and the yield statement.

Generator Function:

def lineandlast(listish, first=True, last=True):
    iterator = iter(listish)
    lastline = next(iterator)

    if first:
      yield lastline, None

    for line in iterator:
        yield line, lastline
        lastline = line

    if last:
        yield None, lastline

Example:

for line, last in lineandlast([1,2,3,4,5]):
    print(line, last)

Output:

1 None
2 1
3 2
4 3
5 4
None 5

Notes:
There are two keyword arguments, first and last, which can be used to control the output of the first and last items on the output example above.

September 12, 2014 by eligarber

Programming

Hi, this is Eli. One of our school assignments is to completely learn our math tables by heart. We wrote a python program to help us practice with this. This is a program for multiplication tables 1-12.

Ezra and I are learning how to program in python. Dad helped us get the structure of this program right, but we entered into IDLE and debugged it ourselves.

Here is an example of the program in action:

When you activate the program it asks you how many problems you want – you can go from 1 to 144 problems.

When you are done with your problems it tells you how long you took and how many problems you got right and how many problems you got right and how long it took you to answer each problem.

That’s all for now!

March 4, 2013 by Jason Garber

Introducing FileStruct (for Python)

FileStruct is a lightweight and fast file-cache / file-server designed for web-applications. It solves the problems of “where do I save all of those uploads” that has been encountered time and time again. FileStruct uses the local filesystem, but in a sensible way (keeping permissions sane), and with the ability to secure it to a reasonable level.

https://github.com/appcove/FileStruct/

Here is a simple example of taking an image upload, resizing, and saving it:

with client.TempDir() as TempDir:
   open(TempDir.FilePath('upload.jpg'), 'wb').write(mydata)
   TempDir.ResizeImage('upload.jpg', 'resize.jpg', '100x100')
   hash1 = TempDir.Save('upload.jpg')
   hash2 = TempDir.Save('resize.jpg')

Design Goals

Immutable Files

FileStruct is designed to work with files represented by the SHA-1 hash of their contents. This means that all files in FileStruct are immutable.

High Performance

FileStruct is designed as a local repository of file data accessable (read/write) by an application or web application. All operations are local I/O operations and therefore, very fast.

Where possible, streaming hash functions are used to prevent iterating over a file twice.

Direct serving from Nginx

FileStruct is designed so that Nginx can serve files directly from it’s Data directory using an X-Accel-Redirect header. For more information on this Nginx configuration directive, see http://wiki.nginx.org/XSendfile

Assuming that nginx runs under nginx user and file database is owned by the fileserver group, nginx needs to be in thefileserver group to serve files:

# usermod -a -G fileserver nginx

Secure

FileStruct is designed to be as secure as your hosting configuration. Where possible, a dedicated user should be allocated to read/write to FileStruct, and the database directory restricted to this user.

Simple

FileStruct is designed to be incredibly simple to use.

File Manipulaion

FileStruct is designed to simplify common operations on files, especially uploaded files. Image resizing for thumbnails is supported.

Temporary File Management

FileStruct is designed to simplify the use of Temp Files in an application. The API supports creation of a temporary directory, placing files in it, Ingesting files into FileStruct, and deleting the directory when completed (or retaining it in the event of an error)

Garbage Collection

FileStruct is designed to retain files until garbage collection is performed. Garbage collection consists of telling FileStruct what files you are interested in keeping, and having it move the remaining files to the trash.

Backup and Sync with Rsync

FileStruct is designed to work seamlessly with rsync for backups and restores.

Atomic operations

At the point a file is inserted or removed from FileStruct, it is a filesystem move operation. This means that under no circumstances will a file exist in FileStruct that has contents that do not match the name of the file.

No MetaData

FileStruct is not designed to store MetaData. It is designed to store file content. There may be several “files” which refer to the same content. empty.log, empty.txt, and empty.ini may all refer to the empty fileData/da/39/da39a3ee5e6b4b0d3255bfef95601890afd80709. However, this file will be retained as long as any aspect of the application still uses it.

Automatic De-Duplication

Because file content is stored in files with the hash of the content, automatic file-level de-duplication occurs. When a file is pushed to FileStruct that already exists, there is no need to write it again.

This carries the distinct benifit of being able to use the same FileStruct database across multiple projects if desired, because the content of file Data/da/39/da39a3ee5e6b4b0d3255bfef95601890afd80709 is always the same, regardless of the application that placed it there.

Note: In the event that multiple instances or applications use the same database, the garbage collection routine MUST take all references to a given hash into account, across all applications that use the database. Otherwise, it would be easy to delete data that should be retained.

October 18, 2012 by Jason Garber

Simulating ENUM in PostgreSQL using CHECK expression

PostgreSQL is a very powerful database. One of the things that seems missing when moving from MySQL is the ability to simply create an enumeration. ENUM is nice when you have a programmatically-semantic set of values for a field.

In PostgreSQL, you have several choices. But one simple one is to create a Check expression, like follows. Skip the IS NULL part if you don’t want the field nullable.

ALTER TABLE 
   "public"."CallCenter_Transfer"
ADD CONSTRAINT 
   "CallCenter_Transfer_TransferStatus"
   CHECK (
      "TransferStatus" IS NULL 
       OR 
      "TransferStatus" = ANY(ARRAY['TransferComplete', 'TransferFailedNoAnswer', 'TransferFailedProspectLost'])
      )

June 27, 2012 by Jason Garber

Install Root Certificate Authority (CA) on Windows XP

As a developer, it is great to be able to provide my own SSL certificates for preview and development sites. To this end, I’ve created my own Certificate Authority (CA), and need to import it into my computer and any client computer in order to avoid the scary SSL Certificate Warnings.

Applies to Windows XP. Not sure about Windows 7.

September 25, 2010 by Jason Garber

Python: ‘tuple’ object is not callable

This can be a bit of an obscure error, if you run into it… It looks like this:

File ".../CCRM/Content.py", line 202, in Page_Update
    ('Nav1'       , Data.Nav1),
TypeError: 'tuple' object is not callable

In reality, it’s typically caused by accidentally forgetting a comma from the line before:

    Page_MNID = App.DB.Value('''
      UPDATE
        "Dashboard"."Page"
      SET
        [Field=Value]
      WHERE True
        AND "Page_MNID" = $Page_MNID
      ''',
      ('ScriptPath' , Data.ScriptPath)
      ('Nav1'       , Data.Nav1),
      ('Nav2_Icon'  , Data.Nav2_Icon),
      ('Nav2_Label' , Data.Nav2_Label),
      ('Title'      , Data.Title),
      ('Active'     , Data.Active),
      Page_MNID     = Data.Page_MNID,
      )

Notice that line 9 is missing a comma at the end? That causes python to see this:

tuple_object = ("ScriptPath", Data.ScriptPath")
tuple_object("Nav1" , Data.Nav1)  #eg, next tuple looks like params

Solution? Just add the comma :)

December 8, 2009 by Jason Garber

Python has keyword only parameters!

Maybe I should have known this about Python by now, but…

Little did I know (was wishing for it today), but there IS a way to specify that arguments MUST be keyword only.

>>> def foo(a,b,*,c,d):
...     print(a,b,c,d)
...
>>> foo(1,2,3,4)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: foo() takes exactly 2 positional arguments (4 given)

In this case, everything after the * is required to be keyword only. This means that you can safely amend the function definition to re-order or even add keyword arguments.

Likewise, the following requires ALL keyword arguments.

>>> def foo(*,a,b):
...     print(a,b)
...
>>> foo(1,2)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: foo() takes exactly 0 positional arguments (2 given)

>>> foo(a=1, b=2)
1 2

The Gahooa Perspective

Thoughts on Life, Engineering, Technology, Business, and more…

programming