Module xg
[hide private]
[frames] | no frames]

Module xg

source code

PyXG provides a Python interface to Apple's Xgrid.

Xgrid is Apple's software for building and managing clusters of Macintosh computers for use in high performance computation. See http://www.apple.com/server/macosx/technology/xgrid.html for more details.

This module wraps the xgrid command line in Mac OS X. It will not work with the Technonogy Previews of Xgrid. The command line is wrapped in this module as the goal is to provide an interface to Xgrid that can be used from an interactive Python prompt. The Cocoa API for Xgrid (XgridFoundation) is based on an event-loop paradigm and is less well suited for interactive work. If you want to use Xgrid and Python from within a Cocoa application, you should use XgridFoundation and PyObjC.

Features

  1. Use Xgrid from within Python scripts as well as in interactive Python sessions.
  2. Create, submit and manage simple (one task) and batch (many task) Xgrid jobs using Python's elegant syntax.
  3. Work with multiple Xgrid controllers simultaneouly.
  4. List available grids for each controller and query their status.

Quick Start

Import xg, create a Connection and Controller object:

>>> from xg import *
>>> conn = Connection(hostname='xgrid.work.com',password='secret')
>>> cont = Controller(conn)

List the grids managed by the controller:

>>> cont.gridIDs()
(0, 3)
>>> cont.grids()
[<Grid with gridID = 0>, <Grid with gridID = 3>]

Work with the default grid, listing active jobs:

>>> g = cont.grid(0)
>>> g.jobIDs()
(229, 230, 231, 232)
>>> g.printJobs()
##################################################
id   Date Started             Status     CPU Power 
##################################################
229  2005-12-22 11:18:47 -0800 Finished   0         
230  2005-12-22 11:18:50 -0800 Finished   0         
231  2005-12-22 11:18:52 -0800 Finished   0         
232  2005-12-22 11:18:55 -0800 Finished   0

Get a job from the default grid and work with it:

>>> j = g.job(229)      
>>> j.printInfo()
{
    name:  /usr/bin/cal
    jobStatus:  Finished
    taskCount:  1
    undoneTaskCount:  0
    percentDone:  100
}
>>> j.printInfo(verbose=False)
229  2005-12-22 11:18:47 -0800 Finished   0   
>>> j.printSpecification()
{
    applicationIdentifier :  com.apple.xgrid.cli
    taskSpecifications :  {0 = {arguments = (); command = "/usr/bin/cal"; }; }
    name :  /usr/bin/cal
    inputFiles :  {}
    submissionIdentifier :  abc
}

Get job results:

>>> j.results(stdout="job229.out",stderr="job229.err")
Job stdout saved in file: job229.out

Use a Grid object to submit a single task job:

>>> j = g.submit(cmd='/usr/bin/cal')
Job submitted with id:  234
>>> j.printInfo(verbose=False)
234  2005-12-22 13:09:52 -0800 Finished   0
Classes [hide private]
  XgridError
Xgrid exception class.
  InvalidIdentifier
Xgrid exception for invalid job or grid identifiers.
  InvalidGridIdentifier
Invalid grid identifier exception.
  InvalidJobIdentifier
Invalid job identifier exception.
  InvalidAction
Invalid action exception.
  InvalidIdentifierType
Invalid job or grid identifier type.
  Connection
Track information needed to connect to an Xgrid controller.
  JobManager
Manage a set of Xgrid jobs.
  GridManager
Manage the grids of a given Xgrid controller.
  Controller
A class for working with an Xgrid controller.
  Grid
A class for working with jobs on a specific Xgrid grid.
  Job
A class for working with an Xgrid job.
  JobSpecification
A class used for constructing multi-task batch jobs.
Functions [hide private]
 
autorelease(func)
A decorator to properly release ObjC object instances.
source code
 
xgridParse(cmd='xgrid -grid list')
Submits and parses output from the xgrid command line.
source code
 
processID(id)
Makes sure that the id is a unicode string
source code
float
detectPlatform()
Detect the version of Mac OS.
source code
Variables [hide private]
  PYXGRID_DEBUG = False
  VERSION = '0.3.0'
  defaultXgridHostname = None
  defaultXgridPassword = ''
  __package__ = None
Function Details [hide private]

autorelease(func)

source code 

A decorator to properly release ObjC object instances.

Anytime an ObjC object instance is created, an NSAutoReleasePool needs to be available. PyObjC will create one but it won't get drained very often in Mac OS X Leopard (10.5) and earlier. This is especially a problem if many PyXG jobs are created within a loop. In order to prevent memory leaking, the call(s) to PyXG within the loop should be decorated with this decorator.

Parameters:
  • func - The function to be decorated.

xgridParse(cmd='xgrid -grid list')

source code 

Submits and parses output from the xgrid command line.

The output of the xgrid CLI is a (sometimes illegal) old-style plist. This function runs an xgrid command and parses the output of the command into a valid NSDictionary (a Python dict).

To handle the illegal plists returned by the xgrid CLI, we use the xGridPropertyList: method of NSString (defined above).

See the xgrid man pages for details on the xgrid command. This fuction will return a nested Python structure that reflects the output of the xgrid command.

detectPlatform()

source code 

Detect the version of Mac OS.

Returns: float
Platform version