py.path

Note: The ‘py’ library is in “maintenance mode” and so is not recommended for new projects. Please check out pathlib or pathlib2 for path operations.

The ‘py’ lib provides a uniform high-level api to deal with filesystems and filesystem-like interfaces: py.path. It aims to offer a central object to fs-like object trees (reading from and writing to files, adding files/directories, examining the types and structure, etc.), and out-of-the-box provides a number of implementations of this API.

py.path.local - local file system path

basic interactive example

The first and most obvious of the implementations is a wrapper around a local filesystem. It’s just a bit nicer in usage than the regular Python APIs, and of course all the functionality is bundled together rather than spread over a number of modules.

>>> import py
>>> temppath = py.path.local('py.path_documentation')
>>> foopath = temppath.join('foo') # get child 'foo' (lazily)
>>> foopath.check() # check if child 'foo' exists
False
>>> foopath.write('bar') # write some data to it
>>> foopath.check()
True
>>> foopath.read()
'bar'
>>> foofile = foopath.open() # return a 'real' file object
>>> foofile.read(1)
'b'

reference documentation

class py._path.local.LocalPath(path=None, expanduser=False)[source]

object oriented interface to os.path and other local filesystem related information.

exception ImportMismatchError[source]

raised on pyimport() if there is a mismatch of __file__’s

msg

exception message

name

module name

path

module path

with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

samefile(other)[source]

return True if ‘other’ references the same file as ‘self’.

remove(rec=1, ignore_errors=False)[source]

remove a file or directory (or a directory tree if rec=1). if ignore_errors is True, errors while removing directories will be ignored.

computehash(hashtype='md5', chunksize=524288)[source]

return hexdigest of hashvalue for this file.

new(**kw)[source]

create a modified version of this path. the following keyword arguments modify various path parts:

a:/some/path/to/a/file.ext
xx                           drive
xxxxxxxxxxxxxxxxx            dirname
                  xxxxxxxx   basename
                  xxxx       purebasename
                       xxx   ext
dirpath(*args, **kwargs)[source]

return the directory path joined with any given path arguments.

join(*args, **kwargs)[source]

return a new path by appending all ‘args’ as path components. if abs=1 is used restart from root if any of the args is an absolute path.

open(mode='r', ensure=False, encoding=None)[source]

return an opened file with the given mode.

If ensure is True, create parent directories if needed.

check(**kw)[source]

check a path for existence and properties.

Without arguments, return True if the path exists, otherwise False.

valid checkers:

file=1    # is a file
file=0    # is not a file (may not even exist)
dir=1     # is a dir
link=1    # is a link
exists=1  # exists

You can specify multiple checker definitions, for example:

path.check(file=1, link=1)  # a link pointing to a file
listdir(fil=None, sort=None)[source]

list directory contents, possibly filter by the given fil func and possibly sorted.

size()[source]

return size of the underlying file object

mtime()[source]

return last modification time of the path.

copy(target, mode=False, stat=False)[source]

copy path to target.

If mode is True, will copy copy permission from path to target. If stat is True, copy permission, last modification time, last access time, and flags from path to target.

rename(target)[source]

rename this path to target.

dump(obj, bin=1)[source]

pickle object into path location

mkdir(*args)[source]

create & return the directory joined with args.

write_binary(data, ensure=False)[source]

write binary data into path. If ensure is True create missing parent directories.

write_text(data, encoding, ensure=False)[source]

write text data into path using the specified encoding. If ensure is True create missing parent directories.

write(data, mode='w', ensure=False)[source]

write data into path. If ensure is True create missing parent directories.

ensure(*args, **kwargs)[source]

ensure that an args-joined path exists (by default as a file). if you specify a keyword argument ‘dir=True’ then the path is forced to be a directory path.

stat(raising=True)[source]

Return an os.stat() tuple.

lstat()[source]

Return an os.lstat() tuple.

setmtime(mtime=None)[source]

set modification time for the given path. if ‘mtime’ is None (the default) then the file’s mtime is set to current time.

Note that the resolution for ‘mtime’ is platform dependent.

chdir()[source]

change directory to self and return old current directory

as_cwd()[source]

Return a context manager, which changes to the path’s dir during the managed “with” context. On __enter__ it returns the old dir, which might be None.

realpath()[source]

return a new path which contains no symbolic links.

atime()[source]

return last access time of the path.

chmod(mode, rec=0)[source]

change permissions to the given mode. If mode is an integer it directly encodes the os-specific modes. if rec is True perform recursively.

pypkgpath()[source]

return the Python package path by looking for the last directory upwards which still contains an __init__.py. Return None if a pkgpath can not be determined.

pyimport(modname=None, ensuresyspath=True)[source]

return path as an imported python module.

If modname is None, look for the containing package and construct an according module name. The module will be put/looked up in sys.modules. if ensuresyspath is True then the root dir for importing the file (taking __init__.py files into account) will be prepended to sys.path if it isn’t there already. If ensuresyspath==”append” the root dir will be appended if it isn’t already contained in sys.path. if ensuresyspath is False no modification of syspath happens.

Special value of ensuresyspath==”importlib” is intended purely for using in pytest, it is capable only of importing separate .py files outside packages, e.g. for test suite without any __init__.py file. It effectively allows having same-named test modules in different places and offers mild opt-in via this option. Note that it works only in recent versions of python.

sysexec(*argv, **popen_opts)[source]

return stdout text from executing a system child process, where the ‘self’ path points to executable. The process is directly invoked and not through a system shell.

classmethod sysfind(name, checker=None, paths=None)[source]

return a path object found by looking at the systems underlying PATH specification. If the checker is not None it will be invoked to filter matching paths. If a binary cannot be found, None is returned Note: This is probably not working on plain win32 systems but may work on cygwin.

classmethod get_temproot()[source]

return the system’s temporary directory (where tempfiles are usually created in)

classmethod mkdtemp(rootdir=None)[source]

return a Path object pointing to a fresh new temporary directory (which we created ourself).

basename

basename part of path.

bestrelpath(dest)

return a string which is a relative path from self (assumed to be a directory) to dest such that self.join(bestrelpath) == dest and if not such path can be determined return dest.

chown(user, group, rec=0)

change ownership to the given user and group. user and group may be specified by a number or by a name. if rec is True change ownership recursively.

common(other)

return the common part shared with the other path or None if there is no common part.

dirname

dirname part of path.

ensure_dir(*args)

ensure the path joined with args is a directory.

ext

extension of the path (including the ‘.’).

fnmatch(pattern)

return true if the basename/fullname matches the glob-‘pattern’.

valid pattern characters:

*       matches everything
?       matches any single character
[seq]   matches any character in seq
[!seq]  matches any char not in seq

If the pattern contains a path-separator then the full path is used for pattern matching and a ‘*’ is prepended to the pattern.

if the pattern doesn’t contain a path-separator the pattern is only matched against the basename.

load()

(deprecated) return object unpickled from self.read()

classmethod make_numbered_dir(prefix='session-', rootdir=None, keep=3, lock_timeout=172800)[source]

return unique directory with a number greater than the current maximum one. The number is assumed to start directly after prefix. if keep is true directories with a number less than (maxnum-keep) will be removed. If .lock files are used (lock_timeout non-zero), algorithm is multi-process safe.

mklinkto(oldname)

posix style hard link to another name.

mksymlinkto(value, absolute=1)

create a symbolic link with the given value (pointing to another name).

move(target)

move this path to target.

parts(reverse=False)

return a root-first list of all ancestor directories plus the path itself.

purebasename

pure base name of the path.

read(mode='r')

read and return a bytestring from reading the path.

read_binary()

read and return a bytestring from reading the path.

read_text(encoding)

read and return a Unicode string from reading the path.

readlines(cr=1)

read and return a list of lines from the path. if cr is False, the newline will be removed from the end of each line.

return value of a symbolic link.

relto(relpath)

return a string which is the relative part of the path to the given ‘relpath’.

visit(fil=None, rec=None, ignore=<class 'py._path.common.NeverRaised'>, bf=False, sort=False)

yields all paths below the current one

fil is a filter (glob pattern or callable), if not matching the path will not be yielded, defaulting to None (everything is returned)

rec is a filter (glob pattern or callable) that controls whether a node is descended, defaulting to None

ignore is an Exception class that is ignoredwhen calling dirlist() on any of the paths (by default, all exceptions are reported)

bf if True will cause a breadthfirst search instead of the default depthfirst. Default: False

sort if True will sort entries within each directory level.

py.path.svnurl and py.path.svnwc

Two other py.path implementations that the py lib provides wrap the popular Subversion revision control system: the first (called ‘svnurl’) by interfacing with a remote server, the second by wrapping a local checkout. Both allow you to access relatively advanced features such as metadata and versioning, and both in a way more user-friendly manner than existing other solutions.

Some example usage of py.path.svnurl:

.. >>> import py
.. >>> if not py.test.config.option.urlcheck: raise ValueError('skipchunk')
>>> url = py.path.svnurl('http://codespeak.net/svn/py')
>>> info = url.info()
>>> info.kind
'dir'
>>> firstentry = url.log()[-1]
>>> import time
>>> time.strftime('%Y-%m-%d', time.gmtime(firstentry.date))
'2004-10-02'

Example usage of py.path.svnwc:

.. >>> if not py.test.config.option.urlcheck: raise ValueError('skipchunk')
>>> temp = py.path.local('py.path_documentation')
>>> wc = py.path.svnwc(temp.join('svnwc'))
>>> wc.checkout('http://codespeak.net/svn/py/dist/py/path/local')
>>> wc.join('local.py').check()
True

Common vs. specific API, Examples

All Path objects support a common set of operations, suitable for many use cases and allowing to transparently switch the path object within an application (e.g. from “local” to “svnwc”). The common set includes functions such as path.read() to read all data from a file, path.write() to write data, path.listdir() to get a list of directory entries, path.check() to check if a node exists and is of a particular type, path.join() to get to a (grand)child, path.visit() to recursively walk through a node’s children, etc. Only things that are not common on ‘normal’ filesystems (yet), such as handling metadata (e.g. the Subversion “properties”) require using specific APIs.

A quick ‘cookbook’ of small examples that will be useful ‘in real life’, which also presents parts of the ‘common’ API, and shows some non-common methods:

Searching .txt files

Search for a particular string inside all files with a .txt extension in a specific directory.

>>> dirpath = temppath.ensure('testdir', dir=True)
>>> dirpath.join('textfile1.txt').write('foo bar baz')
>>> dirpath.join('textfile2.txt').write('frob bar spam eggs')
>>> subdir = dirpath.ensure('subdir', dir=True)
>>> subdir.join('textfile1.txt').write('foo baz')
>>> subdir.join('textfile2.txt').write('spam eggs spam foo bar spam')
>>> results = []
>>> for fpath in dirpath.visit('*.txt'):
...     if 'bar' in fpath.read():
...         results.append(fpath.basename)
>>> results.sort()
>>> results
['textfile1.txt', 'textfile2.txt', 'textfile2.txt']

Working with Paths

This example shows the py.path features to deal with filesystem paths Note that the filesystem is never touched, all operations are performed on a string level (so the paths don’t have to exist, either):

>>> p1 = py.path.local('/foo/bar')
>>> p2 = p1.join('baz/qux')
>>> p2 == py.path.local('/foo/bar/baz/qux')
True
>>> sep = py.path.local.sep
>>> p2.relto(p1).replace(sep, '/') # os-specific path sep in the string
'baz/qux'
>>> p2.bestrelpath(p1).replace(sep, '/')
'../..'
>>> p2.join(p2.bestrelpath(p1)) == p1
True
>>> p3 = p1 / 'baz/qux' # the / operator allows joining, too
>>> p2 == p3
True
>>> p4 = p1 + ".py"
>>> p4.basename == "bar.py"
True
>>> p4.ext == ".py"
True
>>> p4.purebasename == "bar"
True

This should be possible on every implementation of py.path, so regardless of whether the implementation wraps a UNIX filesystem, a Windows one, or a database or object tree, these functions should be available (each with their own notion of path seperators and dealing with conversions, etc.).

Checking path types

Now we will show a bit about the powerful ‘check()’ method on paths, which allows you to check whether a file exists, what type it is, etc.:

>>> file1 = temppath.join('file1')
>>> file1.check() # does it exist?
False
>>> file1 = file1.ensure(file=True) # 'touch' the file
>>> file1.check()
True
>>> file1.check(dir=True) # is it a dir?
False
>>> file1.check(file=True) # or a file?
True
>>> file1.check(ext='.txt') # check the extension
False
>>> textfile = temppath.ensure('text.txt', file=True)
>>> textfile.check(ext='.txt')
True
>>> file1.check(basename='file1') # we can use all the path's properties here
True

Setting svn-properties

As an example of ‘uncommon’ methods, we’ll show how to read and write properties in an py.path.svnwc instance:

.. >>> if not py.test.config.option.urlcheck: raise ValueError('skipchunk')
>>> wc.propget('foo')
''
>>> wc.propset('foo', 'bar')
>>> wc.propget('foo')
'bar'
>>> len(wc.status().prop_modified) # our own props
1
>>> msg = wc.revert() # roll back our changes
>>> len(wc.status().prop_modified)
0

SVN authentication

Some uncommon functionality can also be provided as extensions, such as SVN authentication:

.. >>> if not py.test.config.option.urlcheck: raise ValueError('skipchunk')
>>> auth = py.path.SvnAuth('anonymous', 'user', cache_auth=False,
...             interactive=False)
>>> wc.auth = auth
>>> wc.update() # this should work
>>> path = wc.ensure('thisshouldnotexist.txt')
>>> try:
...     path.commit('testing')
... except py.process.cmdexec.Error, e:
...     pass
>>> 'authorization failed' in str(e)
True

Known problems / limitations

  • The SVN path objects require the “svn” command line, there is currently no support for python bindings. Parsing the svn output can lead to problems, particularly regarding if you have a non-english “locales” setting.
  • While the path objects basically work on windows, there is no attention yet on making unicode paths work or deal with the famous “8.3” filename issues.