API Documentation

These are the major NiceLib classes and functions of which you should know:

Header Processing API

nicelib.build_lib(header_info, lib_name, module_name, filedir, ignored_headers=(), ignore_system_headers=False, preamble=None, token_hooks=(), ast_hooks=(), hook_groups=(), debug_file=None, logbuf=None, load_dump_file=False, save_dump_file=False, pack=None, override=False)

Build a low-level Python wrapper of a C lib

Parameters
header_infostr, dict, or None

Path to header file. If None, uses only the header source given by preamble.

Paths can use os.environ, as described below. Info is provided in the form of a dict which must contain a ‘header’ key whose value is either a str containing a single header name, or a tuple of such strings.

There are also some other optional entries:

The 'path' value must be a str or tuple of strs that are directories which will be searched for the given header(s). Any headers that are specified with a leading slash are considered absolute and are not affected by this path.

The 'predef' value is a str which is the name or path of a header file which will be used to populate the predefined preprocessor macros that are ordinarily provided by the compiler on a per-system basis. If provided, this overrides the default header that NiceLib uses for your system.

lib_namestr or dict

Name of compiled library file, e.g. 'mylib.dll'

module_namestr

Name of module to create. Must be in the format '_*lib', e.g. '_mylib'

filedirstr

Path indicating the directory where the generated module will be saved. If filedir points to an existing file, that file’s directory is used. Usually you would pass the __file__ attribute from your build module.

ignored_headerssequence of strs

Names of headers to ignore; #includes containing these will be skipped.

ignore_system_headersbool

If True, skip inclusion of headers specified with angle brackets, e.g. #include <stdarg.h> Header files specified with double quotes are processed as ususal. Default is False.

preamblestr

C source to insert before the headers specified by header_info. Useful for including typedefs that would otherwise be hard to reach due to an end user missing headers.

token_hookssequence of functions

Token hook functions. See process_headers() for more info.

ast_hookssequence of functions

AST hook functions. See process_headers() for more info.

hook_groupsstr or sequence of strs

Hook groups. See process_headers() for more info.

debug_filestr

File to write a partially-processed header to just before it is parsed by pycparser. Useful for debugging the preprocessor when pycparser’s parser chokes on its output.

logbufwriteable buffer

IO buffer to write() common log output to. By default this output will logged using the logging stdlib module, at the info log level. You can use sys.stdout to perform ordinary printing.

load_dump_filebool

Save the list of tokens resulting from preprocessing to ‘token_dump.pkl’. See save_dump_file for more info.

save_dump_filebool

Ignore header_paths and load the already-preprocessed tokens from ‘token_dump.pkl’. This can significantly speed up your turnaround time when debugging really large header sets when writing and debugging hooks.

pack: int

Forwared to FFI.cdef. Controls the packing of structs if necessary. This has to be done manually since CFFI ignores #pragma pack and gcc directives. See the CFFI documentation for more information.

override: bool

Forwarded to FFI.cdef. If True, allows repeated declarations; the final declaration will override any others. Otherwise, repeated declarations are treated as an error.

Notes

header_info and lib_name can each be a dict that maps from a platform to the corresponding path or name, allowing cross-platform support. The keys are matched against sys.platform and can use globbing, i.e. 'linux*' will match anything starting with 'linux'.

The path or paths provided by header_info may use items in os.environ. For example, '{PROGRAMFILES}\\header.h' will be formatted with os.environ['PROGRAMFILES'].

nicelib.process.process_headers(header_paths, predef_path=None, update_cb=None, ignored_headers=(), ignore_system_headers=False, debug_file=None, preamble=None, token_hooks=(), ast_hooks=(), hook_groups=(), return_ast=False, load_dump_file=False, save_dump_file=False)

Preprocess header(s) and split into a cleaned header and macros

Parameters
header_pathsstr or sequence of strs

Paths of the headers

ignored_headerssequence of strs

Names of headers to ignore; #includes containing these will be skipped.

ignore_system_headersbool

If True, skip inclusion of headers specified with angle brackets, e.g. #include <stdarg.h> Header files specified with double quotes are processed as ususal. Default is False.

debug_filestr

File to write a partially-processed header to just before it is parsed by pycparser. Useful for debugging the preprocessor when pycparser’s parser chokes on its output.

preamblestr

C source to insert before the headers specified by header_paths. Useful for including typedefs that would otherwise be hard to reach due to an end user missing headers. token_hooks : sequence of functions Hook functions to be run on the already preprocessed token stream. Each function should accept and return a sequence of Tokens. These are applied after any builtin token hooks.

ast_hookssequence of functions

Hook functions to be run on chunks of the header’s C AST. After preprocessing and running the token hooks, the tokens are grouped and joined to form a sequence of chunks called “external declarations” (declarations, typedefs, and function definitions). Each chunk is parsed by pycparser, then passed through the list of AST hook functions to transform it. These are applied after any builtin AST hooks.

AST hook functions take the parsed FileAST and the persistent CParser instance as arguments, and return a transformed FileAST. It is useful to have a reference to the parser to add phony typedefs if necessary.

hook_groupsstr or sequence of strs

Predefined hook groups to use. Each hook group enables certain builtin hooks that are commonly used together. The only hook group available for now is ‘C++’.

‘C++’(declspec_hook, extern_c_hook, enum_type_hook, CPPTypedefAdder)

Hooks for converting C++-only headers into C syntax understandable by cffi.

load_dump_filebool

Save the list of tokens resulting from preprocessing to ‘token_dump.pkl’. See save_dump_file for more info.

save_dump_filebool

Ignore header_paths and load the already-preprocessed tokens from ‘token_dump.pkl’. This can significantly speed up your turnaround time when debugging really large header sets when writing and debugging hooks.

Returns
header_srcstr

Cleaned header C-code.

macro_rcstr

Extracted macros expressed as Python source code.

Token Hooks

These functions can all be used in the token_hooks passed to build_lib() or process_headers()

nicelib.process.cdecl_hook(tokens)

Removes cdecl, _cdecl, and __cdecl

Enabled by default.

nicelib.process.stdcall_hook(tokens)

Replace __stdcall and WINAPI with volatile volatile const

Enabled by default.

This technique is stolen from cffi.

nicelib.process.declspec_hook(tokens)

Removes all occurences of __declspec(...)

nicelib.process.inline_hook(tokens)

Removes _inline, __inline, and __forceinline

nicelib.process.extern_c_hook(tokens)

Removes extern "C" { ... } while keeping the block’s contents

nicelib.process.enum_type_hook(tokens)
nicelib.process.asm_hook(tokens)

Remove _asm and __asm and their blocks

nicelib.process.vc_pragma_hook(tokens)

Remove __pragma(...)

nicelib.process.struct_func_hook(tokens)

Removes function definitions from inside struct definitions.

Once we’re inside the struct’s curly braces, we look member-by-member. We push each next token into a buffer, if we see an open brace then we assume this member is a funcdef. The funcdef is over after the matching close brace and an optional semicolon. Otherwise, if we see a semicolon outside of any nesting, that’s the end of an ordinary member declaration.

NOTE: Does not deal with nested structs that have methods (if that’s even allowed).

If we see a funcdef, throw away all its tokens. If we see a member declaration, pass the tokens on.

nicelib.process.add_line_directive_hook(tokens)

Adds line directives to indicate where each token came from

Enabled by default.

Token Hook Helpers

These functions and classes are useful for writing your own custom token hooks:

class nicelib.process.TokenType(value)

Enum of token types for C Preprocessor

BLOCK_COMMENT = 11

Block comment (inside /* */)

CHAR_CONST = 5

Char literal (e.g. 'c')

DEFINED = 1

‘defined’

HEADER_NAME = 6

Header name (inside <angle brackets>)

IDENTIFIER = 2

Identifier

LINE_COMMENT = 10

Line comment (starts with //)

NEWLINE = 8

Newline char

NUMBER = 3

Numeric literal

PUNCTUATOR = 7

Punctuation token

STRING_CONST = 4

String literal (e.g. "my string")

WHITESPACE = 9

Non-newline whitespace

nicelib.process.remove_pattern(tokens, pattern)

Convenience function that works like modify_pattern, but you only specify targets

nicelib.process.modify_pattern(tokens, pattern)

Helper function for easily matching and modifying patterns in a token stream

Parameters
tokenssequence of Tokens

Input token stream to process

patternsequence of tuples

Pattern to match, as a sequence of (keep, target) pairs. Targets are compared against tokens; when the first target is matched, we then try matching the rest in order. If the whole pattern is matched, each target’s corresponding keep indicates whether the token should be kept in the sequence, or discarded. keep is a string, where ‘k’ means ‘keep’, ‘d’ means discard and ‘a’ means add.

A target can be either a string or a TokenType.

There is also some functionality for dealing with blocks enclosed in curly braces {}. For more advanced functionality, check out ParseHelper.

Passing a sequence of the type ((keep_start, '{'), ('keep_end', '~~}~~')) will keep the opening { according to keep_start. keep_end is a two letter string, where the first letter indicates whether the contents of the block enclosed in braces should be kept, and the second indicates if the closing } should be kept.

class nicelib.process.ParseHelper(tokens)

Helper class for processing token streams

Allows you to easily read until specific tokens or the ends of blocks, etc.

__init__(tokens)
peek()

Peek at the next token

Returns None at the end of the token stream.

peek_true_token()

Peek at next true token

“True” tokens are non-whitespace and non-comment tokens.

Returns None at the end of the token stream.

pop()

Pop and return next token

Raises StopIteration if we’re already at the end of the token stream.

read_to(tokens, discard=False)

Read to the given token and consume it

Raises StopIteration if we’re already at the end of the token stream.

read_to_depth(depth, discard=False)

Read until the specified nesting depth or the end of the token stream

If discard is False, returns a list of the tokens seen before either reaching the desired depth or end-of-stream.

If discard is True, returns True on reaching the desired depth, and raises StopIteration on end-of-stream.

Always raises StopIteration if we’re already at the end of the token stream.

Seeing a (, {, or [ increases the depth, and seeing a ), }, or ] decreases it. The current depth is available via the depth attribute.

read_until(tokens, discard=False)

Read until the given token, but do not consume it

Raises StopIteration if we’re already at the end of the token stream.

AST Hooks

process.add_typedef_hook(parse_func)

Wraps enum/struct/union definitions in a typedef if they lack one

Useful for C++ headers where the typedefs are implicit.

AST Hook Helpers

class nicelib.process.TreeModifier

A special type of visitor class that modifies an AST in place

Subclass this and implement the various visit_X methods which transform nodes of each type. You can then instantiate the class and use it to implement an AST hook.

Its visit_X methods must return a value, which correspond to the transformed node. If the node is contained in a parent node’s list and its visit_X method returns None, it will be removed from the list. This modification/removal is implemented via generic_visit, so you can override it on a per-nodetype basis.

The X in visit_X is the node type’s name, e.g. visit_Enum. See pycparser.c_ast or pycparser._c_ast.cfg for all the available types of nodes, and pycparser.c_ast.NodeVisitor to see the base visitor class.

generic_visit(node)

Called if no explicit visitor function exists for a node. Implements preorder visiting of the node.

visit(node)

Visit a node.

Mid-Level Binding API

nicelib.load_lib(name, pkg=None, dir=None, builder=None, kwargs={})

Load a low-level lib module, building it first if required.

If name is 'foo', tries to import a module named _foolib. If the module can’t be located, load_lib tries to build it.

Parameters
namestr

The name of the library. A name of 'foo' would try to import the module _foolib.

pkgstr

The package within which to look for the module. If None, looks for a top-level module.

dirstr

A directory within which to search for the module, before checking the system path. Useful when working with local modules not within a package.

builderstr, optional

The name of the module whose build() function is used to generate _foolib.py By default, it is assumed to be _build_foo (where ‘foo’ is the value of name).

kwargsdict, optional

Keyword args to be passed to build().

Returns
libLibInfo
class nicelib.LibInfo(lib_module=None, prefix=None)

Opaque object containing library information

class nicelib.nicelib.NiceLib

Base class for mid-level library wrappers

Provides a nice interface for quickly defining mid-level library wrappers. You define a subclass for each specific library (.dll/.so file).

Attributes
_info_

A LibInfo object that contains access to the underlying library and macros. Required (unless you are using the old-style _ffi_, _ffilib_, and _defs_ attributes)

_ffi_

FFI instance variable. Required if not using _info_

_ffilib_

FFI library opened with dlopen(). Required if not using _info_.

_defs_

dict containing the Python-equivalent macros defined in the header file(s). Optional and only used if not using _info_.

_prefix_str or sequence of strs, optional

Prefix(es) to strip from the library function names. E.g. If the library has functions named like SDK_Func(), you can set _prefix_ to 'SDK_', and access them as Func(). If more than one prefix is given, they are tried in order for each signature until the appropraite function is found.

_ret_function or str, optional

RetHandler to handle the return values of each C function. By default, the return value will be appended to the end of the Python return values. The return handler function takes the C function’s return value (often an error/success code) as its only argument. If the wrapper returns a non-None value, it will be appended to the wrapped function’s return values.

NiceLib defines the simple handlers ret_return() and ret_ignore() for convenience.

_struct_maker_function, optional

Function that is called to create an FFI struct of the given type. Mainly useful for odd libraries that require you to always fill out some field of the struct, like its size in bytes

_buflen_int, optional

The default length for buffers. This can be overridden on a per-argument basis in the argument’s spec string, e.g 'len=64' will make a 64-byte buffer.

_use_numpy_bool, optional

If true, convert output args marked as ‘arr’ to numpy arrays. Obviously requires numpy to be installed.

class nicelib.nicelib.Sig(*arg_strs, **flags)
__init__(*arg_strs, **flags)

Create a signature specification.

Parameters
arg_strsstrings

Strings defining the input-output signature of the underlying C function being wrapped. There is a one-to-one correspondence between arg strings and the C function’s args.

flags

Flags(settings) to be applied to this function.

class nicelib.nicelib.NiceObject(*args)

Base class for object-like mid-level library wrappers

Attributes
_prefix_str or sequence of strs, optional

Prefix(es) to strip from the library “method” names. E.g. If the library has functions named like SDK_Obj_Func(), you can set _prefix_ to 'SDK_Obj_', and access them as Func(). If more than one prefix is given, they are tried in order for each signature until the appropraite function is found.

_ret_function or str, optional

RetHandler to handle the return values of each C function. By default, the return value will be appended to the end of the Python return values. The return handler function takes the C function’s return value (often an error/success code) as its only argument. If the wrapper returns a non-None value, it will be appended to the wrapped function’s return values.

NiceLib defines the simple handlers ret_return() and ret_ignore() for convenience.

_struct_maker_function, optional

Function that is called to create an FFI struct of the given type. Mainly useful for odd libraries that require you to always fill out some field of the struct, like its size in bytes

_buflen_int, optional

The default length for buffers. This can be overridden on a per-argument basis in the argument’s spec string, e.g 'len=64' will make a 64-byte buffer.

_use_numpy_bool, optional

If true, convert output args marked as ‘arr’ to numpy arrays. Obviously requires numpy to be installed.

__init__(*args)
class nicelib.nicelib.RetHandler(func=None, name=None, num_retvals=None)

Decorator class for creating return handlers

__init__(func=None, name=None, num_retvals=None)
nicelib.nicelib.ret_return(func)

Append the return value to the wrapped function’s return values.

nicelib.nicelib.ret_ignore(func)

Ignore the return value.

nicelib.generate_bindings(header_info, outfile, prefix=(), add_ret_ignore=False, niceobj_prefix={}, fill_handle=True, **kwds)

Generate a skeleton library wrapper.

Grabs all the function declarations from the given header(s), generating a partially-implemented NiceLib wrapper that can be uncommented and filled in as you go. Supports NiceObjects and stripping of prefixes.

Parameters
header_info, **kwds

These get passed directly to process_headers()

outfilestr or file-like object

File (or filename) where output is written.

prefixstr or tuple of strs, optional

Prefix(es) to strip from toplevel functions.

add_ret_ignorebool, optional

Automatically add the ‘ignore’ return value wrapper for void C functions. False by default.

niceobj_prefixdict, optional

Mapping from NiceObject name to its function prefix. If a function has this prefix, it will be considered a ‘method’ of the given NiceObject. These prefixes are checked before the top-level prefixes.

fill_handlebool, optional

If True, automatically set the first argument of every NiceObject function to 'in'. True by default.