API Documentation¶
These are the major NiceLib classes and functions of which you should know:
Header Processing API¶
- nicelib.build_lib(header_info, lib_name, module_name, filedir, ignored_headers=(), ignore_system_headers=False, preamble=None, token_hooks=(), ast_hooks=(), hook_groups=(), debug_file=None, logbuf=None, load_dump_file=False, save_dump_file=False, pack=None, override=False)¶
Build a low-level Python wrapper of a C lib
- Parameters
- header_infostr, dict, or None
Path to header file. If None, uses only the header source given by
preamble
.Paths can use
os.environ
, as described below. Info is provided in the form of a dict which must contain a ‘header’ key whose value is either a str containing a single header name, or a tuple of such strings.There are also some other optional entries:
The
'path'
value must be a str or tuple of strs that are directories which will be searched for the given header(s). Any headers that are specified with a leading slash are considered absolute and are not affected by this path.The
'predef'
value is a str which is the name or path of a header file which will be used to populate the predefined preprocessor macros that are ordinarily provided by the compiler on a per-system basis. If provided, this overrides the default header that NiceLib uses for your system.- lib_namestr or dict
Name of compiled library file, e.g.
'mylib.dll'
- module_namestr
Name of module to create. Must be in the format
'_*lib'
, e.g.'_mylib'
- filedirstr
Path indicating the directory where the generated module will be saved. If
filedir
points to an existing file, that file’s directory is used. Usually you would pass the__file__
attribute from your build module.- ignored_headerssequence of strs
Names of headers to ignore;
#include
s containing these will be skipped.- ignore_system_headersbool
If True, skip inclusion of headers specified with angle brackets, e.g.
#include <stdarg.h>
Header files specified with double quotes are processed as ususal. Default is False.- preamblestr
C source to insert before the headers specified by
header_info
. Useful for including typedefs that would otherwise be hard to reach due to an end user missing headers.- token_hookssequence of functions
Token hook functions. See
process_headers()
for more info.- ast_hookssequence of functions
AST hook functions. See
process_headers()
for more info.- hook_groupsstr or sequence of strs
Hook groups. See
process_headers()
for more info.- debug_filestr
File to write a partially-processed header to just before it is parsed by
pycparser
. Useful for debugging the preprocessor whenpycparser
’s parser chokes on its output.- logbufwriteable buffer
IO buffer to write() common log output to. By default this output will logged using the
logging
stdlib module, at theinfo
log level. You can usesys.stdout
to perform ordinary printing.- load_dump_filebool
Save the list of tokens resulting from preprocessing to ‘token_dump.pkl’. See save_dump_file for more info.
- save_dump_filebool
Ignore
header_paths
and load the already-preprocessed tokens from ‘token_dump.pkl’. This can significantly speed up your turnaround time when debugging really large header sets when writing and debugging hooks.- pack: int
Forwared to
FFI.cdef
. Controls the packing of structs if necessary. This has to be done manually since CFFI ignores#pragma pack
and gcc directives. See the CFFI documentation for more information.- override: bool
Forwarded to
FFI.cdef
. If True, allows repeated declarations; the final declaration will override any others. Otherwise, repeated declarations are treated as an error.
Notes
header_info
andlib_name
can each be a dict that maps from a platform to the corresponding path or name, allowing cross-platform support. The keys are matched againstsys.platform
and can use globbing, i.e.'linux*'
will match anything starting with'linux'
.The path or paths provided by
header_info
may use items inos.environ
. For example,'{PROGRAMFILES}\\header.h'
will be formatted withos.environ['PROGRAMFILES']
.
- nicelib.process.process_headers(header_paths, predef_path=None, update_cb=None, ignored_headers=(), ignore_system_headers=False, debug_file=None, preamble=None, token_hooks=(), ast_hooks=(), hook_groups=(), return_ast=False, load_dump_file=False, save_dump_file=False)¶
Preprocess header(s) and split into a cleaned header and macros
- Parameters
- header_pathsstr or sequence of strs
Paths of the headers
- ignored_headerssequence of strs
Names of headers to ignore;
#include
s containing these will be skipped.- ignore_system_headersbool
If True, skip inclusion of headers specified with angle brackets, e.g.
#include <stdarg.h>
Header files specified with double quotes are processed as ususal. Default is False.- debug_filestr
File to write a partially-processed header to just before it is parsed by
pycparser
. Useful for debugging the preprocessor whenpycparser
’s parser chokes on its output.- preamblestr
C source to insert before the headers specified by
header_paths
. Useful for including typedefs that would otherwise be hard to reach due to an end user missing headers. token_hooks : sequence of functions Hook functions to be run on the already preprocessed token stream. Each function should accept and return a sequence ofToken
s. These are applied after any builtin token hooks.- ast_hookssequence of functions
Hook functions to be run on chunks of the header’s C AST. After preprocessing and running the token hooks, the tokens are grouped and joined to form a sequence of chunks called “external declarations” (declarations, typedefs, and function definitions). Each chunk is parsed by
pycparser
, then passed through the list of AST hook functions to transform it. These are applied after any builtin AST hooks.AST hook functions take the parsed
FileAST
and the persistentCParser
instance as arguments, and return a transformedFileAST
. It is useful to have a reference to the parser to add phony typedefs if necessary.- hook_groupsstr or sequence of strs
Predefined hook groups to use. Each hook group enables certain builtin hooks that are commonly used together. The only hook group available for now is ‘C++’.
- ‘C++’(declspec_hook, extern_c_hook, enum_type_hook, CPPTypedefAdder)
Hooks for converting C++-only headers into C syntax understandable by
cffi
.
- load_dump_filebool
Save the list of tokens resulting from preprocessing to ‘token_dump.pkl’. See save_dump_file for more info.
- save_dump_filebool
Ignore
header_paths
and load the already-preprocessed tokens from ‘token_dump.pkl’. This can significantly speed up your turnaround time when debugging really large header sets when writing and debugging hooks.
- Returns
- header_srcstr
Cleaned header C-code.
- macro_rcstr
Extracted macros expressed as Python source code.
Token Hooks¶
These functions can all be used in the token_hooks
passed to build_lib()
or process_headers()
- nicelib.process.cdecl_hook(tokens)¶
Removes
cdecl
,_cdecl
, and__cdecl
Enabled by default.
- nicelib.process.stdcall_hook(tokens)¶
Replace
__stdcall
andWINAPI
withvolatile volatile const
Enabled by default.
This technique is stolen from
cffi
.
- nicelib.process.declspec_hook(tokens)¶
Removes all occurences of
__declspec(...)
- nicelib.process.inline_hook(tokens)¶
Removes
_inline
,__inline
, and__forceinline
- nicelib.process.extern_c_hook(tokens)¶
Removes
extern "C" { ... }
while keeping the block’s contents
- nicelib.process.enum_type_hook(tokens)¶
- nicelib.process.asm_hook(tokens)¶
Remove
_asm
and__asm
and their blocks
- nicelib.process.vc_pragma_hook(tokens)¶
Remove
__pragma(...)
- nicelib.process.struct_func_hook(tokens)¶
Removes function definitions from inside struct definitions.
Once we’re inside the struct’s curly braces, we look member-by-member. We push each next token into a buffer, if we see an open brace then we assume this member is a funcdef. The funcdef is over after the matching close brace and an optional semicolon. Otherwise, if we see a semicolon outside of any nesting, that’s the end of an ordinary member declaration.
NOTE: Does not deal with nested structs that have methods (if that’s even allowed).
If we see a funcdef, throw away all its tokens. If we see a member declaration, pass the tokens on.
- nicelib.process.add_line_directive_hook(tokens)¶
Adds line directives to indicate where each token came from
Enabled by default.
Token Hook Helpers¶
These functions and classes are useful for writing your own custom token hooks:
- class nicelib.process.TokenType(value)¶
Enum of token types for C Preprocessor
- BLOCK_COMMENT = 11¶
Block comment (inside
/*
*/
)
- CHAR_CONST = 5¶
Char literal (e.g.
'c'
)
- DEFINED = 1¶
‘defined’
- HEADER_NAME = 6¶
Header name (inside
<angle brackets>
)
- IDENTIFIER = 2¶
Identifier
- LINE_COMMENT = 10¶
Line comment (starts with
//
)
- NEWLINE = 8¶
Newline char
- NUMBER = 3¶
Numeric literal
- PUNCTUATOR = 7¶
Punctuation token
- STRING_CONST = 4¶
String literal (e.g.
"my string"
)
- WHITESPACE = 9¶
Non-newline whitespace
- nicelib.process.remove_pattern(tokens, pattern)¶
Convenience function that works like
modify_pattern
, but you only specify targets
- nicelib.process.modify_pattern(tokens, pattern)¶
Helper function for easily matching and modifying patterns in a token stream
- Parameters
- tokenssequence of Tokens
Input token stream to process
- patternsequence of tuples
Pattern to match, as a sequence of (keep, target) pairs. Targets are compared against tokens; when the first target is matched, we then try matching the rest in order. If the whole pattern is matched, each target’s corresponding
keep
indicates whether the token should be kept in the sequence, or discarded.keep
is a string, where ‘k’ means ‘keep’, ‘d’ means discard and ‘a’ means add.A
target
can be either a string or aTokenType
.There is also some functionality for dealing with blocks enclosed in curly braces {}. For more advanced functionality, check out
ParseHelper
.Passing a sequence of the type
((keep_start, '{'), ('keep_end', '~~}~~'))
will keep the opening{
according tokeep_start
.keep_end
is a two letter string, where the first letter indicates whether the contents of the block enclosed in braces should be kept, and the second indicates if the closing}
should be kept.
- class nicelib.process.ParseHelper(tokens)¶
Helper class for processing token streams
Allows you to easily read until specific tokens or the ends of blocks, etc.
- __init__(tokens)¶
- peek()¶
Peek at the next token
Returns None at the end of the token stream.
- peek_true_token()¶
Peek at next true token
“True” tokens are non-whitespace and non-comment tokens.
Returns None at the end of the token stream.
- pop()¶
Pop and return next token
Raises StopIteration if we’re already at the end of the token stream.
- read_to(tokens, discard=False)¶
Read to the given token and consume it
Raises StopIteration if we’re already at the end of the token stream.
- read_to_depth(depth, discard=False)¶
Read until the specified nesting depth or the end of the token stream
If
discard
is False, returns a list of the tokens seen before either reaching the desired depth or end-of-stream.If
discard
is True, returns True on reaching the desired depth, and raises StopIteration on end-of-stream.Always raises StopIteration if we’re already at the end of the token stream.
Seeing a
(
,{
, or[
increases the depth, and seeing a)
,}
, or]
decreases it. The current depth is available via thedepth
attribute.
- read_until(tokens, discard=False)¶
Read until the given token, but do not consume it
Raises StopIteration if we’re already at the end of the token stream.
AST Hooks¶
- process.add_typedef_hook(parse_func)¶
Wraps enum/struct/union definitions in a typedef if they lack one
Useful for C++ headers where the typedefs are implicit.
AST Hook Helpers¶
- class nicelib.process.TreeModifier¶
A special type of visitor class that modifies an AST in place
Subclass this and implement the various
visit_X
methods which transform nodes of each type. You can then instantiate the class and use it to implement an AST hook.Its
visit_X
methods must return a value, which correspond to the transformed node. If the node is contained in a parent node’s list and itsvisit_X
method returns None, it will be removed from the list. This modification/removal is implemented viageneric_visit
, so you can override it on a per-nodetype basis.The
X
invisit_X
is the node type’s name, e.g.visit_Enum
. Seepycparser.c_ast
orpycparser._c_ast.cfg
for all the available types of nodes, andpycparser.c_ast.NodeVisitor
to see the base visitor class.- generic_visit(node)¶
Called if no explicit visitor function exists for a node. Implements preorder visiting of the node.
- visit(node)¶
Visit a node.
Mid-Level Binding API¶
- nicelib.load_lib(name, pkg=None, dir=None, builder=None, kwargs={})¶
Load a low-level lib module, building it first if required.
If
name
is'foo'
, tries to import a module named_foolib
. If the module can’t be located,load_lib
tries to build it.- Parameters
- namestr
The name of the library. A name of
'foo'
would try to import the module_foolib
.- pkgstr
The package within which to look for the module. If None, looks for a top-level module.
- dirstr
A directory within which to search for the module, before checking the system path. Useful when working with local modules not within a package.
- builderstr, optional
The name of the module whose
build()
function is used to generate_foolib.py
By default, it is assumed to be_build_foo
(where ‘foo’ is the value ofname
).- kwargsdict, optional
Keyword args to be passed to
build()
.
- Returns
- lib
LibInfo
- lib
- class nicelib.LibInfo(lib_module=None, prefix=None)¶
Opaque object containing library information
- class nicelib.nicelib.NiceLib¶
Base class for mid-level library wrappers
Provides a nice interface for quickly defining mid-level library wrappers. You define a subclass for each specific library (.dll/.so file).
- Attributes
- _info_
A
LibInfo
object that contains access to the underlying library and macros. Required (unless you are using the old-style_ffi_
,_ffilib_
, and_defs_
attributes)- _ffi_
FFI instance variable. Required if not using
_info_
- _ffilib_
FFI library opened with
dlopen()
. Required if not using_info_
.- _defs_
dict
containing the Python-equivalent macros defined in the header file(s). Optional and only used if not using_info_
.- _prefix_str or sequence of strs, optional
Prefix(es) to strip from the library function names. E.g. If the library has functions named like
SDK_Func()
, you can set_prefix_
to'SDK_'
, and access them asFunc()
. If more than one prefix is given, they are tried in order for each signature until the appropraite function is found.- _ret_function or str, optional
RetHandler
to handle the return values of each C function. By default, the return value will be appended to the end of the Python return values. The return handler function takes the C function’s return value (often an error/success code) as its only argument. If the wrapper returns a non-None value, it will be appended to the wrapped function’s return values.NiceLib defines the simple handlers
ret_return()
andret_ignore()
for convenience.- _struct_maker_function, optional
Function that is called to create an FFI struct of the given type. Mainly useful for odd libraries that require you to always fill out some field of the struct, like its size in bytes
- _buflen_int, optional
The default length for buffers. This can be overridden on a per-argument basis in the argument’s spec string, e.g
'len=64'
will make a 64-byte buffer.- _use_numpy_bool, optional
If true, convert output args marked as ‘arr’ to
numpy
arrays. Obviously requiresnumpy
to be installed.
- class nicelib.nicelib.Sig(*arg_strs, **flags)¶
- __init__(*arg_strs, **flags)¶
Create a signature specification.
- Parameters
- arg_strsstrings
Strings defining the input-output signature of the underlying C function being wrapped. There is a one-to-one correspondence between arg strings and the C function’s args.
- flags
Flags(settings) to be applied to this function.
- class nicelib.nicelib.NiceObject(*args)¶
Base class for object-like mid-level library wrappers
- Attributes
- _prefix_str or sequence of strs, optional
Prefix(es) to strip from the library “method” names. E.g. If the library has functions named like
SDK_Obj_Func()
, you can set_prefix_
to'SDK_Obj_'
, and access them asFunc()
. If more than one prefix is given, they are tried in order for each signature until the appropraite function is found.- _ret_function or str, optional
RetHandler
to handle the return values of each C function. By default, the return value will be appended to the end of the Python return values. The return handler function takes the C function’s return value (often an error/success code) as its only argument. If the wrapper returns a non-None value, it will be appended to the wrapped function’s return values.NiceLib defines the simple handlers
ret_return()
andret_ignore()
for convenience.- _struct_maker_function, optional
Function that is called to create an FFI struct of the given type. Mainly useful for odd libraries that require you to always fill out some field of the struct, like its size in bytes
- _buflen_int, optional
The default length for buffers. This can be overridden on a per-argument basis in the argument’s spec string, e.g
'len=64'
will make a 64-byte buffer.- _use_numpy_bool, optional
If true, convert output args marked as ‘arr’ to
numpy
arrays. Obviously requiresnumpy
to be installed.
- __init__(*args)¶
- class nicelib.nicelib.RetHandler(func=None, name=None, num_retvals=None)¶
Decorator class for creating return handlers
- __init__(func=None, name=None, num_retvals=None)¶
- nicelib.nicelib.ret_return(func)¶
Append the return value to the wrapped function’s return values.
- nicelib.nicelib.ret_ignore(func)¶
Ignore the return value.
- nicelib.generate_bindings(header_info, outfile, prefix=(), add_ret_ignore=False, niceobj_prefix={}, fill_handle=True, **kwds)¶
Generate a skeleton library wrapper.
Grabs all the function declarations from the given header(s), generating a partially-implemented NiceLib wrapper that can be uncommented and filled in as you go. Supports NiceObjects and stripping of prefixes.
- Parameters
- header_info, **kwds
These get passed directly to
process_headers()
- outfilestr or file-like object
File (or filename) where output is written.
- prefixstr or tuple of strs, optional
Prefix(es) to strip from toplevel functions.
- add_ret_ignorebool, optional
Automatically add the ‘ignore’ return value wrapper for void C functions. False by default.
- niceobj_prefixdict, optional
Mapping from NiceObject name to its function prefix. If a function has this prefix, it will be considered a ‘method’ of the given NiceObject. These prefixes are checked before the top-level prefixes.
- fill_handlebool, optional
If True, automatically set the first argument of every NiceObject function to
'in'
. True by default.