struct assignment python

Structured arrays #

Introduction #.

Structured arrays are ndarrays whose datatype is a composition of simpler datatypes organized as a sequence of named fields . For example,

Here x is a one-dimensional array of length two whose datatype is a structure with three fields: 1. A string of length 10 or less named ‘name’, 2. a 32-bit integer named ‘age’, and 3. a 32-bit float named ‘weight’.

If you index x at position 1 you get a structure:

You can access and modify individual fields of a structured array by indexing with the field name:

Structured datatypes are designed to be able to mimic ‘structs’ in the C language, and share a similar memory layout. They are meant for interfacing with C code and for low-level manipulation of structured buffers, for example for interpreting binary blobs. For these purposes they support specialized features such as subarrays, nested datatypes, and unions, and allow control over the memory layout of the structure.

Users looking to manipulate tabular data, such as stored in csv files, may find other pydata projects more suitable, such as xarray, pandas, or DataArray. These provide a high-level interface for tabular data analysis and are better optimized for that use. For instance, the C-struct-like memory layout of structured arrays in numpy can lead to poor cache behavior in comparison.

Structured Datatypes #

A structured datatype can be thought of as a sequence of bytes of a certain length (the structure’s itemsize ) which is interpreted as a collection of fields. Each field has a name, a datatype, and a byte offset within the structure. The datatype of a field may be any numpy datatype including other structured datatypes, and it may also be a subarray data type which behaves like an ndarray of a specified shape. The offsets of the fields are arbitrary, and fields may even overlap. These offsets are usually determined automatically by numpy, but can also be specified.

Structured Datatype Creation #

Structured datatypes may be created using the function numpy.dtype . There are 4 alternative forms of specification which vary in flexibility and conciseness. These are further documented in the Data Type Objects reference page, and in summary they are:

A list of tuples, one tuple per field

Each tuple has the form (fieldname, datatype, shape) where shape is optional. fieldname is a string (or tuple if titles are used, see Field Titles below), datatype may be any object convertible to a datatype, and shape is a tuple of integers specifying subarray shape.

If fieldname is the empty string '' , the field will be given a default name of the form f# , where # is the integer index of the field, counting from 0 from the left:

The byte offsets of the fields within the structure and the total structure itemsize are determined automatically.

A string of comma-separated dtype specifications

In this shorthand notation any of the string dtype specifications may be used in a string and separated by commas. The itemsize and byte offsets of the fields are determined automatically, and the field names are given the default names f0 , f1 , etc.

A dictionary of field parameter arrays

This is the most flexible form of specification since it allows control over the byte-offsets of the fields and the itemsize of the structure.

The dictionary has two required keys, ‘names’ and ‘formats’, and four optional keys, ‘offsets’, ‘itemsize’, ‘aligned’ and ‘titles’. The values for ‘names’ and ‘formats’ should respectively be a list of field names and a list of dtype specifications, of the same length. The optional ‘offsets’ value should be a list of integer byte-offsets, one for each field within the structure. If ‘offsets’ is not given the offsets are determined automatically. The optional ‘itemsize’ value should be an integer describing the total size in bytes of the dtype, which must be large enough to contain all the fields.

Offsets may be chosen such that the fields overlap, though this will mean that assigning to one field may clobber any overlapping field’s data. As an exception, fields of numpy.object_ type cannot overlap with other fields, because of the risk of clobbering the internal object pointer and then dereferencing it.

The optional ‘aligned’ value can be set to True to make the automatic offset computation use aligned offsets (see Automatic Byte Offsets and Alignment ), as if the ‘align’ keyword argument of numpy.dtype had been set to True.

The optional ‘titles’ value should be a list of titles of the same length as ‘names’, see Field Titles below.

A dictionary of field names

The keys of the dictionary are the field names and the values are tuples specifying type and offset:

This form was discouraged because Python dictionaries did not preserve order in Python versions before Python 3.6. Field Titles may be specified by using a 3-tuple, see below.

Manipulating and Displaying Structured Datatypes #

The list of field names of a structured datatype can be found in the names attribute of the dtype object:

The dtype of each individual field can be looked up by name:

The field names may be modified by assigning to the names attribute using a sequence of strings of the same length.

The dtype object also has a dictionary-like attribute, fields , whose keys are the field names (and Field Titles , see below) and whose values are tuples containing the dtype and byte offset of each field.

Both the names and fields attributes will equal None for unstructured arrays. The recommended way to test if a dtype is structured is with if dt.names is not None rather than if dt.names , to account for dtypes with 0 fields.

The string representation of a structured datatype is shown in the “list of tuples” form if possible, otherwise numpy falls back to using the more general dictionary form.

Automatic Byte Offsets and Alignment #

Numpy uses one of two methods to automatically determine the field byte offsets and the overall itemsize of a structured datatype, depending on whether align=True was specified as a keyword argument to numpy.dtype .

By default ( align=False ), numpy will pack the fields together such that each field starts at the byte offset the previous field ended, and the fields are contiguous in memory.

If align=True is set, numpy will pad the structure in the same way many C compilers would pad a C-struct. Aligned structures can give a performance improvement in some cases, at the cost of increased datatype size. Padding bytes are inserted between fields such that each field’s byte offset will be a multiple of that field’s alignment, which is usually equal to the field’s size in bytes for simple datatypes, see PyArray_Descr.alignment . The structure will also have trailing padding added so that its itemsize is a multiple of the largest field’s alignment.

Note that although almost all modern C compilers pad in this way by default, padding in C structs is C-implementation-dependent so this memory layout is not guaranteed to exactly match that of a corresponding struct in a C program. Some work may be needed, either on the numpy side or the C side, to obtain exact correspondence.

If offsets were specified using the optional offsets key in the dictionary-based dtype specification, setting align=True will check that each field’s offset is a multiple of its size and that the itemsize is a multiple of the largest field size, and raise an exception if not.

If the offsets of the fields and itemsize of a structured array satisfy the alignment conditions, the array will have the ALIGNED flag set.

A convenience function numpy.lib.recfunctions.repack_fields converts an aligned dtype or array to a packed one and vice versa. It takes either a dtype or structured ndarray as an argument, and returns a copy with fields re-packed, with or without padding bytes.

Field Titles #

In addition to field names, fields may also have an associated title , an alternate name, which is sometimes used as an additional description or alias for the field. The title may be used to index an array, just like a field name.

To add titles when using the list-of-tuples form of dtype specification, the field name may be specified as a tuple of two strings instead of a single string, which will be the field’s title and field name respectively. For example:

When using the first form of dictionary-based specification, the titles may be supplied as an extra 'titles' key as described above. When using the second (discouraged) dictionary-based specification, the title can be supplied by providing a 3-element tuple (datatype, offset, title) instead of the usual 2-element tuple:

The dtype.fields dictionary will contain titles as keys, if any titles are used. This means effectively that a field with a title will be represented twice in the fields dictionary. The tuple values for these fields will also have a third element, the field title. Because of this, and because the names attribute preserves the field order while the fields attribute may not, it is recommended to iterate through the fields of a dtype using the names attribute of the dtype, which will not list titles, as in:

Union types #

Structured datatypes are implemented in numpy to have base type numpy.void by default, but it is possible to interpret other numpy types as structured types using the (base_dtype, dtype) form of dtype specification described in Data Type Objects . Here, base_dtype is the desired underlying dtype, and fields and flags will be copied from dtype . This dtype is similar to a ‘union’ in C.

Indexing and Assignment to Structured arrays #

Assigning data to a structured array #.

There are a number of ways to assign values to a structured array: Using python tuples, using scalar values, or using other structured arrays.

Assignment from Python Native Types (Tuples) #

The simplest way to assign values to a structured array is using python tuples. Each assigned value should be a tuple of length equal to the number of fields in the array, and not a list or array as these will trigger numpy’s broadcasting rules. The tuple’s elements are assigned to the successive fields of the array, from left to right:

Assignment from Scalars #

A scalar assigned to a structured element will be assigned to all fields. This happens when a scalar is assigned to a structured array, or when an unstructured array is assigned to a structured array:

Structured arrays can also be assigned to unstructured arrays, but only if the structured datatype has just a single field:

Assignment from other Structured Arrays #

Assignment between two structured arrays occurs as if the source elements had been converted to tuples and then assigned to the destination elements. That is, the first field of the source array is assigned to the first field of the destination array, and the second field likewise, and so on, regardless of field names. Structured arrays with a different number of fields cannot be assigned to each other. Bytes of the destination structure which are not included in any of the fields are unaffected.

Assignment involving subarrays #

When assigning to fields which are subarrays, the assigned value will first be broadcast to the shape of the subarray.

Indexing Structured Arrays #

Accessing individual fields #.

Individual fields of a structured array may be accessed and modified by indexing the array with the field name.

The resulting array is a view into the original array. It shares the same memory locations and writing to the view will modify the original array.

This view has the same dtype and itemsize as the indexed field, so it is typically a non-structured array, except in the case of nested structures.

If the accessed field is a subarray, the dimensions of the subarray are appended to the shape of the result:

Accessing Multiple Fields #

One can index and assign to a structured array with a multi-field index, where the index is a list of field names.

The behavior of multi-field indexes changed from Numpy 1.15 to Numpy 1.16.

The result of indexing with a multi-field index is a view into the original array, as follows:

Assignment to the view modifies the original array. The view’s fields will be in the order they were indexed. Note that unlike for single-field indexing, the dtype of the view has the same itemsize as the original array, and has fields at the same offsets as in the original array, and unindexed fields are merely missing.

In Numpy 1.15, indexing an array with a multi-field index returned a copy of the result above, but with fields packed together in memory as if passed through numpy.lib.recfunctions.repack_fields .

The new behavior as of Numpy 1.16 leads to extra “padding” bytes at the location of unindexed fields compared to 1.15. You will need to update any code which depends on the data having a “packed” layout. For instance code such as:

will need to be changed. This code has raised a FutureWarning since Numpy 1.12, and similar code has raised FutureWarning since 1.7.

In 1.16 a number of functions have been introduced in the numpy.lib.recfunctions module to help users account for this change. These are numpy.lib.recfunctions.repack_fields . numpy.lib.recfunctions.structured_to_unstructured , numpy.lib.recfunctions.unstructured_to_structured , numpy.lib.recfunctions.apply_along_fields , numpy.lib.recfunctions.assign_fields_by_name , and numpy.lib.recfunctions.require_fields .

The function numpy.lib.recfunctions.repack_fields can always be used to reproduce the old behavior, as it will return a packed copy of the structured array. The code above, for example, can be replaced with:

Furthermore, numpy now provides a new function numpy.lib.recfunctions.structured_to_unstructured which is a safer and more efficient alternative for users who wish to convert structured arrays to unstructured arrays, as the view above is often intended to do. This function allows safe conversion to an unstructured type taking into account padding, often avoids a copy, and also casts the datatypes as needed, unlike the view. Code such as:

can be made safer by replacing with:

Assignment to an array with a multi-field index modifies the original array:

This obeys the structured array assignment rules described above. For example, this means that one can swap the values of two fields using appropriate multi-field indexes:

Indexing with an Integer to get a Structured Scalar #

Indexing a single element of a structured array (with an integer index) returns a structured scalar:

Unlike other numpy scalars, structured scalars are mutable and act like views into the original array, such that modifying the scalar will modify the original array. Structured scalars also support access and assignment by field name:

Similarly to tuples, structured scalars can also be indexed with an integer:

Thus, tuples might be thought of as the native Python equivalent to numpy’s structured types, much like native python integers are the equivalent to numpy’s integer types. Structured scalars may be converted to a tuple by calling numpy.ndarray.item :

Viewing Structured Arrays Containing Objects #

In order to prevent clobbering object pointers in fields of object type, numpy currently does not allow views of structured arrays containing objects.

Structure Comparison and Promotion #

If the dtypes of two void structured arrays are equal, testing the equality of the arrays will result in a boolean array with the dimensions of the original arrays, with elements set to True where all fields of the corresponding structures are equal:

NumPy will promote individual field datatypes to perform the comparison. So the following is also valid (note the 'f4' dtype for the 'a' field):

To compare two structured arrays, it must be possible to promote them to a common dtype as returned by numpy.result_type and np.promote_types . This enforces that the number of fields, the field names, and the field titles must match precisely. When promotion is not possible, for example due to mismatching field names, NumPy will raise an error. Promotion between two structured dtypes results in a canonical dtype that ensures native byte-order for all fields:

The resulting dtype from promotion is also guaranteed to be packed, meaning that all fields are ordered contiguously and any unnecessary padding is removed:

Note that the result prints without offsets or itemsize indicating no additional padding. If a structured dtype is created with align=True ensuring that dtype.isalignedstruct is true, this property is preserved:

When promoting multiple dtypes, the result is aligned if any of the inputs is:

The < and > operators always return False when comparing void structured arrays, and arithmetic and bitwise operations are not supported.

Changed in version 1.23: Before NumPy 1.23, a warning was given and False returned when promotion to a common dtype failed. Further, promotion was much more restrictive: It would reject the mixed float/integer comparison example above.

Record Arrays #

As an optional convenience numpy provides an ndarray subclass, numpy.recarray that allows access to fields of structured arrays by attribute instead of only by index. Record arrays use a special datatype, numpy.record , that allows field access by attribute on the structured scalars obtained from the array. The numpy.rec module provides functions for creating recarrays from various objects. Additional helper functions for creating and manipulating structured arrays can be found in numpy.lib.recfunctions .

The simplest way to create a record array is with numpy.rec.array :

numpy.rec.array can convert a wide variety of arguments into record arrays, including structured arrays:

The numpy.rec module provides a number of other convenience functions for creating record arrays, see record array creation routines .

A record array representation of a structured array can be obtained using the appropriate view :

For convenience, viewing an ndarray as type numpy.recarray will automatically convert to numpy.record datatype, so the dtype can be left out of the view:

To get back to a plain ndarray both the dtype and type must be reset. The following view does so, taking into account the unusual case that the recordarr was not a structured type:

Record array fields accessed by index or by attribute are returned as a record array if the field has a structured type but as a plain ndarray otherwise.

Note that if a field has the same name as an ndarray attribute, the ndarray attribute takes precedence. Such fields will be inaccessible by attribute but will still be accessible by index.

Recarray Helper Functions #

Collection of utilities to manipulate structured arrays.

Most of these functions were initially implemented by John Hunter for matplotlib. They have been rewritten and extended for convenience.

Add new fields to an existing array.

The names of the fields are given with the names arguments, the corresponding values with the data arguments. If a single field is appended, names , data and dtypes do not have to be lists but just values.

Input array to extend.

String or sequence of strings corresponding to the names of the new fields.

Array or sequence of arrays storing the fields to add to the base.

Datatype or sequence of datatypes. If None, the datatypes are estimated from the data .

Filling value used to pad missing data on the shorter arrays.

Whether to return a masked array or not.

Whether to return a recarray (MaskedRecords) or not.

Apply function ‘func’ as a reduction across fields of a structured array.

This is similar to apply_along_axis , but treats the fields of a structured array as an extra axis. The fields are all first cast to a common type following the type-promotion rules from numpy.result_type applied to the field’s dtypes.

Function to apply on the “field” dimension. This function must support an axis argument, like np.mean, np.sum, etc.

Structured array for which to apply func.

Result of the recution operation

Assigns values from one structured array to another by field name.

Normally in numpy >= 1.14, assignment of one structured array to another copies fields “by position”, meaning that the first field from the src is copied to the first field of the dst, and so on, regardless of field name.

This function instead copies “by field name”, such that fields in the dst are assigned from the identically named field in the src. This applies recursively for nested structures. This is how structure assignment worked in numpy >= 1.6 to <= 1.13.

The source and destination arrays during assignment.

If True, fields in the dst for which there was no matching field in the src are filled with the value 0 (zero). This was the behavior of numpy <= 1.13. If False, those fields are not modified.

Return a new array with fields in drop_names dropped.

Nested fields are supported.

Changed in version 1.18.0: drop_fields returns an array with 0 fields if all fields are dropped, rather than returning None as it did previously.

Input array

String or sequence of strings corresponding to the names of the fields to drop.

Whether to return a recarray or a mrecarray ( asrecarray=True ) or a plain ndarray or masked array with flexible dtype. The default is False.

Find the duplicates in a structured array along a given key

Name of the fields along which to check the duplicates. If None, the search is performed by records

Whether masked data should be discarded or considered as duplicates.

Whether to return the indices of the duplicated values.

Flatten a structured data-type description.

Returns a dictionary with fields indexing lists of their parent fields.

This function is used to simplify access to fields nested in other fields.

Input datatype

Last processed field name (used internally during recursion).

Dictionary of parent fields (used interbally during recursion).

Returns the field names of the input datatype as a tuple. Input datatype must have fields otherwise error is raised.

Returns the field names of the input datatype as a tuple. Input datatype must have fields otherwise error is raised. Nested structure are flattened beforehand.

Join arrays r1 and r2 on key key .

The key should be either a string or a sequence of string corresponding to the fields used to join the array. An exception is raised if the key field cannot be found in the two input arrays. Neither r1 nor r2 should have any duplicates along key : the presence of duplicates will make the output quite unreliable. Note that duplicates are not looked for by the algorithm.

A string or a sequence of strings corresponding to the fields used for comparison.

Structured arrays.

If ‘inner’, returns the elements common to both r1 and r2. If ‘outer’, returns the common elements as well as the elements of r1 not in r2 and the elements of not in r2. If ‘leftouter’, returns the common elements and the elements of r1 not in r2.

String appended to the names of the fields of r1 that are present in r2 but absent of the key.

String appended to the names of the fields of r2 that are present in r1 but absent of the key.

Dictionary mapping field names to the corresponding default values.

Whether to return a MaskedArray (or MaskedRecords is asrecarray==True ) or a ndarray.

Whether to return a recarray (or MaskedRecords if usemask==True ) or just a flexible-type ndarray.

The output is sorted along the key.

A temporary array is formed by dropping the fields not in the key for the two arrays and concatenating the result. This array is then sorted, and the common entries selected. The output is constructed by filling the fields with the selected entries. Matching is not preserved if there are some duplicates…

Merge arrays field by field.

Sequence of arrays

Whether to collapse nested fields.

Without a mask, the missing value will be filled with something, depending on what its corresponding type:

-1 for integers

-1.0 for floating point numbers

'-' for characters

'-1' for strings

True for boolean values

XXX: I just obtained these values empirically

Returns a new numpy.recarray with fields in drop_names dropped.

Join arrays r1 and r2 on keys. Alternative to join_by, that always returns a np.recarray.

equivalent function

Fills fields from output with fields from input, with support for nested structures.

Input array.

Output array.

output should be at least the same size as input

Rename the fields from a flexible-datatype ndarray or recarray.

Input array whose fields must be modified.

Dictionary mapping old field names to their new version.

Re-pack the fields of a structured array or dtype in memory.

The memory layout of structured datatypes allows fields at arbitrary byte offsets. This means the fields can be separated by padding bytes, their offsets can be non-monotonically increasing, and they can overlap.

This method removes any overlaps and reorders the fields in memory so they have increasing byte offsets, and adds or removes padding bytes depending on the align option, which behaves like the align option to numpy.dtype .

If align=False , this method produces a “packed” memory layout in which each field starts at the byte the previous field ended, and any padding bytes are removed.

If align=True , this methods produces an “aligned” memory layout in which each field’s offset is a multiple of its alignment, and the total itemsize is a multiple of the largest alignment, by adding padding bytes as needed.

array or dtype for which to repack the fields.

If true, use an “aligned” memory layout, otherwise use a “packed” layout.

If True, also repack nested structures.

Copy of a with fields repacked, or a itself if no repacking was needed.

Casts a structured array to a new dtype using assignment by field-name.

This function assigns from the old to the new array by name, so the value of a field in the output array is the value of the field with the same name in the source array. This has the effect of creating a new ndarray containing only the fields “required” by the required_dtype.

If a field name in the required_dtype does not exist in the input array, that field is created and set to 0 in the output array.

array to cast

datatype for output array

array with the new dtype, with field values copied from the fields in the input array with the same name

Superposes arrays fields by fields

Sequence of input arrays.

Whether automatically cast the type of the field to the maximum.

Converts an n-D structured array into an (n+1)-D unstructured array.

The new array will have a new last dimension equal in size to the number of field-elements of the input array. If not supplied, the output datatype is determined from the numpy type promotion rules applied to all the field datatypes.

Nested fields, as well as each element of any subarray fields, all count as a single field-elements.

Structured array or dtype to convert. Cannot contain object datatype.

The dtype of the output unstructured array.

If true, always return a copy. If false, a view is returned if possible, such as when the dtype and strides of the fields are suitable and the array subtype is one of np.ndarray , np.recarray or np.memmap .

Changed in version 1.25.0: A view can now be returned if the fields are separated by a uniform stride.

See casting argument of numpy.ndarray.astype . Controls what kind of data casting may occur.

Unstructured array with one more dimension.

Converts an n-D unstructured array into an (n-1)-D structured array.

The last dimension of the input array is converted into a structure, with number of field-elements equal to the size of the last dimension of the input array. By default all output fields have the input array’s dtype, but an output structured dtype with an equal number of fields-elements can be supplied instead.

Nested fields, as well as each element of any subarray fields, all count towards the number of field-elements.

Unstructured array or dtype to convert.

The structured dtype of the output array

If dtype is not supplied, this specifies the field names for the output dtype, in order. The field dtypes will be the same as the input array.

Whether to create an aligned memory layout.

See copy argument to numpy.ndarray.astype . If true, always return a copy. If false, and dtype requirements are satisfied, a view is returned.

Structured array with fewer dimensions.

Python Numerical Methods

This notebook contains an excerpt from the Python Programming and Numerical Methods - A Guide for Engineers and Scientists , the content is also available at Berkeley Python Numerical Methods .

The copyright of the book belongs to Elsevier. We also have this interactive book online for a better learning experience. The code is released under the MIT license . If you find this content useful, please consider supporting the work on Elsevier or Amazon !

< 2.0 Variables and Basic Data Structures | Contents | 2.2 Data Structure - Strings >

Variables and Assignment ¶

When programming, it is useful to be able to store information in variables. A variable is a string of characters and numbers associated with a piece of information. The assignment operator , denoted by the “=” symbol, is the operator that is used to assign values to variables in Python. The line x=1 takes the known value, 1, and assigns that value to the variable with name “x”. After executing this line, this number will be stored into this variable. Until the value is changed or the variable deleted, the character x behaves like the value 1.

TRY IT! Assign the value 2 to the variable y. Multiply y by 3 to show that it behaves like the value 2.

A variable is more like a container to store the data in the computer’s memory, the name of the variable tells the computer where to find this value in the memory. For now, it is sufficient to know that the notebook has its own memory space to store all the variables in the notebook. As a result of the previous example, you will see the variable “x” and “y” in the memory. You can view a list of all the variables in the notebook using the magic command %whos .

TRY IT! List all the variables in this notebook

Note that the equal sign in programming is not the same as a truth statement in mathematics. In math, the statement x = 2 declares the universal truth within the given framework, x is 2 . In programming, the statement x=2 means a known value is being associated with a variable name, store 2 in x. Although it is perfectly valid to say 1 = x in mathematics, assignments in Python always go left : meaning the value to the right of the equal sign is assigned to the variable on the left of the equal sign. Therefore, 1=x will generate an error in Python. The assignment operator is always last in the order of operations relative to mathematical, logical, and comparison operators.

TRY IT! The mathematical statement x=x+1 has no solution for any value of x . In programming, if we initialize the value of x to be 1, then the statement makes perfect sense. It means, “Add x and 1, which is 2, then assign that value to the variable x”. Note that this operation overwrites the previous value stored in x .

There are some restrictions on the names variables can take. Variables can only contain alphanumeric characters (letters and numbers) as well as underscores. However, the first character of a variable name must be a letter or underscores. Spaces within a variable name are not permitted, and the variable names are case-sensitive (e.g., x and X will be considered different variables).

TIP! Unlike in pure mathematics, variables in programming almost always represent something tangible. It may be the distance between two points in space or the number of rabbits in a population. Therefore, as your code becomes increasingly complicated, it is very important that your variables carry a name that can easily be associated with what they represent. For example, the distance between two points in space is better represented by the variable dist than x , and the number of rabbits in a population is better represented by nRabbits than y .

Note that when a variable is assigned, it has no memory of how it was assigned. That is, if the value of a variable, y , is constructed from other variables, like x , reassigning the value of x will not change the value of y .

EXAMPLE: What value will y have after the following lines of code are executed?

WARNING! You can overwrite variables or functions that have been stored in Python. For example, the command help = 2 will store the value 2 in the variable with name help . After this assignment help will behave like the value 2 instead of the function help . Therefore, you should always be careful not to give your variables the same name as built-in functions or values.

TIP! Now that you know how to assign variables, it is important that you learn to never leave unassigned commands. An unassigned command is an operation that has a result, but that result is not assigned to a variable. For example, you should never use 2+2 . You should instead assign it to some variable x=2+2 . This allows you to “hold on” to the results of previous commands and will make your interaction with Python must less confusing.

You can clear a variable from the notebook using the del function. Typing del x will clear the variable x from the workspace. If you want to remove all the variables in the notebook, you can use the magic command %reset .

In mathematics, variables are usually associated with unknown numbers; in programming, variables are associated with a value of a certain type. There are many data types that can be assigned to variables. A data type is a classification of the type of information that is being stored in a variable. The basic data types that you will utilize throughout this book are boolean, int, float, string, list, tuple, dictionary, set. A formal description of these data types is given in the following sections.

Python Enhancement Proposals

• Python »
• PEP Index »

PEP 636 – Structural Pattern Matching: Tutorial

Matching sequences, matching multiple patterns, matching specific values, matching multiple values, adding a wildcard, composing patterns, or patterns, capturing matched sub-patterns, adding conditions to patterns, adding a ui: matching objects, matching positional attributes, matching against constants and enums, going to the cloud: mappings, matching builtin classes, appendix a – quick intro.

This PEP is a tutorial for the pattern matching introduced by PEP 634 .

PEP 622 proposed syntax for pattern matching, which received detailed discussion both from the community and the Steering Council. A frequent concern was about how easy it would be to explain (and learn) this feature. This PEP addresses that concern providing the kind of document which developers could use to learn about pattern matching in Python.

This is considered supporting material for PEP 634 (the technical specification for pattern matching) and PEP 635 (the motivation and rationale for having pattern matching and design considerations).

For readers who are looking more for a quick review than for a tutorial, see Appendix A .

As an example to motivate this tutorial, you will be writing a text adventure. That is a form of interactive fiction where the user enters text commands to interact with a fictional world and receives text descriptions of what happens. Commands will be simplified forms of natural language like get sword , attack dragon , go north , enter shop or buy cheese .

Your main loop will need to get input from the user and split it into words, let’s say a list of strings like this:

The next step is to interpret the words. Most of our commands will have two words: an action and an object. So you may be tempted to do the following:

The problem with that line of code is that it’s missing something: what if the user types more or fewer than 2 words? To prevent this problem you can either check the length of the list of words, or capture the ValueError that the statement above would raise.

You can use a matching statement instead:

The match statement evaluates the “subject” (the value after the match keyword), and checks it against the pattern (the code next to case ). A pattern is able to do two different things:

• Verify that the subject has certain structure. In your case, the [action, obj] pattern matches any sequence of exactly two elements. This is called matching
• It will bind some names in the pattern to component elements of your subject. In this case, if the list has two elements, it will bind action = subject[0] and obj = subject[1] .

If there’s a match, the statements inside the case block will be executed with the bound variables. If there’s no match, nothing happens and the statement after match is executed next.

Note that, in a similar way to unpacking assignments, you can use either parenthesis, brackets, or just comma separation as synonyms. So you could write case action, obj or case (action, obj) with the same meaning. All forms will match any sequence (for example lists or tuples).

Even if most commands have the action/object form, you might want to have user commands of different lengths. For example, you might want to add single verbs with no object like look or quit . A match statement can (and is likely to) have more than one case :

The match statement will check patterns from top to bottom. If the pattern doesn’t match the subject, the next pattern will be tried. However, once the first matching pattern is found, the body of that case is executed, and all further cases are ignored. This is similar to the way that an if/elif/elif/... statement works.

Your code still needs to look at the specific actions and conditionally execute different logic depending on the specific action (e.g., quit , attack , or buy ). You could do that using a chain of if/elif/elif/... , or using a dictionary of functions, but here we’ll leverage pattern matching to solve that task. Instead of a variable, you can use literal values in patterns (like "quit" , 42 , or None ). This allows you to write:

A pattern like ["get", obj] will match only 2-element sequences that have a first element equal to "get" . It will also bind obj = subject[1] .

As you can see in the go case, we also can use different variable names in different patterns.

Literal values are compared with the == operator except for the constants True , False and None which are compared with the is operator.

A player may be able to drop multiple items by using a series of commands drop key , drop sword , drop cheese . This interface might be cumbersome, and you might like to allow dropping multiple items in a single command, like drop key sword cheese . In this case you don’t know beforehand how many words will be in the command, but you can use extended unpacking in patterns in the same way that they are allowed in assignments:

This will match any sequences having “drop” as its first elements. All remaining elements will be captured in a list object which will be bound to the objects variable.

This syntax has similar restrictions as sequence unpacking: you can not have more than one starred name in a pattern.

You may want to print an error message saying that the command wasn’t recognized when all the patterns fail. You could use the feature we just learned and write case [*ignored_words] as your last pattern. There’s however a much simpler way:

This special pattern which is written _ (and called wildcard) always matches but it doesn’t bind any variables.

Note that this will match any object, not just sequences. As such, it only makes sense to have it by itself as the last pattern (to prevent errors, Python will stop you from using it before).

This is a good moment to step back from the examples and understand how the patterns that you have been using are built. Patterns can be nested within each other, and we have been doing that implicitly in the examples above.

There are some “simple” patterns (“simple” here meaning that they do not contain other patterns) that we’ve seen:

• Capture patterns (stand-alone names like direction , action , objects ). We never discussed these separately, but used them as part of other patterns.
• Literal patterns (string literals, number literals, True , False , and None )
• The wildcard pattern _

Until now, the only non-simple pattern we have experimented with is the sequence pattern. Each element in a sequence pattern can in fact be any other pattern. This means that you could write a pattern like ["first", (left, right), _, *rest] . This will match subjects which are a sequence of at least three elements, where the first one is equal to "first" and the second one is in turn a sequence of two elements. It will also bind left=subject[1][0] , right=subject[1][1] , and rest = subject[3:]

Going back to the adventure game example, you may find that you’d like to have several patterns resulting in the same outcome. For example, you might want the commands north and go north to be equivalent. You may also desire to have aliases for get X , pick up X and pick X up for any X.

The | symbol in patterns combines them as alternatives. You could for example write:

This is called an or pattern and will produce the expected result. Patterns are tried from left to right; this may be relevant to know what is bound if more than one alternative matches. An important restriction when writing or patterns is that all alternatives should bind the same variables. So a pattern [1, x] | [2, y] is not allowed because it would make unclear which variable would be bound after a successful match. [1, x] | [2, x] is perfectly fine and will always bind x if successful.

The first version of our “go” command was written with a ["go", direction] pattern. The change we did in our last version using the pattern ["north"] | ["go", "north"] has some benefits but also some drawbacks in comparison: the latest version allows the alias, but also has the direction hardcoded, which will force us to actually have separate patterns for north/south/east/west. This leads to some code duplication, but at the same time we get better input validation, and we will not be getting into that branch if the command entered by the user is "go figure!" instead of a direction.

We could try to get the best of both worlds doing the following (I’ll omit the aliased version without “go” for brevity):

This code is a single branch, and it verifies that the word after “go” is really a direction. But the code moving the player around needs to know which one was chosen and has no way to do so. What we need is a pattern that behaves like the or pattern but at the same time does a capture. We can do so with an as pattern :

The as-pattern matches whatever pattern is on its left-hand side, but also binds the value to a name.

The patterns we have explored above can do some powerful data filtering, but sometimes you may wish for the full power of a boolean expression. Let’s say that you would actually like to allow a “go” command only in a restricted set of directions based on the possible exits from the current_room. We can achieve that by adding a guard to our case. Guards consist of the if keyword followed by any expression:

The guard is not part of the pattern, it’s part of the case. It’s only checked if the pattern matches, and after all the pattern variables have been bound (that’s why the condition can use the direction variable in the example above). If the pattern matches and the condition is truthy, the body of the case executes normally. If the pattern matches but the condition is falsy, the match statement proceeds to check the next case as if the pattern hadn’t matched (with the possible side-effect of having already bound some variables).

Your adventure is becoming a success and you have been asked to implement a graphical interface. Your UI toolkit of choice allows you to write an event loop where you can get a new event object by calling event.get() . The resulting object can have different type and attributes according to the user action, for example:

• A KeyPress object is generated when the user presses a key. It has a key_name attribute with the name of the key pressed, and some other attributes regarding modifiers.
• A Click object is generated when the user clicks the mouse. It has an attribute position with the coordinates of the pointer.
• A Quit object is generated when the user clicks on the close button for the game window.

Rather than writing multiple isinstance() checks, you can use patterns to recognize different kinds of objects, and also apply patterns to its attributes:

A pattern like Click(position=(x, y)) only matches if the type of the event is a subclass of the Click class. It will also require that the event has a position attribute that matches the (x, y) pattern. If there’s a match, the locals x and y will get the expected values.

A pattern like KeyPress() , with no arguments will match any object which is an instance of the KeyPress class. Only the attributes you specify in the pattern are matched, and any other attributes are ignored.

The previous section described how to match named attributes when doing an object match. For some objects it could be convenient to describe the matched arguments by position (especially if there are only a few attributes and they have a “standard” ordering). If the classes that you are using are named tuples or dataclasses, you can do that by following the same order that you’d use when constructing an object. For example, if the UI framework above defines their class like this:

then you can rewrite your match statement above as:

The (x, y) pattern will be automatically matched against the position attribute, because the first argument in the pattern corresponds to the first attribute in your dataclass definition.

Other classes don’t have a natural ordering of their attributes so you’re required to use explicit names in your pattern to match with their attributes. However, it’s possible to manually specify the ordering of the attributes allowing positional matching, like in this alternative definition:

The __match_args__ special attribute defines an explicit order for your attributes that can be used in patterns like case Click((x,y)) .

Your pattern above treats all mouse buttons the same, and you have decided that you want to accept left-clicks, and ignore other buttons. While doing so, you notice that the button attribute is typed as a Button which is an enumeration built with enum.Enum . You can in fact match against enumeration values like this:

This will work with any dotted name (like math.pi ). However an unqualified name (i.e. a bare name with no dots) will be always interpreted as a capture pattern, so avoid that ambiguity by always using qualified constants in patterns.

You have decided to make an online version of your game. All of your logic will be in a server, and the UI in a client which will communicate using JSON messages. Via the json module, those will be mapped to Python dictionaries, lists and other builtin objects.

Our client will receive a list of dictionaries (parsed from JSON) of actions to take, each element looking for example like these:

• {"text": "The shop keeper says 'Ah! We have Camembert, yes sir'", "color": "blue"}
• If the client should make a pause {"sleep": 3}
• To play a sound {"sound": "filename.ogg", "format": "ogg"}

Until now, our patterns have processed sequences, but there are patterns to match mappings based on their present keys. In this case you could use:

The keys in your mapping pattern need to be literals, but the values can be any pattern. As in sequence patterns, all subpatterns have to match for the general pattern to match.

You can use **rest within a mapping pattern to capture additional keys in the subject. Note that if you omit this, extra keys in the subject will be ignored while matching, i.e. the message {"text": "foo", "color": "red", "style": "bold"} will match the first pattern in the example above.

The code above could use some validation. Given that messages came from an external source, the types of the field could be wrong, leading to bugs or security issues.

Any class is a valid match target, and that includes built-in classes like bool str or int . That allows us to combine the code above with a class pattern. So instead of writing {"text": message, "color": c} we can use {"text": str() as message, "color": str() as c} to ensure that message and c are both strings. For many builtin classes (see PEP 634 for the whole list), you can use a positional parameter as a shorthand, writing str(c) rather than str() as c . The fully rewritten version looks like this:

A match statement takes an expression and compares its value to successive patterns given as one or more case blocks. This is superficially similar to a switch statement in C, Java or JavaScript (and many other languages), but much more powerful.

The simplest form compares a subject value against one or more literals:

Note the last block: the “variable name” _ acts as a wildcard and never fails to match.

You can combine several literals in a single pattern using | (“or”):

Patterns can look like unpacking assignments, and can be used to bind variables:

Study that one carefully! The first pattern has two literals, and can be thought of as an extension of the literal pattern shown above. But the next two patterns combine a literal and a variable, and the variable binds a value from the subject ( point ). The fourth pattern captures two values, which makes it conceptually similar to the unpacking assignment (x, y) = point .

If you are using classes to structure your data you can use the class name followed by an argument list resembling a constructor, but with the ability to capture attributes into variables:

You can use positional parameters with some builtin classes that provide an ordering for their attributes (e.g. dataclasses). You can also define a specific position for attributes in patterns by setting the __match_args__ special attribute in your classes. If it’s set to (“x”, “y”), the following patterns are all equivalent (and all bind the y attribute to the var variable):

Patterns can be arbitrarily nested. For example, if we have a short list of points, we could match it like this:

We can add an if clause to a pattern, known as a “guard”. If the guard is false, match goes on to try the next case block. Note that value capture happens before the guard is evaluated:

Several other key features:

• Like unpacking assignments, tuple and list patterns have exactly the same meaning and actually match arbitrary sequences. An important exception is that they don’t match iterators or strings. (Technically, the subject must be an instance of collections.abc.Sequence .)
• Sequence patterns support wildcards: [x, y, *rest] and (x, y, *rest) work similar to wildcards in unpacking assignments. The name after * may also be _ , so (x, y, *_) matches a sequence of at least two items without binding the remaining items.
• Mapping patterns: {"bandwidth": b, "latency": l} captures the "bandwidth" and "latency" values from a dict. Unlike sequence patterns, extra keys are ignored. A wildcard **rest is also supported. (But **_ would be redundant, so it is not allowed.)
• Subpatterns may be captured using the as keyword: case ( Point ( x1 , y1 ), Point ( x2 , y2 ) as p2 ): ...
• Most literals are compared by equality, however the singletons True , False and None are compared by identity.
• Patterns may use named constants. These must be dotted names to prevent them from being interpreted as capture variable: from enum import Enum class Color ( Enum ): RED = 0 GREEN = 1 BLUE = 2 match color : case Color . RED : print ( "I see red!" ) case Color . GREEN : print ( "Grass is green" ) case Color . BLUE : print ( "I'm feeling the blues :(" )

This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.

Source: https://github.com/python/peps/blob/main/peps/pep-0636.rst

Last modified: 2023-09-09 17:39:29 GMT

Destructuring in Python

Destructuring (also called unpacking) is where we take a collection, like a list or a tuple, and we break it up into individual values. This allows us to do things like destructuring assignments, where we assign values to several variables at once from a single collection.

Today let's talk about:

• Standard destructuring assignments, with tuples or lists

Destructuring dictionaries in Python

• Using destructuring in for loops
• How to ignore certain values when destructuring
• How to collect values left over when destructuring, using *

Standard destructuring assignments

Python, like many programming languages, allows us to assign more than one variable at a time on a single line. We just have to provide the same number of values on both sides of the assignment. For example:

Here we assign the value 5 to x , and 11 to y . The values are assigned entirely based on order, so if we switch the variable order, or the order of the values we intend to assign, we will end up with different results.

How this works is fairly straightforward, but what's less obvious is that this is an example of destructuring. What if I wrote it like this?

One thing that a lot of newer Python programmers in particular don't realise is that parentheses have nothing at all to do with tuples. In fact, it's the commas which tell Python something is a tuple: we just add brackets for readability in a lot of cases. In some instances the brackets are actually necessary, in order to isolate the tuple from the syntax around it, such as when we put a tuple inside a list. However, in this case the brackets are still not part of the tuple syntax. Adding brackets around a single number doesn't turn it into a tuple.

With all that in mind, we actually end up destructuring a tuple in the example above, splitting the tuple (5, 11) into its component values, so that we can bind those values to the two variable names.

We're not limited to just tuples here. We can also destructure a list, for example, or even a set. You're unlikely to want to perform a destructuring assignment with a set, however, since the order is not guaranteed, and we therefore end up with variables we don't really know the values of.

If we try to destructure a collection with more or fewer values than we provide variables, we end up with a ValueError .

A dictionary is a collection of key-value pairs, so when destructuring a dictionary, things can get a bit confusing!

Now, before you run that code, think about what the values of x and y should be!

The answer is that x = "name" and y = "age" . That's because by default when we use a dictionary as an unidimensional collection (such as a list or tuple), what you get back are the keys. It's the same if you call list(my_dict) , you get back ["name", "age"] .

Often when we talk about "dictionary destructuring", what most people refer to is the **kwargs part of functions. If you want to learn more about that, check out Day 17 of our 30 Days of Python free course, Flexible Functions with *args and **kwargs** .

If you wanted to destructure the dictionary values only, you can do:

Note that destructuring dictionaries only works well in modern Python versions (3.7+), because dictionaries are ordered collections now. In older Python versions, dictionaries were unordered (so items could move around). It didn't make sense to do dictionary destructuring then.

Destructuring in for loops

A couple of months back, we wrote a snippet post on enumerate , which is a vital component in writing good, Pythonic loops. In case you're not familiar, the syntax for enumerate looks like this:

enumerate takes an iterable collection as an argument and returns an enumerate object containing a tuple for each item in the collection. Each tuple contains a counter value, which increments with each iteration, along with a value from the provided collection.

As you can see in the example above, we provide two variable names when creating our for loop: counter and letter . This is actually a very common example of destructuring. For each tuple in the enumerate object, the first value gets assigned to counter , and the second value is assigned to letter .

Once again, there isn't any magic going on here with the variable names, and the assignment is entirely based on the order of the values. If we were to switch the position of counter and letter , we'd end up with some confusing variable names.

It's possible to do this type of destructuring with as many values as we like, and this is not limited to the enumerate function. For example, we could do something like this:

Here we break apart each tuple in the people list, assigning the values to name , age , and profession respectively.

This is a lot better than something like the code below, where we rely on indices instead of good descriptive names:

Raymond Hettinger — one of the core Python developers — said something in one of his talks that really stuck with me. He said that in Python, you should basically never be referring to items by their index: there is nearly always a better way.

Destructuring is a good example of one of those better ways.

Ignoring Values

So, what do we do if we have a collection of values and we don't want to assign all of them? We can use an _ in place of a variable name.

For example, if we take one of the tuples from the the people list above, and we only care about the name and profession, we can do the following:

This would also work just as well inside a loop, and can also be used when we don't care about any of the values. An example might be when using range to ensure a set number of iterations.

Using * to Collect Values

In some situations, we might want to isolate one or two values in a collection, and then keep the other items together. We featured an example of a situation like this our post on list rotation .

In Python, we can use the * operator to collect leftover values when performing a destructuring assignment. For example, we might have a list of numbers, and we want to grab the first number, and then assign the remaining numbers to a second variable:

Here, the first value ( 1 ) is assigned to head , while the rest of the numbers end up in a new list called tail .

We can also do this the other way around, creating a new list with everything but the last value, and assigning the last value to its own variable.

This is interesting, but unless we need to preserve the original list, we already have the pop method for stuff like this. However, we can do something else with this syntax. We can can assign any number of variables, and then gather up the remainder. We might grab the first and last items, and then gather up the middle, for example:

Alternatively, we might want to grab the first three items and then bundle up the rest:

There are tonnes of possibilities.

Wrapping Up

I hope you learnt something new from this short post on destructuring in Python!

This isn't actually the end of the story, as there are also ways to pack and unpack collections using * and ** , so feel free to read this article to learn more.

If you're learning Python and you find this kind of content interesting, be sure to follow us on Twitter or sign up to our mailing list to stay up to date with all out content. There's a form at the bottom of the page if you're interested.

We also just did a big update to our Complete Python Course , so check that out if you're interested in getting to an advanced level in Python. We have a 30 day money back guarantee, so you really have nothing to lose by giving it a try. We'd love to have you!

• Python »
• 3.12.3 Documentation »
• The Python Standard Library »
• Generic Operating System Services »
• ctypes — A foreign function library for Python
• Theme Auto Light Dark |

ctypes — A foreign function library for Python ¶

Source code: Lib/ctypes

ctypes is a foreign function library for Python. It provides C compatible data types, and allows calling functions in DLLs or shared libraries. It can be used to wrap these libraries in pure Python.

ctypes tutorial ¶

Note: The code samples in this tutorial use doctest to make sure that they actually work. Since some code samples behave differently under Linux, Windows, or macOS, they contain doctest directives in comments.

Note: Some code samples reference the ctypes c_int type. On platforms where sizeof(long) == sizeof(int) it is an alias to c_long . So, you should not be confused if c_long is printed if you would expect c_int — they are actually the same type.

Loading dynamic link libraries ¶

ctypes exports the cdll , and on Windows windll and oledll objects, for loading dynamic link libraries.

You load libraries by accessing them as attributes of these objects. cdll loads libraries which export functions using the standard cdecl calling convention, while windll libraries call functions using the stdcall calling convention. oledll also uses the stdcall calling convention, and assumes the functions return a Windows HRESULT error code. The error code is used to automatically raise an OSError exception when the function call fails.

Changed in version 3.3: Windows errors used to raise WindowsError , which is now an alias of OSError .

Here are some examples for Windows. Note that msvcrt is the MS standard C library containing most standard C functions, and uses the cdecl calling convention:

Windows appends the usual .dll file suffix automatically.

Accessing the standard C library through cdll.msvcrt will use an outdated version of the library that may be incompatible with the one being used by Python. Where possible, use native Python functionality, or else import and use the msvcrt module.

On Linux, it is required to specify the filename including the extension to load a library, so attribute access can not be used to load libraries. Either the LoadLibrary() method of the dll loaders should be used, or you should load the library by creating an instance of CDLL by calling the constructor:

Accessing functions from loaded dlls ¶

Functions are accessed as attributes of dll objects:

Note that win32 system dlls like kernel32 and user32 often export ANSI as well as UNICODE versions of a function. The UNICODE version is exported with an W appended to the name, while the ANSI version is exported with an A appended to the name. The win32 GetModuleHandle function, which returns a module handle for a given module name, has the following C prototype, and a macro is used to expose one of them as GetModuleHandle depending on whether UNICODE is defined or not:

windll does not try to select one of them by magic, you must access the version you need by specifying GetModuleHandleA or GetModuleHandleW explicitly, and then call it with bytes or string objects respectively.

Sometimes, dlls export functions with names which aren’t valid Python identifiers, like "??2@YAPAXI@Z" . In this case you have to use getattr() to retrieve the function:

On Windows, some dlls export functions not by name but by ordinal. These functions can be accessed by indexing the dll object with the ordinal number:

Calling functions ¶

You can call these functions like any other Python callable. This example uses the rand() function, which takes no arguments and returns a pseudo-random integer:

On Windows, you can call the GetModuleHandleA() function, which returns a win32 module handle (passing None as single argument to call it with a NULL pointer):

ValueError is raised when you call an stdcall function with the cdecl calling convention, or vice versa:

To find out the correct calling convention you have to look into the C header file or the documentation for the function you want to call.

On Windows, ctypes uses win32 structured exception handling to prevent crashes from general protection faults when functions are called with invalid argument values:

There are, however, enough ways to crash Python with ctypes , so you should be careful anyway. The faulthandler module can be helpful in debugging crashes (e.g. from segmentation faults produced by erroneous C library calls).

None , integers, bytes objects and (unicode) strings are the only native Python objects that can directly be used as parameters in these function calls. None is passed as a C NULL pointer, bytes objects and strings are passed as pointer to the memory block that contains their data ( char * or wchar_t * ). Python integers are passed as the platforms default C int type, their value is masked to fit into the C type.

Before we move on calling functions with other parameter types, we have to learn more about ctypes data types.

Fundamental data types ¶

ctypes defines a number of primitive C compatible data types:

The constructor accepts any object with a truth value.

All these types can be created by calling them with an optional initializer of the correct type and value:

Since these types are mutable, their value can also be changed afterwards:

Assigning a new value to instances of the pointer types c_char_p , c_wchar_p , and c_void_p changes the memory location they point to, not the contents of the memory block (of course not, because Python bytes objects are immutable):

You should be careful, however, not to pass them to functions expecting pointers to mutable memory. If you need mutable memory blocks, ctypes has a create_string_buffer() function which creates these in various ways. The current memory block contents can be accessed (or changed) with the raw property; if you want to access it as NUL terminated string, use the value property:

The create_string_buffer() function replaces the old c_buffer() function (which is still available as an alias). To create a mutable memory block containing unicode characters of the C type wchar_t , use the create_unicode_buffer() function.

Calling functions, continued ¶

Note that printf prints to the real standard output channel, not to sys.stdout , so these examples will only work at the console prompt, not from within IDLE or PythonWin :

As has been mentioned before, all Python types except integers, strings, and bytes objects have to be wrapped in their corresponding ctypes type, so that they can be converted to the required C data type:

Calling variadic functions ¶

On a lot of platforms calling variadic functions through ctypes is exactly the same as calling functions with a fixed number of parameters. On some platforms, and in particular ARM64 for Apple Platforms, the calling convention for variadic functions is different than that for regular functions.

On those platforms it is required to specify the argtypes attribute for the regular, non-variadic, function arguments:

Because specifying the attribute does not inhibit portability it is advised to always specify argtypes for all variadic functions.

Calling functions with your own custom data types ¶

You can also customize ctypes argument conversion to allow instances of your own classes be used as function arguments. ctypes looks for an _as_parameter_ attribute and uses this as the function argument. The attribute must be an integer, string, bytes, a ctypes instance, or an object with an _as_parameter_ attribute:

If you don’t want to store the instance’s data in the _as_parameter_ instance variable, you could define a property which makes the attribute available on request.

Specifying the required argument types (function prototypes) ¶

It is possible to specify the required argument types of functions exported from DLLs by setting the argtypes attribute.

argtypes must be a sequence of C data types (the printf() function is probably not a good example here, because it takes a variable number and different types of parameters depending on the format string, on the other hand this is quite handy to experiment with this feature):

Specifying a format protects against incompatible argument types (just as a prototype for a C function), and tries to convert the arguments to valid types:

If you have defined your own classes which you pass to function calls, you have to implement a from_param() class method for them to be able to use them in the argtypes sequence. The from_param() class method receives the Python object passed to the function call, it should do a typecheck or whatever is needed to make sure this object is acceptable, and then return the object itself, its _as_parameter_ attribute, or whatever you want to pass as the C function argument in this case. Again, the result should be an integer, string, bytes, a ctypes instance, or an object with an _as_parameter_ attribute.

Return types ¶

By default functions are assumed to return the C int type. Other return types can be specified by setting the restype attribute of the function object.

The C prototype of time() is time_t time(time_t *) . Because time_t might be of a different type than the default return type int , you should specify the restype attribute:

The argument types can be specified using argtypes :

To call the function with a NULL pointer as first argument, use None :

Here is a more advanced example, it uses the strchr() function, which expects a string pointer and a char, and returns a pointer to a string:

If you want to avoid the ord("x") calls above, you can set the argtypes attribute, and the second argument will be converted from a single character Python bytes object into a C char:

You can also use a callable Python object (a function or a class for example) as the restype attribute, if the foreign function returns an integer. The callable will be called with the integer the C function returns, and the result of this call will be used as the result of your function call. This is useful to check for error return values and automatically raise an exception:

WinError is a function which will call Windows FormatMessage() api to get the string representation of an error code, and returns an exception. WinError takes an optional error code parameter, if no one is used, it calls GetLastError() to retrieve it.

Please note that a much more powerful error checking mechanism is available through the errcheck attribute; see the reference manual for details.

Passing pointers (or: passing parameters by reference) ¶

Sometimes a C api function expects a pointer to a data type as parameter, probably to write into the corresponding location, or if the data is too large to be passed by value. This is also known as passing parameters by reference .

ctypes exports the byref() function which is used to pass parameters by reference. The same effect can be achieved with the pointer() function, although pointer() does a lot more work since it constructs a real pointer object, so it is faster to use byref() if you don’t need the pointer object in Python itself:

Structures and unions ¶

Structures and unions must derive from the Structure and Union base classes which are defined in the ctypes module. Each subclass must define a _fields_ attribute. _fields_ must be a list of 2-tuples , containing a field name and a field type .

The field type must be a ctypes type like c_int , or any other derived ctypes type: structure, union, array, pointer.

Here is a simple example of a POINT structure, which contains two integers named x and y , and also shows how to initialize a structure in the constructor:

You can, however, build much more complicated structures. A structure can itself contain other structures by using a structure as a field type.

Here is a RECT structure which contains two POINTs named upperleft and lowerright :

Nested structures can also be initialized in the constructor in several ways:

Field descriptor s can be retrieved from the class , they are useful for debugging because they can provide useful information:

ctypes does not support passing unions or structures with bit-fields to functions by value. While this may work on 32-bit x86, it’s not guaranteed by the library to work in the general case. Unions and structures with bit-fields should always be passed to functions by pointer.

Structure/union alignment and byte order ¶

By default, Structure and Union fields are aligned in the same way the C compiler does it. It is possible to override this behavior by specifying a _pack_ class attribute in the subclass definition. This must be set to a positive integer and specifies the maximum alignment for the fields. This is what #pragma pack(n) also does in MSVC.

ctypes uses the native byte order for Structures and Unions. To build structures with non-native byte order, you can use one of the BigEndianStructure , LittleEndianStructure , BigEndianUnion , and LittleEndianUnion base classes. These classes cannot contain pointer fields.

Bit fields in structures and unions ¶

It is possible to create structures and unions containing bit fields. Bit fields are only possible for integer fields, the bit width is specified as the third item in the _fields_ tuples:

Arrays are sequences, containing a fixed number of instances of the same type.

The recommended way to create array types is by multiplying a data type with a positive integer:

Here is an example of a somewhat artificial data type, a structure containing 4 POINTs among other stuff:

Instances are created in the usual way, by calling the class:

The above code print a series of 0 0 lines, because the array contents is initialized to zeros.

Initializers of the correct type can also be specified:

Pointer instances are created by calling the pointer() function on a ctypes type:

Pointer instances have a contents attribute which returns the object to which the pointer points, the i object above:

Note that ctypes does not have OOR (original object return), it constructs a new, equivalent object each time you retrieve an attribute:

Assigning another c_int instance to the pointer’s contents attribute would cause the pointer to point to the memory location where this is stored:

Pointer instances can also be indexed with integers:

Assigning to an integer index changes the pointed to value:

It is also possible to use indexes different from 0, but you must know what you’re doing, just as in C: You can access or change arbitrary memory locations. Generally you only use this feature if you receive a pointer from a C function, and you know that the pointer actually points to an array instead of a single item.

Behind the scenes, the pointer() function does more than simply create pointer instances, it has to create pointer types first. This is done with the POINTER() function, which accepts any ctypes type, and returns a new type:

Calling the pointer type without an argument creates a NULL pointer. NULL pointers have a False boolean value:

ctypes checks for NULL when dereferencing pointers (but dereferencing invalid non- NULL pointers would crash Python):

Type conversions ¶

Usually, ctypes does strict type checking. This means, if you have POINTER(c_int) in the argtypes list of a function or as the type of a member field in a structure definition, only instances of exactly the same type are accepted. There are some exceptions to this rule, where ctypes accepts other objects. For example, you can pass compatible array instances instead of pointer types. So, for POINTER(c_int) , ctypes accepts an array of c_int:

In addition, if a function argument is explicitly declared to be a pointer type (such as POINTER(c_int) ) in argtypes , an object of the pointed type ( c_int in this case) can be passed to the function. ctypes will apply the required byref() conversion in this case automatically.

To set a POINTER type field to NULL , you can assign None :

Sometimes you have instances of incompatible types. In C, you can cast one type into another type. ctypes provides a cast() function which can be used in the same way. The Bar structure defined above accepts POINTER(c_int) pointers or c_int arrays for its values field, but not instances of other types:

For these cases, the cast() function is handy.

The cast() function can be used to cast a ctypes instance into a pointer to a different ctypes data type. cast() takes two parameters, a ctypes object that is or can be converted to a pointer of some kind, and a ctypes pointer type. It returns an instance of the second argument, which references the same memory block as the first argument:

So, cast() can be used to assign to the values field of Bar the structure:

Incomplete Types ¶

Incomplete Types are structures, unions or arrays whose members are not yet specified. In C, they are specified by forward declarations, which are defined later:

The straightforward translation into ctypes code would be this, but it does not work:

because the new class cell is not available in the class statement itself. In ctypes , we can define the cell class and set the _fields_ attribute later, after the class statement:

Let’s try it. We create two instances of cell , and let them point to each other, and finally follow the pointer chain a few times:

Callback functions ¶

ctypes allows creating C callable function pointers from Python callables. These are sometimes called callback functions .

First, you must create a class for the callback function. The class knows the calling convention, the return type, and the number and types of arguments this function will receive.

The CFUNCTYPE() factory function creates types for callback functions using the cdecl calling convention. On Windows, the WINFUNCTYPE() factory function creates types for callback functions using the stdcall calling convention.

Both of these factory functions are called with the result type as first argument, and the callback functions expected argument types as the remaining arguments.

I will present an example here which uses the standard C library’s qsort() function, that is used to sort items with the help of a callback function. qsort() will be used to sort an array of integers:

qsort() must be called with a pointer to the data to sort, the number of items in the data array, the size of one item, and a pointer to the comparison function, the callback. The callback will then be called with two pointers to items, and it must return a negative integer if the first item is smaller than the second, a zero if they are equal, and a positive integer otherwise.

So our callback function receives pointers to integers, and must return an integer. First we create the type for the callback function:

To get started, here is a simple callback that shows the values it gets passed:

The result:

Now we can actually compare the two items and return a useful result:

As we can easily check, our array is sorted now:

The function factories can be used as decorator factories, so we may as well write:

Make sure you keep references to CFUNCTYPE() objects as long as they are used from C code. ctypes doesn’t, and if you don’t, they may be garbage collected, crashing your program when a callback is made.

Also, note that if the callback function is called in a thread created outside of Python’s control (e.g. by the foreign code that calls the callback), ctypes creates a new dummy Python thread on every invocation. This behavior is correct for most purposes, but it means that values stored with threading.local will not survive across different callbacks, even when those calls are made from the same C thread.

Accessing values exported from dlls ¶

Some shared libraries not only export functions, they also export variables. An example in the Python library itself is the Py_Version , Python runtime version number encoded in a single constant integer.

ctypes can access values like this with the in_dll() class methods of the type. pythonapi is a predefined symbol giving access to the Python C api:

An extended example which also demonstrates the use of pointers accesses the PyImport_FrozenModules pointer exported by Python.

Quoting the docs for that value:

This pointer is initialized to point to an array of _frozen records, terminated by one whose members are all NULL or zero. When a frozen module is imported, it is searched in this table. Third-party code could play tricks with this to provide a dynamically created collection of frozen modules.

So manipulating this pointer could even prove useful. To restrict the example size, we show only how this table can be read with ctypes :

We have defined the _frozen data type, so we can get the pointer to the table:

Since table is a pointer to the array of struct_frozen records, we can iterate over it, but we just have to make sure that our loop terminates, because pointers have no size. Sooner or later it would probably crash with an access violation or whatever, so it’s better to break out of the loop when we hit the NULL entry:

The fact that standard Python has a frozen module and a frozen package (indicated by the negative size member) is not well known, it is only used for testing. Try it out with import __hello__ for example.

Surprises ¶

There are some edges in ctypes where you might expect something other than what actually happens.

Consider the following example:

Hm. We certainly expected the last statement to print 3 4 1 2 . What happened? Here are the steps of the rc.a, rc.b = rc.b, rc.a line above:

Note that temp0 and temp1 are objects still using the internal buffer of the rc object above. So executing rc.a = temp0 copies the buffer contents of temp0 into rc ‘s buffer. This, in turn, changes the contents of temp1 . So, the last assignment rc.b = temp1 , doesn’t have the expected effect.

Keep in mind that retrieving sub-objects from Structure, Unions, and Arrays doesn’t copy the sub-object, instead it retrieves a wrapper object accessing the root-object’s underlying buffer.

Another example that may behave differently from what one would expect is this:

Objects instantiated from c_char_p can only have their value set to bytes or integers.

Why is it printing False ? ctypes instances are objects containing a memory block plus some descriptor s accessing the contents of the memory. Storing a Python object in the memory block does not store the object itself, instead the contents of the object is stored. Accessing the contents again constructs a new Python object each time!

Variable-sized data types ¶

ctypes provides some support for variable-sized arrays and structures.

The resize() function can be used to resize the memory buffer of an existing ctypes object. The function takes the object as first argument, and the requested size in bytes as the second argument. The memory block cannot be made smaller than the natural memory block specified by the objects type, a ValueError is raised if this is tried:

This is nice and fine, but how would one access the additional elements contained in this array? Since the type still only knows about 4 elements, we get errors accessing other elements:

Another way to use variable-sized data types with ctypes is to use the dynamic nature of Python, and (re-)define the data type after the required size is already known, on a case by case basis.

ctypes reference ¶

Finding shared libraries ¶.

When programming in a compiled language, shared libraries are accessed when compiling/linking a program, and when the program is run.

The purpose of the find_library() function is to locate a library in a way similar to what the compiler or runtime loader does (on platforms with several versions of a shared library the most recent should be loaded), while the ctypes library loaders act like when a program is run, and call the runtime loader directly.

The ctypes.util module provides a function which can help to determine the library to load.

Try to find a library and return a pathname. name is the library name without any prefix like lib , suffix like .so , .dylib or version number (this is the form used for the posix linker option -l ). If no library can be found, returns None .

The exact functionality is system dependent.

On Linux, find_library() tries to run external programs ( /sbin/ldconfig , gcc , objdump and ld ) to find the library file. It returns the filename of the library file.

Changed in version 3.6: On Linux, the value of the environment variable LD_LIBRARY_PATH is used when searching for libraries, if a library cannot be found by any other means.

Here are some examples:

On macOS, find_library() tries several predefined naming schemes and paths to locate the library, and returns a full pathname if successful:

On Windows, find_library() searches along the system search path, and returns the full pathname, but since there is no predefined naming scheme a call like find_library("c") will fail and return None .

If wrapping a shared library with ctypes , it may be better to determine the shared library name at development time, and hardcode that into the wrapper module instead of using find_library() to locate the library at runtime.

Loading shared libraries ¶

There are several ways to load shared libraries into the Python process. One way is to instantiate one of the following classes:

Instances of this class represent loaded shared libraries. Functions in these libraries use the standard C calling convention, and are assumed to return int .

On Windows creating a CDLL instance may fail even if the DLL name exists. When a dependent DLL of the loaded DLL is not found, a OSError error is raised with the message “[WinError 126] The specified module could not be found”. This error message does not contain the name of the missing DLL because the Windows API does not return this information making this error hard to diagnose. To resolve this error and determine which DLL is not found, you need to find the list of dependent DLLs and determine which one is not found using Windows debugging and tracing tools.

Changed in version 3.12: The name parameter can now be a path-like object .

Microsoft DUMPBIN tool – A tool to find DLL dependents.

Windows only: Instances of this class represent loaded shared libraries, functions in these libraries use the stdcall calling convention, and are assumed to return the windows specific HRESULT code. HRESULT values contain information specifying whether the function call failed or succeeded, together with additional error code. If the return value signals a failure, an OSError is automatically raised.

Changed in version 3.3: WindowsError used to be raised, which is now an alias of OSError .

Windows only: Instances of this class represent loaded shared libraries, functions in these libraries use the stdcall calling convention, and are assumed to return int by default.

The Python global interpreter lock is released before calling any function exported by these libraries, and reacquired afterwards.

Instances of this class behave like CDLL instances, except that the Python GIL is not released during the function call, and after the function execution the Python error flag is checked. If the error flag is set, a Python exception is raised.

Thus, this is only useful to call Python C api functions directly.

All these classes can be instantiated by calling them with at least one argument, the pathname of the shared library. If you have an existing handle to an already loaded shared library, it can be passed as the handle named parameter, otherwise the underlying platforms dlopen() or LoadLibrary() function is used to load the library into the process, and to get a handle to it.

The mode parameter can be used to specify how the library is loaded. For details, consult the dlopen(3) manpage. On Windows, mode is ignored. On posix systems, RTLD_NOW is always added, and is not configurable.

The use_errno parameter, when set to true, enables a ctypes mechanism that allows accessing the system errno error number in a safe way. ctypes maintains a thread-local copy of the systems errno variable; if you call foreign functions created with use_errno=True then the errno value before the function call is swapped with the ctypes private copy, the same happens immediately after the function call.

The function ctypes.get_errno() returns the value of the ctypes private copy, and the function ctypes.set_errno() changes the ctypes private copy to a new value and returns the former value.

The use_last_error parameter, when set to true, enables the same mechanism for the Windows error code which is managed by the GetLastError() and SetLastError() Windows API functions; ctypes.get_last_error() and ctypes.set_last_error() are used to request and change the ctypes private copy of the windows error code.

The winmode parameter is used on Windows to specify how the library is loaded (since mode is ignored). It takes any value that is valid for the Win32 API LoadLibraryEx flags parameter. When omitted, the default is to use the flags that result in the most secure DLL load, which avoids issues such as DLL hijacking. Passing the full path to the DLL is the safest way to ensure the correct library and dependencies are loaded.

Changed in version 3.8: Added winmode parameter.

Flag to use as mode parameter. On platforms where this flag is not available, it is defined as the integer zero.

Flag to use as mode parameter. On platforms where this is not available, it is the same as RTLD_GLOBAL .

The default mode which is used to load shared libraries. On OSX 10.3, this is RTLD_GLOBAL , otherwise it is the same as RTLD_LOCAL .

Instances of these classes have no public methods. Functions exported by the shared library can be accessed as attributes or by index. Please note that accessing the function through an attribute caches the result and therefore accessing it repeatedly returns the same object each time. On the other hand, accessing it through an index returns a new object each time:

The following public attributes are available, their name starts with an underscore to not clash with exported function names:

The system handle used to access the library.

The name of the library passed in the constructor.

Shared libraries can also be loaded by using one of the prefabricated objects, which are instances of the LibraryLoader class, either by calling the LoadLibrary() method, or by retrieving the library as attribute of the loader instance.

Class which loads shared libraries. dlltype should be one of the CDLL , PyDLL , WinDLL , or OleDLL types.

__getattr__() has special behavior: It allows loading a shared library by accessing it as attribute of a library loader instance. The result is cached, so repeated attribute accesses return the same library each time.

Load a shared library into the process and return it. This method always returns a new instance of the library.

These prefabricated library loaders are available:

Creates CDLL instances.

Windows only: Creates WinDLL instances.

Windows only: Creates OleDLL instances.

Creates PyDLL instances.

For accessing the C Python api directly, a ready-to-use Python shared library object is available:

An instance of PyDLL that exposes Python C API functions as attributes. Note that all these functions are assumed to return C int , which is of course not always the truth, so you have to assign the correct restype attribute to use these functions.

Loading a library through any of these objects raises an auditing event ctypes.dlopen with string argument name , the name used to load the library.

Accessing a function on a loaded library raises an auditing event ctypes.dlsym with arguments library (the library object) and name (the symbol’s name as a string or integer).

In cases when only the library handle is available rather than the object, accessing a function raises an auditing event ctypes.dlsym/handle with arguments handle (the raw library handle) and name .

Foreign functions ¶

As explained in the previous section, foreign functions can be accessed as attributes of loaded shared libraries. The function objects created in this way by default accept any number of arguments, accept any ctypes data instances as arguments, and return the default result type specified by the library loader. They are instances of a private class:

Base class for C callable foreign functions.

Instances of foreign functions are also C compatible data types; they represent C function pointers.

This behavior can be customized by assigning to special attributes of the foreign function object.

Assign a ctypes type to specify the result type of the foreign function. Use None for void , a function not returning anything.

It is possible to assign a callable Python object that is not a ctypes type, in this case the function is assumed to return a C int , and the callable will be called with this integer, allowing further processing or error checking. Using this is deprecated, for more flexible post processing or error checking use a ctypes data type as restype and assign a callable to the errcheck attribute.

Assign a tuple of ctypes types to specify the argument types that the function accepts. Functions using the stdcall calling convention can only be called with the same number of arguments as the length of this tuple; functions using the C calling convention accept additional, unspecified arguments as well.

When a foreign function is called, each actual argument is passed to the from_param() class method of the items in the argtypes tuple, this method allows adapting the actual argument to an object that the foreign function accepts. For example, a c_char_p item in the argtypes tuple will convert a string passed as argument into a bytes object using ctypes conversion rules.

New: It is now possible to put items in argtypes which are not ctypes types, but each item must have a from_param() method which returns a value usable as argument (integer, string, ctypes instance). This allows defining adapters that can adapt custom objects as function parameters.

Assign a Python function or another callable to this attribute. The callable will be called with three or more arguments:

result is what the foreign function returns, as specified by the restype attribute.

func is the foreign function object itself, this allows reusing the same callable object to check or post process the results of several functions.

arguments is a tuple containing the parameters originally passed to the function call, this allows specializing the behavior on the arguments used.

The object that this function returns will be returned from the foreign function call, but it can also check the result value and raise an exception if the foreign function call failed.

This exception is raised when a foreign function call cannot convert one of the passed arguments.

On Windows, when a foreign function call raises a system exception (for example, due to an access violation), it will be captured and replaced with a suitable Python exception. Further, an auditing event ctypes.set_exception with argument code will be raised, allowing an audit hook to replace the exception with its own.

Some ways to invoke foreign function calls may raise an auditing event ctypes.call_function with arguments function pointer and arguments .

Function prototypes ¶

Foreign functions can also be created by instantiating function prototypes. Function prototypes are similar to function prototypes in C; they describe a function (return type, argument types, calling convention) without defining an implementation. The factory functions must be called with the desired result type and the argument types of the function, and can be used as decorator factories, and as such, be applied to functions through the @wrapper syntax. See Callback functions for examples.

The returned function prototype creates functions that use the standard C calling convention. The function will release the GIL during the call. If use_errno is set to true, the ctypes private copy of the system errno variable is exchanged with the real errno value before and after the call; use_last_error does the same for the Windows error code.

Windows only: The returned function prototype creates functions that use the stdcall calling convention. The function will release the GIL during the call. use_errno and use_last_error have the same meaning as above.

The returned function prototype creates functions that use the Python calling convention. The function will not release the GIL during the call.

Function prototypes created by these factory functions can be instantiated in different ways, depending on the type and number of the parameters in the call:

Returns a foreign function at the specified address which must be an integer.

Create a C callable function (a callback function) from a Python callable .

Returns a foreign function exported by a shared library. func_spec must be a 2-tuple (name_or_ordinal, library) . The first item is the name of the exported function as string, or the ordinal of the exported function as small integer. The second item is the shared library instance.

Returns a foreign function that will call a COM method. vtbl_index is the index into the virtual function table, a small non-negative integer. name is name of the COM method. iid is an optional pointer to the interface identifier which is used in extended error reporting.

COM methods use a special calling convention: They require a pointer to the COM interface as first argument, in addition to those parameters that are specified in the argtypes tuple.

The optional paramflags parameter creates foreign function wrappers with much more functionality than the features described above.

paramflags must be a tuple of the same length as argtypes .

Each item in this tuple contains further information about a parameter, it must be a tuple containing one, two, or three items.

The first item is an integer containing a combination of direction flags for the parameter:

1 Specifies an input parameter to the function. 2 Output parameter. The foreign function fills in a value. 4 Input parameter which defaults to the integer zero.

The optional second item is the parameter name as string. If this is specified, the foreign function can be called with named parameters.

The optional third item is the default value for this parameter.

The following example demonstrates how to wrap the Windows MessageBoxW function so that it supports default parameters and named arguments. The C declaration from the windows header file is this:

Here is the wrapping with ctypes :

The MessageBox foreign function can now be called in these ways:

A second example demonstrates output parameters. The win32 GetWindowRect function retrieves the dimensions of a specified window by copying them into RECT structure that the caller has to supply. Here is the C declaration:

Functions with output parameters will automatically return the output parameter value if there is a single one, or a tuple containing the output parameter values when there are more than one, so the GetWindowRect function now returns a RECT instance, when called.

Output parameters can be combined with the errcheck protocol to do further output processing and error checking. The win32 GetWindowRect api function returns a BOOL to signal success or failure, so this function could do the error checking, and raises an exception when the api call failed:

If the errcheck function returns the argument tuple it receives unchanged, ctypes continues the normal processing it does on the output parameters. If you want to return a tuple of window coordinates instead of a RECT instance, you can retrieve the fields in the function and return them instead, the normal processing will no longer take place:

Utility functions ¶

Returns the address of the memory buffer as integer. obj must be an instance of a ctypes type.

Raises an auditing event ctypes.addressof with argument obj .

Returns the alignment requirements of a ctypes type. obj_or_type must be a ctypes type or instance.

Returns a light-weight pointer to obj , which must be an instance of a ctypes type. offset defaults to zero, and must be an integer that will be added to the internal pointer value.

byref(obj, offset) corresponds to this C code:

The returned object can only be used as a foreign function call parameter. It behaves similar to pointer(obj) , but the construction is a lot faster.

This function is similar to the cast operator in C. It returns a new instance of type which points to the same memory block as obj . type must be a pointer type, and obj must be an object that can be interpreted as a pointer.

This function creates a mutable character buffer. The returned object is a ctypes array of c_char .

init_or_size must be an integer which specifies the size of the array, or a bytes object which will be used to initialize the array items.

If a bytes object is specified as first argument, the buffer is made one item larger than its length so that the last element in the array is a NUL termination character. An integer can be passed as second argument which allows specifying the size of the array if the length of the bytes should not be used.

Raises an auditing event ctypes.create_string_buffer with arguments init , size .

This function creates a mutable unicode character buffer. The returned object is a ctypes array of c_wchar .

init_or_size must be an integer which specifies the size of the array, or a string which will be used to initialize the array items.

If a string is specified as first argument, the buffer is made one item larger than the length of the string so that the last element in the array is a NUL termination character. An integer can be passed as second argument which allows specifying the size of the array if the length of the string should not be used.

Raises an auditing event ctypes.create_unicode_buffer with arguments init , size .

Windows only: This function is a hook which allows implementing in-process COM servers with ctypes. It is called from the DllCanUnloadNow function that the _ctypes extension dll exports.

Windows only: This function is a hook which allows implementing in-process COM servers with ctypes. It is called from the DllGetClassObject function that the _ctypes extension dll exports.

Windows only: return the filename of the VC runtime library used by Python, and by the extension modules. If the name of the library cannot be determined, None is returned.

If you need to free memory, for example, allocated by an extension module with a call to the free(void *) , it is important that you use the function in the same library that allocated the memory.

Windows only: Returns a textual description of the error code code . If no error code is specified, the last error code is used by calling the Windows api function GetLastError.

Windows only: Returns the last error code set by Windows in the calling thread. This function calls the Windows GetLastError() function directly, it does not return the ctypes-private copy of the error code.

Returns the current value of the ctypes-private copy of the system errno variable in the calling thread.

Raises an auditing event ctypes.get_errno with no arguments.

Windows only: returns the current value of the ctypes-private copy of the system LastError variable in the calling thread.

Raises an auditing event ctypes.get_last_error with no arguments.

Same as the standard C memmove library function: copies count bytes from src to dst . dst and src must be integers or ctypes instances that can be converted to pointers.

Same as the standard C memset library function: fills the memory block at address dst with count bytes of value c . dst must be an integer specifying an address, or a ctypes instance.

Create and return a new ctypes pointer type. Pointer types are cached and reused internally, so calling this function repeatedly is cheap. type must be a ctypes type.

Create a new pointer instance, pointing to obj . The returned object is of the type POINTER(type(obj)) .

Note: If you just want to pass a pointer to an object to a foreign function call, you should use byref(obj) which is much faster.

This function resizes the internal memory buffer of obj , which must be an instance of a ctypes type. It is not possible to make the buffer smaller than the native size of the objects type, as given by sizeof(type(obj)) , but it is possible to enlarge the buffer.

Set the current value of the ctypes-private copy of the system errno variable in the calling thread to value and return the previous value.

Raises an auditing event ctypes.set_errno with argument errno .

Windows only: set the current value of the ctypes-private copy of the system LastError variable in the calling thread to value and return the previous value.

Raises an auditing event ctypes.set_last_error with argument error .

Returns the size in bytes of a ctypes type or instance memory buffer. Does the same as the C sizeof operator.

Return the byte string at void *ptr . If size is specified, it is used as size, otherwise the string is assumed to be zero-terminated.

Raises an auditing event ctypes.string_at with arguments ptr , size .

Windows only: this function is probably the worst-named thing in ctypes. It creates an instance of OSError . If code is not specified, GetLastError is called to determine the error code. If descr is not specified, FormatError() is called to get a textual description of the error.

Changed in version 3.3: An instance of WindowsError used to be created, which is now an alias of OSError .

Return the wide-character string at void *ptr . If size is specified, it is used as the number of characters of the string, otherwise the string is assumed to be zero-terminated.

Raises an auditing event ctypes.wstring_at with arguments ptr , size .

Data types ¶

This non-public class is the common base class of all ctypes data types. Among other things, all ctypes type instances contain a memory block that hold C compatible data; the address of the memory block is returned by the addressof() helper function. Another instance variable is exposed as _objects ; this contains other Python objects that need to be kept alive in case the memory block contains pointers.

Common methods of ctypes data types, these are all class methods (to be exact, they are methods of the metaclass ):

This method returns a ctypes instance that shares the buffer of the source object. The source object must support the writeable buffer interface. The optional offset parameter specifies an offset into the source buffer in bytes; the default is zero. If the source buffer is not large enough a ValueError is raised.

Raises an auditing event ctypes.cdata/buffer with arguments pointer , size , offset .

This method creates a ctypes instance, copying the buffer from the source object buffer which must be readable. The optional offset parameter specifies an offset into the source buffer in bytes; the default is zero. If the source buffer is not large enough a ValueError is raised.

This method returns a ctypes type instance using the memory specified by address which must be an integer.

This method, and others that indirectly call this method, raises an auditing event ctypes.cdata with argument address .

This method adapts obj to a ctypes type. It is called with the actual object used in a foreign function call when the type is present in the foreign function’s argtypes tuple; it must return an object that can be used as a function call parameter.

All ctypes data types have a default implementation of this classmethod that normally returns obj if that is an instance of the type. Some types accept other objects as well.

This method returns a ctypes type instance exported by a shared library. name is the name of the symbol that exports the data, library is the loaded shared library.

Common instance variables of ctypes data types:

Sometimes ctypes data instances do not own the memory block they contain, instead they share part of the memory block of a base object. The _b_base_ read-only member is the root ctypes object that owns the memory block.

This read-only variable is true when the ctypes data instance has allocated the memory block itself, false otherwise.

This member is either None or a dictionary containing Python objects that need to be kept alive so that the memory block contents is kept valid. This object is only exposed for debugging; never modify the contents of this dictionary.

This non-public class is the base class of all fundamental ctypes data types. It is mentioned here because it contains the common attributes of the fundamental ctypes data types. _SimpleCData is a subclass of _CData , so it inherits their methods and attributes. ctypes data types that are not and do not contain pointers can now be pickled.

Instances have a single attribute:

This attribute contains the actual value of the instance. For integer and pointer types, it is an integer, for character types, it is a single character bytes object or string, for character pointer types it is a Python bytes object or string.

When the value attribute is retrieved from a ctypes instance, usually a new object is returned each time. ctypes does not implement original object return, always a new object is constructed. The same is true for all other ctypes object instances.

Fundamental data types, when returned as foreign function call results, or, for example, by retrieving structure field members or array items, are transparently converted to native Python types. In other words, if a foreign function has a restype of c_char_p , you will always receive a Python bytes object, not a c_char_p instance.

Subclasses of fundamental data types do not inherit this behavior. So, if a foreign functions restype is a subclass of c_void_p , you will receive an instance of this subclass from the function call. Of course, you can get the value of the pointer by accessing the value attribute.

These are the fundamental ctypes data types:

Represents the C signed char datatype, and interprets the value as small integer. The constructor accepts an optional integer initializer; no overflow checking is done.

Represents the C char datatype, and interprets the value as a single character. The constructor accepts an optional string initializer, the length of the string must be exactly one character.

Represents the C char * datatype when it points to a zero-terminated string. For a general character pointer that may also point to binary data, POINTER(c_char) must be used. The constructor accepts an integer address, or a bytes object.

Represents the C double datatype. The constructor accepts an optional float initializer.

Represents the C long double datatype. The constructor accepts an optional float initializer. On platforms where sizeof(long double) == sizeof(double) it is an alias to c_double .

Represents the C float datatype. The constructor accepts an optional float initializer.

Represents the C signed int datatype. The constructor accepts an optional integer initializer; no overflow checking is done. On platforms where sizeof(int) == sizeof(long) it is an alias to c_long .

Represents the C 8-bit signed int datatype. Usually an alias for c_byte .

Represents the C 16-bit signed int datatype. Usually an alias for c_short .

Represents the C 32-bit signed int datatype. Usually an alias for c_int .

Represents the C 64-bit signed int datatype. Usually an alias for c_longlong .

Represents the C signed long datatype. The constructor accepts an optional integer initializer; no overflow checking is done.

Represents the C signed long long datatype. The constructor accepts an optional integer initializer; no overflow checking is done.

Represents the C signed short datatype. The constructor accepts an optional integer initializer; no overflow checking is done.

Represents the C size_t datatype.

Represents the C ssize_t datatype.

Added in version 3.2.

Represents the C time_t datatype.

Added in version 3.12.

Represents the C unsigned char datatype, it interprets the value as small integer. The constructor accepts an optional integer initializer; no overflow checking is done.

Represents the C unsigned int datatype. The constructor accepts an optional integer initializer; no overflow checking is done. On platforms where sizeof(int) == sizeof(long) it is an alias for c_ulong .

Represents the C 8-bit unsigned int datatype. Usually an alias for c_ubyte .

Represents the C 16-bit unsigned int datatype. Usually an alias for c_ushort .

Represents the C 32-bit unsigned int datatype. Usually an alias for c_uint .

Represents the C 64-bit unsigned int datatype. Usually an alias for c_ulonglong .

Represents the C unsigned long datatype. The constructor accepts an optional integer initializer; no overflow checking is done.

Represents the C unsigned long long datatype. The constructor accepts an optional integer initializer; no overflow checking is done.

Represents the C unsigned short datatype. The constructor accepts an optional integer initializer; no overflow checking is done.

Represents the C void * type. The value is represented as integer. The constructor accepts an optional integer initializer.

Represents the C wchar_t datatype, and interprets the value as a single character unicode string. The constructor accepts an optional string initializer, the length of the string must be exactly one character.

Represents the C wchar_t * datatype, which must be a pointer to a zero-terminated wide character string. The constructor accepts an integer address, or a string.

Represent the C bool datatype (more accurately, _Bool from C99). Its value can be True or False , and the constructor accepts any object that has a truth value.

Windows only: Represents a HRESULT value, which contains success or error information for a function or method call.

Represents the C PyObject * datatype. Calling this without an argument creates a NULL PyObject * pointer.

The ctypes.wintypes module provides quite some other Windows specific data types, for example HWND , WPARAM , or DWORD . Some useful structures like MSG or RECT are also defined.

Structured data types ¶

Abstract base class for unions in native byte order.

Abstract base class for unions in big endian byte order.

Added in version 3.11.

Abstract base class for unions in little endian byte order.

Abstract base class for structures in big endian byte order.

Abstract base class for structures in little endian byte order.

Structures and unions with non-native byte order cannot contain pointer type fields, or any other data types containing pointer type fields.

Abstract base class for structures in native byte order.

Concrete structure and union types must be created by subclassing one of these types, and at least define a _fields_ class variable. ctypes will create descriptor s which allow reading and writing the fields by direct attribute accesses. These are the

A sequence defining the structure fields. The items must be 2-tuples or 3-tuples. The first item is the name of the field, the second item specifies the type of the field; it can be any ctypes data type.

For integer type fields like c_int , a third optional item can be given. It must be a small positive integer defining the bit width of the field.

Field names must be unique within one structure or union. This is not checked, only one field can be accessed when names are repeated.

It is possible to define the _fields_ class variable after the class statement that defines the Structure subclass, this allows creating data types that directly or indirectly reference themselves:

The _fields_ class variable must, however, be defined before the type is first used (an instance is created, sizeof() is called on it, and so on). Later assignments to the _fields_ class variable will raise an AttributeError.

It is possible to define sub-subclasses of structure types, they inherit the fields of the base class plus the _fields_ defined in the sub-subclass, if any.

An optional small integer that allows overriding the alignment of structure fields in the instance. _pack_ must already be defined when _fields_ is assigned, otherwise it will have no effect. Setting this attribute to 0 is the same as not setting it at all.

An optional sequence that lists the names of unnamed (anonymous) fields. _anonymous_ must be already defined when _fields_ is assigned, otherwise it will have no effect.

The fields listed in this variable must be structure or union type fields. ctypes will create descriptors in the structure type that allows accessing the nested fields directly, without the need to create the structure or union field.

Here is an example type (Windows):

The TYPEDESC structure describes a COM data type, the vt field specifies which one of the union fields is valid. Since the u field is defined as anonymous field, it is now possible to access the members directly off the TYPEDESC instance. td.lptdesc and td.u.lptdesc are equivalent, but the former is faster since it does not need to create a temporary union instance:

It is possible to define sub-subclasses of structures, they inherit the fields of the base class. If the subclass definition has a separate _fields_ variable, the fields specified in this are appended to the fields of the base class.

Structure and union constructors accept both positional and keyword arguments. Positional arguments are used to initialize member fields in the same order as they are appear in _fields_ . Keyword arguments in the constructor are interpreted as attribute assignments, so they will initialize _fields_ with the same name, or create new attributes for names not present in _fields_ .

Arrays and pointers ¶

Abstract base class for arrays.

The recommended way to create concrete array types is by multiplying any ctypes data type with a non-negative integer. Alternatively, you can subclass this type and define _length_ and _type_ class variables. Array elements can be read and written using standard subscript and slice accesses; for slice reads, the resulting object is not itself an Array .

A positive integer specifying the number of elements in the array. Out-of-range subscripts result in an IndexError . Will be returned by len() .

Specifies the type of each element in the array.

Array subclass constructors accept positional arguments, used to initialize the elements in order.

Private, abstract base class for pointers.

Concrete pointer types are created by calling POINTER() with the type that will be pointed to; this is done automatically by pointer() .

If a pointer points to an array, its elements can be read and written using standard subscript and slice accesses. Pointer objects have no size, so len() will raise TypeError . Negative subscripts will read from the memory before the pointer (as in C), and out-of-range subscripts will probably crash with an access violation (if you’re lucky).

Specifies the type pointed to.

Returns the object to which to pointer points. Assigning to this attribute changes the pointer to point to the assigned object.

Table of Contents

• Loading dynamic link libraries
• Accessing functions from loaded dlls
• Calling functions
• Fundamental data types
• Calling functions, continued
• Calling variadic functions
• Calling functions with your own custom data types
• Specifying the required argument types (function prototypes)
• Return types
• Passing pointers (or: passing parameters by reference)
• Structures and unions
• Structure/union alignment and byte order
• Bit fields in structures and unions
• Type conversions
• Incomplete Types
• Callback functions
• Accessing values exported from dlls
• Variable-sized data types
• Finding shared libraries
• Loading shared libraries
• Foreign functions
• Function prototypes
• Utility functions
• Structured data types
• Arrays and pointers

Previous topic

errno — Standard errno system symbols

Concurrent Execution

• Report a Bug
• Show Source

C Functions

C structures, c structures (structs).

Structures (also called structs) are a way to group several related variables into one place. Each variable in the structure is known as a member of the structure.

Unlike an array , a structure can contain many different data types (int, float, char, etc.).

Create a Structure

You can create a structure by using the struct keyword and declare each of its members inside curly braces:

To access the structure, you must create a variable of it.

Use the struct keyword inside the main() method, followed by the name of the structure and then the name of the structure variable:

Create a struct variable with the name "s1":

Access Structure Members

To access members of a structure, use the dot syntax ( . ):

Now you can easily create multiple structure variables with different values, using just one structure:

What About Strings in Structures?

Remember that strings in C are actually an array of characters, and unfortunately, you can't assign a value to an array like this:

An error will occur:

However, there is a solution for this! You can use the strcpy() function and assign the value to s1.myString , like this:

Simpler Syntax

You can also assign values to members of a structure variable at declaration time, in a single line.

Just insert the values in a comma-separated list inside curly braces {} . Note that you don't have to use the strcpy() function for string values with this technique:

Note: The order of the inserted values must match the order of the variable types declared in the structure (13 for int, 'B' for char, etc).

Copy Structures

You can also assign one structure to another.

In the following example, the values of s1 are copied to s2:

Modify Values

If you want to change/modify a value, you can use the dot syntax ( . ).

And to modify a string value, the strcpy() function is useful again:

Modifying values are especially useful when you copy structure values:

Ok, so, how are structures useful?

Imagine you have to write a program to store different information about Cars, such as brand, model, and year. What's great about structures is that you can create a single "Car template" and use it for every cars you make. See below for a real life example.

Real-Life Example

Use a structure to store different information about Cars:

C Exercises

Test yourself with exercises.

Fill in the missing part to create a Car structure:

Start the Exercise

COLOR PICKER

Contact Sales

If you want to use W3Schools services as an educational institution, team or enterprise, send us an e-mail: [email protected]

Report Error

If you want to report an error, or if you want to make a suggestion, send us an e-mail: [email protected]

Top Tutorials

Top references, top examples, get certified.

Proposal: Annotate types in multiple assignment

In the latest version of Python (3.12.3), type annotation for single variable assignment is available:

However, in some scenarios like when we want to annotate the tuple of variables in return, the syntax of type annotation is invalid:

In this case, I propose two new syntaxes to support this feature:

• Annotate directly after each variable:
• Annotate the tuple of return:

In other programming languages, as I know, Julia and Rust support this feature in there approaches:

I’m pretty sure this has already been suggested. Did you go through the mailing list and searched for topics here? Without doing that, there’s nothing to discuss here. (Besides linking to them).

Secondly, try to not edit posts, but post a followup. Some people read these topics in mailing list mode and don’t see your edits.

• https://mail.python.org
• https://mail.python.org/archives

For reference, PEP 526 has a note about this in the “Rejected/Postponed Proposals” section:

Allow type annotations for tuple unpacking: This causes ambiguity: it’s not clear what this statement means: x, y: T Are x and y both of type T , or do we expect T to be a tuple type of two items that are distributed over x and y , or perhaps x has type Any and y has type T ? (The latter is what this would mean if this occurred in a function signature.) Rather than leave the (human) reader guessing, we forbid this, at least for now.

Personally I think the meaning of this is rather clear, especially when combined with an assignment, and I would like to see this.

Thank you for your valuable response, both regarding the discussion convention for Python development and the history of this feature.

I have found a related topic here: https://mail.python.org/archives/list/[email protected]/thread/5NZNHBDWK6EP67HSK4VNDTZNIVUOXMRS/

Here’s the part I find unconvincing:

Under what circumstances will fun() be hard to annotate, but a, b will be easy?

It’s better to annotate function arguments and return values, not variables. The preferred scenario is that fun() has a well-defined return type, and the type of a, b can be inferred (there is no reason to annotate it). This idea is presupposing there are cases where that’s difficult, but I’d like to see some examples where that applies.

Does this not work?

You don’t need from __future__ as of… 3.9, I think?

3.10 if you want A | B too: PEP 604 , although I’m not sure which version the OP is using and 3.9 hasn’t reached end of life yet.

We can’t always infer it, so annotating a variable is sometimes necessary or useful. But if the function’s return type is annotated then a, b = fun() allows type-checkers to infer the types of a and b . This stuff isn’t built in to Python and is evolving as typing changes, so what was inferred in the past might be better in the future.

So my question above was: are there any scenarios where annotating the function is difficult, but annotating the results would be easy? That seems like the motivating use case.

Would it be a solution to put it on the line above? And not allow assigning on the same line? Then it better mirrors function definitions.

It’s a long thread, so it might have been suggested already.

Actually, in cases where the called function differs from the user-defined function, we should declare the types when assignment unpacking.

Here is a simplified MWE:

NOTE: In PyTorch, the __call__ function is internally wrapped from forward .

Can’t you write this? That’s shorter than writing the type annotations.

This is the kind of example I was asking for, thanks. Is the problem that typing tools don’t trace the return type through the call because the wrapping isn’t in python?

I still suggest to read the thread you linked, like I’m doing right now.

The __call__ function is not the same as forward . There might be many other preprocessing and postprocessing steps involved inside it.

Yeah, quite a bit of pre-processing in fact… unless you don’t have hooks by the looks of it:

Related Topics

• Guidelines to Write Experiences
• Write Interview Experience
• Write Work Experience
• Write Admission Experience
• Write Campus Experience
• Write Engineering Experience
• Write Coaching Experience
• Write Professional Degree Experience
• Write Govt. Exam Experiences
• TCS Interview Experience( Off Campus-2020)
• TCS Ninja Interview Experience (Off-Campus) 2022
• TCS Interview Experience (On-Campus) 2023
• TCS Interview Experience (Off-Campus)
• LTI Interview Experience (Off-Campus) 2022
• TCS Ninja Interview Experience (On-Campus) 2023
• TCS Interview Experience (On-Campus 2019)
• TCS Interview Experience | Set 20 (On Campus)
• TCS Interview Experience | Set 1 (On-Campus 2014)
• TCS Interview Experience (On-Campus)
• TCS Interview Experience | Set 41 (On-Campus)
• TCS Ninja Interview Experience (On-Campus) 2022-23
• TCS Digital Interview Experience (Off-Campus) 2021
• TCS Digital Interview Experience 2023 (Off-Campus)
• TCS Interview Experience | Set 40 (On-Campus)
• TCS Interview experience - Off campus drive via NQT
• TCS NQT Interview Experience (Oct 2020)

TCS NQT Interview Experience (Off-Campus) 2024

In recent years, the TCS National Qualifier Test (NQT) has become a gateway for many aspiring candidates to kickstart their careers in the tech industry. However, the coding section of the exam has been a subject of debate among candidates due to its ambiguous and sometimes inadequately described questions. In this article, we’ll dissect one such coding question from the TCS NQT conducted on April 26th, 2024, shedding light on the issues with its description and providing a detailed example.

The Problem:

One of the questions posed in the coding section of the TCS NQT on April 26th, 2024, revolved around finding all subarrays of a given array whose sum of elements is equal to a specified value, let’s call it K.

The problem statement lacked clarity, especially concerning how input should be handled when the size of the array is not provided upfront. Moreover, it failed to specify how input containing non-integer characters should be managed, leading to confusion among candidates.

An Example:

Consider the following input scenario:

Input: 1 2 3 4 5 -4 -3 ,10

In this example, the input consists of integers separated by spaces, with a comma separating the array from the specified value of K, which is 10. However, the problem statement does not clarify how to handle non-integer characters like commas in the input.

The Solution:

To address the ambiguity in the problem statement, let’s provide a solution along with the necessary input handling. Below is the code snippet demonstrating how to handle input until a non-integer character is encountered:

Conclusion:

The tcs nqt coding questions often present challenges due to their vague descriptions and ambiguous requirements. as demonstrated in this article, candidates need to be equipped with not only coding skills but also the ability to interpret and clarify unclear problem statements. it’s crucial for examination authorities to provide well-defined questions to ensure a fair and accurate assessment of candidates’ abilities., exact solution for the above question :, please login to comment..., similar reads.

• TCS-coding-questions
• Write It Up 2024
• Experiences
• Interview Experiences

Improve your Coding Skills with Practice

What kind of Experience do you want to share?