Introduction

Mahotas is a computer vision library for the Python Programming Language (versions 2.5 and up, including version 3 and up). It operates on numpy arrays[1]. Therefore, it uses all the infrastructure built by that project for storing information and performing basic manipulations and computations. In particular, unlike libraries written in the C Language or in Java [2][3], Mahotas does not need to define a new image data structure, but uses the numpy array structure. Many basic manipulation functionality that would otherwise be part of a computer vision library are handled by numpy. For example, computing averages and other simple statistics, handling multi-channel images, converting between types (integer and floating point images are supported by mahotas) can all be performed with numpy builtin functionality. For the users, this has the additional advantage that they do not need to learn new interfaces.

Mahotas contains over 100 functions with functionality ranging from traditional image filtering and morphological operations to more modern wavelet decompositions and local feature computations. Additionally, by integrating into the Python numeric ecosystem, users can use other packages in a seamless way. In particular, mahotas does not implement any machine learning functionality, but rather advises the user to use another, specialised package, such as scikits-learn or milk.

Python is a natural “glue” language: it is easy to use state-of-the-art libraries written in multiple languages [4]. Mahotas itself is a mix of high-level Python and low-level C++. This achieves a good balance between speed and ease of implementation.

Version 1.0.2 of mahotas has been released recently and this is now a mature, well-tested package (the first versions were made available over 4 years ago, although the package was not named mahotas then). Mahotas runs and is used on different versions of Unix (including Linux, SunOS, and FreeBSD), Mac OS X, and Windows.

Interface

The interface is a procedural interface, with no global state. All functions work independently of each other (there is code sharing at the implementation level, but this is hidden from the user). The main functionality is grouped into the following categories:

Surf: Speeded-up Robust Features [5]. This includes both keypoint detection and descriptor computation.

Features: Global feature descriptors. In particular, Haralick texture features [6], Zernike moments, local binary patterns [7], and threshold adjacency statistics (both the original [8] and the parameter-free versions [5]).

Wavelet: Haar and Daubechies wavelets [10]. Forward and inverse transforms are supported.

Morphological functions: Erosion and dilation, as well as some more complex operations built on these. There are both binary and grayscale implementations of these operators.

Watershed: Seeded watershed and distance map transforms [11].

Filtering: Gaussian filtering, edge finding, and general convolutions.

Polygon operations: Convex hull, polygon drawing.

Numpy arrays contain data of a specific type, such unsigned 8 bit integer or floating point numbers. While natural colour images are typically 8 bits, scientific data is often larger (12 and 16 bit formats are common). Processing can generate floating point images. For example, a common normalization procedure is to subtract from each pixel location the overall pixel value mean; the result will be a floating point image even if the original image was integral. Mahotas works on all datatypes. This is performed without any extra memory copies. Mahotas is heavily optimised for both speed and memory usage (it can be used with very large arrays).

There are a few interface conventions which apply to many functions. When meaningful, a structuring element is used to define neighbourhoods or adjacency relationships (morphological functions, in particular, use this convention). Generally, the default is to use a cross as the default if no structuring filter is given.

When a new image is to be returned, functions take an argument named out where the output will be stored. This argument is often much more restricted in type. In particular, it must be a contiguous array. Since this is a performance feature (its purpose is to avoid extra memory allocation), it is natural that the interface is less flexible (accessing a contiguous array is much more efficient than a non-contiguous one).

Examples of Use

Code for this and other examples is present in the mahotas source distribution under the demos/ directory. In this example, we load an image, find SURF interest points, and compute descriptors.

We start by importing the necessary packages, including numpy and mahotas. We also use scipy.cluster, to demonstrate how the mahotas output can integrate with a machine learning package.

	import numpy as np 
	import mahotas as mh 
	from mahotas.features import surf 
	from scipy.cluster import vq 
				

The first step is to load the image and convert to 8 bit numbers. In this case, the conversion is done using standard numpy methods, namely astype. We use the function mahotas.demos.image_path to access a demonstration image.

	import mahotas.demos 
	impath = mh.demos.image_path('luispedro.jpg') 
	f = mh.imread(impath, as_grey=True) 
	f = f.astype(np.uint8) 
				

We can now compute SURF interest points and descriptors.

	spoints = surf.surf(f, 4, 6, 2)
				

The surf.surf function returns both the descriptors and their meta data. We use numpy operations to retain only the descriptors (the meta data is in the first five positions):

	descrs = spoints[:,6:]
				

Using scipy.cluster.vq, we cluster the descriptors into five groups. The function kmeans2 returns two values: the centroids, which we ignore; and the cluster ids, which we will use below to assign colours:

	_,cids = vq.kmeans2(vq.whiten(descrs), 5) 
				

Finally, we can show the points in different colours. In order to avoid a very cluttered image, we will only plot the first 64 regions.

	colors = np.array([ 
		[ 255,  25,   1], 
		[203,  77,  37], 
		[151, 129,  56], 
		[ 99, 181,  52], 
		[ 47, 233,   5]]) 
	f2 = surf.show_surf(f, spoints[:64], cids, colors) 
				

The show_surf only builds the image as a multi-channel (one for each colour) image. Using matplotlib [12], we finally display the image as Fig. 1.

	colors = np.array([ 
	[ 255,  25,   1], 
	[203,  77,  37], 
	[151, 129,  56], 
	[ 99, 181,  52], 
	[ 47, 233,   5]]) 
	f2 = surf.show_surf(f, spoints[:64], cids, colors) 
				

The easy interaction with matplotlib is another way in which we benefit from the numpy-based ecosystem. Mahotas does not need to support interacting with a graphical system to display images.

Implementation

Mahotas is mostly written in C++, but this is completely hidden from the user as there are hand-written Python wrappers for all functions.The main reason that mahotas is implemented in C++ (and not in C, which is the language of the Python interpreter) is to use templates. Almost all C++ functionality is split across 2 functions:

1. A py_function which uses the Python C API to get arguments and check them.
2. A template function <dtype> which works for the type dtype performing the actual operation.

So, for example, this is how erode is implemented. py_erode consists mostly of boiler-plate code:

	PyObject* py_erode(PyObject* self, PyObject* args) { 
		PyArrayObject* array; 
		PyArrayObject* Bc; 
		PyArrayObject* output; 
		if (!PyArg_ParseTuple(args,"OOO",&array,&Bc,&output) || 
		!numpy::are_arrays(array, Bc, output) || 
		!numpy::same_shape(array, output) || 
		!numpy::equiv_typenums(array, Bc, output) || 
		PyArray_NDIM(array) != PyArray_NDIM(Bc) 
		) { 
		PyErr_SetString(PyExc_RuntimeError, TypeErrorMsg); 
		return NULL; 
		} 
		holdref r_o(output); 
	
	#define HANDLE(type) \ 
		erode>type<(numpy::aligned_array>type<(output), \ 
			numpy::aligned_array>type<(array), \ 
			numpy::aligned_array>type<(Bc)); 
		SAFE_SWITCH_ON_INTEGER_TYPES_OF(array); 
	#undef HANDLE 
	... 					
				

This function retrieves the arguments, performs some sanity checks, performs a bit of initialization, and finally, switches in the input type with the help of the SAFE_SWITCH_ON_INTEGER_TYPES macro, which call the right specialisation of the template that does the actual work. In this example erode implements erosion:

	template>typename T<
	void erode(numpy::aligned_array>T< res 
				numpy::aligned_array>T< array, 
				numpy::aligned_array>T< Bc) { 
			gil_release nogil; 
			const int N = res.size(); 
			typename numpy::aligned_array>T<:iterator
							iter = array.begin(); 
			filter_iterator>T< filter(array.raw_array(),
					Bc.raw_array(), 
				ExtendNearest,
					is_bool(T())); 
			const int N2 = filter.size(); 
			T* rpos = res.data(); 
										
			for (int i = 0; 
				i != N; 
					++i, ++rpos, filter.iterate_both(iter)) { 
			T value = std::numeric_limits>T<::max(); 
			for (int j = 0; j != N2; ++j) { 
				T arr_val = T(); 
				filter.retrieve(iter, j, arr_val); 
				value = std::min>T<(value,
							erode_sub(arr_val, filter[j])); 
			} 
			*rpos = value; 
		}
	}	
				

The template machinery makes the functions that use it very simple and easy to read. The only downside is that there is some expansion of code size when the compiler instantiates the function for the several integer and floating point types. Given the small size of these functions, the total size of the compiled library is reasonable (circa 6MiB on an Intel-based 64 bit system for the whole library).

In the snippet above, you can see some other C++ machinery:

gil_release: This is a “resource-acquisition is object initialisation” (raii)4 object that releases the Python global interpreter lock (gil)5 in its constructor and gets it back in its destructor. Normally, the template function will release the gil after the Python-specific code is done. This allows several mahotas functions to run concurrently.

array: This is a thin wrapper around PyArrayObject, the raw numpy data type, which has iterators which resemble the C++ standard library. It also handles type-casting internally, making the code type-safer. This is also a raii object in terms of managing Python reference counts. In mahotas debug builds, this object additionally adds several checks to all the memory acesses.

filter_iterator: This was adapted from code in the scipy.ndimage packages and it is useful to iterate over an image and use a centered filter around each pixel (it keeps track of all of the boundary conditions).

The inner loop is as direct an implementation of erosion as one would wish for: for each pixel in the image, look at its neighbours, subtract the filter value, and compute the minimum of this operation.

Efficiency

Table 1 shows timings for different operations. These were normalized to multiples of the time it takes to go over the image and find its maximum pixel value (using the expression numpy.max(image)). The measurements shown were obtained on an Intel 64 bit system, running Ubuntu Linux. Due to the normalization, measurements obtained on another system (Intel 32 bits running Mac OS) were qualitatively similar.

The comparison is against Pymorph [13], which is a pure Python implementation of some of the same functions; scikits-image, which is a similar project to mahotas, but with a heavier emphasis on the use of Cython [14]; and OpenCV, which is a C++ library with automatically generated Python wrappers.

OpenCV is the fastest library, but this comes at the cost of some flexibility. Arguments to its functions must be of the exact expected type and it is possible to crash the interpreter if types do match the expected type (in the other libraries, including mahotas, all types are checked and an exception is generated which can be caught by user code). This is particularly relevant for interactive use as the user is often exploring and is willing to pay the speed cost of a few extra type checks to avoid a hard-crash.

Distribution and Installation

In keeping with the philosophy of blending in with the ecosystem, Mahotas uses the standard Python build machinery and distribution channels. Building and installing from source code is done using python setup.py install

Alternatively, Python based package managers (such as easy_install or pip) can be used (mahotas works well with these systems).

For compiling from source, a C++ compiler is needed, as well as the development headers for Python and numpy.

There are binary packages available for Windows, maintained by Christoph Gohlke, and for FreeBSD and Linux Frugalware through their respective package systems.

Quality Control

Mahotas includes a complete automated suite of unit tests, which tests all functionality and include several regression tests. There are no known bugs in version 1.0.2. Occasional bugs discovered in previous released versions have been corrected before the next release.

The development is completely open-source and development versions are available. Many users have submitted bug reports and fixes.

Operating system

Mahotas runs and is used on different versions of Unix (including Linux, SunOS, and FreeBSD), Mac OS X, and Windows.

Programming Language

Mahotas works in Python (minimal version is 2.5, but mahotas works with all more recent versions, including version in the Python 3 series).

Additional system requirements

None at runtime. Compilation from source requires a C++ compiler and the Python development headers.

Dependencies

At runtime, mahotas requires numpy to be present and installed.

List of contriubutors

Luis Pedro Coelho (Carnegie Mellon University and Instituto de Medicina Molecular), Zachary Pincus (Stanford University), Peter J. Verveer (European Molecular Biology Laboratory), Davis King (Northrop Grumman ES), Robert Webb (Carnegie Mellon University), Matthew Goodman (University of Texas at Austin), K.-Michael Aye (University of Bern), Rita Simões (University of Twente), Joe Kington (University of Wisconsin), Christoph Gohlke (University of California, Irvine), Lukas Bossard (ETH Zurich), and Sandro Knauss (University of Bremen).

Code Repository

Archive