I implemented a function in C, that takes an array and creates vectors containing certain elements:
double** delay_vectors(const double* ts, int L, int tau, int p) {
int i, j;
// Allocate contiguous memory for v_i_list
double **v_i_list = (double **)malloc((L-tau*(p-1)) * sizeof(double *));
v_i_list[0] = (double *)malloc((L-tau*(p-1)) * p * sizeof(double));
for (int i = 1; i < (L-tau*(p-1)); i++) {
v_i_list[i] = v_i_list[i - 1] + p;
}
for (i = 0; i < L - tau * (p - 1); ++i) {
for (j = 0; j < p; ++j) {
v_i_list[i][j] = ts[i + j * tau];
}
}
return v_i_list;
}
For this function, I created a python wrapper, which takes a numpy array and returns the 2d array to numpy again:
static PyObject* delay_vectors_wrapper(PyObject* self, PyObject* args) {
PyObject *input_array;
PyArrayObject *input_array_np;
int tau, p;
// Parse the input arguments
if (!PyArg_ParseTuple(args, "Oii", &input_array, &tau, &p)) {
return NULL;
}
// Convert the input object to a NumPy array
input_array_np = (PyArrayObject *)PyArray_FROM_OTF(input_array, NPY_DOUBLE, NPY_ARRAY_IN_ARRAY);
if (input_array_np == NULL) {
return NULL;
}
int ndim = PyArray_NDIM(input_array_np);
if (ndim != 1) {
PyErr_SetString(PyExc_ValueError, "Input array must be 1-dimensional.");
Py_DECREF(input_array_np);
return NULL;
}
int L = PyArray_DIM(input_array_np, 0);
// Get data pointer from the NumPy array
double *ts = (double *)PyArray_DATA(input_array_np);
double** result = delay_vectors(ts, L, tau, p);
// Convert the C result to a np array
npy_intp dims[2] = {L - tau * (p - 1), p};
PyObject* output_array = PyArray_SimpleNewFromData(2, dims, NPY_DOUBLE, result);
Py_DECREF(input_array_np);
return output_array;
}
As you can see, I am not freeing any memory except for “input_array_np”. Freeing result would lead to an error, since, as I understand “PyArray_SimpleNewFromData” creates a wrapper around the data pointed to. I tested this function using valgrind and it shows no memory leaks. However, if I use this function in a loop e.g. in
ts = np.random.random(10**5)
for i in range(500):
dv = test.delay_vectors(ts,2,3)
I can see, that the consumed memory increases. I suspect this is due to the fact that everytime the function is called, it allocates a new potion of memory to store result in, which does not get freed even if the array is overwritten in Python, but I sadly can´t think of another way to do it.
I tried giving ownership of output_array
to numpy by using PyArray_ENABLEFLAGS(output_array, NPY_ARRAY_OWNDATA);
before returning to get pythons garbage collector to free it, but I can`t seem to get it working. I am also very new to writing C-code, so if you have any further suggestions regarding memory allocation, I would appreciate it.