The thread pool

The thread pool — The thread pool

Synopsis

#include "thread-pool.h"


void *              (*TPGetTaskFunc)                    (void *qargs);
void                (*TPWorkFunc)                       (void *work,
                                                         int cookie);
void                (*TPFinalFunc)                      (void *qargs,
                                                         void *work);

int                 run_threads                         (int n_threads,
                                                         TPWorkFunc work,
                                                         TPGetTaskFunc get_task,
                                                         TPFinalFunc final,
                                                         void *queue_args,
                                                         int max,
                                                         int cpu_num,
                                                         int cpu_groupsize,
                                                         int cpu_offset);
extern pthread_mutex_t stderr_lock;
signed int          get_status_label                    (void);

Description

The thread pool helps when running many tasks in parallel. It takes care of starting and stopping threads, and presents a relatively simple interface to the individual programs.

Details

TPGetTaskFunc ()

void *              (*TPGetTaskFunc)                    (void *qargs);

This function is called, non-reentrantly, to get a new work item to give to your work function. The stuff you need to generate the new work item should have been stored in qargs which was passed to run_threads().

qargs :

The queue_args pointer which was given to run_threads(). Returns: A pointer which will be passed to the worker function.

TPWorkFunc ()

void                (*TPWorkFunc)                       (void *work,
                                                         int cookie);

This function is called, reentrantly, for each work item.

work :

The queue_args pointer which was given to run_threads().

cookie :

A small integral number which is guaranteed to be unique among all currently running threads.

TPFinalFunc ()

void                (*TPFinalFunc)                      (void *qargs,
                                                         void *work);

This function is called, non-reentrantly, after each work item has been completed. A typical use might be to update some counters inside qargs according to fields withing work which were filled by your 'work' function.

qargs :

The queue_args pointer which was given to run_threads().

work :

The pointer which was returned by your get_task function.

run_threads ()

int                 run_threads                         (int n_threads,
                                                         TPWorkFunc work,
                                                         TPGetTaskFunc get_task,
                                                         TPFinalFunc final,
                                                         void *queue_args,
                                                         int max,
                                                         int cpu_num,
                                                         int cpu_groupsize,
                                                         int cpu_offset);

'get_task' will be called every time a worker is idle. It returns either NULL, indicating that no further work is available, or a pointer which will be passed to 'work'.

'final' will be called once per image, and will be given both queue_args and the last task pointer.

'get_task' and 'final' will be called only under lock, and so do NOT need to be re-entrant or otherwise thread safe. 'work', of course, needs to be thread safe.

Work will stop after 'max' tasks have been processed whether get_task returned NULL or not. If "max" is zero, all tasks will be processed.

n_threads :

The number of threads to run in parallel

work :

The function to be called to do the work

get_task :

The function which will determine the next unassigned task

final :

The function which will be called to clean up after a task

queue_args :

A pointer to any data required to determine the next task

max :

Stop calling get_task after starting this number of jobs

cpu_num :

The number of CPUs in the system

cpu_groupsize :

The group size into which the CPUs are grouped

cpu_offset :

The CPU group number at which to start pinning threads

Returns :

The number of tasks completed.

stderr_lock

extern pthread_mutex_t stderr_lock;

get_status_label ()

signed int          get_status_label                    (void);