Job system

Job system for asynchronous work with optional worker threads. Each job provides a process function and an optional callback. - The process function performs the work and may run on a worker thread. - The callback runs on the main thread and can interact with non-threaded systems (for example Lua). Jobs can have dependencies. A parent job will only run after all child jobs have finished.

Namespace: JobSystem
Include: #include <>
TYPES
FJobCallback The callback to process the user data.
FJobProcess The callback to process the user data.
HJob Job handle
HJobContext Job system context
ENUMS
JobSystemResult job result enumeration
JobSystemStatus job status enumeration
STRUCTS
struct Job Job parameters
struct JobSystemCreateParams creation parameters
FUNCTIONS
JobSystemResult JobSystemCancelJob(HJobContext context, HJob job) cancel a job (and its children)
HJobContext JobSystemCreate(JobSystemCreateParams* create_params) create job system context
HJob JobSystemCreateJob(HJobContext context, Job* job) create a job
void JobSystemDestroy(HJobContext context) destroy job system context
void* JobSystemGetContext(HJobContext context, HJob job) get the context from a job
void* JobSystemGetData(HJobContext context, HJob job) get the data from a job
uint32_t JobSystemGetWorkerCount(HJobContext context) get worker count
JobSystemResult JobSystemPushJob(HJobContext context, HJob job) push a job onto the work queue
JobSystemResult JobSystemSetParent(HJobContext context, HJob child, HJob parent) set a job as a dependency
void JobSystemUpdate(HJobContext context, uint64_t time_limit) update job system

Functions

JobSystemCancelJob

JobSystemResult JobSystemCancelJob(HJobContext context, HJob job)

cancel a job (and its children)

PARAMETERS

HJobContext context the job system context
HJob job the job to cancel

RETURNS

JobSystemResult Returns JOBSYSTEM_RESULT_OK if finished, JOBSYSTEM_RESULT_CANCELED if canceled, or JOBSYSTEM_RESULT_PENDING if the job (or any child) is still in flight

EXAMPLES

How to wait until a job has been cancelled or finished 
enum JobSystemResult jr = JobSystemCancelJob(job_context, hjob);
while (JOBSYSTEM_RESULT_PENDING == jr)
{
    dmTime::Sleep(1000);
    jr = JobSystemCancelJob(job_context, hjob);
}

JobSystemCreate

HJobContext JobSystemCreate(JobSystemCreateParams* create_params)

Create a new job system context.

PARAMETERS

JobSystemCreateParams* create_params creation parameters

RETURNS

HJobContext job system context, or null on failure

JobSystemCreateJob

HJob JobSystemCreateJob(HJobContext context, Job* job)

create a job

PARAMETERS

HJobContext context the job system context
Job* job the job creation parameters. This pointer is not stored, the data is copied as-is.

RETURNS

HJob returns the job if successful. 0 otherwise.

JobSystemDestroy

void JobSystemDestroy(HJobContext context)

Destroy a job system context and free its resources.

PARAMETERS

HJobContext context job system context

JobSystemGetContext

void* JobSystemGetContext(HJobContext context, HJob job)

get the context from a job

PARAMETERS

HJobContext context the job system context
HJob job the job

RETURNS

void* The registered context pointer. 0 if the job handle is invalid.

JobSystemGetData

void* JobSystemGetData(HJobContext context, HJob job)

get the data from a job

PARAMETERS

HJobContext context the job system context
HJob job the job

RETURNS

void* The registered data pointer. 0 if the job handle is invalid.

JobSystemGetWorkerCount

uint32_t JobSystemGetWorkerCount(HJobContext context)

get worker count

PARAMETERS

HJobContext context job system context

RETURNS

uint32_t number of worker threads

JobSystemPushJob

JobSystemResult JobSystemPushJob(HJobContext context, HJob job)

push a job onto the work queue

PARAMETERS

HJobContext context the job system context
HJob job the job to add to the work queue

RETURNS

JobSystemResult return JOBSYSTEM_RESULT_OK if job was pushed

JobSystemSetParent

JobSystemResult JobSystemSetParent(HJobContext context, HJob child, HJob parent)

set a job as a dependency

PARAMETERS

HJobContext context the job system context
HJob child the child job
HJob parent the parent job

RETURNS

JobSystemResult return JOBSYSTEM_RESULT_OK if job was pushed

JobSystemUpdate

void JobSystemUpdate(HJobContext context, uint64_t time_limit)

Flush finished jobs and invoke callbacks on the main thread. In single-threaded mode, this also processes jobs on the calling thread up to the time limit.

PARAMETERS

HJobContext context job system context
uint64_t time_limit max time (microseconds) to process jobs in single-threaded mode; 0 processes at most one job

Structs

Job

TYPE

struct Job

Job parameters

MEMBERS

FJobProcess m_Process function to process the job. Called from a worker thread.
FJobCallback m_Callback function to process the job. Called from main thread.
void* m_Context the user context for the callbacks
void* m_Data the user data for the callbacks

JobSystemCreateParams

TYPE

struct JobSystemCreateParams

creation parameters

MEMBERS

const char* m_ThreadNamePrefix thread name prefix for worker threads
uint8_t m_ThreadCount number of worker threads (set to 0 to spawn no threads)

Types

FJobCallback

void FJobCallback(HJobContext context, HJob job, JobSystemStatus status, void* user_context, void* user_data, int32_t user_result)

The callback to process the user data.

PARAMETERS

HJobContext context the job system context
HJob job the job
JobSystemStatus status the job status. JOBSYSTEM_STATUS_FINISHED if ok, or JOBSYSTEM_STATUS_CANCELED if it was canceled
void* user_context the user context
void* user_data the user data
int32_t user_result if status == JOBSYSTEM_STATUS_FINISHED, the user result returned from the process function. otherwise 0

FJobProcess

int32_t FJobProcess(HJobContext context, HJob job, void* user_context, void* user_data)

The callback to process the user data.

PARAMETERS

HJobContext context the job system context
HJob job the job
void* user_context the user context
void* user_data the user data

RETURNS

int32_t user defined result. This is passed to the FJobCallback function.

HJob

void HJob()

Job handle

PARAMETERS

HJobContext

void HJobContext()

Job system context

PARAMETERS

Enums

JobSystemResult

job result enumeration

JOBSYSTEM_RESULT_OK 0
JOBSYSTEM_RESULT_ERROR 1
JOBSYSTEM_RESULT_INVALID_HANDLE 2
JOBSYSTEM_RESULT_CANCELED 3
JOBSYSTEM_RESULT_PENDING 4

JobSystemStatus

job status enumeration

JOBSYSTEM_STATUS_FREE 0
JOBSYSTEM_STATUS_CREATED 1
JOBSYSTEM_STATUS_QUEUED 2
JOBSYSTEM_STATUS_PROCESSING 3
JOBSYSTEM_STATUS_FINISHED 4
JOBSYSTEM_STATUS_CANCELED 5