I/O Request Packet. More...

Collaboration diagram for I/O Request Packet:

Detailed Description

I/O Request Packet.

The I/O Request Packet (IRP) is a lock-less, self-contained, layered, continuation-passing request that acts as the primary primitive used by the kernel for asynchronous operations.

The IRP is designed to be generic enough to be used by any system in the kernel, however it is primarily used by the I/O ring system.

Warning: While the cancellation or completion of an IRP is thread safe, the setup of an IRP is not (as in pushing layers to it). It is assumed that only one thread is manipulating an IRP during its setup.

Completion

Todo:: Write the IRP documentation.

Cancellation

Cancelling an IRP can intuitively be considered equivalent to forcing the last completion to fail, thus resulting in all the other completions to fail as well.

The current owner of a IRP is responsible for handling cancellation by specifying a cancellation callback via irp_set_cancel(). The current owner being the last target of a irp_call() or irp_call_direct().

When an IRP is cancelled or timed out the cancellation callback will be invoked and atomically exchanged with a IRP_CANCELLED sentinel value. At which point the owner should perform whatever logic is needed to cancel the IRP, if it is not possible immediately cancel the IRP it should return ERR.

Below is an example of how to implement a completion with an associated cancellation callback:

void my_completion(irp_t* irp, void* ctx)
{
    // Do stuff...
 
    irp_complete(irp);
}
 
uint64_t my_cancel(irp_t* irp)
{
    // Cancellation callback is automatically cleared.
 
    if (irp->err == ETIMEDOUT)
    {
        // We timed out.
    }
    if (irp->err == ECANCELED)
    {
        // We were explicitly cancelled.
    }
 
    // Do stuff...
 
    uint64_t result = ...;
    if (result == ERR) // If an error occurs we can reassign the cancellation callback.
    {
        irp_set_cancel(irp, my_cancel);
        return ERR;
    }
 
    return 0;
}

Error Values

The IRP system uses the err field to indicate both the current state of the IRP as well as any error that may have occurred during its processing.

Included below are a list of "special" values which the IRP system will set:

EOK: Operation completed successfully.
ECANCELED: Operation was cancelled.
ETIMEDOUT: Operation timed out.

See also: I/O Subsystem for the ring system.; Wikipedia for more information about IRPs.; Microsoft _IRP for information on how Windows NT implements IRPs.

Data Structures
struct	irp_frame_t
	IRP stack frame structure. More...

struct	irp_pool_t

struct	irp_vtable_t
	IRP vtable structure. More...

struct	irp_t
	I/O Request Packet structure. More...

struct	irp_pool
	Request pool structure. More...

Macros
#define	IRP_CANCELLED ((irp_cancel_t)1)
	Sentinel value indicating that the IRP has been cancelled.

#define	IRP_MJ_READ 0

#define	IRP_MJ_WRITE 1

#define	IRP_MJ_POLL 2

#define	IRP_MJ_MAX 3

#define	IRP_MN_NORMAL 0

#define	IRP_ARGS_MAX 4
	The maximum number of 64-bit arguments in an `irp_frame_t`.

#define	IRP_FRAME_MAX 5
	The maximum number of frames in a IRP stack.

Typedefs
typedef void(*	irp_complete_t) (irp_t irp, void ctx)
	IRP complete callback type.

typedef uint64_t(*	irp_cancel_t) (irp_t *irp)
	IRP cancellation callback type.

typedef uint16_t	irp_major_t

typedef uint16_t	irp_minor_t

typedef void(*	irp_func_t) (irp_t *irp)
	IRP function type.

typedef struct irp	irp_t

Functions
struct	ALIGNED (64) irp

irp_pool_t *	irp_pool_new (size_t size, process_t process, void ctx)
	Allocate a new IRP pool.

void	irp_pool_free (irp_pool_t *pool)
	Free a IRP pool.

void	irp_timeout_add (irp_t *irp, clock_t timeout)
	Add an IRP to a per-CPU timeout queue.

void	irp_timeout_remove (irp_t *irp)
	Remove an IRP from its per-CPU timeout queue.

void	irp_timeouts_check (void)
	Check and handle expired IRP timeouts on the current CPU.

irp_t *	irp_new (irp_pool_t *pool)
	Allocate a new IRP from a pool.

mdl_t *	irp_get_mdl (irp_t irp, const void addr, size_t size)
	Retrieve a memory descriptor list and associate it with an IRP.

static irp_pool_t *	irp_get_pool (irp_t *irp)
	Retrieve the IRP pool that an IRP was allocated from.

static void *	irp_get_ctx (irp_t *irp)
	Retrieve the context of the IRP pool that an IRP was allocated from.

static process_t *	irp_get_process (irp_t *irp)
	Retrieve the process that owns an IRP.

static irp_t *	irp_chain_next (irp_t *irp)
	Retrieve the next IRP in a chain and advance the chain.

uint64_t	irp_cancel (irp_t *irp)
	Attempt to cancel an IRP.

static irp_cancel_t	irp_set_cancel (irp_t *irp, irp_cancel_t cancel)
	Set the cancellation callback for an IRP.

static irp_frame_t *	irp_current (irp_t *irp)
	Retrieve the current frame in the IRP stack.

static irp_frame_t *	irp_next (irp_t *irp)
	Retrieve the next frame in the IRP stack.

static void	irp_copy_to_next (irp_t *irp)
	Copy the current frame in the IRP stack to the next.

static void	irp_skip (irp_t *irp)
	Skip the current stack frame, meaning the next call will run in the same stack frame.

void	irp_call (irp_t irp, vnode_t vnode)
	Send an IRP to a specified vnode.

void	irp_call_direct (irp_t *irp, irp_func_t func)
	Send an IRP to a specified function directly.

void	irp_complete (irp_t *irp)
	Complete the current frame in the IRP stack.

static void	irp_set_complete (irp_t irp, irp_complete_t complete, void ctx)
	Set the completion callback and context for the next frame in the IRP stack.

static void	irp_error (irp_t *irp, uint8_t err)
	Helper to set an error code and complete the IRP.

static void	irp_prepare_generic (irp_t *irp, irp_major_t major, uint64_t arg0, uint64_t arg1, uint64_t arg2, uint64_t arg3)
	Prepares the next IRP stack frame for a generic operation.

static void	irp_prepare_read (irp_t irp, mdl_t buffer, void *data, uint64_t off, uint32_t len, mode_t mode)
	Prepares the next IRP stack frame for a read operation.

static void	irp_prepare_write (irp_t irp, mdl_t buffer, void *data, uint64_t off, uint32_t len, mode_t mode)
	Prepares the next IRP stack frame for a write operation.

Macro Definition Documentation

◆ IRP_CANCELLED

#define IRP_CANCELLED ((irp_cancel_t)1)

Sentinel value indicating that the IRP has been cancelled.

Definition at line 127 of file irp.h.

◆ IRP_MJ_READ

#define IRP_MJ_READ 0

Definition at line 130 of file irp.h.

◆ IRP_MJ_WRITE

#define IRP_MJ_WRITE 1

Definition at line 131 of file irp.h.

◆ IRP_MJ_POLL

#define IRP_MJ_POLL 2

Definition at line 132 of file irp.h.

◆ IRP_MJ_MAX

#define IRP_MJ_MAX 3

Definition at line 133 of file irp.h.

◆ IRP_MN_NORMAL

#define IRP_MN_NORMAL 0

Definition at line 136 of file irp.h.

◆ IRP_ARGS_MAX

#define IRP_ARGS_MAX 4

The maximum number of 64-bit arguments in an irp_frame_t.

Definition at line 138 of file irp.h.

◆ IRP_FRAME_MAX

#define IRP_FRAME_MAX 5

The maximum number of frames in a IRP stack.

Definition at line 175 of file irp.h.

Typedef Documentation

◆ irp_complete_t

typedef void(* irp_complete_t) (irp_t *irp, void *ctx)

IRP complete callback type.

Parameters

irp	The IRP.
ctx	The contxt pointer from the `irp_frame_t` structure.

Definition at line 114 of file irp.h.

◆ irp_cancel_t

typedef uint64_t(* irp_cancel_t) (irp_t *irp)

IRP cancellation callback type.

Parameters

irp The IRP.

Returns: On success, 0. On failure, ERR.

Definition at line 122 of file irp.h.

◆ irp_major_t

typedef uint16_t irp_major_t

Definition at line 129 of file irp.h.

◆ irp_minor_t

typedef uint16_t irp_minor_t

Definition at line 135 of file irp.h.

◆ irp_func_t

typedef void(* irp_func_t) (irp_t *irp)

IRP function type.

Definition at line 227 of file irp.h.

◆ irp_t

irp_t

Definition at line 23 of file irp.h.

Function Documentation

◆ ALIGNED()

struct ALIGNED ( 64 )

< Used to store the IRP in various lists.

< Used to store the IRP in the timeout queue.

< Cancellation callback, must be atomic to ensure an IRP is only cancelled once.

< The time at which the IRP will be removed from a timeout queue.

< A preallocated memory descriptor list for use by the IRP.

< Index of the IRP in its pool.

< Index of the next IRP in a chain or in the free list.

< The CPU whose timeout queue the IRP is in.

< The error code of the operation, also used to specify its current state.

< The index of the current frame in the stack.

< The frame stack, grows downwards.

Definition at line 173 of file irp.h.

◆ irp_pool_new()

irp_pool_t * irp_pool_new	(	size_t	size,
		process_t *	process,
		void *	ctx
	)

Allocate a new IRP pool.

Parameters

size	The amount of requests to allocate.
process	The process that will own the IRPs allocated from this pool.
ctx	The context of the IRP pool.

Returns: On success, a pointer to the new IRP pool. On failure, NULL and errno is set.

Definition at line 32 of file irp.c.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ irp_pool_free()

void irp_pool_free ( irp_pool_t * pool )

Free a IRP pool.

Parameters

pool	The IRP pool to free.

Definition at line 62 of file irp.c.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ irp_timeout_add()

void irp_timeout_add	(	irp_t *	irp,
		clock_t	timeout
	)

Add an IRP to a per-CPU timeout queue.

Parameters

irp	The IRP to add.
timeout	The timeout of the IRP.

Definition at line 67 of file irp.c.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ irp_timeout_remove()

void irp_timeout_remove ( irp_t * irp )

Remove an IRP from its per-CPU timeout queue.

Parameters

irp	The IRP to remove.

Definition at line 97 of file irp.c.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ irp_timeouts_check()

void irp_timeouts_check ( void )

Check and handle expired IRP timeouts on the current CPU.

Definition at line 160 of file irp.c.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ irp_new()

irp_t * irp_new ( irp_pool_t * pool )

Allocate a new IRP from a pool.

The pool that the IRP was allocated from, and its context, can be retrieved using the irp_get_pool() function.

Parameters

pool	The IRP pool.

Returns: On success, a pointer to the allocated IRP. On failure, NULL and errno is set.

Definition at line 211 of file irp.c.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ irp_get_mdl()

mdl_t * irp_get_mdl	(	irp_t *	irp,
		const void *	addr,
		size_t	size
	)

Retrieve a memory descriptor list and associate it with an IRP.

All MDLs associated with a IRP will be cleaned up when finished.

Parameters

irp	The IRP to associate the MDL with.
addr	The virtual address of the memory region to add to the MDL, or `NULL` for a blank MDL.
size	The size of the memory region.

Returns: On success, a pointer to the MDL. On failure, NULL and errno is set.

Definition at line 241 of file irp.c.

Here is the call graph for this function:

◆ irp_get_pool()

static irp_pool_t * irp_get_pool ( irp_t * irp )

inlinestatic

Retrieve the IRP pool that an IRP was allocated from.

Parameters

irp The IRP.

Returns: The IRP pool.

Definition at line 304 of file irp.h.

Here is the caller graph for this function:

◆ irp_get_ctx()

static void * irp_get_ctx ( irp_t * irp )

inlinestatic

Retrieve the context of the IRP pool that an IRP was allocated from.

Parameters

irp The IRP.

Returns: The context.

Definition at line 315 of file irp.h.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ irp_get_process()

static process_t * irp_get_process ( irp_t * irp )

inlinestatic

Retrieve the process that owns an IRP.

Parameters

irp The IRP.

Returns: The process.

Definition at line 326 of file irp.h.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ irp_chain_next()

static irp_t * irp_chain_next ( irp_t * irp )

inlinestatic

Retrieve the next IRP in a chain and advance the chain.

Parameters

irp	The current IRP.

Returns: The next IRP, or NULL if there is no next IRP.

Definition at line 337 of file irp.h.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ irp_cancel()

uint64_t irp_cancel ( irp_t * irp )

Attempt to cancel an IRP.

Parameters

irp	The IRP to cancel.

Returns: On success, 0. On failure, ERR and errno is set.

Definition at line 331 of file irp.c.

Here is the call graph for this function:

◆ irp_set_cancel()

static irp_cancel_t irp_set_cancel	(	irp_t *	irp,
		irp_cancel_t	cancel
	)

inlinestatic

Set the cancellation callback for an IRP.

Parameters

irp	The IRP.
cancel	The cancellation callback.

Returns: The previous cancellation callback.

Definition at line 366 of file irp.h.

Here is the caller graph for this function:

◆ irp_current()

static irp_frame_t * irp_current ( irp_t * irp )

inlinestatic

Retrieve the current frame in the IRP stack.

Parameters

irp	The IRP to retrieve the frame from.

Returns: The current frame.

Definition at line 385 of file irp.h.

Here is the caller graph for this function:

◆ irp_next()

static irp_frame_t * irp_next ( irp_t * irp )

inlinestatic

Retrieve the next frame in the IRP stack.

Parameters

irp	The IRP to retrieve the frame from.

Returns: The next frame, or NULL if we are at the bottom of the stack.

Definition at line 397 of file irp.h.

Here is the caller graph for this function:

◆ irp_copy_to_next()

static void irp_copy_to_next ( irp_t * irp )

inlinestatic

Copy the current frame in the IRP stack to the next.

Parameters

irp The IRP.

Definition at line 411 of file irp.h.

Here is the call graph for this function:

◆ irp_skip()

static void irp_skip ( irp_t * irp )

inlinestatic

Skip the current stack frame, meaning the next call will run in the same stack frame.

Parameters

irp The IRP.

Definition at line 433 of file irp.h.

Here is the call graph for this function:

◆ irp_call()

void irp_call	(	irp_t *	irp,
		vnode_t *	vnode
	)

Send an IRP to a specified vnode.

Will advance the IRP stack.

Parameters

irp	The IRP to send.
vnode	The vnode to associated with the next IRP stack frame.

Definition at line 283 of file irp.c.

Here is the call graph for this function:

◆ irp_call_direct()

void irp_call_direct	(	irp_t *	irp,
		irp_func_t	func
	)

Send an IRP to a specified function directly.

Will advance the IRP stack.

Parameters

irp	The IRP to send.
func	The function to call.

Definition at line 314 of file irp.c.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ irp_complete()

void irp_complete ( irp_t * irp )

Complete the current frame in the IRP stack.

If the current frame does not have a completion, it will automatically complete the next frame in the stack.

If the last frame is reached, the IRP is considered finished. Which will causing its resources to be freed and for the IRP to be returned to its pool.

Parameters

irp	The IRP to complete.

Definition at line 355 of file irp.c.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ irp_set_complete()

static void irp_set_complete	(	irp_t *	irp,
		irp_complete_t	complete,
		void *	ctx
	)

inlinestatic

Set the completion callback and context for the next frame in the IRP stack.

Parameters

irp	The IRP to set.
complete	The completion callback.
ctx	The context pointer to pass to the completion callback.

Definition at line 484 of file irp.h.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ irp_error()

static void irp_error	(	irp_t *	irp,
		uint8_t	err
	)

inlinestatic

Helper to set an error code and complete the IRP.

Parameters

irp	The IRP to error.
err	The error code to set.

Definition at line 497 of file irp.h.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ irp_prepare_generic()

static void irp_prepare_generic	(	irp_t *	irp,
		irp_major_t	major,
		uint64_t	arg0,
		uint64_t	arg1,
		uint64_t	arg2,
		uint64_t	arg3
	)

inlinestatic

Prepares the next IRP stack frame for a generic operation.

Parameters

irp	The IRP.
major	The major function number.
arg0	Generic argument 0.
arg1	Generic argument 1.
arg2	Generic argument 2.
arg3	Generic argument 3.

Definition at line 513 of file irp.h.

Here is the call graph for this function:

◆ irp_prepare_read()

static void irp_prepare_read	(	irp_t *	irp,
		mdl_t *	buffer,
		void *	data,
		uint64_t	off,
		uint32_t	len,
		mode_t	mode
	)

inlinestatic

Prepares the next IRP stack frame for a read operation.

Parameters

irp	The IRP.
buffer	The memory descriptor list to read into.
data	The buffer to read into.
off	The offset in the file to read from.
len	The number of bytes to read.
mode	The mode.

Definition at line 536 of file irp.h.

Here is the call graph for this function:

◆ irp_prepare_write()

static void irp_prepare_write	(	irp_t *	irp,
		mdl_t *	buffer,
		void *	data,
		uint64_t	off,
		uint32_t	len,
		mode_t	mode
	)

inlinestatic

Prepares the next IRP stack frame for a write operation.

Parameters

irp	The IRP.
buffer	The memory descriptor list to write from.
data	The buffer to write from.
off	The offset in the file to write to.
len	The number of bytes to write.
mode	The mode.

Definition at line 559 of file irp.h.

Here is the call graph for this function:

Detailed Description

Completion

Cancellation

Error Values

Data Structures

Macros

Typedefs

Functions

Macro Definition Documentation

◆ IRP_CANCELLED

◆ IRP_MJ_READ

◆ IRP_MJ_WRITE

◆ IRP_MJ_POLL

◆ IRP_MJ_MAX

◆ IRP_MN_NORMAL

◆ IRP_ARGS_MAX

◆ IRP_FRAME_MAX

Typedef Documentation

◆ irp_complete_t

◆ irp_cancel_t

◆ irp_major_t

◆ irp_minor_t

◆ irp_func_t

◆ irp_t

Function Documentation

◆ ALIGNED()

◆ irp_pool_new()

◆ irp_pool_free()

◆ irp_timeout_add()

◆ irp_timeout_remove()

◆ irp_timeouts_check()

◆ irp_new()

◆ irp_get_mdl()

◆ irp_get_pool()

◆ irp_get_ctx()

◆ irp_get_process()

◆ irp_chain_next()

◆ irp_cancel()

◆ irp_set_cancel()

◆ irp_current()

◆ irp_next()

◆ irp_copy_to_next()

◆ irp_skip()

◆ irp_call()

◆ irp_call_direct()

◆ irp_complete()

◆ irp_set_complete()

◆ irp_error()

◆ irp_prepare_generic()

◆ irp_prepare_read()

◆ irp_prepare_write()