Address Space handling. More...

Collaboration diagram for Space:

Detailed Description

Address Space handling.

Data Structures
struct	space_callback_t

struct	space_pinned_page_t
	Pinned page structure. More...

struct	space_t
	Virtual address space structure. More...

struct	space_mapping_t
	Helper structure for managing address space mappings. More...

struct	vmm_callback_t
	Space callback structure. More...

Macros
#define	SPACE_TLB_SHOOTDOWN_TIMEOUT (CLOCKS_PER_SEC)
	The maximum time to wait for the acknowledgements from other CPU's before panicking.

Typedefs
typedef void(*	space_callback_func_t) (void *private)
	Space callback function.

Enumerations
enum	space_flags_t { SPACE_NONE = 0 , SPACE_USE_PMM_BITMAP = 1 << 0 , SPACE_MAP_KERNEL_BINARY = 1 << 1 , SPACE_MAP_KERNEL_HEAP = 1 << 2 , SPACE_MAP_IDENTITY = 1 << 3 }
	Flags for space initialization. More...

Functions
uint64_t	space_init (space_t *space, uintptr_t startAddress, uintptr_t endAddress, space_flags_t flags)
	Initializes a virtual address space.

void	space_deinit (space_t *space)
	Deinitializes a virtual address space.

void	space_load (space_t *space)
	Loads a virtual address space.

uint64_t	space_pin (space_t space, const void address, uint64_t length, stack_pointer_t *userStack)
	Pins pages within a region of the address space.

uint64_t	space_pin_terminated (space_t space, const void address, const void terminator, uint8_t objectSize, uint64_t maxCount, stack_pointer_t userStack)
	Pins a region of memory terminated by a terminator value.

void	space_unpin (space_t space, const void address, uint64_t length)
	Unpins pages in a region previously pinned with `space_pin()` or `space_pin_string()`.

uint64_t	space_check_access (space_t space, const void addr, uint64_t length)
	Checks if a virtual memory region is within the allowed address range of the space.

uint64_t	space_mapping_start (space_t space, space_mapping_t mapping, void virtAddr, void physAddr, uint64_t length, pml_flags_t flags)
	Prepare for changes to the address space mappings.

pml_callback_id_t	space_alloc_callback (space_t space, uint64_t pageAmount, space_callback_func_t func, void private)
	Allocate a callback.

void	space_free_callback (space_t *space, pml_callback_id_t callbackId)
	Free a callback.

void	space_tlb_shootdown (space_t space, void virtAddr, uint64_t pageAmount)
	Performs a TLB shootdown for a region of the address space, and wait for acknowledgements.

void *	space_mapping_end (space_t space, space_mapping_t mapping, errno_t err)
	Performs cleanup after changes to the address space mappings.

bool	space_is_mapped (space_t space, const void virtAddr, uint64_t length)
	Checks if a virtual memory region is fully mapped.

uint64_t	space_user_page_count (space_t *space)
	Get the number of user pages allocated in the address space.

Macro Definition Documentation

◆ SPACE_TLB_SHOOTDOWN_TIMEOUT

#define SPACE_TLB_SHOOTDOWN_TIMEOUT (CLOCKS_PER_SEC)

The maximum time to wait for the acknowledgements from other CPU's before panicking.

Definition at line 106 of file space.h.

Typedef Documentation

◆ space_callback_func_t

typedef void(* space_callback_func_t) (void *private)

Space callback function.

Definition at line 43 of file space.h.

Enumeration Type Documentation

◆ space_flags_t

enum space_flags_t

Flags for space initialization.

Enumerator
SPACE_NONE
SPACE_USE_PMM_BITMAP	Use the PMM bitmap to allocate the page table, this is really only for the kernel page table as it must be within a 32 bit boundary because the smp trampoline loads it as a dword.
SPACE_MAP_KERNEL_BINARY	Map the kernel binary into the address space.
SPACE_MAP_KERNEL_HEAP	Map the kernel heap into the address space.
SPACE_MAP_IDENTITY	Map the identity mapped physical memory into the address space.

Definition at line 27 of file space.h.

Function Documentation

◆ space_init()

uint64_t space_init	(	space_t *	space,
		uintptr_t	startAddress,
		uintptr_t	endAddress,
		space_flags_t	flags
	)

Initializes a virtual address space.

Parameters

space	The address space to initialize.
startAddress	The starting address for allocations in this address space.
endAddress	The ending address for allocations in this address space.
flags	Flags to control the initialization behavior.

Returns

On success, 0. On failure, ERR and errno is set to:

EINVAL: Invalid parameters.
ENOMEM: Not enough memory to initialize the address space.

Definition at line 64 of file space.c.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ space_deinit()

void space_deinit ( space_t * space )

Deinitializes a virtual address space.

Parameters

space The address space to deinitialize.

Definition at line 122 of file space.c.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ space_load()

void space_load ( space_t * space )

Loads a virtual address space.

Must be called with interrupts disabled.

Will do nothing if the space is already loaded.

Parameters

space The address space to load.

Definition at line 159 of file space.c.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ space_pin()

uint64_t space_pin	(	space_t *	space,
		const void *	address,
		uint64_t	length,
		stack_pointer_t *	userStack
	)

Pins pages within a region of the address space.

Used to prevent TOCTOU attacks, where a system call provides some user space region, the kernel then checks that its mapped and after that check a seperate thread in the user space process unmaps or modifies that regions mappings while the kernel is still using it.

Our solution is to pin any user space pages that are accessed or modified during the syscall, meaning that a special flag is set in the address spaces page tables that prevent those pages from being unmapped or modified until they are unpinned which happens when the syscall is finished in space_unpin().

If the region is not fully mapped, or the region is not within the spaces startAddress and endAddress range, the function will fail.

If any page in the region is already at its maximum pin depth, the calling thread will block until the page is unpinned by another thread.

If a user stack is provided and the region to pin is both unmapped and within the stack region, memory will be allocated and mapped to the relevant region in the user stack. This is needed as its possible for a user space process to pass an address to a system call that is in its user stack but not yet mapped. For example, it could create a big buffer on its stack then pass it to a syscall without first accessing it, meaning no page fault would have occurred to map the pages.

Parameters

space	The target address space.
address	The address to pin, can be `NULL` if length is 0.
length	The length of the region pointed to by `address`, in bytes.
userStack	Pointer to the user stack of the calling thread, can be `NULL, see above. @return On success,`0`. On failure,`ERR`and`errno`is set to: -`EINVAL`: Invalid parameters. -`EOVERFLOW`: Address overflow. -`EFAULT`: The region is not fully mapped or within the provided user stack. -`ENOMEM`: Not enough memory.

Definition at line 330 of file space.c.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ space_pin_terminated()

uint64_t space_pin_terminated	(	space_t *	space,
		const void *	address,
		const void *	terminator,
		uint8_t	objectSize,
		uint64_t	maxCount,
		stack_pointer_t *	userStack
	)

Pins a region of memory terminated by a terminator value.

Pins pages in the address space starting from address up to maxSize bytes or until the specified terminator is found.

Used for null-terminated strings or other buffers that have a specific terminator.

Parameters

space	The target address space.
address	The starting address of the region to pin.
terminator	The terminator value to search for.
objectSize	The size of each object to compare against the terminator, in bytes.
maxCount	The maximum number of objects to scan before failing.
userStack	Pointer to the user stack of the calling thread, can be `NULL`, see `space_pin()`.

Returns

On success, the number of bytes pinned, not including the terminator. On failure, ERR and errno is set to:

EINVAL: Invalid parameters.
EOVERFLOW: Address overflow.
EFAULT: The region is not fully mapped or within the provided user stack.
ENOMEM: Not enough memory.

Definition at line 379 of file space.c.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ space_unpin()

void space_unpin	(	space_t *	space,
		const void *	address,
		uint64_t	length
	)

Unpins pages in a region previously pinned with space_pin() or space_pin_string().

Will wake up any threads waiting to pin the same pages.

Parameters

space	The target address space.
address	The address of the region to unpin, can be `NULL` if length is 0.
length	The length of the region pointed to by `address`, in bytes.

Definition at line 452 of file space.c.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ space_check_access()

uint64_t space_check_access	(	space_t *	space,
		const void *	addr,
		uint64_t	length
	)

Checks if a virtual memory region is within the allowed address range of the space.

Checks that the given memory region is within the startAddress and endAddress range of the space, really only used in system calls that might access unmapped user space memory for example mmap(), in such cases we dont want to pin the "buffer" since we expect that it is not yet mapped.

Parameters

space	The target address space.
addr	The starting address of the memory region, can be `NULL` if length is 0.
length	The length of the memory region, in bytes.

Returns

On success, 0. On failure, ERR and errno is set to:

EINVAL: Invalid parameters.
EOVERFLOW: Address overflow.
EFAULT: The region is outside the allowed address range.

Definition at line 478 of file space.c.

Here is the caller graph for this function:

◆ space_mapping_start()

uint64_t space_mapping_start	(	space_t *	space,
		space_mapping_t *	mapping,
		void *	virtAddr,
		void *	physAddr,
		uint64_t	length,
		pml_flags_t	flags
	)

Prepare for changes to the address space mappings.

Will return with the spaces mutex acquired for writing, which must be released by calling space_mapping_end().

If flags & PML_USER then the addresses must be in the user space range.

Note: Handling page faults to grow stacks requires mapping memory, this means that if we were to run out of memory while executing this function, it could lead to a deadlock. To avoid this, this function will call stack_pointer_poke() to ensure that sufficient stack space is available.

Parameters

space	The target address space.
mapping	Will be filled with parsed information about the mapping.
virtAddr	The virtual address the mapping will apply to. Can be `NULL` to let the kernel choose an address.
physAddr	The physical address to map from. Can be `NULL`.
length	The length of the virtual memory region to modify, in bytes.
flags	The page table flags for the mapping.

Returns

On success, 0. On failure, ERR and errno is set to:

EINVAL: Invalid parameters.
EOVERFLOW: Address overflow.
EFAULT: The addresses are outside the allowed range.
ENOMEM: Not enough memory.

Definition at line 528 of file space.c.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ space_alloc_callback()

pml_callback_id_t space_alloc_callback	(	space_t *	space,
		uint64_t	pageAmount,
		space_callback_func_t	func,
		void *	private
	)

Allocate a callback.

Must be called between space_mapping_start() and space_mapping_end().

When pageAmount number of pages with this callback ID are unmapped or the address space is freed, the callback function will be called with the provided private data.

Parameters

space	The target address space.
pageAmount	The number of pages the callback is responsible for.
func	The callback function.
private	Private data to pass to the callback function.

Returns: On success, returns the callback ID. On failure, returns PML_MAX_CALLBACK.

Definition at line 598 of file space.c.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ space_free_callback()

void space_free_callback	(	space_t *	space,
		pml_callback_id_t	callbackId
	)

Free a callback.

Must be called between space_mapping_start() and space_mapping_end().

Allows the callback ID to be reused. The callback function will not be called.

Parameters

space	The target address space.
callbackId	The callback ID to free.

Definition at line 639 of file space.c.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ space_tlb_shootdown()

void space_tlb_shootdown	(	space_t *	space,
		void *	virtAddr,
		uint64_t	pageAmount
	)

Performs a TLB shootdown for a region of the address space, and wait for acknowledgements.

Must be called between space_mapping_start() and space_mapping_end().

This will cause all CPUs that have the address space loaded to invalidate their TLB entries for the specified region.

Will not affect the current CPU's TLB, that is handled by the page_table_t directly when modifying page table entries.

Todo:: Currently this does a busy wait for acknowledgements. Use a wait queue?

Parameters

space	The target address space.
virtAddr	The starting virtual address of the region.
pageAmount	The number of pages in the region.

Definition at line 669 of file space.c.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ space_mapping_end()

void * space_mapping_end	(	space_t *	space,
		space_mapping_t *	mapping,
		errno_t	err
	)

Performs cleanup after changes to the address space mappings.

Must be called after space_mapping_start().

Parameters

space	The target address space.
mapping	The parsed information about the mapping.
err	The error code, if 0 then no error.

Returns: If err is EOK, returns the virtual address of the mapping. If err is not EOK, returns NULL and errno is set to err.

Definition at line 739 of file space.c.

Here is the call graph for this function:

Here is the caller graph for this function:

◆ space_is_mapped()

bool space_is_mapped	(	space_t *	space,
		const void *	virtAddr,
		uint64_t	length
	)

Checks if a virtual memory region is fully mapped.

Parameters

space	The target address space.
virtAddr	The virtual address of the memory region.
length	The length of the memory region, in bytes.

Returns: true if the entire region is mapped, false otherwise.

Definition at line 759 of file space.c.

Here is the call graph for this function:

◆ space_user_page_count()

uint64_t space_user_page_count ( space_t * space )

Get the number of user pages allocated in the address space.

Will count the number of pages with the PML_OWNED flag set in user space.

Parameters

space The target address space.

Returns: The number of user pages mapped.

Definition at line 766 of file space.c.

Here is the call graph for this function:

Here is the caller graph for this function:

Detailed Description

Data Structures

Macros

Typedefs

Enumerations

Functions

Macro Definition Documentation

◆ SPACE_TLB_SHOOTDOWN_TIMEOUT

Typedef Documentation

◆ space_callback_func_t

Enumeration Type Documentation

◆ space_flags_t

Function Documentation

◆ space_init()

◆ space_deinit()

◆ space_load()

◆ space_pin()

◆ space_pin_terminated()

◆ space_unpin()

◆ space_check_access()

◆ space_mapping_start()

◆ space_alloc_callback()

◆ space_free_callback()

◆ space_tlb_shootdown()

◆ space_mapping_end()

◆ space_is_mapped()

◆ space_user_page_count()