OS Development Notes #3: UEFI Memory and exit
2023-08-30 | osdev rust programming uefi boot acpi memoryRefactoring
First let's clean up the code before the main.rs file gets bloated, if not already done. Cargo can run in workspaces where we can make libraries that's included in other libraries and runtimes.
In kattegat (not in src), run:
cargo new uefi --lib
cargo new kernel
and we move the src to kernel for now overwriting main.rs, but not .cargo and Cargo.toml. For now, remove all the contents in uefi/src/lib.rs.
We change the run.sh script to point to kernel instead:
...
-netdev user,id=n0,tftp=target/x86_64-unknown-uefi/debug,bootfile=kernel.efi
and change the main Cargo.toml in kattegat as:
[workspace]
members = ["kernel", "uefi"]
and in kernel/Cargo.toml we add uefi as a dependency:
[dependencies]
uefi = { path = "../uefi" }
and running run.sh again, it should be as before.
We separate all the code to the uefi library, and in kernel, reference to the library instead of direct. Some structs might have to become public.
Copy the kernel/src/main.rs to uefi/src/lib.rs and remove no_main and the panic function in the uefi library, and the opposite in the kernel executable.
For the uefi we rename the efi_main to:
pub fn get_boot_info(_image_handle: Handle, system_table: *mut SystemTable) -> usize {
...
0
}
And we have to make SystemTable public accesible.
For the kernel we then import that necessary uefi libraries, and update using get_boot_info instead:
use uefi::{Handle, SystemTable};
#[no_mangle]
extern "C" fn efi_main(image_handle: Handle, system_table: *mut SystemTable) -> usize {
let status = uefi::get_boot_info(image_handle, system_table);
if status != 0 {
return status;
}
loop {}
}
And we can remove the imports of core libraries in the kernel. The library can be designed as we want, so we wont go into details on the division of code. It might be a good exercise to refactor the code to create an understanding of the rust modularity file structure, if you are not already familiar with it.
ACPI tables
According to Wikipedia:
Advanced Configuration and Power Interface (ACPI) is an open standard that operating systems can use to discover and configure computer hardware components, to perform power management (e.g. putting unused hardware components to sleep), auto configuration (e.g. Plug and Play and hot swapping), and status monitoring.
It was first released in 1996, but in 2013 the UEFI Forum took over, and we have now the convenience of accessing this interface through the System Table, and using one of the most helpful Rust unsafe feature slice::from_raw_parts that returns an array slice from a pointer and the size, when both are known to be a table of the same type:
use core::slice;
struct SystemTable {
...
configuration_table: *const ConfigurationTable,
}
impl SystemTable {
...
pub fn acpi_tables(&self) -> &[ConfigurationTable] {
unsafe { slice::from_raw_parts(self.configuration_table, self.number_of_table_entries) }
}
}
where we define each entry in the ConfigurationTable as having a GUID (refer to the GOP article last time) and the pointer to the address fitting this GUID, called the RSDP (Root System Description Pointer):
#[repr(C)]
pub struct ConfigurationTable {
pub guid: Guid,
pub address: *const c_void,
}
There a several GUID to look up in this table, who will define what kind of ACPI industry standard we are working with. The most interesting if we have to work with ACPI 1.0 or ACPI 2.0 (preferred):
/// Entry pointing to the old ACPI 1 RSDP.
pub const ACPI_GUID: Guid = Guid {
data1: 0xeb9d2d30,
data2: 0x2d88,
data3: 0x11d3,
data4: [0x9a, 0x16, 0x00, 0x90, 0x27, 0x3f, 0xc1, 0x4d],
};
/// Entry pointing to the ACPI 2 RSDP.
pub const ACPI2_GUID: Guid = Guid {
data1: 0x8868e871,
data2: 0xe4f1,
data3: 0x11d3,
data4: [0xbc, 0x22, 0x00, 0x80, 0xc7, 0x3c, 0x88, 0x81],
};
To use matching on the table, we let the Guid struct derive Eq and PartialEq:
#[derive(Copy, Clone, PartialEq, Eq)]
#[repr(C)]
pub struct Guid {
...
}
and implement a function to retrieve the RSDP:
pub enum RSDPType {
Acpi1,
Acpi2,
Other,
}
pub fn find_rsdp(acpi_tables: &[ConfigurationTable]) -> (RSDPType, *const c_void) {
let mut rsdp_address = None;
for table in acpi_tables {
match table.guid {
ACPI_GUID => {
// fallback ACPI < 2.0
rsdp_address = Some((RSDPType::Acpi1, table.address));
}
ACPI2_GUID => {
// if ACPI >= 2.0 we have what we want
return (RSDPType::Acpi2, table.address);
}
_ => {
// only BIOS available, not expected on a modern UEFI
if rsdp_address.is_none() {
rsdp_address = Some((RSDPType::Other, table.address));
}
}
}
}
rsdp_address.expect("ACPI RSDP not found")
}
We'll use this address to get access to the ACPI tables in the next article.
We add it to the get_boot_info function as well:
pub fn get_boot_info(_image_handle: Handle, system_table: *mut SystemTable) -> usize {
...
let (rsdp_type, rsdp) = find_rsdp((*system_table).acpi_tables());
0
}
Memory map
At last we get current memory map, where we have a defined usage of the memmory from what we have spent on UEFI services and our own unikernel. We access the memory map for two reasons:
- We need to have a
map_keyfrom the memory map to exit the boot services - We might want to know which areas not to overwrite. This includes the kernel, the framebuffer and ACPI tables.
We change the following functions in the BootServices struct. We need to allocate a pool to create a writable buffer for the memory map, as we don't know how big the map is, We use u8 instead of c_void as we will make this into an slice array of characters, and then we don't need to cast it:
pub struct BootServices {
...
get_memory_map: extern "efiapi" fn(
memory_map_size: &mut usize,
memory_map: *mut MemoryDescriptor,
map_key: &mut usize,
descriptor_size: &mut usize,
descriptor_version: &mut u32,
) -> usize,
allocate_pool:
extern "efiapi" fn(pool_type: MemoryType, size: usize, buffer: &mut *mut u8) -> usize,
...
}
based on the UEFI specification where we have a MemoryDescriptor and a MemoryType:
#[repr(C)]
pub struct MemoryDescriptor {
pub memory_type: MemoryType,
_padding: u32,
pub physical_start: u64,
pub virtual_start: u64,
pub number_of_pages: u64,
pub memory_attribute: u64,
}
#[repr(u32)]
pub enum MemoryType {
_Reserved = 0,
// The code portions of a loaded UEFI application (Free after exit)
LoaderCode,
// The data portions of a loaded UEFI application and the default data allocation type used by a UEFI application to allocate pool memory (Free after exit)
LoaderData,
// The code portions of a loaded UEFI Boot Service Driver (Free after exit)
BootServicesCode,
// The data portions of a loaded UEFI Boot Serve Driver, and the default data allocation type used by a UEFI Boot Service Driver to allocate pool memory. (Free after exit)
BootServicesData,
// The code portions of a loaded UEFI Runtime Driver
RuntimeServicesCode,
// The data portions of a loaded UEFI Runtime Driver and the default data allocation type used by a UEFI Runtime Driver to allocate pool memory.
RuntimeServicesData,
// Free (unallocated) memory
Conventional,
// Memory in which errors have been detected
Unusable,
// Memory that holds the ACPI tables.
ACPIReclaim,
// Address space reserved for use by the firmware.
ACPINonVolatile,
// Used by system firmware to request that a memory-mapped IO region be mapped by the OS to a virtual address so it can be accessed by EFI runtime services
MMIO,
// System memory-mapped IO region that is used to translate memory cycles to IO cycles by the processor.
MMIOPortSpace,
// Address space reserved by the firmware for code that is part of the processor
PALCode,
// A memory region that operates as EfiConventionalMemory. However, it happens to also support byte-addressable non-volatility.
PersistentMemory,
// A memory region that represents unaccepted memory, that must be accepted by the boot target before it can be used. Unless otherwise noted, all other EFI memory types are accepted. For platforms that support unaccepted memory, all unaccepted valid memory will be reported as unaccepted in the memory map. Unreported physical address ranges must be treated as not-present memory.
Unaccepted,
}
As we see from the description, the Conventional memory type is the free memory we can use for future usage, and as we exit the boot services immediately after, we can also use LoaderCode, LoaderData, BootServicesCode and BootServicesData as well.
We need to define a storage for the memory map first, which is done by running get_memory_map() twice, to see the result of the map size, and create a slice array for the second run:
use core::slice;
pub fn get_memory(bs: &BootServices) -> Result<(), usize> {
let (mmap_size, entry_size) = bs.memory_map_size()?;
let max_size = mmap_size + 8 * entry_size;
let mmap_storage = {
let ptr = bs.allocate_pool(MemoryType::LoaderData, max_size)?;
unsafe { slice::from_raw_parts_mut(ptr, max_size) }
};
Ok(())
}
We need to know the size of an MemoryDescriptor to create eight additional entries to allocate.
with the implementations for BootServices:
impl BootServices {
...
pub fn memory_map_size(&self) -> Result<(usize, usize), usize> {
let mut map_size = 0;
let mut map_key = 0;
let mut entry_size = 0;
let mut entry_version = 0;
let status = (self.get_memory_map)(
&mut map_size,
ptr::null_mut(),
&mut map_key,
&mut entry_size,
&mut entry_version,
);
if status != 0x8000000000000005 {
return Err(status)
}
Ok((map_size, entry_size))
}
pub fn allocate_pool(&self, mem_ty: MemoryType, size: usize) -> Result<*mut u8, usize> {
let mut buffer = ptr::null_mut();
let status = (self.allocate_pool)(mem_ty, size, &mut buffer);
if status != 0 {
return Err(status);
}
Ok(buffer)
}
}
For getting the memory map size, we will run into the EFI_BUFFER_TOO_SMALL error that is valued as 0x8000000000000005, as we entered a null pointer for the buffer. We expect the error, since we get the correct map_size returned together with the entry_size that defines the size of a MemoryDescriptor.
We then make a function that takes in this storage, instead of using a null pointer, where we now have access to the map_key and the memory map for usage later on. This is implemented in BootServices:
impl BootServices {
...
pub fn get_memory_map<'buf>(
&self,
buffer: &'buf mut [u8],
) -> Result<(
usize,
impl ExactSizeIterator<Item = &'buf MemoryDescriptor> + Clone,
), usize> {
let mut map_size = buffer.len();
let map_buffer = buffer.as_mut_ptr().cast::<MemoryDescriptor>();
let mut map_key = 0;
let mut entry_size = 0;
let mut entry_version = 0;
let status = (self.get_memory_map)(
&mut map_size,
map_buffer,
&mut map_key,
&mut entry_size,
&mut entry_version,
);
if status != 0 {
return Err(status);
}
let len = map_size / entry_size;
Ok((
map_key,
MemoryMapIter {
buffer,
entry_size,
index: 0,
len,
},
))
}
}
As we have each entry size, the buffer and the length, we can use it as an iterator:
#[derive(Clone)]
struct MemoryMapIter<'buf> {
buffer: &'buf [u8],
entry_size: usize,
index: usize,
len: usize,
}
impl<'buf> Iterator for MemoryMapIter<'buf> {
type Item = &'buf MemoryDescriptor;
fn next(&mut self) -> Option<Self::Item> {
if self.index < self.len {
let ptr = self.buffer.as_ptr() as usize + self.entry_size * self.index;
self.index += 1;
let descriptor = unsafe { &*(ptr as *const MemoryDescriptor) };
Some(descriptor)
} else {
None
}
}
}
impl ExactSizeIterator for MemoryMapIter<'_> {}
We finish up the get_memory function with returning these values for the get_boot_info function:
pub fn get_memory(
bs: &BootServices,
) -> Result<
(
usize,
impl ExactSizeIterator + Iterator<Item = &MemoryDescriptor> + Clone,
),
usize,
> {
...
bs.get_memory_map(mmap_storage)
}
Exit boot services
So in the end of our get_boot_info function, we now a use for the image handle and we use the map key, found before:
pub fn get_boot_info(image_handle: Handle, system_table: *mut SystemTable) -> usize {
...
match get_memory(&bs) {
Ok((map_key, _memory_map)) => {
if let Err(status) = bs.exit_boot_services(image_handle, map_key) {
return status;
}
}
Err(status) => return status,
}
}
and in the BootServices implementation, basically a one-to-one call to the EFI API function, adding a Rust style result:
struct BootServices {
...
exit_boot_services: extern "efiapi" fn(image_handle: Handle, map_key: usize) -> usize,
...
}
impl BootServices {
...
pub fn exit_boot_services(&self, image: Handle, memory_map_key: usize) -> Result<(), usize> {
let status = (self.exit_boot_services)(image, memory_map_key);
if status != 0 {
return Err(status)
}
Ok(())
}
}
Retrospective
Although we are done with UEFI at this point, it might be prudent to rewrite all the usize error codes to a rust-style error deriving the Error trait for easier handling of errors and debugging what error code comes out.
I am planning to put the code and notes on gitlab later on, so there is possibility to get feedback on errors or better implementations.
Update 2023-09-04
I fixed some errors with the Bootloader and added the code on Gitlab with public access.
Next time
Last time I mentioned to not work on UART or writing to the screen, but I see the necessarity soon to be able to show something, mostly for debugging. At this point, it could already be interesting to see how the memory map looks like, and we actually got access to it.
UART will be fine for usage on a virtual machine like QEMU or having a serial port access to a computer, but writing to the screen can give debugging information on real hardware.
Especially when diving into the ACPI tables and CPUID information about the system, it might be good to get some kind of verification that the virtual machine is setup correctly and/or hardware information is the expected.
So next time we will dive into writing characters on the screen, using the framebuffer.