Testing

[utterances-bot]

This is a general purpose comment thread for the Testing post.

[leggettc18]

I'm not sure if I did something wrong, but by following along with this guide exactly I was receiving an error about fmt being undeclared after setting up the CompareMessage implementation. I fixed it by adding use core::fmt; and use core::fmt::Write; at the top of the file, and increasing the PANIC_LINE by 2 to compensate. Did I miss a step along the way or is there a missing step in the guide?

[phil-opp]

@leggettc18 You did nothing wrong, I forgot to mention these imports. Fixed in 1f97103. Thanks for reporting!

[Lorithan]

I'm unable to run cargo xtest. Instead I keep keeping the message

error[E0463]: can't find crate for std
|
= note: the x86_64-blog_os-1954185736335484285 target may not be installed

cargo xbuild works fine, though.

[AntoineSebert]

Try cargo xtest --target x86_64-blog_os.json.

I have the same problem but couldn't figure out why the file isn't automatically found.

[phil-opp]

@AntoineSebert Do you have a build.target key defined in your .cargo/config file?

@Beidah Sounds like forgot a #![no_std] attribute (perhaps the conditional attribute in your lib.rs?). Or you accidentally use a dependency that uses std.

[AntoineSebert]

@phil-opp Yes it's there. The problem seems to have disappread today, and anyway it's not a big deal.

[AntoineSebert]

The recent improvements in the test framework now produce text outputs that are not colored in green, thus making the test result less visible within the terminal.
Is there a way to specify a color for the text when calling print/println macros ?

[phil-opp]

Yes, there is! You can use ANSI escape codes for coloring. For example, you can create a type to print something in green:

use core::fmt;

struct Green(&'static str);

impl fmt::Display for Green {
    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { 
        write!(f, "\x1B[32m")?; // prefix code
        write!(f, "{}", self.0)?;
        write!(f, "\x1B[0m")?; // postfix code
        Ok(())
    }
}

Now you can do serial_println!("{}", Green("[ok]")); at the end of your test.

I opened #603 for adding this to the post.

[bytesnail]

In the process of adding tests, I encountered some strange errors. In this test, I simply created a writer, but failed, and judging from the error message, it seems that it did not execute any tests at all, because I did not see the information printed by test_runner or panic_handler on the console.

lws@lsw:/document/project/flandre-os/FlandreOS$ cargo xtest test_default_writer
   Compiling flandre_os v0.0.1 (/document/project/flandre-os/FlandreOS)
    Finished dev [unoptimized + debuginfo] target(s) in 0.33s
     Running target/x86_64-flandre_os/debug/deps/flandre_os-d120248f92b70111
Building bootloader
   Compiling bootloader v0.6.0 (/work/tool/rust/cargo/registry/src/mirrors.ustc.edu.cn-61ef6e0cd06fb9b8/bootloader-0.6.0)
    Finished release [optimized + debuginfo] target(s) in 0.72s
Running: `qemu-system-x86_64 -drive format=raw,file=/document/project/flandre-os/FlandreOS/target/x86_64-flandre_os/debug/deps/bootimage-flandre_os-d120248f92b70111.bin -device isa-debug-exit,iobase=0xf4,iosize=0x04 -serial stdio -display none test_default_writer`
qemu-system-x86_64: -display none: drive with bus=0, unit=0 (index=0) exists
error: test failed, to rerun pass '--lib'

[bytesnail]

I did all kinds of printing tests and the printing function seemed to work normally, but when I tried to add some tests to ensure that these functions would not be damaged in future modifications, I got the above errors.

[bjorn3]

I think passing a test name to cargo xtest doesnt work. Try running just cargo xtest.

[phil-opp]

I think passing a test name to cargo xtest doesnt work. Try running just cargo xtest.

Yes, as you see from the "Running: …" output, the test_default_writer is directly passed as an argument to QEMU, which does not understand it and exits with an error. Passing the name of a single test is a feature of the default test runner and not available in our bare-metal environment because our kernel does not get any arguments passed when it's started.

What you can do is to run the tests of the library component through cargo xtest --lib and the tests of the binary through cargo xtest --bin blog_os. You can also run an individual integration test through cargo xtest --test test_name.

[xj42]

Now we can run the test using cargo xtest --test panic_handler. We see that it passes, which means that the reported panic info is correct. If we use a wrong line number in PANIC_LINE or panic with an additional character through panic!("{}x", MESSAGE), we see that the test indeed fails.

I tried adding the "x" as you indicated in the above paragraph. The test did not fail. WIth some more fooling about i see that the fn write_str(&mut self, s: &str) -> fmt::Result {} function is called 4 times. The third time it is called, it sets equals to true. Because there is no else setting false, the test does not work as expected.

I have submitted a pull request #610

[phil-opp]

@xj42 Thanks a lot for reporting! I created #611 to fix this.

[ta5een]

Hi, thank you for taking your time and effort into making this brilliant blog! I've learnt so much about operating systems and the Rust language in general! Can't wait to see how this blog unfolds and what new things you'll bring.

I just wanted to ask if it is the expected behaviour for serial_print! and serial_println! to only work in test mode? With cargo xtest, the serial print macros work as expected; they print to my host's terminal. However, with cargo xrun, nothing is shown in my host's terminal, nor my QEMU instance. Oddly enough, the system hasn't panicked, so it somehow worked (but the bytes don't show up).

It seems to me that perhaps this was the desired behaviour? If so, is there a way for me to send data to the serial port while the QEMU instance is running as normal?

[zhanlidegui]

[package.metadata.bootimage] test-args = ["-device", "isa-debug-exit,iobase=0xf4,iosize=0x04","-serial", "stdio","-display","none"] test-success-exit-code = 33 test-timeout = 120

This setting is only valid for testing

[wintertempest]

The section where we try to print the names of the test functions is not functioning correctly for me. Whenever I use core::any::type_name::(), it only prints the actual type, which is &dyn core::ops::function::Fn() instead of the function's name.

[bjorn3]

Can you show your Testable implementation? And your test_runner?

[wintertempest]

Hi, thanks for replying, this is my implementation

[wintertempest]

pub trait Testable {
fn run(&self) -> ();
}

impl Testable for T
where
T: Fn(),
{
fn run(&self) {
serial_print!("{}...\t", core::any::type_name::());
self();
serial_println!("[ok]");
}
}

#[cfg(test)]
fn test_runner(tests: &[&dyn Fn()]) {
serial_println!("Demarage de {} tests", tests.len());
for test in tests {
test.run();
}
serial_println!("Fin des tests");
exit_qemu(QemuExitCode::Success);
}

[bjorn3]

You need to change the function signature of test_runner to accept &[&dyn Testable] instead of &[&dyn Fn()].

[wintertempest]

Thank you very much, it was indeed the issue.

[Cooper-Pilot]

#[cfg(test)] never returns true when compiling with cargo t.
This is my cargo.toml:

[package]
name = "xarbaros"
version = "0.1.0"
edition = "2018"

#[profile.dev]
#panic = "abort"
#[profile.release]
#panic = "abort"

[dependencies.lazy_static]
version = "1.0"
features = ["spin_no_std"]

[dependencies]
spin = "0.5.2"
volatile = "0.2.6"
bootloader = "0.9.23"

main.rs:

#![no_std]
#![no_main]

#![feature(custom_test_frameworks)]
#![test_runner(crate::test_runner)]

#![reexport_test_harness_main = "test_main"]

use core::{panic::PanicInfo, fmt::Write};
mod vga_buffer;

#[no_mangle]
pub extern "C" fn _start() -> ! {
    println!("Hello World{}", "!");
    
    #[cfg(test)]
    test_main();
    
    loop {}
}

#[panic_handler]
fn panic(_info: &PanicInfo) -> ! {
    loop {}
}

#[cfg(test)]
pub fn test_runner(tests: &[&dyn Fn()]) {
    println!("Running {} tests", tests.len());
    for test in tests {
        test();
    }
}

[AtomicGamer9523]

test_main doesn't exist

• it doesn't look like you have any tests to run

[Cooper-Pilot]

When I change #[cfg(test)] test_main(); to #[cfg(test)] println!("test");, I expect it to print “test” to the buffer when ran with cargo t, but the results are exactly the same as when ran with cargo r, that is it doesn’t print “test”. It seems to just be completely ignoring tests for me.

[toolateralus]

same here, and i also had some meaningless test cases and none ran.

[phil-opp]

Could you try with the post-04 branch of this repo?

[jgarth]

I ran into the same issue because my runner was using cargo objcopy, which rebuilds the executable (without test). Fixed mine by invoking rust-objcopy directly.

[JiaFengYu]

Is it possible to move the testing module such as test_panic_handler, exit_qemu, Testable trait from lib.rs into its own dedicated file? It would make the code more structured and but I've had a problem with test_main(), where I've tried importing it via use crate::test_main() and also included these attributes in the dedicated testing module file:

#![feature(custom_test_frameworks)]
#![test_runner(blog_os::test_runner)]
#![reexport_test_harness_main = "test_main"]

Anyone have any thoughts?

[Vrajs16]

Is there a solution so that if the the test panics, can we continue?

[tsatke]

You would have to be able to catch a panic, which means you would have to implement unwinding in the kernel. You may want to look into https://github.com/nbdd0121/unwinding for that

[vincentberthier]

The step after testing is to get coverage data.
Tryning to run llvm-cov on the tests fails however:

error: `profiler_builtins` crate (required by compiler options) is not compatible with crate attribute `#![no_core]`

error[E0463]: can't find crate for `profiler_builtins`
  |
  = note: the compiler may have been built without the profiler runtime```

I couldn’t find anything about how to solve that issue. Is there a solution to this?

[dedestem]

cargo test gives this: (signal: 11, SIGSEGV: invalid memory reference)

[Fincap]

If anyone else is experience the same issue that I was with cargo test failing, and the posted workaround of removing panic = "abort" from Cargo.toml not working, I have a workaround that worked for me.

Insert into your .cargo/config.toml in the [unstable] section:
panic-abort-tests = true

Make sure you're using the latest nightly compiler.

[delguoqing]

both lib.rs and basic_boot.rs contain a entry point function "_start", but cargo test still compile. Why is that?

[delguoqing]

ok, I think I know why. When we're running test cases for basic_boot.rs, cfg(test) is not used for lib.rs?

[SimonOneNineEight]

I think there's a minor error, in the panic function it said shouldn't return, but there's no semi-column at the end of the function call. I've tried to copy paste the code into my editor and the lsp tells me to add a semi-column.

// in tests/basic_boot.rs

#![test_runner(blog_os::test_runner)]

#[panic_handler]
fn panic(info: &PanicInfo) -> ! {
    blog_os::test_panic_handler(info)
}

[AtomicGamer9523]

You shouldn't need to add the semicolon, because blog_os::test_panic_handler should never return.

[danielmorozoff]

I am receiving a compilation error whenever I attempt to include the x86_64 = "0.14.2" crate. Has anyone come across this and what was the remedy that alleviated this?

error[E0053]: method `steps_between` has an incompatible type for trait
   --> /Users/danielmorozoff/.cargo/registry/src/index.crates.io-6f17d22bba15001f/x86_64-0.14.12/src/addr.rs:382:51
    |
382 |     fn steps_between(start: &Self, end: &Self) -> Option<usize> {
    |                                                   ^^^^^^^^^^^^^ expected `(usize, Option<usize>)`, found `Option<usize>`
    |
    = note: expected signature `fn(&addr::VirtAddr, &addr::VirtAddr) -> (usize, Option<usize>)`
               found signature `fn(&addr::VirtAddr, &addr::VirtAddr) -> Option<usize>`
help: change the output type to match the trait
    |
382 |     fn steps_between(start: &Self, end: &Self) -> (usize, Option<usize>) {
    |                                                   ~~~~~~~~~~~~~~~~~~~~~~

[kingananas20]

i just tried out the latest crate x86_64 v 0.14.13 and it works

0.14.13 doesn't work anymore but 0.14.12 works for me

[ChocolateLoverRaj]

0.14.13 doesn't work anymore but 0.14.12 works for me

@kingananas20 0.14.13 expects a newer Rust nightly version than 0.14.12. So if you want to use 0.14.13 just upgrade your Rust nightly version.

[Zain-Najeeb]

I am getting this same error, not really sure why either. Ive tried both 0.14.12 and 0.14.13 and rustc nightly version on both 1.84.0 and 1.88.0. Here is my Cargo.toml file if it helps

 
[package]
name = "blog_os"
version = "0.1.0"
edition = "2021"

[dependencies]
bootloader = "0.9.22"
volatile = "=0.2.6" 
spin = "0.5.2"
x86_64 = "0.14.12"

[dependencies.lazy_static]
version = "1.4.0"
features = ["spin_no_std"]

[package.metadata.bootimage]
test-args = ["-device", "isa-debug-exit,iobase=0xf4,iosize=0x04"]

[Zain-Najeeb]

I was able to fix it after doing some testing. Being on version 1.88.0 and with x86_64 version 0.15.2 worked for me

[ChocolateLoverRaj]

I was able to fix it after doing some testing. Being on version 1.88.0 and with x86_64 version 0.15.2 worked for me

@Zain-Najeeb note that 0.15.2 could contain breaking changes. If you are following this tutorial I would recommend keeping your Rust version as it is and using 0.14.13

[UnwhiteCat]

I have a error at the first step trying to run the "cargo test" barely...Has the different error as wrote in this book.

UnwhiteCat@uccomputer ~/Codes/rust/blogos $ cargo test
   Compiling scopeguard v1.2.0
   Compiling bootloader v0.9.29
error[E0152]: duplicate lang item in crate `core` (which `rustc_std_workspace_core` depends on): `sized`
  |
  = note: the lang item is first defined in crate `core` (which `bootloader` depends on)
  = note: first definition in `core` loaded from /home/UnwhiteCat/Codes/rust/blogos/target/x86_64-blog_os/debug/deps/libcore-33e3a870bea73245.rmeta
  = note: second definition in `core` loaded from /home/UnwhiteCat/Codes/rust/blogos/target/x86_64-blog_os/debug/deps/libcore-f389adb7b0a7ac1b.rmeta

error[E0152]: duplicate lang item in crate `core` (which `rustc_std_workspace_core` depends on): `sized`
  |
  = note: the lang item is first defined in crate `core` (which `scopeguard` depends on)
  = note: first definition in `core` loaded from /home/UnwhiteCat/Codes/rust/blogos/target/x86_64-blog_os/debug/deps/libcore-33e3a870bea73245.rmeta
  = note: second definition in `core` loaded from /home/UnwhiteCat/Codes/rust/blogos/target/x86_64-blog_os/debug/deps/libcore-f389adb7b0a7ac1b.rmeta

For more information about this error, try `rustc --explain E0152`.
error: could not compile `scopeguard` (lib) due to 1 previous error
warning: build failed, waiting for other jobs to finish...
error: could not compile `bootloader` (lib) due to 1 previous error

Here is my rustc version.

UnwhiteCat@uccomputer ~/Codes/rust/blogos $ rustc --version
rustc 1.85.0-nightly (6b6a867ae 2024-11-27)

[rctcwyvrn]

#840 (comment)

Delete panic = "abort" from your cargo.toml

[doomed-neko]

for some reason, qemu does not exit after calling exit_qemu(), this only happens when using cargo bootimage but when I use qemu-system-x86_64 and pass the isa-debug-exit,iobase=0xf4,iosize=0x04 argument, it works fine

[doomed-neko]

nevermind, turns out I have set the [package.metadata.bootimage] options in .cargo/config.toml instead of Cargo.toml

[devashish-sood]

When writing the serial_println! macro, why are these cases distinct and more than the cases we had when writing the println! macro?

[RobinToncourt]

If a test fails it prevents the next tests to be executed(by going through the panic handler due to the assert).
To prevent that you can rewrote the assert and assert_eq methods.
Here my solution: https://github.com/RobinToncourt/os/blob/master/src/test_utils.rs
As for the should_panic test, I don't find them relevant as I consider that an operating system should never panic.

[jsanchezpc]

Hi!
I searched and can't find any solutions. Can anybody help me please?

When I try to run cargo test, it doesnt launch qemu, but I can't get to see any errors at all. Here is my code:

#![no_std] // don't link the Rust standard library
#![no_main] // disable all Rust-level entry points
#![feature(custom_test_frameworks)]
#![test_runner(crate::test_runner)]
#![reexport_test_harness_main = "test_main"]

mod vga_buffer;
use core::panic::PanicInfo;

#[cfg(test)]
pub fn test_runner(tests: &[&dyn Fn()]) {
    println!("Running {} tests", tests.len());
    for test in tests {
        test();
    }
}

#[test_case]
fn trivial_assertion() {
    print!("trivial assertion... ");
    assert_eq!(1, 1);
    println!("[ok]");
}

/// This function is called on panic.
#[panic_handler]
fn panic(info: &PanicInfo) -> ! {
    println!("{}", info);
    loop {}
}

#[no_mangle] // don't mangle the name of this function
pub extern "C" fn _start() -> ! {
    // this function is the entry point, since the linker looks for a function
    // named `_start` by default

    println!("Hello World!");

    #[cfg(test)]
    test_main();

    loop {}
}

The output in my console:

> cargo test
Finished `test` profile [unoptimized + debuginfo] target(s) in 0.10s 

All seems good but Qemu wont show up. Why?? Tried to use chatgpt but didnt help at all.

[jsanchezpc]

Ok I just solved the problem. If you have the same issue, try this:
Go to your Cargo.toml file and set the test value to true (in [[bin]] )