Creating UVM Tests Dynamically

Everyone who uses UVM knows that using the library ofter requires large amounts of boilerplate code. Tests are no exception.

A UVM test usually starts a certain sequence during its run phase (but other run-time phases could also be used). Let’s assume that we have three sequences: some_sequence, some_other_sequence and yet_another_sequence, which we each want to start from its own test.

Here’s an example of a test that starts some_sequence:

class test_that_executes_some_sequence extends uvm_test;

  function new(string name, uvm_component parent);
    super.new(name, parent);
  endfunction

  virtual task run_phase(uvm_phase phase);
    some_sequence seq = some_sequence::type_id::create("seq", this);
    phase.raise_objection(this);
    seq.start(null);
    phase.drop_objection(this);
  endtask

  `uvm_component_utils(test_that_executes_some_sequence)

endclass

A test that starts some_other_sequence is:

class test_that_executes_some_other_sequence extends uvm_test;

  function new(string name, uvm_component parent);
    super.new(name, parent);
  endfunction

  virtual task run_phase(uvm_phase phase);
    some_other_sequence seq = some_other_sequence::type_id::create("seq", this);
    phase.raise_objection(this);
    seq.start(null);
    phase.drop_objection(this);
  endtask

  `uvm_component_utils(test_that_executes_some_other_sequence)

endclass

Finally, a test that starts yet_another_sequence is:

class test_that_executes_yet_another_sequence extends uvm_test;

  function new(string name, uvm_component parent);
    super.new(name, parent);
  endfunction

  virtual task run_phase(uvm_phase phase);
    yet_another_sequence seq = yet_another_sequence::type_id::create("seq", this);
    phase.raise_objection(this);
    seq.start(null);
    phase.drop_objection(this);
  endtask

  `uvm_component_utils(test_that_executes_yet_another_sequence)

endclass

That’s a pretty impressive amount of code just to say “start each of the three sequences in its own test”. The amount is not necessarily the problem, but coupled with the fact that there is a lot of duplication makes it become one. It’s not immediately obvious whether the three files differ by more than just the sequence and test names or whether there are any hidden tweaks which make them behave slightly differently.

We would like to achieve something like the following pseudo-code:

create_test_that_starts(some_sequence)
create_test_that_starts(some_other_sequence)
create_test_that_starts(yet_another_sequence)

This way we know that the tests only differ in the sequence being started. Even cooler would be to be able to specify tests using loops:

for seq in [some_sequence, some_other_sequence, yet_another_sequence]:
    create_test_that_starts(seq)

This matches the natural language description of our problem statement: “start each of the three sequences in its own test”.

One possible solution would be to use macros, but these generally suffer from an array of issues, like lack of clarity as to how and where they should be used, poor error messages when writing code that doesn’t compile, lack of debug capabilities, etc.

Another widespread solution for such problems in the industry is to use code generation. I’m generally skeptical of code generation of this sort, which usually means passing the buck to some other half baked scripts. These scripts would be in a different language, so they would require us to either spin off or duplicate information we need in SystemVerilog. They also wouldn’t integrate as well into the overall development and execution flow.

We’ll look at an alternative way of defining tests, which doesn’t suffer from the problems of the “traditional” approaches. We won’t just look at the finished code, but take a journey through the steps required to build it.

Before we begin, let’s do a crash course in how UVM starts tests. UVM uses the value of +UVM_TESTNAME as a type name when it instructs the factory to create the test. Conceptually, it does the following:

string test_type_name = get_value_of_plusarg("UVM_TESTNAME");
uvm_component test = uvm_factory()::get().create_component_by_name(test_type_name);

Users define their test classes and register them with the factory using the uvm_component_utils macro. This makes it possible for the test type name to be used as an argument to +UVM_TESTNAME.

To reduce the amount of code we need, we want to consolidate the duplicated code into a single place. If we look at the tests, they only differ by the type of the seq variable, which holds the sequence to start. We can create a shared template by using a class that is parameterized by the sequence type:

class test_that_executes_sequence_via_param #(type SEQ = int) extends uvm_test;

  function new(string name, uvm_component parent);
    super.new(name, parent);
  endfunction

  virtual task run_phase(uvm_phase phase);
    SEQ seq = SEQ::type_id::create("seq", this);
    phase.raise_objection(this);
    seq.start(null);
    phase.drop_objection(this);
  endtask

  `uvm_component_param_utils(test_that_executes_sequence_via_param #(SEQ))

endclass

While the uvm_component_param_utils macro will register the class with the UVM factory, it will not register it using a type name. Since we don’t have type names, we won’t have anything to pass to +UVM_TESTNAME to select tests.

We could create subclasses of the parameterized class, where we pass the desired sequence type for the SEQ parameter, and register those with the factory. For example, for some_sequence we would have:

class test_that_executes_some_sequence_using_param
    extends test_that_executes_sequence_via_param #(some_sequence);

  function new(string name, uvm_component parent);
    super.new(name, parent);
  endfunction

  `uvm_component_utils(test_that_executes_some_sequence_using_param)

endclass

This is still a pretty large amount of boilerplate. Granted, this is code that might not ever change. We could do a little better, though.

The uvm_component_utils macro maps type names to “real” classes using the uvm_component_registry class. If we were to expand the code for uvm_component_utils(test_that_executes_some_sequence_using_param) we would see uvm_component_registry #(test_that_executes_some_sequence_using_param, "test_that_executes_some_sequence_using_param"). This instructs the factory to create a component of type test_that_executes_some_sequence_using_param whenever the string "test_that_executes_some_sequence_using_param" is supplied as a type name.

The exact way this is done is kind of all over the place. The underlying class that the factory uses is called uvm_object_wrapper. It declares two functions, create_object() and create_component(), which the factory uses to create instances. The uvm_component_registry class extends uvm_object_wrapper and implements its create_component() function, but it also has many other functions related to registering itself with the factory and handling type overrides. A cleaner implementation would have been for uvm_component_registry to be its own class, that doesn’t extend uvm_object_wrapper, that instead creates an instance of a uvm_object_wrapper and registers it with the factory instead of itself. This would have provided a better separation of concerns.

We want to create our own mapping, which instructs the factory to create a component of type test_that_executes_sequence_via_param #(some_sequence) whenever the string "test_that_executes_some_sequence_using_param" is supplied as a type name. The easiest way to do this is by creating own own class that extends uvm_component_registry with the appropriate parameters:

class wrapper_for_test_that_executes_some_sequence_using_param
    extends uvm_component_registry #(test_that_executes_sequence_via_param #(some_sequence), "test_that_executes_some_sequence_using_param");
endclass

This class just has to exist. The code in uvm_component_registry will take care of instructing the UVM factory to produce an instance of the test class parameterized with some_sequence when test_that_executes_some_sequence_using_param is passed as a value for +UVM_TESTNAME.

While using parameterization solves a part of the boilerplate issue by having the aspect of “starting sequences” in a single class, it still requires us to write the wrapper classes necessary for factory registration by hand. We can’t implement any kind of looping over the sequences we want to start from tests, as we set out to do.

We have to transpose our code from using types and parameterization to some kind of dynamic constructs. If we go back to test_test_that_executes_sequence_via_param, we see that the type of sequence is not at all relevant to starting it. All uvm_sequences have a start() task, which can be used by the test to start it:

  virtual task run_phase(uvm_phase phase);
    uvm_sequence seq = create_seq();
    phase.raise_objection(this);
    seq.start(null);
    phase.drop_objection(this);
  endtask

We just need a way of creating sequences without hard-coding the exact type. Luckily, the UVM factory already has us covered here. Remember that uvm_object_utils creates a subclass of uvm_object_wrapper that can create the “real” type:

uvm_sequence seq;
uvm_object_wrapper seq_wrapper = some_sequence::get_type()
$cast(result, seq_wrapper.create_object("seq"));

Instead of hard coding the sequence type in the test, so the test can create an instance of that sequence itself, we can instead supply it with a uvm_object_wrapper that produces that sequence:

class test_that_executes_sequence_via_constructor extends uvm_test;

  local const uvm_object_wrapper seq_wrapper;

  function new(string name, uvm_component parent, uvm_object_wrapper seq_wrapper);
    super.new(name, parent);
    this.seq_wrapper = seq_wrapper;
  endfunction

  virtual task run_phase(uvm_phase phase);
    uvm_sequence seq = create_seq();
    phase.raise_objection(this);
    seq.start(null);
    phase.drop_objection(this);
  endtask

  local function uvm_sequence create_seq();
    uvm_sequence result;
    if (!$cast(result, seq_wrapper.create_object("seq")))
      `uvm_fatal("TEST", "Cannot construct sequence from supplied wrapper")
    return result;
  endfunction

endclass

Notice that we have to check at run time whether the seq_wrapper we got can really produce an object of type uvm_sequence. In other programming languages, which take class parameterization seriously and where it’s not just an afterthought, it is possible to enforce this compile time. This would be done by declaring the seq_wrapper argument as being a uvm_object_wrapper of a type that extends uvm_sequence. The syntax could look something like the following:

function new(uvm_object_wrapper #(? extends uvm_sequence) seq_wrapper)

For reference, in Java this language feature is called an upper bounded wildcard.

The tricky part is figuring out how to supply the sequence wrapper. We’ve already talked about how the factory uses uvm_object_wrappers to perform the actual creation and maps type names to such objects.

Let’s look at how we would create a uvm_object_wrapper that creates a test that starts some_sequence and registers itself with the name "test_that_executes_some_sequence_using_constructor". The first thing we need to implement is the create_component() function:

class wrapper_for_test_that_executes_some_sequence_using_constructor
    extends uvm_object_wrapper;

  virtual function uvm_component create_component(string name, uvm_component parent);
    test_that_executes_sequence_via_constructor result = new(name, parent, some_sequence::get_type());
    return result;
  endfunction

endclass

The uvm_object_wrapper class also provides a get_type_name() function, which is supposed to return the name of type being wrapped. I noticed that if we don’t implement this function, we get some annoying prints in the console, so let’s do this as well:

class wrapper_for_test_that_executes_some_sequence_using_constructor
    extends uvm_object_wrapper;

  virtual function string get_type_name();
    return "test_that_executes_some_sequence_using_constructor";
  endfunction

  // ...

endclass

Last, but not least, we should also register this wrapper with the UVM factory, so it knows how to produce instances when "test_that_executes_some_sequence_using_constructor" is used as a type name. This is more or less a copy/paste of how uvm_component_registry does it:

class wrapper_for_test_that_executes_some_sequence_using_constructor
    extends uvm_object_wrapper;

  local static wrapper_for_test_that_executes_some_sequence_using_constructor me = get();

  local function new();
  endfunction

  static function wrapper_for_test_that_executes_some_sequence_using_constructor get();
    if (me == null) begin
  	  uvm_coreservice_t cs = uvm_coreservice_t::get();
  	  uvm_factory factory = cs.get_factory();
      me = new();
      factory.register(me);
    end
    return me;
  endfunction

  // ...

endclass

It’s worth discussing about how registration is actually triggered. Notice that the me variable is static, but has a initial value. This value is computed during static initialization, which happens very early within a simulation, before any processes (e.g. from initial or always blocks) are executed.

If we were to have to write a uvm_object_wrapper for each of the sequences we want to start, we’d end up with a lot of boilerplate. This is exactly what we want to avoid.

We notice that we can pass the sequence wrapper as an argument to the test wrapper itself:

class wrapper_for_test_that_executes_sequence_using_constructor
    extends uvm_object_wrapper;

  local const uvm_object_wrapper seq_wrapper;

  function new(uvm_object_wrapper seq_wrapper);
    this.seq_wrapper = seq_wrapper;
  endfunction

  virtual function string get_type_name();
    return $sformatf("test_that_executes_%s_using_constructor", seq_wrapper.get_type_name());
  endfunction

  virtual function uvm_component create_component(string name, uvm_component parent);
    test_that_executes_sequence_via_constructor result = new(name, parent, seq_wrapper);
    return result;
  endfunction

endclass

We can extract the code for factory registration to a different class, which is responsible for registering tests for the sequences we want to start:

class tests_that_execute_each_sequence;

  local static bit test_for_some_other_sequence_registered = register_test(some_other_sequence::get_type());
  local static bit test_for_yet_another_sequence_registered = register_test(yet_another_sequence::get_type());

  local static function bit register_test(uvm_object_wrapper seq_wrapper);
    uvm_coreservice_t cs = uvm_coreservice_t::get();
  	uvm_factory factory = cs.get_factory();
    wrapper_for_test_that_executes_sequence_using_constructor test_wrapper = new(seq_wrapper);
    factory.register(test_wrapper);
    return 1;
  endfunction

endclass

Notice that the register_test() function returns a bit, but the value is always 1. This is so that it can be used as an initial value for a static helper variable, so that the function is called during static initialization.

The tests_that_execute_each_sequence class does a bit much, though. It knows how to register tests with the factory, but is also registers the tests we want.

We can extract the code for registration into an own class:

class test_builder;

  local const wrapper_for_test_that_executes_sequence_using_constructor test_wrapper;

  local function new(uvm_object_wrapper seq_wrapper);
    test_wrapper = new(seq_wrapper);
  endfunction

  static function test_builder for_sequence_type(uvm_object_wrapper seq_wrapper);
    test_builder result = new(seq_wrapper);
    return result;
  endfunction

  function bit register();
    uvm_coreservice_t cs = uvm_coreservice_t::get();
  	uvm_factory factory = cs.get_factory();
    factory.register(test_wrapper);
    return 1;
  endfunction

endclass

Registering tests for the sequences is a bit cleaner, because we’re not distracted by the actual test builder code.

class tests_that_execute_each_sequence;

  local static bit test_for_some_other_sequence_registered
      = test_builder::for_sequence_type(some_other_sequence::get_type()).register();
  local static bit test_for_yet_another_sequence_registered
      = test_builder::for_sequence_type(yet_another_sequence::get_type()).register();

endclass

Now it starts to become obvious that we can use a loop to register the tests:

class tests_that_execute_each_sequence;

  local static bit tests_registered = register_tests();

  local static function bit register_tests();
    uvm_object_wrapper seqs[] = '{
        some_sequence::get_type(),
        some_other_sequence::get_type(),
        yet_another_sequence::get_type() };

    foreach (seqs[i])
      void'(test_builder::for_sequence_type(seqs[i]).register());
  endfunction

endclass

While it’s very convenient to use loops, being able to use procedural code to define tests is extremely powerful. It makes it possible to do things like to conditionally define tests based on conditions known to the testbench (for example, whether a feature is present in the DUT) or to read some input file and define tests based on that. The space of possibilities is huge.

To reap the benefits of dynamic test creation, we would want to integrate the code from above into our own projects. The test_builder class and its associated classes feel as if they shouldn’t depend on any kind of user code and that they could be reusable.

One thing a uvm_test also does, that we completely left out of the examples, is to instantiate the rest of the verification environment (i.e. the agents, scoreboards, etc.). The examples we’ve looked at so far completely ignore this fact. We’ll emulate it with an info message in the build phase:

virtual class abstract_test extends uvm_test;

  function new(string name, uvm_component parent);
    super.new(name, parent);
  endfunction

  virtual function void build_phase(uvm_phase phase);
    super.build_phase(phase);
    `uvm_info("TEST", "Building TB...", UVM_NONE)
  endfunction

endclass

The test_builder registers tests of the type test_that_executes_sequence, which only knows how to start sequences, but knows nothing about TB creation. In order to combine the two, test_that_executes_sequence can become a mixin, allowing the user to include it onto their abstract_test class:

class test_that_executes_sequence #(type T = uvm_test) extends T;

    // ...

endclass

Naturally, the parameter has to propagate to the other classes as well:

class wrapper_for_test_that_executes_sequence #(type T = uvm_test) extends uvm_object_wrapper;

  virtual function uvm_component create_component(string name, uvm_component parent);
    test_that_executes_sequence #(T) result = new(name, parent, seq_wrapper);
    return result;
  endfunction

  // ...

endclass

class test_builder #(type T = uvm_test);

  local const wrapper_for_test_that_executes_sequence #(T) test_wrapper;

  static function test_builder #(T) for_sequence_type(uvm_object_wrapper seq_wrapper);
    test_builder #(T) result = new(seq_wrapper);
    return result;
  endfunction

  // ...

endclass

This allows users to layer test building onto their own code:

class tests_that_execute_each_sequence;

  local static function bit register_tests();
    // ...
    foreach (seqs[i])
      void'(test_builder #(abstract_test)::for_sequence_type(seqs[i]).register());
  endfunction

  // ...

endclass

At the moment, I see the test builder as more of a pattern, instead of a reusable library, because there’s not much code there to reuse directly. A full example can be found at https://github.com/verification-gentleman-blog/creating-uvm-tests-dynamically. It can serve as the basis for implementing the pattern in your own projects.

In a future post, I’m going to show you how to use dynamic test creation to layer constraints onto a random test, in order to target certain areas of the design space. Be sure to subscribe if you don’t want to miss it.