Menu

MTRR Emulation: Beginner’s Common Mistake in EPT Setup

2022-01-01 - Computer Architecture, Virtualization Technology

According to Chapter 28.2.7 “EPT and Memory Typing”, Volume 3, Intel 64 and IA-32 Architecture Software Developer’s Manual, the MTRRs have no effect on the memory type used for an access to a guest-physical address. This means the memory type for the guest is solely determined by the memory type indicated in EPT paging structure.

In the early versions of NoirVisor, MTRRs were not considered in setting up EPT paging structures. It was not an issue if it was to be running in a virtual machine in that typically the only MTRR enabled is describing write-back memories. However, things are different on a real machine. By virtue of memory usage on different peripheral hardware, memory types are different on different regions of memory. For example, graphics buffer may require write-combining memories, regular devices would require uncacheable memories, etc.

In this regard, improper setup of memory types could induce cache consistency issues. If an external hardware writes cached memory, then there could be inconsistency between main memory and the copy in cache. Some computers can correct such inconsistency and thereby runs normally. Some may trigger hardware minor logic errors difficult to be diagnosed, which typically result in annoying user experiences. Some may be detected by the processor and throws an Machine-Check exception, abbreviated as #MC, detecting error in L2-cache data-reads.
Let’s rethink the order of logic regarding this #MC exception. The wrong memory type is set by the processor, and the processor thereby caches the region. The external hardware writes this region. By virtue of the lack of cache coherency protocols, the processor has no idea the copy held in cache becomes invalid. Hence, when the caching manager detects the fault, and if it is unable to correct such fault, it will indicate error in cache while reading data.

To address the issue, the solution is to emulate the MTRRs via setting up the memory type fields in the last paging structures of EPT in accordance to the MTRRs. In general, there are three types of MTRRs: the default-type MTRR, fixed MTRRs and variable MTRRs. The default-type MTRR defines the memory type if the memory is described by any other MTRRs. The fixed MTRRs are straightforward: their length are fixed All fixed MTRRs only covers the first MiB region of main memory. Therefore, the first 2MiB memory must be described in 4KiB paging granularity in order to describe fixed MTRRs. The logic of variable MTRRs is a bit different. I am quite curious why Intel do not simply defines it with bases and lengths but with bases and masks instead. The latter has obviously complex purpose of defining memory ranges, yet the alignment requirements of variable MTRRs forbid defining complex memory ranges. so why not simply use length instead of mask in the first place? Anyway, there are two ways to emulate MTRRs: the simple approach and the brute-force approach.
The simple approach to emulate variable MTRRs is to scan the bit in mask forwardly. The lowest bit set indicates the length of the range. The brute-force approach is to traverse all addresses available in EPT. Do a logical-and operation to compare the masked address and the masked base. If they equal, the address is in the range. NoirVisor adopts the brute-force approach to emulate variable MTRRs. In that the approach is brute-force, necessary optimizations are performed: ignoring disabled MTRRs and MTRRs that has the same memory type to default memory type.
Please note that, other than regular MTRRs, the SMRR should also be checked as well. If the capability MSR indicates support to SMRR, then SMRR is present and thereby requires emulating.

In terms of AMD-V, there is no need for emulating MTRRs. According to Chapter 15.25.8, “Combining Memory Types, MTRRs”, Volume 2, AMD64 Architecture Programmer’s Manual, the MTRRs do affect the memory type, unlike the one in Intel VT-x, where MTRRs have no effects on determining memory type for guest physical addresses. By the way, the memory type in NPT is defined by bit-combinations PAT-PCD-PWT and the host PAT register. For example, if an entry clears PAT-PCD-PWT and the lowest three bits in host PAT register is 6, then the memory type is write-back if MTRRs also define this address as write-back.

Leave a Reply

Your email address will not be published. Required fields are marked *