We extend SBI PMU implementation to allow custom PMU device operations
which a platform can use for platform specific quirks.
The custom PMU device operations added by this patch include:
1) Operations to allow a platform implement custom firmware events.
These custom firmware events can be SBI vendor extension related
events or platform specific per-HART events are not possible to
count through HPM CSRs.
2) Operations to allow a platform implement custom way for enabling
(or disabling) an overflow interrupt (e.g. T-Head C9xx).
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Reviewed-by: Atish Patra <atishp@rivosinc.com>
Currently, we have 32 elements (i.e. SBI_PMU_FW_EVENT_MAX) array of
"struct sbi_pmu_fw_event" for each of 128 possible HARTs
(i.e. SBI_HARTMASK_MAX_BITS).
To reduce memory usage of OpenSBI, we update FW counter implementation
as follows:
1) Remove SBI_PMU_FW_EVENT_MAX
2) Remove "struct sbi_pmu_fw_event"
3) Create per-HART bitmap of XLEN bits to track FW counters
which are started on each HART
4) Create per-HART uint64_t array to track values of FW
counters on each HART.
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Reviewed-by: Atish Patra <atishp@rivosinc.com>
The "read a firmware counter" SBI call should only work for firmware
counters so let us replace sbi_pmu_ctr_read() with sbi_pmu_ctr_fw_read()
which works only on firmware counters.
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Reviewed-by: Atish Patra <atishp@rivosinc.com>
sideleg and sedeleg csrs are not part of riscv isa spec
anymore, these csrs were part of N extension which
is removed from the riscv isa specification.
These commits removed all traces of these csrs from
riscv spec (https://github.com/riscv/riscv-isa-manual) -
commit f8d27f805b65 ("Remove or downgrade more references to N extension (#674)")
commit b6cade07034d ("Remove N extension chapter for now")
Signed-off-by: Rahul Pathak <rpathak@ventanamicro.com>
Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
The machine mode GVA field is available if the hypervisor extension is
implemented, and indicates if mtval is a guest virtual address. Add a
gva field to sbi_trap_info for this, and in __sbi_expected_trap_hext,
save mstatus[h].GVA to it, so that gva indicates if tval is a guest
virtual address. If the hypervisor extension is not implemented, always
set gva to 0.
Signed-off-by: Vivian Wang <dramforever@live.com>
Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
The machine mode GVA field is in mstatus for RV64 and mstatush for RV32,
and is available if the hypervisor extension is available. If an
exception occurs, we may need to redirect the trap to HS-mode, in which
case, hstatus.GVA should be set to same as the machine mode GVA bit.
Add MSTATUS_GVA for RV64, MSTATUSH_GVA for RV32, and their SHIFT
encodings. The SHIFT index is helpful in assembly code, since field
extraction can be implemented in only one register. In pseudocode:
- For RV32: gva = (mstatus >> MSTATUS_GVA_SHIFT) & 1;
- For RV64: gva = (mstatush >> MSTATUSH_GVA_SHIFT) & 1;
Signed-off-by: Vivian Wang <dramforever@live.com>
Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Currently, there is no sanity check for firmware event code. We don't see
any issue as Linux kernel driver does a bound check on firmware events
already. However, OpenSBI can't assume sane supervisor mode software
always. Thus, an invalid event idx can cause a buffer overflow error.
For hardware events, the match will fail for invalid event code anyways.
However, a search is unecessary if event code is invalid.
Add a event ID validation function to solve the issue.
Signed-off-by: Atish Patra <atishp@rivosinc.com>
Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
If the ecall SBI_EXT_HSM_HART_START is called it might try to wake the
secondary hart using sbi_ipi_raw_send() to send an IPI to the hart.
This can fail if there is no IPI device but no error is returned from
sbi_ipi_raw_send() so the ecall returns as if the action completed and
the caller continues without noticing (in the case of Linux it just hangs
waiting for the secondary hart to become active)
Fix this by changing sbi_ipi_raw_send() to return and error, and if an
error is returned, then return it via SBI_EXT_HSM_HART_START call.
Signed-off-by: Ben Dooks <ben.dooks@sifive.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Motivation: Suppose a peripheral needs to be configured to transmit
data. There is an SFR bit which indicates that the peripheral is ready
to transmit. The firmware should check the bit and will only transmit
data when the peripheral is ready. When the firmware starts polling the
SFR, the peripheral could be busy transmitting/receiving other data so
the firmware must wait till that completes. Assuming that there is no
other way, the firmware shouldn't wait indefinitely.
The function sbi_timer_waitms_until() will constantly check whether a
certain condition is satisfied, or timeout occurs. It can be used for
the cases when a timeout is required.
Signed-off-by: Adnan Rahman Chowdhury <adnan.chowdhury@sifive.com>
Reviewed-by: Xiang W <wxjstz@126.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Now that SBI v1.0 specification is ratified, we change spec verion
implemented by OpenSBI to v1.0.
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
Reviewed-by: Atish Patra <atishp@rivosinc.com>
Based on architecture review committee feedback, the [m|s|vs]seteienum,
[m|s|vs]clreienum, [m|s|vs]seteipnum, and [m|s|vs]clreipnum CSRs are
removed in the latest AIA draft v0.3.0 specification.
(Refer, https://github.com/riscv/riscv-aia/releases/tag/0.3.0-draft.31)
These CSRs were mostly for software convenience and software can always
use [m|s|vs]iselect and [m|s|vs]ireg CSRs to update the IMSIC interrupt
file bits.
We update the IMSIC programming as-per above to match the latest AIA
draft specification.
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
Reviewed-by: Atish Patra <atishp@rivosinc.com>
Add psuedoinstruction encodings written to mtinst/htinst for faults
caused by implicit memory access for VS-stage address translation
Signed-off-by: dramforever <dramforever@live.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
The suspend code needs to know the resume address for two reasons:
1) Programming some hardware register or management firmware. Here we
assume the hardware/firmware maintains its state between suspends,
so it only needs to be programmed once at startup.
2) When a non-retentive suspend request ends up being retentive, due
to lack of hardware support, pending interrupt, or for some other
reason. However, the behavior here is not platform-dependent, and
this can be handled in the generic hart suspend function.
Since neither situation requires the platform-level suspend function to
know the resume address, stop passing it to that function. Instead,
handle the non-retentive to retentive situation generically.
Reviewed-by: Anup Patel <anup@brainfault.org>
Signed-off-by: Samuel Holland <samuel@sholland.org>
Non-retentive suspend states may require platform-specific actions
during resume. For example, firmware may need to save and restore the
values of custom CSRs. Add a hook to support this.
Reviewed-by: Anup Patel <anup@brainfault.org>
Signed-off-by: Samuel Holland <samuel@sholland.org>
We add platform specific extensions_init() callback which allows
platforms to populate HART extensions for each HART. For example,
the generic platform can populate HART extensions from HART ISA
string described in DeviceTree.
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
Reviewed-by: Atish Patra <atishp@rivosinc.com>
We add sbi_hart_update_extension() function which allow platforms
to enable/disable hart extensions.
Signed-off-by: Anup Patel <anup@brainfault.org>
Reviewed-by: Atish Patra <atishp@rivosinc.com>
Since past few years, we have been using "hart features" in OpenSBI
to represent all optionalities and multi-letter extensions defined
by the RISC-V specifications.
The RISC-V profiles specification has taken a different approach and
started assigning extension names for all optionalities which did not
have any extension name previously.
(Refer, https://github.com/riscv/riscv-profiles/blob/main/profiles.adoc)
Inspired from the RISC-V profiles specification, we convert OpenSBI
hart features into hart extensions. Going forward, we align the
extension naming with RISC-V profiles specification. Currently, only
"time CSR" and "AIA CSR" have not been assigned extension name but
for everything else we have a name.
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
Reviewed-by: Atish Patra <atishp@rivosinc.com>
If a hart implements privileged spec v1.12 (or higher) then we can
safely assume that menvcfg CSR is present and we don't need MENVCFG
as a hart feature.
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
Reviewed-by: Atish Patra <atishp@rivosinc.com>
If a hart implements privileged spec v1.11 (or higher) then we can
safely assume that mcountinhibit CSR is present and we don't need
MCOUNTINHIBT as a hart feature.
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
Reviewed-by: Atish Patra <atishp@rivosinc.com>
If a hart implements privileged spec v1.10 (or higher) then we can
safely assume that [m|s]counteren CSR are present and we don't need
MCOUNTEREN and SCOUNTEREN as hart features.
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
Reviewed-by: Atish Patra <atishp@rivosinc.com>
It is possible to guess privileged spec versions based on the CSRs
that where introduced in different privileged spec versions. In
future, if we are not able guess privileged spec version then we
can have platform provide it.
We add privileged spec version as per-hart feature and try to guess
it based on presence of mcounteren, mcountinhibit, and menvcfg CSRs.
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
Reviewed-by: Atish Patra <atishp@rivosinc.com>
Recently, Sstc extension was ratified. It defines stimecmp which allows
the supervisor mode to directly update the timecmp value without the
need of the SBI call. The hardware also can inject the S-mode timer
interrupt direclty to the supervisor without going through the M-mode.
To maintain backward compatibility with the older software, SBI call
now uses stimecmp directly if the hardware supports.
Implement the Sstc extension.
Signed-off-by: Atish Patra <atishp@rivosinc.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Accordind to the RISC-V privileged specification, the VS filed is
mstatus[10:9] instead of mstatus[24:23]. Modify the MSTATUS_VS
to the correct value.
Reported-by: I-Cheng Cheng <i-cheng.cheng@sifive.com>
Signed-off-by: Vincent Chen <vincent.chen@sifive.com>
Reviewed-by: Xiang W <wxjstz@126.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Extend HART feature detection to discover Smstateen CSRs at boot-time
and configure mstateen envcfg bit depending on availability of
menvcfg CSR.
Signed-off-by: Mayuresh Chitale <mchitale@ventanamicro.com>
Reviewed-by: Xiang W <wxjstz@126.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Smstateen extension provides a mechanism to plug potential
covert channels which are opened by extensions that add to
processor state that may not get context-switched.
Signed-off-by: Mayuresh Chitale <mchitale@ventanamicro.com>
Reviewed-by: Xiang W <wxjstz@126.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
We add the menvcfg CSR as a HART feature and detect it at boot time
using traping mechanism.
Signed-off-by: Atish Patra <atishp@rivosinc.com>
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
Reviewed-by: Xiang W <wxjstz@126.com>
The latest RISC-V privileged specification introduces xenvcfg CSRs
to enable/disable certain features/extensions for lower privilege
modes. This patch adds defines for these new [m|h|s]envcfg CSRs.
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
Reviewed-by: Atish Patra <atishp@rivosinc.com>
Reviewed-by: Xiang W <wxjstz@126.com>
Add static detection to prevent the modification of struct fw_dynamic_info
from forgetting the modification of FW_DYNAMIC_INFO_xxx_OFFSET
Signed-off-by: Xiang W <wxjstz@126.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Add static detection to prevent the modification of struct sbi_scratch
from forgetting the modification of SBI_SCRATCH_xxx_OFFSET
Signed-off-by: Xiang W <wxjstz@126.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Add static detection to prevent the modification of struct sbi_platform
from forgetting the modification of SBI_PLATFORM_xxx_OFFSET
Signed-off-by: Xiang W <wxjstz@126.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Currently, the external interrupt handling is scattered between
sbi_init and sbi_trap. This patch moves all external interrupt
handling into a simple framework called sbi_irqchip.
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
Reviewed-by: Xiang W <wxjstz@126.com>
Reviewed-by: Atish Patra <atishp@rivosinc.com>
We introduce nascent_init() platform callback which will allow
platforms to do very early initialization of platform specific
per-HART CSRs and per-HART devices.
Signed-off-by: Anup Patel <anup.patel@wdc.com>
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
Reviewed-by: Atish Patra <atishp@rivosinc.com>
This patch adds sbi_trap_set_external_irqfn() API which can be used by
OpenSBI platform code to set a callback function for external interrupts.
The RISC-V AIA IMSIC driver will use this API to implement inter-processor
interrupts on-top-of MSIs.
Signed-off-by: Anup Patel <anup.patel@wdc.com>
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
Reviewed-by: Atish Patra <atishp@rivosinc.com>
The RISC-V AIA specification improves handling of per-HART local
interrupts in a backward compatible manner. This patch adds defines
for the new RISC-V AIA CSRs.
Signed-off-by: Anup Patel <anup.patel@wdc.com>
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
Reviewed-by: Atish Patra <atishp@rivosinc.com>
In a systems that provide strings.h and it is included
together with sbi_bitops.h the compilation error appears.
The ffs() and fls() are provided by strings.h
Signed-off-by: Petro Karashchenko <petro.karashchenko@gmail.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
PMU overflow interrupt should be disabled durinig initial configuration of
counters. They should be enabled while starting counters.
Fixes: 730f01bb41 ("lib: sbi: Support sscofpmf extension in OpenSBI")
Signed-off-by: Atish Patra <atishp@rivosinc.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Using "void *" in arithmetic causes errors with strict compiler settings:
"error: pointer of type 'void *' used in arithmetic [-Werror=pointer-arith]"
Avoid these by calculating on "char *" where 1-byte data size is assumed.
Signed-off-by: Jukka Laitinen <jukkax@ssrc.tii.ae>
Reviewed-by: Dong Du <Dd_nirvana@sjtu.edu.cn>
Reviewed-by: Xiang W <wxjstz@126.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
RISC-V privilege specification does not specify how to encode the event ID.
Therefore, each platform is allowed to customize its own encoding rule.
The common encoding methods are as follow, directly assigning a number to an
event, or every bit in the mphmevent CSR controls one specified event or
mixes the above two methods.
To enable OpenSBI to support the above three encoding methods simultaneously,
this patch repurpose the dt property "riscv,raw-event-to-mhpmcounters". The
"riscv,raw-event-to-mhpmcounters" will describes the one or multiple raw
events that could be counted by a set of counters. But, the column number
of "riscv,raw-event-to-mhpmcounters" is extended from 2 to 3. The 1st column
(64bit) is the ID of the raw events. The 2nd column (64bit) represents a
select_mask now to represent the bits used for event ID encoding.
If a platform directly encodes each raw PMU event as a unique ID,
the value of select_mask will be 0xffffffff_ffffffff.
Signed-off-by: Vincent Chen <vincent.chen@sifive.com>
Signed-off-by: Atish Patra<atishp@rivosinc.com>
Reviewed-by: Anup Patel <anup.patel@wdc.com>
BUG and BUG_ON are not informative and are rather lazy interfaces, only
telling the user that something went wrong in a given function, but not
what, requiring the user to find the sources corresponding to their
firmware (which may not be available) and figure out how that BUG(_ON)
was hit. Even SBI_ASSERT in its current form, which does include the
condition that triggered it in the output, isn't necessarily very
informative. In some cases, the error may be fixable by the user, but
they need to know the problem in order to have any hope of fixing it.
It's also a nuisance for developers, whose development trees may have
changed significantly since the release in question being used, and so
line numbers can make it harder for them to understand which error case
a user has hit.
This patch introduces a new sbi_panic function which is printf-like,
allowing detailed error messages to be printed to the console. BUG and
BUG_ON are removed, since the former is just a worse form of sbi_panic
and the latter is a worse version of SBI_ASSERT. Finally, SBI_ASSERT is
augmented to take a set of arguments to pass to sbi_panic on failure,
used like so (sbi_boot_print_hart's current error case, which currently
manually calls sbi_printf and sbi_hart_hang):
SBI_ASSERT(xlen >= 1, ("Error %d getting MISA XLEN\n", xlen));
The existing users of BUG are replaced with calls to sbi_panic along
with informative error messages. BUG_ON and SBI_ASSERT were unused (and,
in the case of SBI_ASSERT, remain unused).
Many existing users of sbi_hart_hang should be converted to use either
sbi_panic or SBI_ASSERT after this commit.
Signed-off-by: Jessica Clarke <jrtc27@jrtc27.com>
Reviewed-by: Anup Patel <anup.patel@wdc.com>
Reviewed-by: Xiang W <wxjstz@126.com>
Reviewed-by: Atish Patra <atishp@rivosinc.com>
A platform may use programmable counters for cycle/instret events.
The priv spec allows that provided that cycle/instret also report those
events in addition to the programmable counters. We should allow that
functionality in OpenSBI.
Reviewed-by: Anup Patel <anup.patel@wdc.com>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Signed-off-by: Atish Patra <atish.patra@wdc.com>
This patch adds sscofpmf extension in pmu module which includes
following things.
1. Enable overflow irq when starting a counter.
2. Setting the correct event filters passed from supervisor.
3. Delegating the overflow interrupt to the supervisor.
4. Add RV32 support for sscofpmf.
Reviewed-by: Anup Patel <anup.patel@wdc.com>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Signed-off-by: Atish Patra <atish.patra@wdc.com>
Sscofpmf ISA extension introduces PMU counter overflow and filtering support.
It introduces a read only `scountovf` csr that can be used to detect if
a hart supports this extension at runtime. However, this feature is only
useful if the hart already supports mcounteren and mcountinhibit.
Add a dynamic detection mechanism and boot time print message if sscofpmf
is present.
Reviewed-by: Anup Patel <anup.patel@wdc.com>
Reviewed-by: Bin Meng <bmeng.cn@gmail.com>
Signed-off-by: Atish Patra <atish.patra@wdc.com>
