As per section 3.7.2 of RISC-V Privileged Specification,
PMP settings must be synchronized with the virtual memory
system after PMP settings have been written.
Signed-off-by: Himanshu Chauhan <hchauhan@ventanamicro.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
For 'reg->order == __riscv_xlen' the term 'BIT(reg->order)' is undefined.
Addresses-Coverity-ID: 1529706 ("Bad bit shift operation")
Signed-off-by: Heinrich Schuchardt <heinrich.schuchardt@canonical.com>
Reviewed-by: Xiang W <wxjstz@126.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
When order is equal to __riscv_xlen, the shift operation will not perform
any operation, which will cause reg->base & (BIT(reg->order) - 1) to always
be 0, and the condition has not been established.
This patch fixes this bug.
Signed-off-by: Xiang W <wxjstz@126.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Split up sbi_ecall_replace so that each extension is in its individual
file.
Also reorganize the corresponding section in lib/sbi/objects.mk so
that it is grouped by extension, now that the object file targets are
split up.
Signed-off-by: Vivian Wang <dramforever@live.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
For each SBI extension, we:
- Add a Kconfig option for it
- Add the extension to sbi_ecall_exts only if the extension is enabled
- Add the corresponding sbi_ecall_* object file only if the extension is
enabled
Special cases are as follows:
- The legacy extensions are lumped together as one 'big' extension, as
has always been the case in OpenSBI code.
- The platform-defined vendor extensions are regarded as one extension.
- The Base extension cannot be disabled.
- sbi_ecall_replace implements multiple extensions, so it's not easy to
avoid linking it in. Enable it always, and use #ifdef to
disable/enable individual extensions.
Signed-off-by: Vivian Wang <dramforever@live.com>
Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Instead of hard-coding the list of extensions in C code, use carray to
generate the list of extensions.
Using carray makes adding and removing extensions slightly cleaner. This
also paves the way for using Kconfig to disable unneeded extensions.
Signed-off-by: Vivian Wang <dramforever@live.com>
Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
This patch generalizes the logic to add a memory range with desired
alignment and flags of consecutive regions to the root domain.
Signed-off-by: Yu Chien Peter Lin <peterlin@andestech.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
The platform-specific extension_init callback is supposed to
set specific things for the platform opensbi is running on.
So it's also the right place to override specific hart_features
if needed - when it's know that autodetection has provided
wrong results for example.
Suggested-by: Atish Patra <atishp@atishpatra.org>
Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Reviewed-by: Guo Ren <guoren@kernel.org>
Reviewed-by: Anup Patel <anup@brainfault.org>
Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Platforms may need to override auto-detected hart features
in their override functions. So move the hart_features
struct to the sbi_hart.h header allowing us to pass it over
to platform-handlers.
Suggested-by: Atish Patra <atishp@atishpatra.org>
Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Reviewed-by: Guo Ren <guoren@kernel.org>
Reviewed-by: Anup Patel <anup@brainfault.org>
Signed-off-by: Heiko Stuebner <heiko@sntech.de>
Don't spread checking for pmu extensions through the code
but instead introduce a sbi-pmu function that other code can
call to get the correct information about the existence of the
pmu interrupt.
Add a sbi_pmu_device override function to allow overridung this
bit as well if needed.
Reviewed-by: Atish Patra <atishp@rivosinc.com>
Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Reviewed-by: Guo Ren <guoren@kernel.org>
Reviewed-by: Anup Patel <anup@brainfault.org>
Signed-off-by: Heiko Stuebner <heiko@sntech.de>
If a particular misaligned load or store cannot be emulated at all, for
the redirected trap, trap.gva is set to 0, but it should be the same as
mstatus[h].GVA of the original trap. Fix this so that if the trap is
destined for HS-mode, hstatus.GVA is then set correctly.
Fixes: 1c4ce74f51 ("lib: sbi: Set gva when creating sbi_trap_info")
Signed-off-by: Vivian Wang <dramforever@live.com>
Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
The GVA bit is in mstatus on RV64, and in mstatush in RV32. Refactor
code handling this in sbi_trap_handler into a helper function to extract
GVA from sbi_trap_regs, so that future code accessing GVA can be
XLEN-agnostic.
Signed-off-by: Vivian Wang <dramforever@live.com>
Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
The sbi_pmu.h should only include minimal required headers whereas
sbi_pmu.c should include all required headers.
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Reviewed-by: Atish Patra <atishp@rivosinc.com>
Let us print the platform PMU device name at the boot-time so that users
know whether the underlying platform has custom per-HART PMU operations.
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Reviewed-by: Atish Patra <atishp@rivosinc.com>
We extend SBI PMU implementation to allow custom PMU device operations
which a platform can use for platform specific quirks.
The custom PMU device operations added by this patch include:
1) Operations to allow a platform implement custom firmware events.
These custom firmware events can be SBI vendor extension related
events or platform specific per-HART events are not possible to
count through HPM CSRs.
2) Operations to allow a platform implement custom way for enabling
(or disabling) an overflow interrupt (e.g. T-Head C9xx).
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Reviewed-by: Atish Patra <atishp@rivosinc.com>
Currently, we have 32 elements (i.e. SBI_PMU_FW_EVENT_MAX) array of
"struct sbi_pmu_fw_event" for each of 128 possible HARTs
(i.e. SBI_HARTMASK_MAX_BITS).
To reduce memory usage of OpenSBI, we update FW counter implementation
as follows:
1) Remove SBI_PMU_FW_EVENT_MAX
2) Remove "struct sbi_pmu_fw_event"
3) Create per-HART bitmap of XLEN bits to track FW counters
which are started on each HART
4) Create per-HART uint64_t array to track values of FW
counters on each HART.
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Reviewed-by: Atish Patra <atishp@rivosinc.com>
As-per SBI specification, all firmware counters are always 64 bits
wide so let us update the SBI PMU implementation to reflect this fact.
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Reviewed-by: Atish Patra <atishp@rivosinc.com>
The "read a firmware counter" SBI call should only work for firmware
counters so let us replace sbi_pmu_ctr_read() with sbi_pmu_ctr_fw_read()
which works only on firmware counters.
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Reviewed-by: Atish Patra <atishp@rivosinc.com>
In case of missing "FENCE.TSO" instruction implementation,
opensbi can emulate the "FENCE.TSO" with "FENCE RW,RW", but
mepc was not incremented to continue from the next instruction
causing infinite trap.
Fixes: cb8271c8 ("lib: sbi_illegal_insn: Add emulation for fence.tso")
Signed-off-by: Rahul Pathak <rpathak@ventanamicro.com>
Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Reviewed-by: Xiang W <wxjstz@126.com>
Reviewed-by: Samuel Holland <samuel@sholland.org>
The privileged spec specifies that on a trap to HS-mode, hstatus.GVA
should be set to 1 if stval is written with a guest virtual address, and
to 0 otherwise. Implement this by setting hstatus.GVA to trap->gva when
redirecting traps to HS-mode.
Signed-off-by: Vivian Wang <dramforever@live.com>
Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
The hypervisor CSRs hstatus, htval, htinst should always be set if the
trap is to be taken in HS-mode, regardless of which mode it came from.
Signed-off-by: Vivian Wang <dramforever@live.com>
Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
In some cases the sbi_trap_info argument passed to sbi_trap_redirect is
created from scratch by filling its fields. Since we previously added a
gva field to struct sbi_trap_info, initialize gva in these cases also.
Suggested-by: Andrew Jones <ajones@ventanamicro.com>
Signed-off-by: Vivian Wang <dramforever@live.com>
Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
The machine mode GVA field is available if the hypervisor extension is
implemented, and indicates if mtval is a guest virtual address. Add a
gva field to sbi_trap_info for this, and in __sbi_expected_trap_hext,
save mstatus[h].GVA to it, so that gva indicates if tval is a guest
virtual address. If the hypervisor extension is not implemented, always
set gva to 0.
Signed-off-by: Vivian Wang <dramforever@live.com>
Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Read long long arguments directly using va_arg. Remove original hack for
RV32 that read a long long arg as two long args.
This un-breaks the case on RV64 where e.g. the long long is followed by
an odd number of ints:
sbi_printf("%d %lld", (int) 1, (long long) 2LL);
Also remove the acnt variable, which is now unused.
Signed-off-by: dramforever <dramforever@live.com>
Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
fw_event_map represents array of firmware events. It should initialized
for maximum number of firmware events not counters.
Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Signed-off-by: Atish Patra <atishp@rivosinc.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
The active_events array is accessed with counter ID passed from the supervisor
software before the counter ID bound check. This may cause a buffer overrun
if a supervisor passes an invalid counter ID.
Fix this by moving the access part after the bound check.
Reported-by: Andrew Jones <ajones@ventanamicro.com>
Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Signed-off-by: Atish Patra <atishp@rivosinc.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Currently, there is no sanity check for firmware event code. We don't see
any issue as Linux kernel driver does a bound check on firmware events
already. However, OpenSBI can't assume sane supervisor mode software
always. Thus, an invalid event idx can cause a buffer overflow error.
For hardware events, the match will fail for invalid event code anyways.
However, a search is unecessary if event code is invalid.
Add a event ID validation function to solve the issue.
Signed-off-by: Atish Patra <atishp@rivosinc.com>
Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
The current implementation computes the possible counter range
by doing a left shift of counter base. However, this may overflow depending
on the counter base value. In case of overflow, the highest counter id
may be computed incorrectly. As per the SBI specification, the respective
function should return an error if any of the counter is not valid.
Fix the counter index check by avoiding left shifting while doing the
sanity checks. Without the shift, the implementation just iterates
over the counter mask and computes the correct counter index by adding
the base to it.
Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Signed-off-by: Atish Patra <atishp@rivosinc.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
printc would happily write to 'out' even when 'out_len' was zero,
potentially overflowing buffers. Rework printc to not do that and
also ensure the null byte is written at the last position when
necessary, as stated in the snprintf man page. Also, panic if
sprintf or snprintf are called with NULL output strings (except
the special case of snprintf having a NULL output string and
a zero output size, allowing it to be used to get the number of
characters that would have been written). Finally, rename a
goto label which clashed with 'out'.
Fixes: 9e8ff05cb6 ("Initial commit.")
Signed-off-by: Andrew Jones <ajones@ventanamicro.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
MPV bit is set when the value of next_virt boolean variable equals
true. Since the value of next_virt is either 0 or 1, we can set
MPV bit without if-else logic.
Signed-off-by: Che-Chia Chang <alvinga@andestech.com>
Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
If the ecall SBI_EXT_HSM_HART_START is called it might try to wake the
secondary hart using sbi_ipi_raw_send() to send an IPI to the hart.
This can fail if there is no IPI device but no error is returned from
sbi_ipi_raw_send() so the ecall returns as if the action completed and
the caller continues without noticing (in the case of Linux it just hangs
waiting for the secondary hart to become active)
Fix this by changing sbi_ipi_raw_send() to return and error, and if an
error is returned, then return it via SBI_EXT_HSM_HART_START call.
Signed-off-by: Ben Dooks <ben.dooks@sifive.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Motivation: Suppose a peripheral needs to be configured to transmit
data. There is an SFR bit which indicates that the peripheral is ready
to transmit. The firmware should check the bit and will only transmit
data when the peripheral is ready. When the firmware starts polling the
SFR, the peripheral could be busy transmitting/receiving other data so
the firmware must wait till that completes. Assuming that there is no
other way, the firmware shouldn't wait indefinitely.
The function sbi_timer_waitms_until() will constantly check whether a
certain condition is satisfied, or timeout occurs. It can be used for
the cases when a timeout is required.
Signed-off-by: Adnan Rahman Chowdhury <adnan.chowdhury@sifive.com>
Reviewed-by: Xiang W <wxjstz@126.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
OpenSBI explicitly assumes that there is no pmu hardware counter with
index 1: hardware uses that bit for TM control. So OpenSBI filters
out that index in sanity checks. However OpenSBI also excludes that
counter when reports total amount of hardware counters to Linux. As
a result, Linux uses incomplete counters mask excluding the highest
available counter.
Return accurate number of counters, update the firmware counter
starting index, fix range checks that include num_hw_ctrs.
The simple test is to make sure that there is no counter multiplexing
in the following command:
$ perf stat -e \
r8000000000000000,r8000000000000001,r8000000000000002,r8000000000000003, \
r8000000000000004,r8000000000000005,r8000000000000006,r8000000000000007, \
r8000000000000008,r8000000000000009,r800000000000000a,r800000000000000b, \
r800000000000000c,r800000000000000d,r800000000000000e,r800000000000000f \
ls
Note that 16 firmware events with 16 counters won't require multiplexing.
Signed-off-by: Sergey Matyukevich <geomatsi@gmail.com>
Signed-off-by: Atish Patra <atishp@rivosinc.com>
Reviewed-by: Atish Patra <atishp@rivosinc.com>
We should not change trap->tval to mepc because mtval already points to
the faulting portion of the emulated instruction fetch, which is also
what stval is expected to be.
In addition, htinst is only allowed to be zero for instruction access
faults or page faults, and is only allowed to be zero or a
psuedoinstruction for instruction guest-page faults. Fix trap->tinst for
these cases.
Signed-off-by: dramforever <dramforever@live.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
If there is an exception while emulating a misaligned load/store, fixup
uptrap.tinst before redirecting. Otherwise, HS-mode software may receive
an htinst describing the lbu/sb instruction that faulted during
emulation[1].
[1]: https://github.com/riscv-software-src/opensbi/issues/258
Signed-off-by: dramforever <dramforever@live.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
The suspend code needs to know the resume address for two reasons:
1) Programming some hardware register or management firmware. Here we
assume the hardware/firmware maintains its state between suspends,
so it only needs to be programmed once at startup.
2) When a non-retentive suspend request ends up being retentive, due
to lack of hardware support, pending interrupt, or for some other
reason. However, the behavior here is not platform-dependent, and
this can be handled in the generic hart suspend function.
Since neither situation requires the platform-level suspend function to
know the resume address, stop passing it to that function. Instead,
handle the non-retentive to retentive situation generically.
Reviewed-by: Anup Patel <anup@brainfault.org>
Signed-off-by: Samuel Holland <samuel@sholland.org>
Non-retentive suspend states may require platform-specific actions
during resume. For example, firmware may need to save and restore the
values of custom CSRs. Add a hook to support this.
Reviewed-by: Anup Patel <anup@brainfault.org>
Signed-off-by: Samuel Holland <samuel@sholland.org>
HS-mode software can choose what exceptions to delegate to VS-mode using
the hedeleg CSR. Synthetic VS/VU-mode exceptions should also honor
hedeleg. They should be redirected to VS-mode if and only if delegated
by HS-mode.
Signed-off-by: dramforever <dramforever@live.com>
Reviewed-by: Xiang W <wxjstz@126.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
The device's timer_value callback is already the right prototype to use
for the get_time_val function pointer.
Signed-off-by: Samuel Holland <samuel@sholland.org>
Reviewed-by: Anup Patel <anup@brainfault.org>
If the csr's operation comes from M mode, it should not be forwarded
to low-privilege processing, this patch fixes this problem.
Signed-off-by: Xiang W <wxjstz@126.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
We add platform specific extensions_init() callback which allows
platforms to populate HART extensions for each HART. For example,
the generic platform can populate HART extensions from HART ISA
string described in DeviceTree.
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
Reviewed-by: Atish Patra <atishp@rivosinc.com>
We add sbi_hart_update_extension() function which allow platforms
to enable/disable hart extensions.
Signed-off-by: Anup Patel <anup@brainfault.org>
Reviewed-by: Atish Patra <atishp@rivosinc.com>
Currently, the hart_detect_features() is called everytime a hart
is stopped and started again which is unnecessary work.
We update hart_detect_features() to detect hart features only
once for each hart.
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
Reviewed-by: Atish Patra <atishp@rivosinc.com>
