Rather than hand-rolling scaled pointer arithmetic with casts and
shifts, let the compiler do so by indexing an array of GPRs, taking
advantage of the language's type system to scale based on whatever type
the register happens to be. This makes it easier to support CHERI where
the registers are capabilities, not plain integers, and so this pointer
arithmetic would need to change (and currently REGBYTES is both the size
of a register and the size of an integer word upstream).
Signed-off-by: Jessica Clarke <jrtc27@jrtc27.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20250709232932.37622-1-jrtc27@jrtc27.com
Signed-off-by: Anup Patel <anup@brainfault.org>
Align SBI_TRAP_CONTEXT_SIZE to a multiple of 16 bytes. If it is not
aligned to 16 bytes for RV64, it can create performance problems.
Aligning it correctly can fix the performance issues.
Signed-off-by: Raj Vishwanathan <Raj.Vishwanathan@gmail.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Previous privilege mode retrieval from mstatus is done at different
places, factorize it rather than copy/pasting it again.
Signed-off-by: Clément Léger <cleger@rivosinc.com>
Reviewed-by: Samuel Holland <samuel.holland@sifive.com>
The same pattern is used at multiple places to verify in which mode
the exception was actually taken. Factorize it.
Signed-off-by: Clément Léger <cleger@rivosinc.com>
Reviewed-by: Samuel Holland <samuel.holland@sifive.com>
Club the struct sbi_trap_regs and struct sbi_trap_info a new
struct sbi_trap_context (aka trap context) which must be saved
by low-level trap handler before calling sbi_trap_handler().
To track nested traps, the struct sbi_scratch points to the current
trap context and the trap context has pointer to pervious context
of previous trap.
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
Reviewed-by: Samuel Holland <samuel.holland@sifive.com>
Tested-by: Samuel Holland <samuel.holland@sifive.com>
Over the years, no uses of sbi_trap_exit() have been found so remove
it and also remove related code from fw_base.S and sbi_scratch.h.
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
Reviewed-by: Samuel Holland <samuel.holland@sifive.com>
Tested-by: Samuel Holland <samuel.holland@sifive.com>
In the only places this value is used, it duplicates mepc from
struct sbi_trap_regs.
Signed-off-by: Samuel Holland <samuel.holland@sifive.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
sbi_trap_redirect now uses const pointer to `trap`.
This ensures the caller that we never change `trap` in sbi_trap_redirect.
Signed-off-by: Bo Gan <ganboing@gmail.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
The GVA bit is in mstatus on RV64, and in mstatush in RV32. Refactor
code handling this in sbi_trap_handler into a helper function to extract
GVA from sbi_trap_regs, so that future code accessing GVA can be
XLEN-agnostic.
Signed-off-by: Vivian Wang <dramforever@live.com>
Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
The machine mode GVA field is available if the hypervisor extension is
implemented, and indicates if mtval is a guest virtual address. Add a
gva field to sbi_trap_info for this, and in __sbi_expected_trap_hext,
save mstatus[h].GVA to it, so that gva indicates if tval is a guest
virtual address. If the hypervisor extension is not implemented, always
set gva to 0.
Signed-off-by: Vivian Wang <dramforever@live.com>
Reviewed-by: Andrew Jones <ajones@ventanamicro.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Currently, the external interrupt handling is scattered between
sbi_init and sbi_trap. This patch moves all external interrupt
handling into a simple framework called sbi_irqchip.
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
Reviewed-by: Xiang W <wxjstz@126.com>
Reviewed-by: Atish Patra <atishp@rivosinc.com>
This patch adds sbi_trap_set_external_irqfn() API which can be used by
OpenSBI platform code to set a callback function for external interrupts.
The RISC-V AIA IMSIC driver will use this API to implement inter-processor
interrupts on-top-of MSIs.
Signed-off-by: Anup Patel <anup.patel@wdc.com>
Signed-off-by: Anup Patel <apatel@ventanamicro.com>
Reviewed-by: Atish Patra <atishp@rivosinc.com>
There are two copies of the same abnormal exit code, this patch deletes one
Signed-off-by: Xiang W <wxjstz@126.com>
Reviewed-by: Anup Patel <anup.patel@wdc.com>
GCC has already a predefined macro __ASSEMBLER__ therefore, it can be
used without the need to define a new flag with -D__ASSEMBLY__.
This is useful when adding the library to projects having a build
system such one can build without the need to make changes.
THe build system does not use the Makefile in the sources tree.
Signed-off-by: Marouene Boubakri <marouene.boubakri@nxp.com>
Signed-off-by: Anup Patel <anup.patel@wdc.com>
All members in struct sbi_trap_regs are of machine word size so
no need to pack this structure. This also allows GCC to generate
more optimized instruction sequence when accessing members of
struct sbi_trap_regs.
Reported-by: Paul Campbell <taniwha@gmail.com>
Signed-off-by: Anup Patel <anup.patel@wdc.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
Reviewed-by: Atish Patra <atish.patra@wdc.com>
We introduce sbi_trap_exit() API which can help non-firmware
(i.e. generic or platform) code to force exit trap/interrupt
handling and resume execution at context pointed by parameter
"const struct sbi_trap_regs *regs".
This new sbi_trap_exit() API will help Keystone Enclave project
to resume execution of enclave from custom SBI call handler.
Signed-off-by: Anup Patel <anup.patel@wdc.com>
Reviewed-by: Atish Patra <atish.patra@wdc.com>
This patch simplify sbi_trap_handler() API as follows:
1. Remove current hartid local variable because sbi_trap_handler()
itself does not need it.
2. Remove scratch parameter because none of the functions directly
called by sbi_trap_handler() require it.
Signed-off-by: Anup Patel <anup.patel@wdc.com>
Reviewed-by: Atish Patra <atish.patra@wdc.com>
The scratch parameter of sbi_trap_redirect() is not used hence we
remove it.
Signed-off-by: Anup Patel <anup.patel@wdc.com>
Reviewed-by: Atish Patra <atish.patra@wdc.com>
This patch optimize unpriv load/store implementation by having
dedicated unpriv trap handler (just like KVM RISC-V).
As a result of this optimization:
1. We have reduced roughly 13+ instruction in all unpriv load/store
functions. The reduced instruction also include two function calls.
2. Per-HART trap info pointer in scratch space is now redundant
hence removed.
3. The sbi_trap_handler() is now much cleaner because we don't have
to handle unpriv load/store traps.
Signed-off-by: Anup Patel <anup.patel@wdc.com>
Reviewed-by: Atish Patra <atish.patra@wdc.com>
We have two new trap CSRs namely mtval2 and mtinst when
RISC-V hypervisor extension is available hence we extend
struct sbi_trap_info accordingly.
Signed-off-by: Anup Patel <anup.patel@wdc.com>
Reviewed-by: Alistair Francis <alistair.francis@wdc.com>
The unpriv APIs can be useful to external firmware and out-of-tree
platform support code.
This patch adds "sbi_" prefix to unpriv load/store APIs and rename
struct riscv_unpriv to struct sbi_trap_info everywhere. We also
place struct sbi_trap_info in sbi/sbi_trap.h so that we can use
it for sbi_trap_redirect() as well.
Overall, this patch will make naming of unpriv APIs consistent
with other OpenSBI APIs.
Signed-off-by: Anup Patel <anup.patel@wdc.com>
Reviewed-by: Atish Patra <atish.patra@wdc.com>
When hypervisor extension is available, we can get traps from VS/VU
modes. We should be able to force redirect some of these traps to
HS-mode. In other words, we should be able forward traps from VS/VU
mode to HS-mode using sbi_trap_redirect() hence this patch.
Signed-off-by: Atish Patra <atish.patra@wdc.com>
Signed-off-by: Anup Patel <anup.patel@wdc.com>
Signed-off-by: Alistair Francis <alistair.francis@wdc.com>
Noisy commit, no functional changes.
Generated with an current upstream clang-format and:
clang-format -i $(find . -name \*.[ch])
Signed-off-by: Olof Johansson <olof@lixom.net>
One of the shortcomings of clang-format is that it doesn't allow
for aligned define tables, which is used for a number of constants.
Add annotation to disable the automatic formatting where needed.
Signed-off-by: Olof Johansson <olof@lixom.net>
This patch updates copyright header in all files as follows:
1. Makes "SPDX-License-Identifier: BSD-2-Clause" as first line
2. Change copyright year to 2019 for Western Digital
Signed-off-by: Anup Patel <anup.patel@wdc.com>
This patch adds doxygen style documentation for struct sbi_trap_regs
and related macros/defines/functions.
Signed-off-by: Anup Patel <anup.patel@wdc.com>
The struct sbi_trap_regs related defines RISCV_TRAP_REGS_xyz should
be in sbi_trap.h so that we can keep these defines in-sync with changes
in struct sbi_trap_regs.
Signed-off-by: Anup Patel <anup.patel@wdc.com>
Currently, we fail with error SBI_ENOTSUPP when we are not able
to handle illegal instruction trap. Instead, we should just
redirect illegal instruction trap to S-mode when not handled.
This redirection of illegal instruction trap will help lazy
save/restore of floating point registers to work correctly in
Linux kernel.
Signed-off-by: Anup Patel <anup.patel@wdc.com>
