The MPXY framework currently stores registered channels in a global
list and exposes them to all callers. However, the intended model is
that each channel is assigned to a single supervisor domain at boot and
remains owned by that domain for its lifetime.
Introduce fixed owner-domain tracking in struct sbi_mpxy_channel and
make channel lookup and enumeration domain-aware. A channel is now
visible only when accessed from its owning domain. Also require the
owner to be set before channel registration.
This allows MPXY to support systems where the same hart may be reused by
multiple domains while keeping channel ownership and visibility fixed to
the domain that owns the service.
Signed-off-by: Oza Pawandeep <pawandeep.oza@oss.qualcomm.com>
Reviewed-by: Rahul Pathak <rahul.pathak@oss.qualcomm.com>
Link: https://lore.kernel.org/r/20260721215758.695736-1-pawandeep.oza@oss.qualcomm.com
Signed-off-by: Anup Patel <anup@brainfault.org>
Add an irq_state field to struct sbi_irqchip_hwirq_data with a single
IRQ_ENABLED flag (bit 0) to track whether a hardware interrupt has been
enabled via the irqchip framework. Set IRQ_ENABLED in
sbi_irqchip_unmask_hwirq() when the unmask callback is invoked.
Add sbi_irqchip_get_irq_state() as a private inline accessor and expose
sbi_irqchip_is_irq_enabled() as a public API for drivers to query the
enabled state of a hardware interrupt by chip pointer and hwirq number.
Refactor imsic_local_eix_update() to operate on a single interrupt ID
instead of a base+count range, simplifying the CSR bit manipulation to
a direct BIT(id) write without the inner loop. Update all call sites
accordingly.
Use sbi_irqchip_is_irq_enabled() in imsic_warm_irqchip_init() to
restore per-EIID EIE CSR state on warm boot and HSM resume based on
the saved irq_state, replacing the previous blanket disable of all
interrupts. This ensures that EIIDs enabled during hotplug/warminit
cycle are correctly re-enabled on the resuming hart without requiring
software to re-register or re-unmask each interrupt.
Signed-off-by: Oza Pawandeep <pawandeep.oza@oss.qualcomm.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20260721214833.687361-6-pawandeep.oza@oss.qualcomm.com
Signed-off-by: Anup Patel <anup@brainfault.org>
Add parent_unique_id fields to struct aplic_data for
IMSIC parent linking in MSI mode. Store parent IMSIC unique_id
during FDT parsing of the APLIC node.
Add aplic_is_msi_mode() complementing aplic_is_direct_mode() to
consolidate delivery mode detection. Add APLIC_TARGET_EIID() macro
for packing the EIID field into the TARGET register.
Add parent_unique_id and parent_irq_map fields to struct aplic_data.
parent_unique_id identifies the upstream IMSIC irqchip device resolved
via sbi_irqchip_find_device() during hwirq_setup. parent_irq_map is
a per-source array allocated at cold init time to track the EIID
assigned by the IMSIC for each APLIC source.
Restore aplic_writel_msicfg() and re-introduce MSI address register
programming in aplic_init(), gated on aplic_is_msi_mode(). Set the
DOMAINCFG_DM bit to switch the hardware to MSI delivery mode when
no IDC structures are present.
Add aplic_program_msi_target() to pack hart_index, guest_index, and
EIID into the APLIC_TARGET register. Add aplic_write_msi() as the
sbi_irqchip write_msi callback that extracts EIID and hart_index from
the MSI message and calls aplic_program_msi_target(). Add
aplic_msi_callback() as the MSI receive callback that dispatches to
sbi_irqchip_process_hwirq() on the APLIC chip.
Extend aplic_hwirq_setup() with an MSI path that resolves the parent
IMSIC chip by parent_unique_id, registers an MSI route via
sbi_irqchip_register_msi(), and stores the allocated EIID in
parent_irq_map for the source being configured.
Extend aplic_hwirq_set_affinity() with an MSI path that delegates
affinity reprogramming to the parent IMSIC chip via
sbi_irqchip_set_affinity() using the stored parent_irq_map entry.
Guard warm_init, process_hwirqs, and hwirq_eoi with early returns in
MSI mode as interrupt delivery and acknowledgement are handled by the
IMSIC in that configuration.
Signed-off-by: Oza Pawandeep <pawandeep.oza@oss.qualcomm.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20260721214833.687361-3-pawandeep.oza@oss.qualcomm.com
Signed-off-by: Anup Patel <anup@brainfault.org>
Currently, the trap delegation setup is part of sbi_hart.c and
called via sbi_hart_init() whereas the per-hart trap (aka exception
and interrupt) handling is part of sbi_trap.c.
Move trap delegation init to sbi_trap.c and call it via new
sbi_trap_init() from both cold boot and warm boot path. This
way trap delegation setup is in same place as trap handling.
Signed-off-by: Anup Patel <anup.patel@oss.qualcomm.com>
Link: https://lore.kernel.org/r/20260613132129.2135128-2-anup.patel@oss.qualcomm.com
Signed-off-by: Anup Patel <anup@brainfault.org>
Fix the following issues with misaligned vector load/store:
a. Stack overflow: the mask[VLEN_MAX / 8] variable consumes 8K stack
space, given VLEN_MAX=65536, overflowing the default-sized stack.
There's no need to fetch the whole mask in one go, instead, make it
on-demand. Use a 128-byte mask as local buffer to hold the sliding
window of mask. For rvv load, this is allowed -- from the spec:
"The destination vector register group for a masked vector
instruction cannot overlap the source mask register (v0),
unless the destination vector register is being written with
a mask value (e.g., compares) or the scalar result of a reduction"
We don't need to worry about the mask getting overwritten.
b. Maintain the value of vstart upon abort (uptrap) to avoid duplicate
work. After fault resolution, the instruction can restart from the
faulting vstart. For Fault-Only-First loads, reset vstart to 0, as
previously done so, to conform to spec.
c. Explicitly set VS dirty in VSSTATUS with SET_VS_DIRTY() if faulting
from V=1, and if any vector register, including vstart/vl/vtype, gets
changed in the handler. It can add 1 unnecessary op to set VS dirty
in M/SSTATUS (not VSSTATUS), where the HW already did, but for code
simplicity, do it anyway. The overhead should be negligible.
Signed-off-by: Bo Gan <ganboing@gmail.com>
Tested-by: Anirudh Srinivasan <asrinivasan@oss.tenstorrent.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20260609060024.706-5-ganboing@gmail.com
Signed-off-by: Anup Patel <anup@brainfault.org>
sbi_load/store_loop read/write variable-length buffer unprivileged.
Both function use the widest aligned 8/4/2/1 byte load/stores in each
loop to reduce the total number of iterations.
Also switch the scalar/vector misaligned handlers to make use of such
functions to simplify code.
Miscellaneous: remove the unnecessary [taddr] in inline assembly
Signed-off-by: Bo Gan <ganboing@gmail.com>
Tested-by: Anirudh Srinivasan <asrinivasan@oss.tenstorrent.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20260609060024.706-4-ganboing@gmail.com
Signed-off-by: Anup Patel <anup@brainfault.org>
The load/store address offset between the uptrap and the orig_trap
can be derived by orig_trap->tval - uptrap->tval, thus refactor
the function prototype for simplicity.
For vector load, sbi_misaligned_v_tinst_fixup is introduced. There's
no transformed instruction for vector load/store, so null out tinst
if the fault is not a guest-page fault.
Signed-off-by: Bo Gan <ganboing@gmail.com>
Tested-by: Anirudh Srinivasan <asrinivasan@oss.tenstorrent.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20260609060024.706-3-ganboing@gmail.com
Signed-off-by: Anup Patel <anup@brainfault.org>
The per-HART stack size for exception/interrupt handling is currently
hardcoded to 8192 bytes in SBI_PLATFORM_DEFAULT_HART_STACK_SIZE. This
may not be sufficient for platforms with deeper call stacks (e.g. those
enabling additional SBI extensions) or may be wasteful for minimal
platforms.
Introduce a HART_STACK_SIZE Kconfig option in lib/sbi/Kconfig with a
valid range of 8192 to 1048576 bytes and a default of 8192 bytes to
preserve existing behavior. The SBI_PLATFORM_DEFAULT_HART_STACK_SIZE
macro now resolves to CONFIG_HART_STACK_SIZE, allowing all platforms
to benefit from a single configuration knob without any source changes.
Signed-off-by: Oriol Catalan <oriol.catalan@openchip.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/VI0P192MB3062735A6194BB6DA72083499E002@VI0P192MB3062.EURP192.PROD.OUTLOOK.COM
Signed-off-by: Anup Patel <anup@brainfault.org>
It's wrong to override the emulator callback in sbi_trap_emulate_load/
store. The function must respect the callback function passed in the
parameter. Hence, let the misaligned emulator callback decide when to
use sbi_misaligned_v_ld/st_emulator. To clean up things, also make the
following changes:
- Add the `insn` parameter to the callback. The trapping insn has been
fetched by the caller already, whether transformed or directly loaded,
thus saving the trouble in the callback. Note that you must not rely
on the length of the `insn`, as it can be a transformed one from tinst
- Also the `tcntx` is added, providing the callback with register values
to handle vector insn or other customized insns.
- Clarify that the read/write length (rlen/wlen) can be 0, in which
case it could be a vector load/store or some customized instruction.
The callback is responsible to handle it accordingly.
Also fixed issues in the sbi_misaligned_v_ld/st_emulator:
a. Redirect the trap when OPENSBI_CC_SUPPORT_VECTOR is not available.
b. Ensure the return code is >0 when no faults are redirected.
Fixes: c2acc5e5b0 ("lib: sbi_misaligned_ldst: Add handling of vector load/store")
Signed-off-by: Bo Gan <ganboing@gmail.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Tested-by: Anirudh Srinivasan <asrinivasan@oss.tenstorrent.com>
Link: https://lore.kernel.org/r/20260605113214.242-6-ganboing@gmail.com
Signed-off-by: Anup Patel <anup@brainfault.org>
Instead of parsing ISA extensions separately for each hart in the
generic_extensions_init() function, it is better to parse ISA extensions
for all available harts in the cold boot path. Also, this allows us
to remove fdt_isa_bitmap from scratch space and directly initialize
"extensions" in struct sbi_hart_features for each hart.
Signed-off-by: Anup Patel <anup.patel@oss.qualcomm.com>
Link: https://lore.kernel.org/r/20260521082625.1520870-3-anup.patel@oss.qualcomm.com
Signed-off-by: Anup Patel <anup@brainfault.org>
UBSan (Undefined Behavior Sanitizer) is a tool implemented using
compiler instrumentation at runtime that allows checking for
statements whose output is not deterministic or defined by the C
standard. Compiling and running OpenSBI with UBSan instrumentation
will print a message in the console if any sentence performs such
an action.
Support involves two main components:
1. The UBSan implementation hooks (derived from NetBSD),
used by the compiler to handle the check output.
2. A test suite integrated with the SBI unit test framework to
verify correct operation at runtime.
Usage:
make UBSAN=y PLATFORM=generic ...
The test suite is built when both UBSAN=y and CONFIG_SBIUNIT=y are
enabled.
When UBSan is enabled, FW_PAYLOAD_OFFSET may need to be increased
due to the size increase added by the instrumentation. A
value of 0x400000 has been tested.
UBSan adds runtime overhead and is intended for development builds
only, not for production.
Note: This patch marks __stack_chk_guard in sbi_init.c as a weak
symbol to prevent multiple definition errors at compile time with
UBSan instrumentation enabled. This resolves the conflict
between the .globl definitions in sbi_init.c and test_head.S.
Signed-off-by: Marcos Oduardo <marcos.oduardo@gmail.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20260515163321.2038366-1-marcos.oduardo@gmail.com
Signed-off-by: Anup Patel <anup@brainfault.org>
Add rpmi_event_notification_state enum with disable, enable, and return
current state IDs. Also, add req_state field to rpmi_enable_notification_req
and current_state field to rpmi_enable_notification_resp for RPMI specification
compliance.
Add notification event ID enum and data structures for RPMI Performance
service group events: power change, limit change, and level change.
Signed-off-by: David E. Garcia Porras <david.garcia@aheadcomputing.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20260608125257.3220114-4-anup.patel@oss.qualcomm.com
Signed-off-by: Anup Patel <anup@brainfault.org>
PMP functions that deal with hart PMP CSRs are given a sbi_hart_ prefix,
to distinguish from RISC-V PMP encoding functions.
The is_pmp_entry_mapped() function is changed a little more, to align
with other PMP conventions, and made to return a bool to make it more
obvious that it returns a bool and not an SBI_ return code.
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20260430045528.420437-8-npiggin@gmail.com
Signed-off-by: Anup Patel <anup@brainfault.org>
Add bit-field defines for the RPMI performance domain attributes flags
and fast-channel attributes flags as specified in the RPMI specification.
These are needed by platform implementations that provide RPMI
performance services (e.g. DVFS controllers).
Also add the missing db_write_value field to
rpmi_perf_get_fast_chn_attr_resp to match the RPMI spec layout.
Signed-off-by: David E. Garcia Porras <david.garcia@aheadcomputing.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20260328054347.3706029-5-david.garcia@aheadcomputing.com
Signed-off-by: Anup Patel <anup@brainfault.org>
This patch adds proper support for per-domain floating-point (FP) and
vector (V) contexts in the domain context switch logic. Each domain
now maintains its own FP and vector state, which is saved and restored
during domain switches.
Conditionalize FP and Vector save/restore based on extensions, unconditional
save and restore of floating-point (FP) and Vector registers fails on
generic platform firmware. This firmware must run on multiple platforms
that may lack these extensions.
Address this by conditionally executing FP save/restore only if the underlying
hart supports the F or D extensions. Similarly, perform Vector save/restore
only if the hart supports the Vector extension.
This improves support for multi-domain systems with FP and Vector
extensions, and prevents corruption of FP/Vector state during domain
switches.
Signed-off-by: Dave Patel <dave.patel@riscstar.com>
Signed-off-by: Anup Patel <anup.patel@oss.qualcomm.com>
Link: https://lore.kernel.org/r/20260518083023.997323-4-anup.patel@oss.qualcomm.com
Signed-off-by: Anup Patel <anup@brainfault.org>
Add support for saving and restoring RISC-V floating-point (F/D) extension
state in OpenSBI. This introduces a floating-point context structure and
helper routines to perform full context save and restore.
The floating-point context includes storage for all 32 FPi registers (f0–f31)
along with the fcsr control and status register. The register state is saved
and restored using double-precision load/store instructions (fsd/fld), and
single-precision load/store instructions (fsw/flw) on an RV64 system with
F and D-extension support.
The implementation follows an eager context switching model where the entire
FP state is saved and restored on every context switch. This avoids the need
for trap-based lazy management and keeps the design simple and deterministic.
Signed-off-by: Dave Patel <dave.patel@riscstar.com>
Signed-off-by: Anup Patel <anup.patel@oss.qualcomm.com>
Link: https://lore.kernel.org/r/20260518083023.997323-3-anup.patel@oss.qualcomm.com
Signed-off-by: Anup Patel <anup@brainfault.org>
Eager context switch: Add support for saving and restoring RISC-V vector
extension state in OpenSBI. This introduces a per-hart vector context
structure and helper routines to perform full context save and restore.
The vector context includes vcsr CSRs along with storage for all 32 vector
registers. The register state is saved and restored using byte-wise vector
load/store instructions (vs8r/vl8r).
The implementation follows an eager context switching model where the entire
vector state is saved and restored on every context switch. This provides a
simple and deterministic mechanism without requiring lazy trap-based
management.
Signed-off-by: Dave Patel <dave.patel@riscstar.com>
Signed-off-by: Anup Patel <anup.patel@oss.qualcomm.com>
Link: https://lore.kernel.org/r/20260518083023.997323-2-anup.patel@oss.qualcomm.com
Signed-off-by: Anup Patel <anup@brainfault.org>
The location of the RNMI/E trap vectors in the Smrnmi extension is
implementation-defined, so platforms with vendor-specific NMI vector
mechanisms must install the firmware's NMI entry points themselves.
Add an smrnmi_handlers_init() callback to sbi_platform_operations that
receives the firmware entry points and lets platform code install them
at the hardware-specific vector locations. Two pointers are passed:
- _trap_rnmi_handler: the dedicated RNMI entry point that saves
context using the Smrnmi MN* CSRs and returns via mnret.
- _trap_handler: the regular M-mode trap entry since RNME is taken
as a regular M-mode trap with NMIE=0.
When Smrnmi is present, install the platform's NMI vectors via the new
callback, initialize MNSCRATCH with the per-hart scratch pointer, and
set MNSTATUS.NMIE.
Smrnmi-enabled platforms must register smrnmi_handlers_init; if the
extension is detected but no callback is registered, sbi_panic() is
called since enabling NMIs without handlers in place would route
subsequent traps into nowhere.
Signed-off-by: Evgeny Voevodin <evvoevod@tenstorrent.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/88b1470e1e3348d454b4b995a11a85c01914f7af.1778176768.git.evvoevod@tenstorrent.com
Signed-off-by: Anup Patel <anup@brainfault.org>
Implement basic Resumable NMI (RNMI) handler support for the RISC-V
Smrnmi extension.
The new _trap_rnmi_handler assembly entry point saves context using the
Smrnmi MN* CSRs (MNSCRATCH, MNEPC, MNSTATUS, MNCAUSE) and returns via
mnret. It dispatches to sbi_trap_rnmi_handler(), which optionally calls
a platform-specific ops->rnmi_handler callback for actual NMI
processing. If no platform handler is registered or it fails, the
event is reported as an unhandled NMI.
The RNMI handler reuses the generic trap context structure but stores MN*
CSR values (MNEPC, MNSTATUS, MNCAUSE) into the corresponding generic
fields (mepc, mstatus, cause) for compatibility with existing trap
infrastructure.
Signed-off-by: Evgeny Voevodin <evvoevod@tenstorrent.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/050ae6d2762ba8d5b9dfb3cc1960a23aa3d6c549.1778176768.git.evvoevod@tenstorrent.com
Signed-off-by: Anup Patel <anup@brainfault.org>
It is possible to have platform where an irqchip device targets
a subset of harts and there are multiple irqchip devices to cover
all harts.
To support this scenario:
1) Add target_harts hartmask to struct sbi_irqchip_device which
represents the set of harts targetted by the irqchip device
2) Call warm_init() and process_hwirqs() callbacks of an irqchip
device on a hart only if irqchip device targets that particular
hart
Signed-off-by: Anup Patel <anup.patel@oss.qualcomm.com>
Link: https://lore.kernel.org/r/20260213055342.3124872-6-anup.patel@oss.qualcomm.com
Signed-off-by: Anup Patel <anup@brainfault.org>
