Align the PMU extension implementation with the error codes required
by the SBI v3.0 specification, chapter 11:
- sbi_pmu_counter_start and sbi_pmu_counter_stop (secs 11.9-11.10,
tables 39-42): the start_flags/stop_flags bits 2:(XLEN-1) are
reserved and must be zero, so return SBI_ERR_INVALID_PARAM when any
reserved flag bit is set. Introduce SBI_PMU_START_FLAGS_MASK and
SBI_PMU_STOP_FLAGS_MASK for the valid bits of each function.
- sbi_pmu_counter_start and sbi_pmu_counter_stop (tables 40 and 42):
return SBI_ERR_ALREADY_STARTED / SBI_ERR_ALREADY_STOPPED when the
set of counters includes a counter which is already started or
stopped, instead of ignoring the error returned for each counter.
- sbi_pmu_event_get_info (sec 11.14, table 47): the output word must
indicate whether the event is supported, but firmware events were
only matched against the hardware event map and were always
reported as unsupported. Report a validated firmware event as
supported.
Signed-off-by: David E. Garcia Porras <david.garcia@aheadcomputing.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20260818210023.466462-4-david.garcia@aheadcomputing.com
Signed-off-by: Anup Patel <anup@brainfault.org>
As per section 11.14 of the SBI specification (Function: Get PMU Event
Info, FID #8), Table 47, the event_idx word of an event info entry
only uses BIT[0:19]; BIT[20:31] are reserved for the future purpose
and must be zero. Table 48 further requires the SBI implementation to
return SBI_ERR_INVALID_PARAM if any reserved bit in an event_idx word
is set.
sbi_pmu_event_get_info() does not check the reserved bits, so a
malformed event_idx is silently passed on to pmu_event_validate()
instead of failing the call. Add SBI_PMU_EVENT_IDX_MBZ_MASK covering
the must-be-zero bits and return SBI_ERR_INVALID_PARAM when any of
them are set.
Fixes: e434584216 ("lib: sbi_pmu: Implement SBI PMU event info function")
Signed-off-by: David E. Garcia Porras <david.garcia@aheadcomputing.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20260818210023.466462-2-david.garcia@aheadcomputing.com
Signed-off-by: Anup Patel <anup@brainfault.org>
A secondary hart may not have reached the target sleep state by the time
the primary checks, so the one-shot check could fail spuriously. Poll
each PCS status instead and give up after HART_SLEEP_TIMEOUT_MS.
Rework atcsmu_pcs_is_sleep() into the atcsmu_hart_is_sleep() predicate so
sbi_timer_waitms_until() can drive it.
Signed-off-by: Ben Zong-You Xie <ben717@andestech.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20260728081041.2724668-7-ben717@andestech.com
Signed-off-by: Anup Patel <anup@brainfault.org>
The tinfo probe only discovers trigger types; WARL fields tied off
inside an implemented trigger CSR (e.g. an mcontrol6 trigger without
load/store address match) are not discoverable. sbi_alloc_trigger()
also selects the first free trigger slot irrespective of the requested
trigger type or configuration. Unsupported configurations are thus
silently dropped by the hardware while SBI reports success, where SBI
v3.0 sections 19.4 / 19.5 require SBI_ERR_NOT_SUPPORTED.
Add an optional platform device, struct sbi_dbtr_device, with a
trigger_supported(idx, tdata1, tdata2, tdata3) callback reporting
whether the trigger slot selected by idx (the tselect value) supports
a given configuration. Use it, together with the probed per-slot
type_mask, to:
- allocate only trigger slots supporting the requested configuration
(new dbtr_find_free_slot() helper)
- reject unsupportable install/update requests with
SBI_ERR_NOT_SUPPORTED
- count only supporting slots in sbi_dbtr_num_trig() (SBI v3.0
section 19.1)
The install path now dry-runs the allocation of the whole batch before
programming any trigger, so a partially-installed batch is never left
behind. Without a registered device, only the type matching applies (unchanged behavior).
Fixes: 97f234f15c ("lib: sbi: Introduce the SBI debug triggers extension support")
Signed-off-by: David E. Garcia Porras <david.garcia@aheadcomputing.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20260727183041.258377-2-david.garcia@aheadcomputing.com
Signed-off-by: Anup Patel <anup@brainfault.org>
Some T-Head based processors, for example the Sophgo SG2044, have a
JTLB errata that requires special handling of certain TLB maintenance
instructions (sfence.vma).
Introduce a sbi_tlb_local_operations structure so platforms can provide
custom local TLB flush implementations. Use it to implement the JTLB
workaround on affected SoCs (currently Sophgo SG2044).
Signed-off-by: Xiang W <wangxiang@iscas.ac.cn>
Signed-off-by: Han Gao <gaohan@iscas.ac.cn>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20260617112925.1144190-2-wangxiang@iscas.ac.cn
Signed-off-by: Anup Patel <anup@brainfault.org>
The MPXY framework currently stores registered channels in a global
list and exposes them to all callers. However, the intended model is
that each channel is assigned to a single supervisor domain at boot and
remains owned by that domain for its lifetime.
Introduce fixed owner-domain tracking in struct sbi_mpxy_channel and
make channel lookup and enumeration domain-aware. A channel is now
visible only when accessed from its owning domain. Also require the
owner to be set before channel registration.
This allows MPXY to support systems where the same hart may be reused by
multiple domains while keeping channel ownership and visibility fixed to
the domain that owns the service.
Signed-off-by: Oza Pawandeep <pawandeep.oza@oss.qualcomm.com>
Reviewed-by: Rahul Pathak <rahul.pathak@oss.qualcomm.com>
Link: https://lore.kernel.org/r/20260721215758.695736-1-pawandeep.oza@oss.qualcomm.com
Signed-off-by: Anup Patel <anup@brainfault.org>
Add an irq_state field to struct sbi_irqchip_hwirq_data with a single
IRQ_ENABLED flag (bit 0) to track whether a hardware interrupt has been
enabled via the irqchip framework. Set IRQ_ENABLED in
sbi_irqchip_unmask_hwirq() when the unmask callback is invoked.
Add sbi_irqchip_get_irq_state() as a private inline accessor and expose
sbi_irqchip_is_irq_enabled() as a public API for drivers to query the
enabled state of a hardware interrupt by chip pointer and hwirq number.
Refactor imsic_local_eix_update() to operate on a single interrupt ID
instead of a base+count range, simplifying the CSR bit manipulation to
a direct BIT(id) write without the inner loop. Update all call sites
accordingly.
Use sbi_irqchip_is_irq_enabled() in imsic_warm_irqchip_init() to
restore per-EIID EIE CSR state on warm boot and HSM resume based on
the saved irq_state, replacing the previous blanket disable of all
interrupts. This ensures that EIIDs enabled during hotplug/warminit
cycle are correctly re-enabled on the resuming hart without requiring
software to re-register or re-unmask each interrupt.
Signed-off-by: Oza Pawandeep <pawandeep.oza@oss.qualcomm.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20260721214833.687361-6-pawandeep.oza@oss.qualcomm.com
Signed-off-by: Anup Patel <anup@brainfault.org>
Add parent_unique_id fields to struct aplic_data for
IMSIC parent linking in MSI mode. Store parent IMSIC unique_id
during FDT parsing of the APLIC node.
Add aplic_is_msi_mode() complementing aplic_is_direct_mode() to
consolidate delivery mode detection. Add APLIC_TARGET_EIID() macro
for packing the EIID field into the TARGET register.
Add parent_unique_id and parent_irq_map fields to struct aplic_data.
parent_unique_id identifies the upstream IMSIC irqchip device resolved
via sbi_irqchip_find_device() during hwirq_setup. parent_irq_map is
a per-source array allocated at cold init time to track the EIID
assigned by the IMSIC for each APLIC source.
Restore aplic_writel_msicfg() and re-introduce MSI address register
programming in aplic_init(), gated on aplic_is_msi_mode(). Set the
DOMAINCFG_DM bit to switch the hardware to MSI delivery mode when
no IDC structures are present.
Add aplic_program_msi_target() to pack hart_index, guest_index, and
EIID into the APLIC_TARGET register. Add aplic_write_msi() as the
sbi_irqchip write_msi callback that extracts EIID and hart_index from
the MSI message and calls aplic_program_msi_target(). Add
aplic_msi_callback() as the MSI receive callback that dispatches to
sbi_irqchip_process_hwirq() on the APLIC chip.
Extend aplic_hwirq_setup() with an MSI path that resolves the parent
IMSIC chip by parent_unique_id, registers an MSI route via
sbi_irqchip_register_msi(), and stores the allocated EIID in
parent_irq_map for the source being configured.
Extend aplic_hwirq_set_affinity() with an MSI path that delegates
affinity reprogramming to the parent IMSIC chip via
sbi_irqchip_set_affinity() using the stored parent_irq_map entry.
Guard warm_init, process_hwirqs, and hwirq_eoi with early returns in
MSI mode as interrupt delivery and acknowledgement are handled by the
IMSIC in that configuration.
Signed-off-by: Oza Pawandeep <pawandeep.oza@oss.qualcomm.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20260721214833.687361-3-pawandeep.oza@oss.qualcomm.com
Signed-off-by: Anup Patel <anup@brainfault.org>
Currently, the trap delegation setup is part of sbi_hart.c and
called via sbi_hart_init() whereas the per-hart trap (aka exception
and interrupt) handling is part of sbi_trap.c.
Move trap delegation init to sbi_trap.c and call it via new
sbi_trap_init() from both cold boot and warm boot path. This
way trap delegation setup is in same place as trap handling.
Signed-off-by: Anup Patel <anup.patel@oss.qualcomm.com>
Link: https://lore.kernel.org/r/20260613132129.2135128-2-anup.patel@oss.qualcomm.com
Signed-off-by: Anup Patel <anup@brainfault.org>
Fix the following issues with misaligned vector load/store:
a. Stack overflow: the mask[VLEN_MAX / 8] variable consumes 8K stack
space, given VLEN_MAX=65536, overflowing the default-sized stack.
There's no need to fetch the whole mask in one go, instead, make it
on-demand. Use a 128-byte mask as local buffer to hold the sliding
window of mask. For rvv load, this is allowed -- from the spec:
"The destination vector register group for a masked vector
instruction cannot overlap the source mask register (v0),
unless the destination vector register is being written with
a mask value (e.g., compares) or the scalar result of a reduction"
We don't need to worry about the mask getting overwritten.
b. Maintain the value of vstart upon abort (uptrap) to avoid duplicate
work. After fault resolution, the instruction can restart from the
faulting vstart. For Fault-Only-First loads, reset vstart to 0, as
previously done so, to conform to spec.
c. Explicitly set VS dirty in VSSTATUS with SET_VS_DIRTY() if faulting
from V=1, and if any vector register, including vstart/vl/vtype, gets
changed in the handler. It can add 1 unnecessary op to set VS dirty
in M/SSTATUS (not VSSTATUS), where the HW already did, but for code
simplicity, do it anyway. The overhead should be negligible.
Signed-off-by: Bo Gan <ganboing@gmail.com>
Tested-by: Anirudh Srinivasan <asrinivasan@oss.tenstorrent.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20260609060024.706-5-ganboing@gmail.com
Signed-off-by: Anup Patel <anup@brainfault.org>
sbi_load/store_loop read/write variable-length buffer unprivileged.
Both function use the widest aligned 8/4/2/1 byte load/stores in each
loop to reduce the total number of iterations.
Also switch the scalar/vector misaligned handlers to make use of such
functions to simplify code.
Miscellaneous: remove the unnecessary [taddr] in inline assembly
Signed-off-by: Bo Gan <ganboing@gmail.com>
Tested-by: Anirudh Srinivasan <asrinivasan@oss.tenstorrent.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20260609060024.706-4-ganboing@gmail.com
Signed-off-by: Anup Patel <anup@brainfault.org>
The load/store address offset between the uptrap and the orig_trap
can be derived by orig_trap->tval - uptrap->tval, thus refactor
the function prototype for simplicity.
For vector load, sbi_misaligned_v_tinst_fixup is introduced. There's
no transformed instruction for vector load/store, so null out tinst
if the fault is not a guest-page fault.
Signed-off-by: Bo Gan <ganboing@gmail.com>
Tested-by: Anirudh Srinivasan <asrinivasan@oss.tenstorrent.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20260609060024.706-3-ganboing@gmail.com
Signed-off-by: Anup Patel <anup@brainfault.org>
The per-HART stack size for exception/interrupt handling is currently
hardcoded to 8192 bytes in SBI_PLATFORM_DEFAULT_HART_STACK_SIZE. This
may not be sufficient for platforms with deeper call stacks (e.g. those
enabling additional SBI extensions) or may be wasteful for minimal
platforms.
Introduce a HART_STACK_SIZE Kconfig option in lib/sbi/Kconfig with a
valid range of 8192 to 1048576 bytes and a default of 8192 bytes to
preserve existing behavior. The SBI_PLATFORM_DEFAULT_HART_STACK_SIZE
macro now resolves to CONFIG_HART_STACK_SIZE, allowing all platforms
to benefit from a single configuration knob without any source changes.
Signed-off-by: Oriol Catalan <oriol.catalan@openchip.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/VI0P192MB3062735A6194BB6DA72083499E002@VI0P192MB3062.EURP192.PROD.OUTLOOK.COM
Signed-off-by: Anup Patel <anup@brainfault.org>
It's wrong to override the emulator callback in sbi_trap_emulate_load/
store. The function must respect the callback function passed in the
parameter. Hence, let the misaligned emulator callback decide when to
use sbi_misaligned_v_ld/st_emulator. To clean up things, also make the
following changes:
- Add the `insn` parameter to the callback. The trapping insn has been
fetched by the caller already, whether transformed or directly loaded,
thus saving the trouble in the callback. Note that you must not rely
on the length of the `insn`, as it can be a transformed one from tinst
- Also the `tcntx` is added, providing the callback with register values
to handle vector insn or other customized insns.
- Clarify that the read/write length (rlen/wlen) can be 0, in which
case it could be a vector load/store or some customized instruction.
The callback is responsible to handle it accordingly.
Also fixed issues in the sbi_misaligned_v_ld/st_emulator:
a. Redirect the trap when OPENSBI_CC_SUPPORT_VECTOR is not available.
b. Ensure the return code is >0 when no faults are redirected.
Fixes: c2acc5e5b0 ("lib: sbi_misaligned_ldst: Add handling of vector load/store")
Signed-off-by: Bo Gan <ganboing@gmail.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Tested-by: Anirudh Srinivasan <asrinivasan@oss.tenstorrent.com>
Link: https://lore.kernel.org/r/20260605113214.242-6-ganboing@gmail.com
Signed-off-by: Anup Patel <anup@brainfault.org>
Instead of parsing ISA extensions separately for each hart in the
generic_extensions_init() function, it is better to parse ISA extensions
for all available harts in the cold boot path. Also, this allows us
to remove fdt_isa_bitmap from scratch space and directly initialize
"extensions" in struct sbi_hart_features for each hart.
Signed-off-by: Anup Patel <anup.patel@oss.qualcomm.com>
Link: https://lore.kernel.org/r/20260521082625.1520870-3-anup.patel@oss.qualcomm.com
Signed-off-by: Anup Patel <anup@brainfault.org>
UBSan (Undefined Behavior Sanitizer) is a tool implemented using
compiler instrumentation at runtime that allows checking for
statements whose output is not deterministic or defined by the C
standard. Compiling and running OpenSBI with UBSan instrumentation
will print a message in the console if any sentence performs such
an action.
Support involves two main components:
1. The UBSan implementation hooks (derived from NetBSD),
used by the compiler to handle the check output.
2. A test suite integrated with the SBI unit test framework to
verify correct operation at runtime.
Usage:
make UBSAN=y PLATFORM=generic ...
The test suite is built when both UBSAN=y and CONFIG_SBIUNIT=y are
enabled.
When UBSan is enabled, FW_PAYLOAD_OFFSET may need to be increased
due to the size increase added by the instrumentation. A
value of 0x400000 has been tested.
UBSan adds runtime overhead and is intended for development builds
only, not for production.
Note: This patch marks __stack_chk_guard in sbi_init.c as a weak
symbol to prevent multiple definition errors at compile time with
UBSan instrumentation enabled. This resolves the conflict
between the .globl definitions in sbi_init.c and test_head.S.
Signed-off-by: Marcos Oduardo <marcos.oduardo@gmail.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20260515163321.2038366-1-marcos.oduardo@gmail.com
Signed-off-by: Anup Patel <anup@brainfault.org>
Add rpmi_event_notification_state enum with disable, enable, and return
current state IDs. Also, add req_state field to rpmi_enable_notification_req
and current_state field to rpmi_enable_notification_resp for RPMI specification
compliance.
Add notification event ID enum and data structures for RPMI Performance
service group events: power change, limit change, and level change.
Signed-off-by: David E. Garcia Porras <david.garcia@aheadcomputing.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20260608125257.3220114-4-anup.patel@oss.qualcomm.com
Signed-off-by: Anup Patel <anup@brainfault.org>
PMP functions that deal with hart PMP CSRs are given a sbi_hart_ prefix,
to distinguish from RISC-V PMP encoding functions.
The is_pmp_entry_mapped() function is changed a little more, to align
with other PMP conventions, and made to return a bool to make it more
obvious that it returns a bool and not an SBI_ return code.
Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20260430045528.420437-8-npiggin@gmail.com
Signed-off-by: Anup Patel <anup@brainfault.org>
Add bit-field defines for the RPMI performance domain attributes flags
and fast-channel attributes flags as specified in the RPMI specification.
These are needed by platform implementations that provide RPMI
performance services (e.g. DVFS controllers).
Also add the missing db_write_value field to
rpmi_perf_get_fast_chn_attr_resp to match the RPMI spec layout.
Signed-off-by: David E. Garcia Porras <david.garcia@aheadcomputing.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20260328054347.3706029-5-david.garcia@aheadcomputing.com
Signed-off-by: Anup Patel <anup@brainfault.org>
This patch adds proper support for per-domain floating-point (FP) and
vector (V) contexts in the domain context switch logic. Each domain
now maintains its own FP and vector state, which is saved and restored
during domain switches.
Conditionalize FP and Vector save/restore based on extensions, unconditional
save and restore of floating-point (FP) and Vector registers fails on
generic platform firmware. This firmware must run on multiple platforms
that may lack these extensions.
Address this by conditionally executing FP save/restore only if the underlying
hart supports the F or D extensions. Similarly, perform Vector save/restore
only if the hart supports the Vector extension.
This improves support for multi-domain systems with FP and Vector
extensions, and prevents corruption of FP/Vector state during domain
switches.
Signed-off-by: Dave Patel <dave.patel@riscstar.com>
Signed-off-by: Anup Patel <anup.patel@oss.qualcomm.com>
Link: https://lore.kernel.org/r/20260518083023.997323-4-anup.patel@oss.qualcomm.com
Signed-off-by: Anup Patel <anup@brainfault.org>
Add support for saving and restoring RISC-V floating-point (F/D) extension
state in OpenSBI. This introduces a floating-point context structure and
helper routines to perform full context save and restore.
The floating-point context includes storage for all 32 FPi registers (f0–f31)
along with the fcsr control and status register. The register state is saved
and restored using double-precision load/store instructions (fsd/fld), and
single-precision load/store instructions (fsw/flw) on an RV64 system with
F and D-extension support.
The implementation follows an eager context switching model where the entire
FP state is saved and restored on every context switch. This avoids the need
for trap-based lazy management and keeps the design simple and deterministic.
Signed-off-by: Dave Patel <dave.patel@riscstar.com>
Signed-off-by: Anup Patel <anup.patel@oss.qualcomm.com>
Link: https://lore.kernel.org/r/20260518083023.997323-3-anup.patel@oss.qualcomm.com
Signed-off-by: Anup Patel <anup@brainfault.org>
Eager context switch: Add support for saving and restoring RISC-V vector
extension state in OpenSBI. This introduces a per-hart vector context
structure and helper routines to perform full context save and restore.
The vector context includes vcsr CSRs along with storage for all 32 vector
registers. The register state is saved and restored using byte-wise vector
load/store instructions (vs8r/vl8r).
The implementation follows an eager context switching model where the entire
vector state is saved and restored on every context switch. This provides a
simple and deterministic mechanism without requiring lazy trap-based
management.
Signed-off-by: Dave Patel <dave.patel@riscstar.com>
Signed-off-by: Anup Patel <anup.patel@oss.qualcomm.com>
Link: https://lore.kernel.org/r/20260518083023.997323-2-anup.patel@oss.qualcomm.com
Signed-off-by: Anup Patel <anup@brainfault.org>
Rahul Pathak