sbi_load/store_loop read/write variable-length buffer unprivileged.
Both function use the widest aligned 8/4/2/1 byte load/stores in each
loop to reduce the total number of iterations.
Also switch the scalar/vector misaligned handlers to make use of such
functions to simplify code.
Miscellaneous: remove the unnecessary [taddr] in inline assembly
Signed-off-by: Bo Gan <ganboing@gmail.com>
Tested-by: Anirudh Srinivasan <asrinivasan@oss.tenstorrent.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20260609060024.706-4-ganboing@gmail.com
Signed-off-by: Anup Patel <anup@brainfault.org>
The load/store address offset between the uptrap and the orig_trap
can be derived by orig_trap->tval - uptrap->tval, thus refactor
the function prototype for simplicity.
For vector load, sbi_misaligned_v_tinst_fixup is introduced. There's
no transformed instruction for vector load/store, so null out tinst
if the fault is not a guest-page fault.
Signed-off-by: Bo Gan <ganboing@gmail.com>
Tested-by: Anirudh Srinivasan <asrinivasan@oss.tenstorrent.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20260609060024.706-3-ganboing@gmail.com
Signed-off-by: Anup Patel <anup@brainfault.org>
Rehaul instruction decoding to fix the following issues:
- We assume the XLEN of previous mode is the same as MXLEN. However,
RVC instructions decodes differently in RV32 and RV64, so shouldn't
have assumed that.
- We assume it's a misaligned fault and the load/store offset is 0,
i.e., base address == fault address, but access faults can have
non-0 offset (on HW supporting misaligned accesses), so platform
specific load/store fault handler gets the wrong base address.
- No checking of [63:32] of tinst in RV64, which is explicitly
required by Privileged ISA 19.6.3. Must reject tinst with non-0
high 32 bits.
Thus, fix all the above. For misaligned load/store fault, the address
offset should be 0, but we'll validate that on a DEBUG build. On an
optmized build, we kill the use of base address, and use trap address
instead (same as before), which lets the compiler optimize out imm
parsing and other calculations.
I also analyzed the behavior of misaligned fault handler before fix.
With the following conditions met, it can trigger data corruption:
- HW doesn't transform instruction into tinst.
- HW doesn't support misaligned load/store, and OS doesn't enable
misaligned delegation, thus OpenSBI handler is in effect
- HW supports mixed XLEN, and M mode is running RV64, and the trapping
mode (U/VS/VU) is running RV32.
- The trapping instruction is c.f{l|s}w(sp).
Due to the incorrect insn decoding, the trapping instruction would
mistakenly be decoded as c.{l|s}d(sp). With this fix, c.f{l|s}w(sp)
in RV32 is now emulated correctly.
Validation:
The patch is validated to have fixed the issue with test cases running
on a modified version of QEMU that exposes misaligned faults [1], and
a further modified version that removes tinst transformation [2]. The
S-mode OS is a local build of Debian Trixie 6.12 kernel that enables
COMPAT (RV32), and the U-mode test application exercises all integer
and floating-point load/store (RVIFD64/32+RVC64/32) instructions with
all possible imm values. The patch is also tested on real HW (Sifive
P550/ESWIN EIC7700), which only supports RV64. On P550, the same test
was validated both in U mode and VU mode, where the host runs a 6.12
ESWIN vendor kernel that has some ESWIN SoC device driver patches [3]
applied, and the guest runs the exact same Debian Trixie 6.12 kernel
mentioned above.
[1] https://github.com/ganboing/qemu/tree/ganboing-misalign
[2] https://github.com/ganboing/qemu/tree/ganboing-misalign-no-tinst
[3] https://github.com/sifiveinc/riscv-linux/tree/rel/kernel-6.12/hifive-premier-p550
Fixes: 7219477f7b ("lib: Use MTINST CSR in misaligned load/store emulation")
Fixes: b5ae8e8a65 ("lib: Add misaligned load/store trap handling")
Fixes: 4c112650bb ("lib: sbi: abstract out insn decoding to unify mem fault handlers")
Signed-off-by: Bo Gan <ganboing@gmail.com>
Tested-by: Anirudh Srinivasan <asrinivasan@oss.tenstorrent.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20260605113214.242-8-ganboing@gmail.com
Signed-off-by: Anup Patel <anup@brainfault.org>
It's wrong to override the emulator callback in sbi_trap_emulate_load/
store. The function must respect the callback function passed in the
parameter. Hence, let the misaligned emulator callback decide when to
use sbi_misaligned_v_ld/st_emulator. To clean up things, also make the
following changes:
- Add the `insn` parameter to the callback. The trapping insn has been
fetched by the caller already, whether transformed or directly loaded,
thus saving the trouble in the callback. Note that you must not rely
on the length of the `insn`, as it can be a transformed one from tinst
- Also the `tcntx` is added, providing the callback with register values
to handle vector insn or other customized insns.
- Clarify that the read/write length (rlen/wlen) can be 0, in which
case it could be a vector load/store or some customized instruction.
The callback is responsible to handle it accordingly.
Also fixed issues in the sbi_misaligned_v_ld/st_emulator:
a. Redirect the trap when OPENSBI_CC_SUPPORT_VECTOR is not available.
b. Ensure the return code is >0 when no faults are redirected.
Fixes: c2acc5e5b0 ("lib: sbi_misaligned_ldst: Add handling of vector load/store")
Signed-off-by: Bo Gan <ganboing@gmail.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Tested-by: Anirudh Srinivasan <asrinivasan@oss.tenstorrent.com>
Link: https://lore.kernel.org/r/20260605113214.242-6-ganboing@gmail.com
Signed-off-by: Anup Patel <anup@brainfault.org>
During warm reset, my EIC770X/Hifive Premier P550 can sometimes
encounter memory corruption issue crashing Linux boot. Currently the
issue is mitigated by having a sbi_printf before writing to the reset
register. I analyzed the issue further since then. From the SoC
datasheet[1], it's recommended to implement power-down flow as:
a. Designate a primary core, and let it broadcast requests to other
cores to execute a CEASE insn. Primary core also notifies an
"Externel Agent" to start monitoring.
b. Primary core waits for other cores to CEASE before it CEASEs.
c. "External Agent" waits for primary core to CEASE before resets
the Core Complex.
It's possible that EIC770X can trigger undefined behavior if the core
complex is reset while the harts are actively running. The sbi_printf
in the reset handler effectively hides the problem by delaying the
reset -- by the time sbi_printf finishes, all other harts will have
already landed in the loop in sbi_hsm_hart_wait(), which parks the hart.
Without the sbi_printf, I confirmed that other harts haven't reached
sbi_hsm_hart_wait yet before current hart resets the SoC. (by debugging)
To safely reset, and inspired by the datasheet, the warm reset logic
in eic770x.c now use the current hart as both primary core and the
"External Agent", and other harts as secondary cores. It leverages
the HSM framework and a new eic770x_hsm device to CEASE other harts,
and wait for them to CEASE before resets the SoC. with the sbi_printf
before reset removed, and this logic in place, stress test shows that
the memory corruption issue no longer occurs.
The new eic770x_hsm device is only used for the reset-CEASE logic at
the moment, and may be extended to a fully functional HSM device in
the future.
[1] https://github.com/eswincomputing/EIC7700X-SoC-Technical-Reference-Manual
Fixes: e5797e0688 ("platform: generic: eswin: add EIC7700")
Signed-off-by: Bo Gan <ganboing@gmail.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20260605075708.96-3-ganboing@gmail.com
Signed-off-by: Anup Patel <anup@brainfault.org>
When using debug builds, aka., DEBUG=1, csr_write_num() function can
trigger stack overflow. This is caused by the large amount of macro
expansion of csr_write(...), which, under debug builds, will generate
massive amount of stack variables (tested with GCC 13.2.0). The issue
is masked previously as we didn't have too many csr_write()'s before
commit 55296fd27c, but now, it does overflow the default 4KB stack.
The csr_read(relaxed) macros already use the "register" modifier to
optimize stack usage (perhaps unknowingly?), so this patch just
follows suit.
Fixes: 55296fd27c ("lib: Allow custom CSRs in csr_read_num() and csr_write_num()")
Signed-off-by: Bo Gan <ganboing@gmail.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20251216052528.18896-1-ganboing@gmail.com
Signed-off-by: Anup Patel <anup@brainfault.org>
Hifive Premier P550[1] is a Mini-DTX form factor board with EIC7700X.
It has a STM32F407VET6 onboard MCU acting as the BMC, controlling
ATX power on/off while providing remote management features. The
EIC7700X SoC/SoM communicates with the BMC via UART2, using ESWIN's
protocol. The messages transmitted are fixed sizes (267 bytes), and
depending on the type, can be directional or bi-directional. The
shutdown and cold reboot requests are directional messages from SoC
to BMC (NOTIFY type) with CMD_POWER_OFF or CMD_RESTART. The payload
of shutdown/cold reboot requests should be empty and are ignored by
the BMC at the moment. A HFP (Hifive Premier) specific reset device
is registered in addition to the SoC reset device. For shutdown and
cold reboot, the board-level reset takes precedence.
The definitions of the SoC <-> BMC message protocol is taken from
ESWIN's repo [2]. The only file used from that repo is `hf_common.h`
It's disjunctively dual licensed as (GPL-2.0-only OR BSD-2-Clause),
hence, compatible with the license of OpenSBI. It's heavily modified
and renamed as platform/generic/include/eswin/hfp.h. The author and
copyright in the original file are retained.
Validated shutdown/cold reboot working on Hifive Premier P550.
[1] https://www.sifive.com/boards/hifive-premier-p550#documentation
[2] https://github.com/eswincomputing/hifive-premier-p550-mcu-patches.git
Signed-off-by: Bo Gan <ganboing@gmail.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20251218104243.562667-8-ganboing@gmail.com
Signed-off-by: Anup Patel <anup@brainfault.org>
Previously we assume only 1 UART8250 instance can be used. Now we support
multiple instances by introducing counterpart functions to putc/getc/init
which take an extra *dev parameter, and name them as uart8250_device_xyz()
The original functions without the *dev parameter will operate on the
default instance exactly the same as before, so no changes on the caller
is required.
Note: uart8250_device_init only does device initialization without the
console registration logic.
Signed-off-by: Bo Gan <ganboing@gmail.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20251218104243.562667-7-ganboing@gmail.com
Signed-off-by: Anup Patel <anup@brainfault.org>
Initial platform support for ESWIN Computing EIC7700 based on public SoC
datasheet[1] and tested on HiFive Premier P550. Vendor U-boot/Linux boots
fine, and I've tested Geekbench 6.5.0 Preview and got scores on par with
the vendor OpenSBI. System shutdown/reboot for HiFive Premier P550 and
other boards will be implemented in subsequent commits. At this point,
only SoC-level warm reset is implemented.
The files and functions are intentionally named as eic770x in many places
for future enhancements to support the 2 die version of the same SoC,
namely EIC7702, seen on DC-ROMA AI PC FML13V03 [2]. This patch set only
deals with the single die version, and the assumption is we can be either
die with id=0 or id=1, but there's only a single die in the system, or we
are only using a single die out of 2. However, the way the SoC handles 2-
die greatly affects how we configure it in a 1-die setup. EIC770X address
map has die 0/1 memory regions interleaved (see comments in eic770x.c).
If only 1 die is connected or active, it creates holes in the address map
for those regions corresponding to the remote die. When speculative-
execution or HW prefetcher touches data-cacheable regions that happen to
fall into those holes, it can trigger bus error. Specifically:
- Remote (non-existent) die L3 zero device
- Remote (non-existent) die cached memory region
- Other holes in Memory Port
To make matters worse, EIC770X doesn't have cache coherent DMA, and due
to the fact that the P550 core lacks Svpbmt, the SoC maps main memory
twice as different regions, so it can bypass cache and fetch the data
directly from memory. In address space, we have two memory regions, one
as cached, the other as uncached. Thus, we also need an extra PMP entry
to protect OpenSBI blob from the uncached window. To do this, platform
code requires single_fw_region, otherwise, we'll run out of PMP entries.
EIC770X also have several feature disable/enable CSRs accessible in M
mode. By default many core features such as speculation and HW prefetch
are disabled, and M mode software is responsible of enabling. Hence,
introduce 4 new build time tunable parameters to Kconfig, which reflects
the values get updated to those CSRs:
- ESWIN_EIC770X_FEAT0_CFG
- ESWIN_EIC770X_FEAT1_CFG
- ESWIN_EIC770X_L1_HWPF_CFG
- ESWIN_EIC770X_L2_HWPF_CFG
The default values are somewhat optimal for generic workloads. They are
dumped when running SiFive's vendor OpenSBI, and in addition, with my
own tuning to address the perf regression reported by drmpeg [3]
To build the firmware+u-boot blob, Use the following, and docs [4] for
testing it with UART boot without flashing:
make FW_TEXT_START=0x80000000 \
FW_PAYLOAD_OFFSET=0x200000 \
FW_PAYLOAD_PATH=u-boot-nodtb.bin \
FW_PAYLOAD_FDT_ADDR=0xf8000000 \
FW_FDT_PATH=u-boot.dtb
[1] https://github.com/eswincomputing/EIC7700X-SoC-Technical-Reference-Manual
[2] https://github.com/geerlingguy/sbc-reviews/issues/82
[3] https://forums.sifive.com/t/low-1-core-stream-bandwidth/7274/15
[4] https://github.com/ganboing/EIC770x-Docs/blob/main/p550/bootchain/UART-Boot.md
Signed-off-by: Bo Gan <ganboing@gmail.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20251218104243.562667-6-ganboing@gmail.com
Signed-off-by: Anup Patel <anup@brainfault.org>
By default the OpenSBI itself is covered by 2 memregions for RX/RW
sections. This is required by platforms with Smepmp to enforce
proper permissions in M mode. Note: M-mode only regions can't
have RWX permissions with Smepmp. Platforms with traditional PMPs
won't be able to benefit from it, as both regions are effectively
RWX in M mode, but usually it's harmless to so. Now we provide
these platforms with an option to disable this logic. It saves 1
PMP entry. For platforms really in short of PMPs, it does make a
difference.
Note: Platform requesting single OpenSBI memregion must be using
traditional (old) PMP. We expect the platform code to do
the right thing.
Signed-off-by: Bo Gan <ganboing@gmail.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
Link: https://lore.kernel.org/r/20251218104243.562667-5-ganboing@gmail.com
Signed-off-by: Anup Patel <anup@brainfault.org>
sbi_load/store_access_handler now tries to call platform emulators
if defined. Otherwise, redirects the fault. If the platform code
returns failure, this means the H/S/U has accessed the emulated
devices in an unexpected manner, which is very likely caused by
buggy code in H/S/U. We redirect the fault, so lower privileged
level can get notified, and act accordingly. (E.g., oops in Linux)
We let the handler truly fail if the trap was originated from M mode.
In this case, something must be very wrong and we should just fail.
Signed-off-by: Bo Gan <ganboing@gmail.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
This patch allows the platform to define load/store emulators. This
enables a platform to trap-and-emulate special devices or filter
access to existing physical devices.
Signed-off-by: Bo Gan <ganboing@gmail.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
This patch abstracts out the instruction decoding part of misaligned ld/st
fault handlers, so it can be reused by ld/st access fault handlers.
Also Added lb/lbu/sb decoding. (previously unreachable by misaligned fault)
sbi_trap_emulate_load/store is now the common handler which takes a `emu`
parameter that is responsible for emulating the misaligned or access fault.
The `emu` callback is expected to fixup the fault, and based on the return
code of `emu`, sbi_trap_emulate_load/store will:
r/wlen => the fixup is successful and regs/mepc needs to be updated.
0 => the fixup is successful, but regs/mepc should be left untouched
(this is usually used if `emu` does `sbi_trap_redirect`)
-err => failed, sbi_trap_error will be called
For now, load/store access faults are blindly redirected. It will be
enhanced in the following patches.
Signed-off-by: Bo Gan <ganboing@gmail.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
This simplifies both handlers such that when the handler needs to
redirect the original trap, it's readily available.
Signed-off-by: Bo Gan <ganboing@gmail.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
sbi_trap_redirect now uses const pointer to `trap`.
This ensures the caller that we never change `trap` in sbi_trap_redirect.
Signed-off-by: Bo Gan <ganboing@gmail.com>
Reviewed-by: Anup Patel <anup@brainfault.org>
