Evidence appendix 29 entries, updated 6 September 2026
About this page
A plain list of the claims I make about my own work, each with a link to the original public record: a merged commit, a review thread, or a published DOI. Nothing here is summarised from memory; every line links to a source you can check yourself.
Where an earlier claim of mine turned out to be wrong, the entry says so rather than being quietly deleted. Kernel commits are linked twice, once to git.kernel.org and once to the GitHub mirror of the same tree, because git.kernel.org's bot check can take a while on an older machine.
The Linux kernel mailing list requires patches to be signed off with a real legal name, so kernel and IOMMU work below is under Jiaxing Hu. Everywhere else, including GitHub and this site, I go by my handle, gahingwoo (Ga Hing Woo). "Jiaxing Hu" alone is a common name that returns many unrelated people, so searching gahingwoo or following the direct links below is the more reliable way to verify any of this.
Summary
| Claim | Accepted or checked by | Status |
|---|---|---|
| iommu/rockchip: two fixes | Will Deacon, IOMMU co-maintainer, own Signed-off-by | Mainline, v7.3-rc1 |
| arm64 dts: ArmSoM CM5, CM5-IO | Heiko Stübner, Rockchip SoC maintainer; Acked-by Krzysztof Kozlowski | Mainline, v7.3-rc1 |
| net/phy motorcomm: YT8521 refclk | Jakub Kicinski, net maintainer; Reviewed-by Andrew Lunn; Tested-by Gavin Gao | Mainline, v7.3-rc1 |
| OP-TEE: RK3576 platform port | Reviewed-by Cherry Embedded Solutions; validated on Flipper One, 35,681 subtests, 0 failures | Merged |
| TF-A: secure-boot override removed | Reviewers at STMicroelectronics, Arm, Rockchip | Merged |
| RK3576 NPU enablement, 14 patches | Reviewed-by ×5, Acked-by ×2, Tested-by ×5, four of them on RK3588; the driver's author has said he will review it | v13, under review |
| NPU bring-up preprint, version 2 | Zenodo DOI; five findings cross-confirmed on RK3588 by an independent effort | Published; v1 result withdrawn |
| SoC-Consistency, RKDevelopTool-GUI, kiln | Arch AUR (taotieren); Radxa official documentation | Packaged by others |
Merged upstream 2
- Fixed a copy-paste build-configuration defect that would have silently broken secure-boot on any build without a secure OS. Reviewed by engineers at STMicroelectronics, Arm, and Rockchip. Merged into mainline.
- Ported the open-source Trusted Execution Environment to a new Rockchip chip. Passed all 57 automated checks, independently reproduced on separate hardware, and merged into mainline with a formal Reviewed-by from a Cherry Embedded Solutions maintainer. The platform code itself is in the master branch.
Merged into mainline Linux 3
iommu/rockchip: two fixes applied by the IOMMU co-maintainer
Applied, mainline since v7.3-rc1Found while bringing up the RK3576 NPU. The IOMMU driver only requested a fixed {aclk,iface} clock pair, but the RK3576 NPU's IOMMU instances sit behind additional CBUF/DSU gates, so register writes were silently dropped until those clocks also ran; fixed by taking every clock the devicetree provides instead of a fixed pair. Separately, boot firmware could leave an IOMMU bank in PAGE_FAULT_ACTIVE with no handler, which made that bank ignore CMD_ENABLE_STALL and hang the stall-enable poll; fixed by acknowledging the stale fault first. Split out of the RK3576 NPU RFC below and applied directly by Will Deacon (IOMMU subsystem co-maintainer) into the IOMMU tree, with his own Signed-off-by on both commits. Both are now in Linus's mainline tree. Checked rather than assumed: querying the v7.2 tag's own history for my name returns nothing, and Linus tagged v7.2 on 16 August 2026, so these landed in the 7.3 merge window that opened after it. Neither commit carries a Fixes: line or a stable Cc, so neither is queued for a 7.2.x point release.arm64: dts: rockchip: ArmSoM CM5 and CM5-IO board support, applied by the Rockchip SoC maintainer
Applied, mainline since v7.3-rc1Devicetree support for the ArmSoM CM5 compute module and CM5-IO carrier board, through five review rounds. Sashiko, an automated AI patch-review tool used on the list, flagged a real hardware-sequencing risk in the GMAC0/YT8531 clock ordering, resolved by the motorcomm PHY fix below. The dt-bindings patch also carries an Acked-by from Krzysztof Kozlowski (devicetree bindings maintainer). Applied by Heiko Stübner (Rockchip SoC maintainer) into his own tree on thev7.3-armsoc/dts64branch, with his own Signed-off-by on both commits; he adjusted styling and fixed a couple of dt-checker warnings on the way in. Both have since reached Linus's mainline tree in the 7.3 merge window, and neither is reachable from the v7.2 tag. A third patch corrects two errors I had put in the CM5-IO header comment myself: it namedcombphy0as the USB-C SuperSpeed path whencombphy0is the M.2 PCIe lane and the USB-C path is the USBDP PHY, and it said two USB-A ports when the RTS5411S hub fans the single USB3 lane out to four. Comment only, no functional change. Applied by Heiko Stübner with his own Signed-off-by and now inlinux-next; it is not in Linus's tree yet and is queued for the next merge window, so it is not one of the five commits counted as mainline elsewhere on this site.net: phy: motorcomm: reference-clock fix, applied by the net maintainer
Applied, mainline since v7.3-rc1Redesigned fix for a crystal-less RGMII PHY problem (see the withdrawn dwmac-rk attempt below), enabling the clock from the PHY driver itself. Tested on an ArmSoM CM5-IO: the YT8531 links at 1000 Mbit/s with no MAC-driver or devicetree-binding change. Carried a formal Reviewed-by from Andrew Lunn on v1. A v2 extended the fix to the YT8521 after Gavin Gao hit the identical problem on a different RK3576 board using that PHY, and Gavin came back with a Tested-by on his own hardware (PicoCOM RK3576 ACP). While v2/v3 were in review (Alexey Charkov of Flipper Devices flagged a needless helper function; Jakub Kicinski, the net maintainer, asked for a rebase), a different contributor's equivalent YT8531 fix, for an unrelated board, landed in net-next first as commit 42310a24389c. That's independent confirmation the diagnosis was right, even though someone else's patch text got there first. v4 dropped the now-redundant YT8531 hunk, kept the YT8521 addition with the Reviewed-by and Tested-by carried forward, and was applied by Jakub Kicinski himself into net-next, with his own Signed-off-by. It has since reached Linus's mainline tree in the 7.3 merge window, and is not reachable from the v7.2 tag.
Withdrawn on maintainer guidance 2
iommu/rockchip: implement .flush_iotlb_all
Withdrawn, led to RFC belowFound while bringing up the RK3576 NPU driver: Rockchip's IOMMU never implemented an optional TLB-flush op the hardware could support. Posted a standalone fix; Will Deacon and Robin Murphy (IOMMU subsystem maintainers) pushed back that a standalone flush_iotlb_all isn't the right direction without also moving invalidation into a proper iotlb_sync/gather path. I agreed and withdrew it rather than defend a fix I no longer thought was correct. Heiko Stübner (Rockchip maintainer) used the thread to suggest posting the wider RK3576 bring-up as an RFC instead, which is what became the 9-patch series below.net: stmmac: dwmac-rk: reference-clock fix, wrong layer
Withdrawn, redesigned abovePosted a fix for boards where the RGMII PHY needs a SoC-sourced 25 MHz reference with no local crystal, routing the clock enable through the MAC driver. Maxime Chevallier (Bootlin) and Andrew Lunn (net/phy subsystem maintainer) pointed out that the PHY, not the MAC, should own and request this clock. Agreed, and withdrew the series in favour of fixing it at the correct layer: the motorcomm PHY driver, above, which is what ended up in mainline.
Under review 3
Linux kernel: RK3576 NPU (rocket driver) enablement, PATCH v13
PATCH v13, posted 15 September 2026, no human reply yet · Reviewed-by ×4 · Acked-by ×2 · Tested-by RK3588 ×3 · earned on v7 to v10Series adding RK3576 NPU support across device-tree bindings, the Rockchip power-domain driver, the IOMMU andaccel/rocket. Now at v13, 14 patches, posted 15 September 2026 onnext-20260914. Tomeu Vizoso, who wroteaccel/rocket, replied to v10 on the morning of 31 August: he intends to review this and the other pending rocket series, and asked for a resubmit with Igor Paunovic's clock patch bundled in rather than referenced by aprerequisite-patch-id:trailer, because Sashiko, the automated reviewer used on the list, cannot follow one. v11 went out an hour later carrying that patch as 01/14 under Igor's own name, based onnext-20260814. Sashiko then reviewed all nine mails of the series, which is what v11 was for. v7 was the first revision sent as PATCH rather than RFC: the behaviour every earlier cover letter described as unsolved is solved. The root cause wasPC_TASK_CON's field layout: the RK3576 uses a 16-bit task number where the RK3588-derivedrocket_registers.hassumes 12, so0x00007001was read by the hardware as a request for 28673 tasks with the task-count-clear pulse landing on a reserved bit. Found by taking an ordered trace of every register write in one submit and diffing it against the vendor driver's on the same board, aligned on values rather than on register addresses: exactly one word differed in the whole submit. Rockchip's Chaoyi Chen confirmed the layout on the list three days later, including a fourth control at BIT(18) that a trace could not have named. The claim carried in v1 through v6 that the RK3576's completion interrupt never reaches the GIC was wrong; I sent a correction to the list the same evening, before the next revision, and v7 removes the polled completion path entirely. Review so far: Krzysztof Kozlowski gave a Reviewed-by on the NPU binding, v7 02/10 carried to v8 04/12, plus a separate review with change requests on the power-domain binding; Igor Paunovic gave a Reviewed-by on v8 03/12, reached by rebuilding the three source states at-O1and-O2rather than by reading them, and separately a Tested-by on v7 01/10 on RK3588 hardware I don't own: 2596 inferences and 111048 completion interrupts across three cores, with an oracle that varies the input between runs so a stale output buffer can't pass as a recomputation. That Tested-by was carried forward to v8 01/12 unchanged, and he stated on the thread that it did not extend to the new patch, 02/12. He then withdrew that limit by measuring: on 19 August he ran an induced-reset test on RK3588, could not reproduce the RK3576 non-recovery, and after a differential base run gave a second Tested-by on 02/12 scoped in the tag itself,# RK3588, three cores, induced reset, differential base, JOB_TIMEOUT_MS=2. That exchange also turned up a real bug on my side: a three-run A/B/A table on the ROCK 4D,put_noidlethenput_autosuspendthenput_noidle, scored 0 of 128 with anMMU_DTE_ADDRfault, then 128 of 128 clean, then 0 of 128 again. A fourth run of mine was inconsistent with all three and I declared it void rather than average it in.pm_runtime_put_autosuspend()becomes its own patch in v9. It was drafted carrying Igor's Reported-by; the pre-send review described below established that no such report exists and the tag was dropped. Uwe Kleine-König asked for<linux/device-id/of.h>in place ofmod_devicetable.hand that is taken. Rob Herring's DT bot reporteddt_binding_checkerrors and asked for a re-submit; all three of its reports are in files this series does not touch, Krzysztof Kozlowski said on the thread that they are most likely false positives from unrelated breakage, and v9 was written to carry the base and dependency as a git note under the---line rather than argue the point. It does not. Igor Paunovic noticed while applying the series that the posted mail has no Notes section at all, most likelyformat-patchrun without--notes, so the dependency Rob's bot asked to see recorded in the patch is still not recorded in it. v9 went out on 24 August with thirteen patches, and drew two reviewers v8 never had. Abel Vesa (Qualcomm, power-domain subsystem) gave a Reviewed-by on both pmdomain patches, 08/13 and 09/13, twenty-two minutes after the series was posted. Conor Dooley gave an Acked-by on both devicetree bindings, 06/13 and 07/13, five hours later. A review of all thirteen before posting found one thing that blocks a send: 02/13 introduced the first register accessrocket_reset()has ever made, and that function holds no runtime PM reference, so with the domain down that write takes an async SError, which is the failure two other patches in the same series describe from the other side. It is guarded bypm_runtime_get_if_active()now. The same review removed a Reported-by from 03/13: checkpatch wanted aLink:for it, the report could not be found, and the list archive showed there was none, so the tag would have put an event into the permanent record that never happened. It also caught a blank line betweenSigned-off-byandReviewed-byon 04/13, whichb4reads as the end of the trailer block and which would have silently dropped Igor's tag. Sashiko's review of v11 raised four findings against this series, and the honest split is two, one and one. Two are real and are fixed: 11/14 dereferencedof_device_get_match_data()with no NULL check, used unchecked in eight places; and 03/14 testedpm_runtime_get_if_active() > 0, which is worse than the severity it was filed at, because that call has three answers rather than two. It returns 1 for active, 0 for suspended, and-EINVALwhen runtime PM is not managing the device at all, and this driver's Kconfig carries nodepends on PMwhile its callbacks come throughRUNTIME_PM_OPS, which compiles away atCONFIG_PM=n. On such a build the stub returns-EINVALunconditionally, the test is never true, and theINTERRUPT_MASKwrite never happens, so the protection the patch exists to add is silently absent in exactly the configuration where the fault it guards against could not occur. One finding is open and the code makes the case against it more strongly than the bot did. One is wrong. The rest of what Sashiko reported is pre-existing and belongs to the upstream driver rather than to this series: aniommu_groupreference leak in the job completion path named in all nine mails, a shared IRQ handler touching registers without checking PM state in most of them, and a missingsynchronize_irq()in runtime suspend in three. On 12 September Igor Paunovic withdrew the evidence this entry rested on, by aggregating data he already had. His script kept the scorer output of every inference in every round and never aggregated it; his summaries scored only the one inference issued after a forced autosuspend. Aggregated, the all-0x80 result is in the rounds of nearly every run, on every arm, on all three dates, so it is not a differential signal. It is what a job cancelled by the reset looks like from userspace in this protocol:rocket_reset()completes the detached jobs with-ECANCELED,PREP_BOmaps any positive return ofdma_resv_wait_timeout()to 0 whether or not the fence carries an error, and teflon's output conversion turns the buffer nobody wrote into 0x80. A kprobe run on 12 September puts the run's only two cancellations in its only two all-0x80 rounds, 5.3062 s apart on one clock and 5.3057 s on the other; in the runs where only the scorer output survives, that identification is inference rather than observation. What this retires: the 25 August reading that the silent failure appeared only on the arm without the patches, and with it the claim that the race had at last been observed. What stands, in his words: 45 induced resets on 19 August, 102 on 25 August and 74 on 12 September, every reset recovered, no MMU faults, no lockdep report from rocket or the scheduler in the runs where lockdep was still armed, and of the 420 inferences scored, 384 matched the CPU reference within 1 on all 48 output channels while 36 returned the all-0x80 buffer of a job the reset had cancelled. The protocol bounds; it does not prove. So 2/14 and 3/14 rest on source analysis again; v13 replaced v12's withdrawn reading with the paragraph above and kept his tags. No patch of the fourteen has been applied anywhere. Earlier revisions carried a different and also wrong localisation, a single register namedOPERATION_ENABLE; that turned out to be a consequence rather than a cause. Nine revisions in thirty-eight days, and the review traffic is the reason: Rockchip's Chaoyi Chen replied to four of the nine patches within ninety minutes of the first version going out on 17 July, including “this doesn't look like a correct implementation” on the driver patch, and a correction on the binding that I took, to makesram-supplyconditional on the compatible rather than delete it. The second version drew Krzysztof Kozlowski, Will Deacon, Heiko Stübner and Alexey Charkov; Igor Paunovic has been on it from the third. All nine cover letters are in the list archive, reachable from any thread below. Posted to the mainline kernel list; not yet applied to a maintainer tree. v13 went out on 15 September at 22:43 NZST, onnext-20260914. 3/14 and 4/14 drop the claims Igor retracted, and 2/14 to 4/14 now carry one Tested-by comment,# RK3588, three cores, induced reset, JOB_TIMEOUT_MS=2, with no differential. 3/14 also has the one code change: in v12 its two newPCregister writes sat outsidejob_lock, so a submit from the IRQ thread could re-arm the interrupt mask after the reset had cleared it. They now sit inside ascoped_guard, withsynchronize_irq()left outside. That change postdates Igor's Tested-by on 3/14, and the cover says so and offers to drop the tag. The other thirteen diffs are byte for byte v12's. Where v13 stands, checked 15 September 2026. The thread holds thirty-four messages: thirty-one mine, which is the fifteen mails of the series with the archive listing each more than once, and three fromsashiko-bot. None from a human yet. Sashiko's three are a use-after-free ofpd->resetson the pmdomain probe error path in 10/14, which is new; the asynchronous put in 4/14 again, which the cover already answers; and thepm_runtime_get_if_active()gate in 3/14 again, now with the system-suspend path added. I have not replied to them yet. What changed since v11, and what Sashiko made of it. 3/14 clears the raw interrupt status beside the mask, and keeps the> 0test: a draft that changed it to!= 0is withdrawn, becausepm_runtime_get_conditional()testspower.disable_depthfirst, so-EINVALhides a suspended device rather than excluding one, and this driver reaches that state twice. 10/14 cycles the resets before the settle delay rather than after, which changes code Abel Vesa reviewed; his tag is kept and the cover offers to drop it. 11/14 checks the match data before anything is allocated. 13/14 givesCLK_RKNN_DSU0a rate, because at the 750 mV the ROCK 4D boots with, two cores at once get thirteen to twenty rows of a 5400 row pass wrong, and 594 MHz is clean. 14/14 enables both cores and both IOMMUs, which v11 left to whoever could test it. Sashiko replied to ten of the fourteen and marked most of what it found pre-existing inaccel/rocket; two findings against this series are open, 9/14 forcing the NPU parent domain off at probe and therockchip,rk3568-iommufallback on 8/14. The one finding I had to measure.pm_runtime_put_autosuspend()is asynchronous with a 50 ms delay, so on a workload that submits faster than that the domain would never cycle and 10/14's power-on pulse would never fire. Over one 60.8 s decode of Phi-3.5-mini on a ROCK 4D, genpd's npu domain read 46.8 s active and 14.2 s idle, which sum to the wall clock, with the idle-state usage count up 202 and npu0 and npu1 up 210 and 217. The domain cycles about two hundred times a minute and the pulse fires. Whether it cycles after a timed-out job is the half still unmeasured, because inducing one needs a rebuilt kernel and a flash, and the cover says that rather than letting the measurement cover both halves. That answer is what filled the placeholder the send script had been refusing to send on. Where v12 stood, checked 13 September 2026. The thread holds forty-seven messages: thirty-three mine, ten fromsashiko-bot, and four from Igor Paunovic, whose clock patch is 01/14, which are two mails the archive lists twice. Nothing yet from the driver's author. Igor Paunovic corrected his own test reports, on the thread. Re-running his 19 August protocol against v12 he found that his script had kept the scorer output of every inference per round and never aggregated it, while his summaries scored one inference a round, so a bound he had given and a sentence about 19 August both fall. He then gave the wording he wants carried in v13: 45 induced resets on 19 August, 102 on 25 August and 74 on 12 September, every reset recovered, and of 420 scored inferences 384 matching the CPU reference within 1 on all 48 output channels while 36 returned the all-0x80 buffer of a job the reset had cancelled, with his own caveat kept as the last sentence, that the protocol bounds and does not prove. Those counts are quotable now because he settled them, which is not where they stood a day ago. Where v11 stood, checked 3 September 2026. Three days after it went out, the v11 thread holds forty messages: thirty-one of them mine, nine fromsashiko-bot, and none from a human. Tomeu Vizoso's statement of intent to review was made on the v10 thread and has not yet been followed by a reply on v11. The maintainer-engagement claim in the status line above refers to that earlier statement and to the review traffic on v7 through v10, not to anything v11 has drawn on its own. What Sashiko found, and what I did with it. The bot filed four findings against this series. Two are valid and are fixed in the v12 tree: 11/14 dereferencedof_device_get_match_data()with no NULL check, and 03/14 testedpm_runtime_get_if_active() > 0, which has three answers rather than two, so on aCONFIG_PM=nbuild the stub returns-EINVALand the interrupt-mask write the patch exists to add never happens. That second one is worse than the severity it was filed under. One finding is open pending a board result. One I judge to be wrong and have not acted on. A further set of findings repeats across most of the nine mails and belongs to the upstream driver rather than to this series, including aniommu_groupreference leak in the completion path; those are Tomeu's to take or leave and are not mine to fix in this series. v12 went out on 12 September at 18:51 NZST, v11 with those three commits amended, plus both cores and both IOMMUs enabled on the ROCK 4D.lore.kernel.org: PATCH v13 00/14 lore.kernel.org: PATCH v12 00/14 lore.kernel.org: Igor Paunovic corrects his own 19 and 25 August reports lore.kernel.org: PATCH v11 00/14 lore.kernel.org: Tomeu Vizoso on v10, asking for the resubmit lore.kernel.org: PATCH v9 00/13, the round that drew Abel Vesa and Conor Dooley lore.kernel.org: PATCH v8 00/12, the review round lore.kernel.org: PATCH v7 00/10, for comparison lore.kernel.org, correction: the interrupt claim was wrong lore.kernel.org: Chaoyi Chen (Rockchip) confirms the field layout lore.kernel.org: RFC v1 0/9, the first version, for comparison lore.kernel.org: RFC v6 0/9, the last version carrying the claim I withdrewMesa: RK3576 NPU support in the Teflon delegate, merge request !43804
Posted, not reviewed, not mergedThe first upstream-shaped slice of the userspace half of the stack: five patches, 648 lines over eight files, for one regular convolution on therocket/Teflon path. Posted to freedesktop.org GitLab; not reviewed, not merged. It is narrow on purpose, because the maintainer asked for the bare minimum for a single convolution first, so it declines the depthwise, pointwise and image-input types MobileNet is actually built from and those follow in later series. The end-to-end MobileNet result in the preprint cannot be reproduced from this merge request, only from the development branch listed under Independent projects below.media: rockchip: RFC V4L2 driver for the RK3576 VEPU510 H.264 encoder
Rfc, design pivot after reviewRFC for a from-scratch mainline video-encoder driver. Reviewed by Heiko Stübner (Rockchip kernel maintainer) and Nicolas Dufresne (Collabora, GStreamer/V4L2 maintainer), with Paul Kocialkowski (upstream stateless-codec author) also weighing in. Based on that discussion, decided to abandon the stateful V4L2 interface and converge instead on a DRM/Vulkan-Video-style kernel/userspace split being developed for the sibling RK3588 chip, judged the better long-term direction rather than defend the original design.
Reviews given to other contributors 1
accel/rocket: Reviewed-by given to another contributor, twice, on real bugs
Reviewed-by given ×2As the RK3576 NPU work drew other people to the accel/rocket driver, Igor Paunovic asked before sending a fix, rather than after, and I gave two patches a formal Reviewed-by after checking them properly rather than rubber-stamping. The first (requesting clocks by name instead of by index) I cross-checked against the devicetree binding and my own board-tested RK3576 values. The second (fixing a shared-device lifecycle bug on probe failure) I did not wave through on v1: I traced a real out-of-bounds write in the driver's scheduler array back to a stale "live" core slot left over from a partial probe failure, with the exact one-line fix, and only gave the Reviewed-by once v2 addressed all three points. It came back around: Igor then gave my own RK3576 NPU RFC v3 a Tested-by on RK3588 hardware I don't have, including 4260 bit-identical inference iterations against the stock driver, a genuine two-way review relationship in the subsystem, not a one-off. When Igor posted his own RFC on an RK3588 DVFS hardware constraint, I brought in a third contributor's (Olaf001au) vendor-BSP clock trace to correct an assumption about which clock the driver actually scales, and cross-checked the constraint against RK3576, discussion and analysis rather than a formal tag, but the same pattern of being asked in.
Published research 1
RK3576 NPU bring-up: preprint, version 2 (version 1's central result withdrawn)
Zenodo, DOI · v1 result withdrawn in v2Version 1 was wrong about the thing it was mainly about, and version 2 says so on its first page. Version 1 (July 2026) reported the first int8 convolution from the open driver stack on this chip, described then as byte exact, and then a negative result: that chained-layer compute failed somewhere below what software could reach, that the completion interrupt could not reach the GIC, and that going further needed RTL access or JTAG. Those three are false. The word byte exact is withdrawn too, separately and for a different reason: the convolution is correct on every output channel to within one count, which is not the same word, and counted strictly on three inputs it is 204788, 204760 and 204767 identical pixels of 204800. The cause of the negative result was a register field-layout error in software:PC_TASK_CONcarries a 16-bit task number on the RK3576 where the RK3588-derived register header assumes 12, so the driver's write asked the hardware for 28673 tasks and the count-clear pulse landed on a reserved bit. Version 2 withdraws those claims by name and reports the working stack instead: MobileNet V1 runs end to end on the NPU and returns the CPU's class, 754, with the CPU's top five in the same order. The channel count that reads better, 1000 of 1001 outputs within one count, is a degenerate metric and version 2 says so, because an empty output buffer scores 983 of 1001 against the same reference, measured. The label is what settles it, not the channel count. Most of version 2's length goes on why a strict disprove-first method held a false conclusion for thirty-eight days: an output-buffer oracle that could not fail, a register diff keyed on the address rather than the value, and an audit pointed at the development tree rather than the tree being posted. Version 1 stays retrievable at its own version DOI so the two can be compared. The CNX Software write-up below quotes the withdrawn result and predates the correction; the Beltrami summary is downstream of it. Separately, gregordinary (reverse-engineering the sibling RK3588 by a different route, from Mesa-Teflon capture rather than vendor model files, with no contact with this project) lists five findings of this work as independently cross-confirmed on his own silicon in hisSOURCES.md: IOMMU attach-once rather than per-job, the ping-pong producer/consumer register groups, requant as a load-bearing right shift, per-channel zero-point correction in a weight-buffer tail, and the byte counters being readable on RK3576 where RK3588 nulls them. That cross-validation is of the register model and survives the withdrawal; it was never evidence for the causal claim that version 2 retracts.doi.org/10.5281/zenodo.21990992 (version 2) doi.org/10.5281/zenodo.21348016 (concept DOI, always resolves to the latest version) doi.org/10.5281/zenodo.21348017 (version 1, the withdrawn result) CNX Software coverage (of v1, pre-correction) Istituto Radiotecnico A. Beltrami write-up gregordinary/rockchip-npu-notes: SOURCES.md (five findings cross-confirmed on RK3588)
Independent projects 4
The findings ledger: a public record of how the work was actually done, retractions included
392 rounds, 17+ self-retractionsNewest-first: 177 entries over 7186 lines, 172 of them carrying a date, with numbered rounds running to 392 as of 12 September 2026. The file itself was last written on 12 August at round 121, and every round after that lives in a commit message or an on-board script rather than in the file, with one harness rewrite having deleted 4880 lines of the second of those. Every round since the depthwise work states its decision rule before the run and carries a control that can fail. At least 17 entries exist to retract, withdraw or correct an earlier reading of my own, several caught by their own controls rather than by a later experiment: an on-chip weight-SRAM theory, a ping-pong hypothesis, a stale-TLB theory whose supposed fix turned out never to have run, a power-domain root cause already implemented a month earlier, and a claim that a zero-point surface was being written when nothing was being written at all. A separate provenance file tags every hardware value CONFIRMED or UNVERIFIED with its source line, so a reader can tell which addresses, clock IDs and interrupt numbers were read out of a binding and which were established by observation. This is the artifact behind the preprint, and it is more use than the preprint if what you want to check is method rather than results.mesa-rk3576: the rocket/Teflon userspace every result was measured on
Development branch, not upstreamBranchrk3576, 103 commits of Teflon androcketwork on top of Mesa. Fifteen register and buffer generation faults reported in the preprint were fixed here, each one individually smaller than the resolution of the oracle then in use, and each left behind an environment variable that restores the old behaviour so the fault can be reproduced rather than taken on trust. This is the tree every number in the preprint came from, including the MobileNet classification, which does not run on the posted Mesa branch because that branch declines every convolution in the model. It is not upstream and is not proposed for upstream as it stands; the narrow slice that is proposed is merge request !43804 above.charsiu: an open LLM runtime on the mainline NPU driver
Byte-exact int8 matmul, work in progressOpens/dev/accel/accel0through mainlinerocketdirectly: no Mesa and no vendor runtime anywhere in the execution path. Packs the operands into the hardware's tile layouts, builds the coefficient buffer, emits the register stream and submits it. The int8 matmul is byte exact, and the four 64-wide probes behind that claim are not equally good. The dense and impulse references hold five distinct values and prove little on their own. The bias-ramp and negative-MAC references hold 64 distinct values across 64 channels and match on all 64, which is what makes a match mean the channel was computed rather than that both sides happened to hold the same common byte. An earlier version of this entry, and the project's own README, claimed all four. Version 2 of the preprint corrects that, and so does this line. Weight bandwidth measured at 11.9 GB/s, fitted across seven shapes at 32 chained tasks per job. It also carries a tool that reads a vendor.rkllm's register command streams offline, on a desktop with no board, which is how the closed stack's dispatch plan was read: int4 projections split by output channel across both NPU cores, fp16 attention, and an int8 LM head. That tool also produced a reading that was wrong, and this line used to carry it. It reported that every int4 dispatch in a vendor file is a single row, and five rounds of this project built on that. It tookMfrom the row-count register. The vendor emits an int4 weight matmul as a one-row imageMpixels wide, so on that axis the row count is 1 whateverMis, and reading the pixel count instead says 2816 of 3328 int4 streams in the same file are batched, up toM= 80. The wrong reading survived because it is correct for fp16, where rows and pixels agree in 4940 of 4940 streams: an instrument right about everything it could check and wrong about the one thing being asked. Two more tools carried the same bug, including the one that generated the dispatch table the claim rested on. What int4 computes is not a product of numbers: the hardware multiplies the two operands' fp16 bit patterns as signed 16-bit integers and shifts right 16, exact on all 18 measured points. The first int4 matmul on this stack computed on 20 August, at round 280: 64 of 64 words, every channel, against a CPU reference. What had been stopping it was not the hardware and not the layout, it was charsiu's own packer refusingk >= 16. Three controls carry that result rather than the reference alone: reversing the intra-group order scored 0 of 64, the no-half-k control scored 0 of 64 with NPU values identical to the passing arm, which is the model confirming itself rather than new evidence, and int8 stayed at 64 of 64 in the same run. It is narrow and the page says so: that run was K = 64, N = 64, on the dense probe, whose reference is one of the two weak ones described above. K = 32 came back 16 of 64 from a base-table overrun, and N of 24, 40 and 56 reported every word written but still placed whole groups wrong. Round 284 proposedwbytes/4as the pair stride that would cover them and labelled it a hypothesis rather than a reading, because every table behind it had been swept at N = 64, where that expression and the8×Kalready in use are the same number. The next round tested it and refuted it, and not narrowly:wbytes/4left all three failing cases at zero and broke three that had been passing, 16 of 16, 32 of 32 and 48 of 48 going to 0, 0 and 8.8×Kis right everywhere there is data, the three failures are something else, and the tidy reasoning that produced the alternative did not survive one round. Rows above one work too (14313 of 14336 at M=224, and 200344 of 200704 across a whole 56×56 surface) after 36 rounds in which they were read as broken and the hardware was not: the output surface is[n/atom][m][n%atom], which collapses to plainnat exactly one row, and every correctness run had been done at one row. The int4 port also introduced a defect of my own: a w4a16 job left the NPU unable to start the next one. It is now localised to two registers in the RDMA coefficient fetch group, and two earlier attempts to bisect it were thrown out by their own written controls before a third succeeded. That last line used to read "still not a runtime in the full sense: no model runs end to end". As of 25 August it does, and as of 31 August it is nine models. Llama-3.2-1B runs end to end at four bits through this driver and nothing else: 113 tensors, every projection including the 128256-wide output head, with the generated sentence identical to the one the exact arithmetic writes. Nine of nine models on the card now answer a batched prompt exactly as their own token loop does, Phi-3.5 and Gemma-4-E2B among them. Correcting the single-row reading above turned a refusal into a result:w4a16batches, and the prompt is 3.3x. Three things were wrong at once and none of them shows atm= 1, which is why hundreds of decode rounds never met any of them: the axis, a surface length computed fromrowswhere it needed3 × M, and a read order in which the row and one operand trade places. On the board, 113 tensors are exact atmof 2, 4, 16, 32, 48, 64 and 80, worst relative error 5.10e-05, which is float summation order.m= 8 is still wrong on one tensor shape and is refused rather than trusted. Two further faults were found and fixed in the same stretch, each one a law rather than a quirk of an architecture: an odd batch width has no expression on the accumulator surface at all, which is the whole reason two models were wrong and Llama never was; and the two NPU cores corrupt each other when their batched submits overlap, 13 of 16 runs wrong overlapped against 0 of 16 serialised. What stays on the CPU is the embedding lookup, RMSNorm, RoPE, the attention score, softmax and weighted sum, SwiGLU, the residuals and sampling. The CPU decode loop was written first and is the oracle: the NPU computes the same sum and the two runs have to agree exactly, so "the text looks fine" was never allowed to count.rp2350-tz-tee: minimal TrustZone-M isolation example on the RP2350
Self-verified, SWDThe Pico SDK still can't build a Non-Secure binary that runs under a Secure one (open upstream since pico-examples#708), so there was no minimal Secure/Non-Secure starting point. Built one from scratch on a Pico 2: a Secure world guarding a secret behind NSC veneers, verified over SWD rather than trusted from source: a direct Secure-RAM read from the Non-Secure side is caught as a SecureFault with the expected fault-status bits set, not just "the demo prints the right thing." Also documents a build recipe for TF-M on rpi/rp2350 and the DMA-has-its-own-MPU pitfall (the SAU alone doesn't stop Non-Secure code from using DMA to walk around TrustZone). Extends the TF-A/OP-TEE trusted-execution work above down to the microcontroller class.
Tools in use by others 3
- Catches hardware misconfiguration before it reaches physical devices. Ships rule packs for five chip families, runs in CI pipelines. Published on PyPI and independently packaged into Arch Linux by an unaffiliated maintainer. A stranger ran it against their own Orange Pi 5 (RK3588) devicetree, reported two real false-positive rules in the comments; both were root-caused and shipped in a same-thread point release (v1.2.1).
RKDevelopTool-GUI: cross-platform Rockchip flashing GUI
AUR, in Radxa docsA Qt/PySide6 graphical front-end for Rockchip firmware flashing on macOS and Linux. Independently packaged into the Arch AUR by the same community maintainer as SoC-Consistency (taotieren), and pulled in as an optional dependency of the SoC-Consistency AUR package. The two tools now have a real dependency relationship in the wild, not just two separate uploads. Radxa's own documentation now carries a dedicated page for it under CM3 low-level development, with per-platform install steps, positioned in their docs alongside their own rkdeveloptool CLI.lore-enhancer: browser extension for reading kernel patches on lore.kernel.org
Browser extension, GPL-2.0A Chrome/Edge/Firefox extension that reformats lore.kernel.org's raw single-<pre>message pages into a mail-client-style view: author card, collapsible syntax-highlighted diffs, a clickable thread sidebar in place of ASCII art, and quote folding, with an instant "Raw" toggle back to the original. No network requests, no accounts, no telemetry. Posted to r/kernel and r/embedded, where working kernel and embedded developers responded with genuine positive first impressions in the comments, not just self-reported use.
Third-party validation 7
Independent testing and cross-checking on hardware I don't own
Tested-by on RK3588, plus review that found real bugsIgor Paunovic reviewed and tested the NPU series on an Orange Pi 5 Plus (RK3588), hardware I have no access to. He gave a Tested-by on thejob_lockpatch after six phases per module across all three NPU cores, binding and unbinding cores between phases: 2596 inferences and 111048 completion interrupts, scored by the SHA-256 of tensors that change with the input and stay stable across repeats, so a stale output buffer can't pass as a recomputation. He also reads the patches rather than only running them: he found a comment that had outlived its subject and a shared-path refactor sitting in the wrong patch for bisection. He then produced the one comparison I cannot (the same per-operator scoring across all 27 MobileNet operators on RK3588 silicon) and, in August, showed that my scoring reference was wrong for the model's last operator: it compares againstmax(cpu, zp), which I had justified as right for the operators carrying a fused ReLU6 and wrong for the logits BiasAdd, which has none. Version 2 answers that half, and answers it against me. There are no such operators. Every model in this tree isfused_activation NONE, MobileNet included, its ReLU6 folded into the output quantisation range instead, so those layers carry output zero point 0 and for a uint8 tensor at zero point 0 the clamped and unclamped references are the same reference. What stays open is the other half: the clamp at the output zero point is my own stack's rather than the silicon's, and what inside my stack produces it is narrowed to one column and not closed.Five findings independently cross-confirmed on the sibling SoC
Independent, different method, different silicongregordinary reverse-engineered the RK3588 NPU by a different route (from Mesa-Teflon capture rather than from vendor model files) with no contact with this project, and lists five of my findings as cross-confirmed against his own silicon in hisSOURCES.md: the IOMMU being attached once and detached on power-down rather than per job; the ping-pong producer/consumer register groups and what a misaligned pair does; requantisation being a right shift whose magnitude is load-bearing; a per-channel zero-point correction living in a weight-buffer tail in groups of eight output channels; and thedt_wr/dt_rd/wt_rdbyte counters being readable on RK3576 where RK3588 nulls them. His independent count of 16 CBUF banks on RK3576 against 12 on RK3588 also matches mine. Scope matters here: what he confirms is the register model, and it was never evidence for the causal claim my preprint's version 2 retracts.- A complete EDK2/TianoCore UEFI port for the RK3576, booting a mainline Fedora 44 desktop. Builds for six boards, hardware-verified on two (Radxa ROCK 4D, ArmSoM CM5-IO). The write-up was pinned to the ROCK 4 series section of the official Radxa community forum by Zhang Yuntian, a Radxa engineer, not by me.
kiln: official Radxa tutorial for local LLM/vision on the RK3576 NPU
Third-party, official tutorialA deliberate stopgap, not a mainline driver: builds the vendor GPL rknpu driver out-of-tree against a mainline kernel (linux-7.1.3), drives it with the closed librkllmrt/librknnrt runtimes, and carries the RK3576 NPU clock/power-domain/IOMMU kernel patches from above so the NPU comes up at all without the vendor's 6.1 BSP. Radxa host a full step-by-step tutorial for it under ROCK 4D > Application Development > NPU Usage (install, health checks, local LLM chat, vision inference, and an OpenAI-compatible API server), explicitly labelled as a third-party project rather than official support, but given the same tutorial treatment as Radxa's own guides.- Flipper Devices validated the RK3576 OP-TEE port on Flipper One hardware: 114 xtest cases, 35,681 subtests, zero failures. Credited by name in their official developer documentation.
- Collabora (an established Linux kernel and open-source engineering consultancy) posted their own technical summary of the RK3576 NPU work, unprompted. Heiko Stübner, the Rockchip kernel maintainer, commented on the post directly.
- A community-maintained GitLab tracker, hosted by Collabora, for Rockchip RK3576 mainline enablement status across every subsystem (GPU, ISP, media, NPU, and more). The NPU row cites the RFC series as its reference entry, and the VEPU510 H.264/AVC row was later updated by Sebastian Reichel (Collabora) to cite the video-encoder RFC thread as well, not something I added myself, but how the tracker's maintainers chose to record the state of both.
Community deployment 2
EzySpeech: real-time transcription and translation, Mt Albert Baptist Church
In active use, signed referenceBuilt and deployed for live services; still in use. The church's official website now has a "Live Translation" page describing the deployment directly: 21 languages, at both the 9am and 11am Sunday services, linking straight to the tool. The linked service recording shows a live session. Frank Zhang, the church's accountant and translation team leader, wrote a signed reference letter independently confirming the volunteer role and the deployment, including the real-time multi-language translation the tool now provides during services.A replacement for the Mt Albert Baptist website
Live on the church's own domainA rebuild of the church's site: eight pages, plus twenty redirect stubs standing in for the old Wix addresses so that no link the church has already handed out breaks. Built on the same PatternFly 6 as this site and served from GitHub Pages. The church created new.mabc.org.nz under their own domain and pointed it at my GitHub Pages account, which is a change only they could make; a DNS lookup of that name returns gahingwoo.github.io, so the delegation can be checked without asking either of us. It is not their default site yet: www.mabc.org.nz still serves the Wix original, and the decision to switch is theirs.
Peer collaboration 1
- A shared GitHub organisation I co-run with three students my own age, where I guide the group through embedded systems and firmware development while we make technical decisions together. The org's main shipped project, hbes, is a cross-platform dev-environment bootstrap tool (Debian/RHEL/Arch/macOS, a real TUI, a lockfile-tracked install/uninstall model) with CI running the actual installer on four platforms every push, and its README credits all four of us by handle.