Skip to content

Evidence appendix 29 entries, updated 6 September 2026

About this page

A plain list of the claims I make about my own work, each with a link to the original public record: a merged commit, a review thread, or a published DOI. Nothing here is summarised from memory; every line links to a source you can check yourself.

Where an earlier claim of mine turned out to be wrong, the entry says so rather than being quietly deleted. Kernel commits are linked twice, once to git.kernel.org and once to the GitHub mirror of the same tree, because git.kernel.org's bot check can take a while on an older machine.

The Linux kernel mailing list requires patches to be signed off with a real legal name, so kernel and IOMMU work below is under Jiaxing Hu. Everywhere else, including GitHub and this site, I go by my handle, gahingwoo (Ga Hing Woo). "Jiaxing Hu" alone is a common name that returns many unrelated people, so searching gahingwoo or following the direct links below is the more reliable way to verify any of this.

Summary

ClaimAccepted or checked byStatus
iommu/rockchip: two fixesWill Deacon, IOMMU co-maintainer, own Signed-off-byMainline, v7.3-rc1
arm64 dts: ArmSoM CM5, CM5-IOHeiko Stübner, Rockchip SoC maintainer; Acked-by Krzysztof KozlowskiMainline, v7.3-rc1
net/phy motorcomm: YT8521 refclkJakub Kicinski, net maintainer; Reviewed-by Andrew Lunn; Tested-by Gavin GaoMainline, v7.3-rc1
OP-TEE: RK3576 platform portReviewed-by Cherry Embedded Solutions; validated on Flipper One, 35,681 subtests, 0 failuresMerged
TF-A: secure-boot override removedReviewers at STMicroelectronics, Arm, RockchipMerged
RK3576 NPU enablement, 14 patchesReviewed-by ×5, Acked-by ×2, Tested-by ×5, four of them on RK3588; the driver's author has said he will review itv13, under review
NPU bring-up preprint, version 2Zenodo DOI; five findings cross-confirmed on RK3588 by an independent effortPublished; v1 result withdrawn
SoC-Consistency, RKDevelopTool-GUI, kilnArch AUR (taotieren); Radxa official documentationPackaged by others

Merged upstream 2

Merged into mainline Linux 3

Withdrawn on maintainer guidance 2

  • iommu/rockchip: implement .flush_iotlb_all

    Withdrawn, led to RFC below
    Found while bringing up the RK3576 NPU driver: Rockchip's IOMMU never implemented an optional TLB-flush op the hardware could support. Posted a standalone fix; Will Deacon and Robin Murphy (IOMMU subsystem maintainers) pushed back that a standalone flush_iotlb_all isn't the right direction without also moving invalidation into a proper iotlb_sync/gather path. I agreed and withdrew it rather than defend a fix I no longer thought was correct. Heiko Stübner (Rockchip maintainer) used the thread to suggest posting the wider RK3576 bring-up as an RFC instead, which is what became the 9-patch series below.
    lore.kernel.org, iommu/rockchip: flush_iotlb_all
  • net: stmmac: dwmac-rk: reference-clock fix, wrong layer

    Withdrawn, redesigned above
    Posted a fix for boards where the RGMII PHY needs a SoC-sourced 25 MHz reference with no local crystal, routing the clock enable through the MAC driver. Maxime Chevallier (Bootlin) and Andrew Lunn (net/phy subsystem maintainer) pointed out that the PHY, not the MAC, should own and request this clock. Agreed, and withdrew the series in favour of fixing it at the correct layer: the motorcomm PHY driver, above, which is what ended up in mainline.
    lore.kernel.org: dwmac-rk refclk v2 (withdrawn)

Under review 3

  • Linux kernel: RK3576 NPU (rocket driver) enablement, PATCH v13

    PATCH v13, posted 15 September 2026, no human reply yet · Reviewed-by ×4 · Acked-by ×2 · Tested-by RK3588 ×3 · earned on v7 to v10
    Series adding RK3576 NPU support across device-tree bindings, the Rockchip power-domain driver, the IOMMU and accel/rocket. Now at v13, 14 patches, posted 15 September 2026 on next-20260914. Tomeu Vizoso, who wrote accel/rocket, replied to v10 on the morning of 31 August: he intends to review this and the other pending rocket series, and asked for a resubmit with Igor Paunovic's clock patch bundled in rather than referenced by a prerequisite-patch-id: trailer, because Sashiko, the automated reviewer used on the list, cannot follow one. v11 went out an hour later carrying that patch as 01/14 under Igor's own name, based on next-20260814. Sashiko then reviewed all nine mails of the series, which is what v11 was for. v7 was the first revision sent as PATCH rather than RFC: the behaviour every earlier cover letter described as unsolved is solved. The root cause was PC_TASK_CON's field layout: the RK3576 uses a 16-bit task number where the RK3588-derived rocket_registers.h assumes 12, so 0x00007001 was read by the hardware as a request for 28673 tasks with the task-count-clear pulse landing on a reserved bit. Found by taking an ordered trace of every register write in one submit and diffing it against the vendor driver's on the same board, aligned on values rather than on register addresses: exactly one word differed in the whole submit. Rockchip's Chaoyi Chen confirmed the layout on the list three days later, including a fourth control at BIT(18) that a trace could not have named. The claim carried in v1 through v6 that the RK3576's completion interrupt never reaches the GIC was wrong; I sent a correction to the list the same evening, before the next revision, and v7 removes the polled completion path entirely. Review so far: Krzysztof Kozlowski gave a Reviewed-by on the NPU binding, v7 02/10 carried to v8 04/12, plus a separate review with change requests on the power-domain binding; Igor Paunovic gave a Reviewed-by on v8 03/12, reached by rebuilding the three source states at -O1 and -O2 rather than by reading them, and separately a Tested-by on v7 01/10 on RK3588 hardware I don't own: 2596 inferences and 111048 completion interrupts across three cores, with an oracle that varies the input between runs so a stale output buffer can't pass as a recomputation. That Tested-by was carried forward to v8 01/12 unchanged, and he stated on the thread that it did not extend to the new patch, 02/12. He then withdrew that limit by measuring: on 19 August he ran an induced-reset test on RK3588, could not reproduce the RK3576 non-recovery, and after a differential base run gave a second Tested-by on 02/12 scoped in the tag itself, # RK3588, three cores, induced reset, differential base, JOB_TIMEOUT_MS=2. That exchange also turned up a real bug on my side: a three-run A/B/A table on the ROCK 4D, put_noidle then put_autosuspend then put_noidle, scored 0 of 128 with an MMU_DTE_ADDR fault, then 128 of 128 clean, then 0 of 128 again. A fourth run of mine was inconsistent with all three and I declared it void rather than average it in. pm_runtime_put_autosuspend() becomes its own patch in v9. It was drafted carrying Igor's Reported-by; the pre-send review described below established that no such report exists and the tag was dropped. Uwe Kleine-König asked for <linux/device-id/of.h> in place of mod_devicetable.h and that is taken. Rob Herring's DT bot reported dt_binding_check errors and asked for a re-submit; all three of its reports are in files this series does not touch, Krzysztof Kozlowski said on the thread that they are most likely false positives from unrelated breakage, and v9 was written to carry the base and dependency as a git note under the --- line rather than argue the point. It does not. Igor Paunovic noticed while applying the series that the posted mail has no Notes section at all, most likely format-patch run without --notes, so the dependency Rob's bot asked to see recorded in the patch is still not recorded in it. v9 went out on 24 August with thirteen patches, and drew two reviewers v8 never had. Abel Vesa (Qualcomm, power-domain subsystem) gave a Reviewed-by on both pmdomain patches, 08/13 and 09/13, twenty-two minutes after the series was posted. Conor Dooley gave an Acked-by on both devicetree bindings, 06/13 and 07/13, five hours later. A review of all thirteen before posting found one thing that blocks a send: 02/13 introduced the first register access rocket_reset() has ever made, and that function holds no runtime PM reference, so with the domain down that write takes an async SError, which is the failure two other patches in the same series describe from the other side. It is guarded by pm_runtime_get_if_active() now. The same review removed a Reported-by from 03/13: checkpatch wanted a Link: for it, the report could not be found, and the list archive showed there was none, so the tag would have put an event into the permanent record that never happened. It also caught a blank line between Signed-off-by and Reviewed-by on 04/13, which b4 reads as the end of the trailer block and which would have silently dropped Igor's tag. Sashiko's review of v11 raised four findings against this series, and the honest split is two, one and one. Two are real and are fixed: 11/14 dereferenced of_device_get_match_data() with no NULL check, used unchecked in eight places; and 03/14 tested pm_runtime_get_if_active() > 0, which is worse than the severity it was filed at, because that call has three answers rather than two. It returns 1 for active, 0 for suspended, and -EINVAL when runtime PM is not managing the device at all, and this driver's Kconfig carries no depends on PM while its callbacks come through RUNTIME_PM_OPS, which compiles away at CONFIG_PM=n. On such a build the stub returns -EINVAL unconditionally, the test is never true, and the INTERRUPT_MASK write never happens, so the protection the patch exists to add is silently absent in exactly the configuration where the fault it guards against could not occur. One finding is open and the code makes the case against it more strongly than the bot did. One is wrong. The rest of what Sashiko reported is pre-existing and belongs to the upstream driver rather than to this series: an iommu_group reference leak in the job completion path named in all nine mails, a shared IRQ handler touching registers without checking PM state in most of them, and a missing synchronize_irq() in runtime suspend in three. On 12 September Igor Paunovic withdrew the evidence this entry rested on, by aggregating data he already had. His script kept the scorer output of every inference in every round and never aggregated it; his summaries scored only the one inference issued after a forced autosuspend. Aggregated, the all-0x80 result is in the rounds of nearly every run, on every arm, on all three dates, so it is not a differential signal. It is what a job cancelled by the reset looks like from userspace in this protocol: rocket_reset() completes the detached jobs with -ECANCELED, PREP_BO maps any positive return of dma_resv_wait_timeout() to 0 whether or not the fence carries an error, and teflon's output conversion turns the buffer nobody wrote into 0x80. A kprobe run on 12 September puts the run's only two cancellations in its only two all-0x80 rounds, 5.3062 s apart on one clock and 5.3057 s on the other; in the runs where only the scorer output survives, that identification is inference rather than observation. What this retires: the 25 August reading that the silent failure appeared only on the arm without the patches, and with it the claim that the race had at last been observed. What stands, in his words: 45 induced resets on 19 August, 102 on 25 August and 74 on 12 September, every reset recovered, no MMU faults, no lockdep report from rocket or the scheduler in the runs where lockdep was still armed, and of the 420 inferences scored, 384 matched the CPU reference within 1 on all 48 output channels while 36 returned the all-0x80 buffer of a job the reset had cancelled. The protocol bounds; it does not prove. So 2/14 and 3/14 rest on source analysis again; v13 replaced v12's withdrawn reading with the paragraph above and kept his tags. No patch of the fourteen has been applied anywhere. Earlier revisions carried a different and also wrong localisation, a single register named OPERATION_ENABLE; that turned out to be a consequence rather than a cause. Nine revisions in thirty-eight days, and the review traffic is the reason: Rockchip's Chaoyi Chen replied to four of the nine patches within ninety minutes of the first version going out on 17 July, including “this doesn't look like a correct implementation” on the driver patch, and a correction on the binding that I took, to make sram-supply conditional on the compatible rather than delete it. The second version drew Krzysztof Kozlowski, Will Deacon, Heiko Stübner and Alexey Charkov; Igor Paunovic has been on it from the third. All nine cover letters are in the list archive, reachable from any thread below. Posted to the mainline kernel list; not yet applied to a maintainer tree. v13 went out on 15 September at 22:43 NZST, on next-20260914. 3/14 and 4/14 drop the claims Igor retracted, and 2/14 to 4/14 now carry one Tested-by comment, # RK3588, three cores, induced reset, JOB_TIMEOUT_MS=2, with no differential. 3/14 also has the one code change: in v12 its two new PC register writes sat outside job_lock, so a submit from the IRQ thread could re-arm the interrupt mask after the reset had cleared it. They now sit inside a scoped_guard, with synchronize_irq() left outside. That change postdates Igor's Tested-by on 3/14, and the cover says so and offers to drop the tag. The other thirteen diffs are byte for byte v12's. Where v13 stands, checked 15 September 2026. The thread holds thirty-four messages: thirty-one mine, which is the fifteen mails of the series with the archive listing each more than once, and three from sashiko-bot. None from a human yet. Sashiko's three are a use-after-free of pd->resets on the pmdomain probe error path in 10/14, which is new; the asynchronous put in 4/14 again, which the cover already answers; and the pm_runtime_get_if_active() gate in 3/14 again, now with the system-suspend path added. I have not replied to them yet. What changed since v11, and what Sashiko made of it. 3/14 clears the raw interrupt status beside the mask, and keeps the > 0 test: a draft that changed it to != 0 is withdrawn, because pm_runtime_get_conditional() tests power.disable_depth first, so -EINVAL hides a suspended device rather than excluding one, and this driver reaches that state twice. 10/14 cycles the resets before the settle delay rather than after, which changes code Abel Vesa reviewed; his tag is kept and the cover offers to drop it. 11/14 checks the match data before anything is allocated. 13/14 gives CLK_RKNN_DSU0 a rate, because at the 750 mV the ROCK 4D boots with, two cores at once get thirteen to twenty rows of a 5400 row pass wrong, and 594 MHz is clean. 14/14 enables both cores and both IOMMUs, which v11 left to whoever could test it. Sashiko replied to ten of the fourteen and marked most of what it found pre-existing in accel/rocket; two findings against this series are open, 9/14 forcing the NPU parent domain off at probe and the rockchip,rk3568-iommu fallback on 8/14. The one finding I had to measure. pm_runtime_put_autosuspend() is asynchronous with a 50 ms delay, so on a workload that submits faster than that the domain would never cycle and 10/14's power-on pulse would never fire. Over one 60.8 s decode of Phi-3.5-mini on a ROCK 4D, genpd's npu domain read 46.8 s active and 14.2 s idle, which sum to the wall clock, with the idle-state usage count up 202 and npu0 and npu1 up 210 and 217. The domain cycles about two hundred times a minute and the pulse fires. Whether it cycles after a timed-out job is the half still unmeasured, because inducing one needs a rebuilt kernel and a flash, and the cover says that rather than letting the measurement cover both halves. That answer is what filled the placeholder the send script had been refusing to send on. Where v12 stood, checked 13 September 2026. The thread holds forty-seven messages: thirty-three mine, ten from sashiko-bot, and four from Igor Paunovic, whose clock patch is 01/14, which are two mails the archive lists twice. Nothing yet from the driver's author. Igor Paunovic corrected his own test reports, on the thread. Re-running his 19 August protocol against v12 he found that his script had kept the scorer output of every inference per round and never aggregated it, while his summaries scored one inference a round, so a bound he had given and a sentence about 19 August both fall. He then gave the wording he wants carried in v13: 45 induced resets on 19 August, 102 on 25 August and 74 on 12 September, every reset recovered, and of 420 scored inferences 384 matching the CPU reference within 1 on all 48 output channels while 36 returned the all-0x80 buffer of a job the reset had cancelled, with his own caveat kept as the last sentence, that the protocol bounds and does not prove. Those counts are quotable now because he settled them, which is not where they stood a day ago. Where v11 stood, checked 3 September 2026. Three days after it went out, the v11 thread holds forty messages: thirty-one of them mine, nine from sashiko-bot, and none from a human. Tomeu Vizoso's statement of intent to review was made on the v10 thread and has not yet been followed by a reply on v11. The maintainer-engagement claim in the status line above refers to that earlier statement and to the review traffic on v7 through v10, not to anything v11 has drawn on its own. What Sashiko found, and what I did with it. The bot filed four findings against this series. Two are valid and are fixed in the v12 tree: 11/14 dereferenced of_device_get_match_data() with no NULL check, and 03/14 tested pm_runtime_get_if_active() > 0, which has three answers rather than two, so on a CONFIG_PM=n build the stub returns -EINVAL and the interrupt-mask write the patch exists to add never happens. That second one is worse than the severity it was filed under. One finding is open pending a board result. One I judge to be wrong and have not acted on. A further set of findings repeats across most of the nine mails and belongs to the upstream driver rather than to this series, including an iommu_group reference leak in the completion path; those are Tomeu's to take or leave and are not mine to fix in this series. v12 went out on 12 September at 18:51 NZST, v11 with those three commits amended, plus both cores and both IOMMUs enabled on the ROCK 4D.
    lore.kernel.org: PATCH v13 00/14 lore.kernel.org: PATCH v12 00/14 lore.kernel.org: Igor Paunovic corrects his own 19 and 25 August reports lore.kernel.org: PATCH v11 00/14 lore.kernel.org: Tomeu Vizoso on v10, asking for the resubmit lore.kernel.org: PATCH v9 00/13, the round that drew Abel Vesa and Conor Dooley lore.kernel.org: PATCH v8 00/12, the review round lore.kernel.org: PATCH v7 00/10, for comparison lore.kernel.org, correction: the interrupt claim was wrong lore.kernel.org: Chaoyi Chen (Rockchip) confirms the field layout lore.kernel.org: RFC v1 0/9, the first version, for comparison lore.kernel.org: RFC v6 0/9, the last version carrying the claim I withdrew
  • Mesa: RK3576 NPU support in the Teflon delegate, merge request !43804

    Posted, not reviewed, not merged
    The first upstream-shaped slice of the userspace half of the stack: five patches, 648 lines over eight files, for one regular convolution on the rocket/Teflon path. Posted to freedesktop.org GitLab; not reviewed, not merged. It is narrow on purpose, because the maintainer asked for the bare minimum for a single convolution first, so it declines the depthwise, pointwise and image-input types MobileNet is actually built from and those follow in later series. The end-to-end MobileNet result in the preprint cannot be reproduced from this merge request, only from the development branch listed under Independent projects below.
    gitlab.freedesktop.org: mesa!43804
  • media: rockchip: RFC V4L2 driver for the RK3576 VEPU510 H.264 encoder

    Rfc, design pivot after review
    RFC for a from-scratch mainline video-encoder driver. Reviewed by Heiko Stübner (Rockchip kernel maintainer) and Nicolas Dufresne (Collabora, GStreamer/V4L2 maintainer), with Paul Kocialkowski (upstream stateless-codec author) also weighing in. Based on that discussion, decided to abandon the stateful V4L2 interface and converge instead on a DRM/Vulkan-Video-style kernel/userspace split being developed for the sibling RK3588 chip, judged the better long-term direction rather than defend the original design.
    lore.kernel.org: RFC PATCH 0/3

Reviews given to other contributors 1

  • accel/rocket: Reviewed-by given to another contributor, twice, on real bugs

    Reviewed-by given ×2
    As the RK3576 NPU work drew other people to the accel/rocket driver, Igor Paunovic asked before sending a fix, rather than after, and I gave two patches a formal Reviewed-by after checking them properly rather than rubber-stamping. The first (requesting clocks by name instead of by index) I cross-checked against the devicetree binding and my own board-tested RK3576 values. The second (fixing a shared-device lifecycle bug on probe failure) I did not wave through on v1: I traced a real out-of-bounds write in the driver's scheduler array back to a stale "live" core slot left over from a partial probe failure, with the exact one-line fix, and only gave the Reviewed-by once v2 addressed all three points. It came back around: Igor then gave my own RK3576 NPU RFC v3 a Tested-by on RK3588 hardware I don't have, including 4260 bit-identical inference iterations against the stock driver, a genuine two-way review relationship in the subsystem, not a one-off. When Igor posted his own RFC on an RK3588 DVFS hardware constraint, I brought in a third contributor's (Olaf001au) vendor-BSP clock trace to correct an assumption about which clock the driver actually scales, and cross-checked the constraint against RK3576, discussion and analysis rather than a formal tag, but the same pattern of being asked in.
    lore.kernel.org: clocks-by-name (Reviewed-by given) lore.kernel.org: lifecycle fix v1 (bug found) lore.kernel.org: lifecycle fix v2 (Reviewed-by given) lore.kernel.org: RK3588 DVFS RFC (discussion)

Published research 1

  • RK3576 NPU bring-up: preprint, version 2 (version 1's central result withdrawn)

    Zenodo, DOI · v1 result withdrawn in v2
    Version 1 was wrong about the thing it was mainly about, and version 2 says so on its first page. Version 1 (July 2026) reported the first int8 convolution from the open driver stack on this chip, described then as byte exact, and then a negative result: that chained-layer compute failed somewhere below what software could reach, that the completion interrupt could not reach the GIC, and that going further needed RTL access or JTAG. Those three are false. The word byte exact is withdrawn too, separately and for a different reason: the convolution is correct on every output channel to within one count, which is not the same word, and counted strictly on three inputs it is 204788, 204760 and 204767 identical pixels of 204800. The cause of the negative result was a register field-layout error in software: PC_TASK_CON carries a 16-bit task number on the RK3576 where the RK3588-derived register header assumes 12, so the driver's write asked the hardware for 28673 tasks and the count-clear pulse landed on a reserved bit. Version 2 withdraws those claims by name and reports the working stack instead: MobileNet V1 runs end to end on the NPU and returns the CPU's class, 754, with the CPU's top five in the same order. The channel count that reads better, 1000 of 1001 outputs within one count, is a degenerate metric and version 2 says so, because an empty output buffer scores 983 of 1001 against the same reference, measured. The label is what settles it, not the channel count. Most of version 2's length goes on why a strict disprove-first method held a false conclusion for thirty-eight days: an output-buffer oracle that could not fail, a register diff keyed on the address rather than the value, and an audit pointed at the development tree rather than the tree being posted. Version 1 stays retrievable at its own version DOI so the two can be compared. The CNX Software write-up below quotes the withdrawn result and predates the correction; the Beltrami summary is downstream of it. Separately, gregordinary (reverse-engineering the sibling RK3588 by a different route, from Mesa-Teflon capture rather than vendor model files, with no contact with this project) lists five findings of this work as independently cross-confirmed on his own silicon in his SOURCES.md: IOMMU attach-once rather than per-job, the ping-pong producer/consumer register groups, requant as a load-bearing right shift, per-channel zero-point correction in a weight-buffer tail, and the byte counters being readable on RK3576 where RK3588 nulls them. That cross-validation is of the register model and survives the withdrawal; it was never evidence for the causal claim that version 2 retracts.
    doi.org/10.5281/zenodo.21990992 (version 2) doi.org/10.5281/zenodo.21348016 (concept DOI, always resolves to the latest version) doi.org/10.5281/zenodo.21348017 (version 1, the withdrawn result) CNX Software coverage (of v1, pre-correction) Istituto Radiotecnico A. Beltrami write-up gregordinary/rockchip-npu-notes: SOURCES.md (five findings cross-confirmed on RK3588)

Independent projects 4

  • The findings ledger: a public record of how the work was actually done, retractions included

    392 rounds, 17+ self-retractions
    Newest-first: 177 entries over 7186 lines, 172 of them carrying a date, with numbered rounds running to 392 as of 12 September 2026. The file itself was last written on 12 August at round 121, and every round after that lives in a commit message or an on-board script rather than in the file, with one harness rewrite having deleted 4880 lines of the second of those. Every round since the depthwise work states its decision rule before the run and carries a control that can fail. At least 17 entries exist to retract, withdraw or correct an earlier reading of my own, several caught by their own controls rather than by a later experiment: an on-chip weight-SRAM theory, a ping-pong hypothesis, a stale-TLB theory whose supposed fix turned out never to have run, a power-domain root cause already implemented a month earlier, and a claim that a zero-point surface was being written when nothing was being written at all. A separate provenance file tags every hardware value CONFIRMED or UNVERIFIED with its source line, so a reader can tell which addresses, clock IDs and interrupt numbers were read out of a binding and which were established by observation. This is the artifact behind the preprint, and it is more use than the preprint if what you want to check is method rather than results.
    FINDINGS.md notes/provenance.md
  • mesa-rk3576: the rocket/Teflon userspace every result was measured on

    Development branch, not upstream
    Branch rk3576, 103 commits of Teflon and rocket work on top of Mesa. Fifteen register and buffer generation faults reported in the preprint were fixed here, each one individually smaller than the resolution of the oracle then in use, and each left behind an environment variable that restores the old behaviour so the fault can be reproduced rather than taken on trust. This is the tree every number in the preprint came from, including the MobileNet classification, which does not run on the posted Mesa branch because that branch declines every convolution in the model. It is not upstream and is not proposed for upstream as it stands; the narrow slice that is proposed is merge request !43804 above.
    github.com/gahingwoo/mesa-rk3576
  • charsiu: an open LLM runtime on the mainline NPU driver

    Byte-exact int8 matmul, work in progress
    Opens /dev/accel/accel0 through mainline rocket directly: no Mesa and no vendor runtime anywhere in the execution path. Packs the operands into the hardware's tile layouts, builds the coefficient buffer, emits the register stream and submits it. The int8 matmul is byte exact, and the four 64-wide probes behind that claim are not equally good. The dense and impulse references hold five distinct values and prove little on their own. The bias-ramp and negative-MAC references hold 64 distinct values across 64 channels and match on all 64, which is what makes a match mean the channel was computed rather than that both sides happened to hold the same common byte. An earlier version of this entry, and the project's own README, claimed all four. Version 2 of the preprint corrects that, and so does this line. Weight bandwidth measured at 11.9 GB/s, fitted across seven shapes at 32 chained tasks per job. It also carries a tool that reads a vendor .rkllm's register command streams offline, on a desktop with no board, which is how the closed stack's dispatch plan was read: int4 projections split by output channel across both NPU cores, fp16 attention, and an int8 LM head. That tool also produced a reading that was wrong, and this line used to carry it. It reported that every int4 dispatch in a vendor file is a single row, and five rounds of this project built on that. It took M from the row-count register. The vendor emits an int4 weight matmul as a one-row image M pixels wide, so on that axis the row count is 1 whatever M is, and reading the pixel count instead says 2816 of 3328 int4 streams in the same file are batched, up to M = 80. The wrong reading survived because it is correct for fp16, where rows and pixels agree in 4940 of 4940 streams: an instrument right about everything it could check and wrong about the one thing being asked. Two more tools carried the same bug, including the one that generated the dispatch table the claim rested on. What int4 computes is not a product of numbers: the hardware multiplies the two operands' fp16 bit patterns as signed 16-bit integers and shifts right 16, exact on all 18 measured points. The first int4 matmul on this stack computed on 20 August, at round 280: 64 of 64 words, every channel, against a CPU reference. What had been stopping it was not the hardware and not the layout, it was charsiu's own packer refusing k >= 16. Three controls carry that result rather than the reference alone: reversing the intra-group order scored 0 of 64, the no-half-k control scored 0 of 64 with NPU values identical to the passing arm, which is the model confirming itself rather than new evidence, and int8 stayed at 64 of 64 in the same run. It is narrow and the page says so: that run was K = 64, N = 64, on the dense probe, whose reference is one of the two weak ones described above. K = 32 came back 16 of 64 from a base-table overrun, and N of 24, 40 and 56 reported every word written but still placed whole groups wrong. Round 284 proposed wbytes/4 as the pair stride that would cover them and labelled it a hypothesis rather than a reading, because every table behind it had been swept at N = 64, where that expression and the 8×K already in use are the same number. The next round tested it and refuted it, and not narrowly: wbytes/4 left all three failing cases at zero and broke three that had been passing, 16 of 16, 32 of 32 and 48 of 48 going to 0, 0 and 8. 8×K is right everywhere there is data, the three failures are something else, and the tidy reasoning that produced the alternative did not survive one round. Rows above one work too (14313 of 14336 at M=224, and 200344 of 200704 across a whole 56×56 surface) after 36 rounds in which they were read as broken and the hardware was not: the output surface is [n/atom][m][n%atom], which collapses to plain n at exactly one row, and every correctness run had been done at one row. The int4 port also introduced a defect of my own: a w4a16 job left the NPU unable to start the next one. It is now localised to two registers in the RDMA coefficient fetch group, and two earlier attempts to bisect it were thrown out by their own written controls before a third succeeded. That last line used to read "still not a runtime in the full sense: no model runs end to end". As of 25 August it does, and as of 31 August it is nine models. Llama-3.2-1B runs end to end at four bits through this driver and nothing else: 113 tensors, every projection including the 128256-wide output head, with the generated sentence identical to the one the exact arithmetic writes. Nine of nine models on the card now answer a batched prompt exactly as their own token loop does, Phi-3.5 and Gemma-4-E2B among them. Correcting the single-row reading above turned a refusal into a result: w4a16 batches, and the prompt is 3.3x. Three things were wrong at once and none of them shows at m = 1, which is why hundreds of decode rounds never met any of them: the axis, a surface length computed from rows where it needed 3 × M, and a read order in which the row and one operand trade places. On the board, 113 tensors are exact at m of 2, 4, 16, 32, 48, 64 and 80, worst relative error 5.10e-05, which is float summation order. m = 8 is still wrong on one tensor shape and is refused rather than trusted. Two further faults were found and fixed in the same stretch, each one a law rather than a quirk of an architecture: an odd batch width has no expression on the accumulator surface at all, which is the whole reason two models were wrong and Llama never was; and the two NPU cores corrupt each other when their batched submits overlap, 13 of 16 runs wrong overlapped against 0 of 16 serialised. What stays on the CPU is the embedding lookup, RMSNorm, RoPE, the attention score, softmax and weighted sum, SwiGLU, the residuals and sampling. The CPU decode loop was written first and is the oracle: the NPU computes the same sum and the two runs have to agree exactly, so "the text looks fine" was never allowed to count.
    github.com/gahingwoo/charsiu
  • rp2350-tz-tee: minimal TrustZone-M isolation example on the RP2350

    Self-verified, SWD
    The Pico SDK still can't build a Non-Secure binary that runs under a Secure one (open upstream since pico-examples#708), so there was no minimal Secure/Non-Secure starting point. Built one from scratch on a Pico 2: a Secure world guarding a secret behind NSC veneers, verified over SWD rather than trusted from source: a direct Secure-RAM read from the Non-Secure side is caught as a SecureFault with the expected fault-status bits set, not just "the demo prints the right thing." Also documents a build recipe for TF-M on rpi/rp2350 and the DMA-has-its-own-MPU pitfall (the SAU alone doesn't stop Non-Secure code from using DMA to walk around TrustZone). Extends the TF-A/OP-TEE trusted-execution work above down to the microcontroller class.
    github.com/gahingwoo/rp2350-tz-tee r/embedded thread

Tools in use by others 3

Third-party validation 7

  • Independent testing and cross-checking on hardware I don't own

    Tested-by on RK3588, plus review that found real bugs
    Igor Paunovic reviewed and tested the NPU series on an Orange Pi 5 Plus (RK3588), hardware I have no access to. He gave a Tested-by on the job_lock patch after six phases per module across all three NPU cores, binding and unbinding cores between phases: 2596 inferences and 111048 completion interrupts, scored by the SHA-256 of tensors that change with the input and stay stable across repeats, so a stale output buffer can't pass as a recomputation. He also reads the patches rather than only running them: he found a comment that had outlived its subject and a shared-path refactor sitting in the wrong patch for bisection. He then produced the one comparison I cannot (the same per-operator scoring across all 27 MobileNet operators on RK3588 silicon) and, in August, showed that my scoring reference was wrong for the model's last operator: it compares against max(cpu, zp), which I had justified as right for the operators carrying a fused ReLU6 and wrong for the logits BiasAdd, which has none. Version 2 answers that half, and answers it against me. There are no such operators. Every model in this tree is fused_activation NONE, MobileNet included, its ReLU6 folded into the output quantisation range instead, so those layers carry output zero point 0 and for a uint8 tensor at zero point 0 the clamped and unclamped references are the same reference. What stays open is the other half: the clamp at the output zero point is my own stack's rather than the silicon's, and what inside my stack produces it is narrowed to one column and not closed.
    lore.kernel.org: the v7 thread, including the test reports lore.kernel.org: the v8 thread, induced-reset run and the second Tested-by
  • Five findings independently cross-confirmed on the sibling SoC

    Independent, different method, different silicon
    gregordinary reverse-engineered the RK3588 NPU by a different route (from Mesa-Teflon capture rather than from vendor model files) with no contact with this project, and lists five of my findings as cross-confirmed against his own silicon in his SOURCES.md: the IOMMU being attached once and detached on power-down rather than per job; the ping-pong producer/consumer register groups and what a misaligned pair does; requantisation being a right shift whose magnitude is load-bearing; a per-channel zero-point correction living in a weight-buffer tail in groups of eight output channels; and the dt_wr/dt_rd/wt_rd byte counters being readable on RK3576 where RK3588 nulls them. His independent count of 16 CBUF banks on RK3576 against 12 on RK3588 also matches mine. Scope matters here: what he confirms is the register model, and it was never evidence for the causal claim my preprint's version 2 retracts.
    gregordinary/rockchip-npu-notes: SOURCES.md
  • edk2-rk3576: UEFI port pinned by a Radxa engineer

    Third-party
    A complete EDK2/TianoCore UEFI port for the RK3576, booting a mainline Fedora 44 desktop. Builds for six boards, hardware-verified on two (Radxa ROCK 4D, ArmSoM CM5-IO). The write-up was pinned to the ROCK 4 series section of the official Radxa community forum by Zhang Yuntian, a Radxa engineer, not by me.
    forum.radxa.com: pinned thread github.com/gahingwoo/edk2-rk3576
  • kiln: official Radxa tutorial for local LLM/vision on the RK3576 NPU

    Third-party, official tutorial
    A deliberate stopgap, not a mainline driver: builds the vendor GPL rknpu driver out-of-tree against a mainline kernel (linux-7.1.3), drives it with the closed librkllmrt/librknnrt runtimes, and carries the RK3576 NPU clock/power-domain/IOMMU kernel patches from above so the NPU comes up at all without the vendor's 6.1 BSP. Radxa host a full step-by-step tutorial for it under ROCK 4D > Application Development > NPU Usage (install, health checks, local LLM chat, vision inference, and an OpenAI-compatible API server), explicitly labelled as a third-party project rather than official support, but given the same tutorial treatment as Radxa's own guides.
    github.com/gahingwoo/kiln docs.radxa.com: Kiln tutorial
  • Flipper Devices: production validation of the OP-TEE port

    Third-party
    Flipper Devices validated the RK3576 OP-TEE port on Flipper One hardware: 114 xtest cases, 35,681 subtests, zero failures. Credited by name in their official developer documentation.
    docs.flipper.net/one/dev-log/3
  • Collabora: official post summarising the NPU bring-up

    Third-party
    Collabora (an established Linux kernel and open-source engineering consultancy) posted their own technical summary of the RK3576 NPU work, unprompted. Heiko Stübner, the Rockchip kernel maintainer, commented on the post directly.
    linkedin.com/posts/collabora
  • Rockchip upstream enablement tracker: NPU and VEPU510 reference points

    Third-party
    A community-maintained GitLab tracker, hosted by Collabora, for Rockchip RK3576 mainline enablement status across every subsystem (GPU, ISP, media, NPU, and more). The NPU row cites the RFC series as its reference entry, and the VEPU510 H.264/AVC row was later updated by Sebastian Reichel (Collabora) to cite the video-encoder RFC thread as well, not something I added myself, but how the tracker's maintainers chose to record the state of both.
    gitlab.collabora.com: mainline-status.md

Community deployment 2

Peer collaboration 1

Back to top