RomHacking with Ghidra, Part 1: Loading SNES ROMs

2026-05-08T00:00:00+02:00
Adding to the myriad of (unfinished) projects I’ve still got on the backburner, I’ve recently started looking into RomHacking in order to translate the dialogs of a fighting game. Since my previous reverse-engineering endeavors in Ghidra were a very pleasant experience, I decided it would be great to keep using that tool for this next task.
I loaded the ROM in Ghidra, started it in the Mesen2 emulator, and started to delimit the first string I had found in the debugger. Alas, it was not in its expected place: I could find the exact bytes I was looking for, but they were in a completely different place.
Treachery! Ghidra had perfectly mapped the ELF sections the previous time, why wasn’t it doing the same here? What’s so special about an SNES ROM?
Straight from the shoulders of giants

Fresh from last time’s mistake, I decided to start fresh from what was already there: an archived ghidra-snes-loader repository exists. Let’s see…
Works with Ghidra v9.1 or later only.
So claimed the documentation. Unfortunately, “or later” apparently meant “but before your version”. Still, let’s check the code of that SnesLoader.java. Clearly that’s extending an abstract class, setting some boilerplate configurations through getters, and then the actual loading logic:
class="highlight">
1
2
3
4
5
6
7
8
9
10
11
12
13
14
public class SnesLoader extends AbstractProgramLoader {
  @Override
  public Collection<LoadSpec> findSupportedLoadSpecs(ByteProvider provider) throws IOException {
    // (...)
  }

  @Override
  protected List<Program> loadProgram(ByteProvider provider, String programName,
      DomainFolder programFolder, LoadSpec loadSpec, List<Option> options, MessageLog log,
      Object consumer, TaskMonitor monitor)
      throws IOException, CancelledException {
    // (...)
  }
}
 That all looks very easy. So that findSupportedLoadSpecs obviously sets the language/architecture, and loadProgram will set up the ROM file¹. Pretty basic. What’s next? Ghidra invokes loadProgram, passes it the ROM content in ByteProvider provider, and then it all goes in the memory map, that’s easy, and (as we’ll see later, dear reader!) the Ghidra extension model is actually pretty pleasant to work with.
 Supporting the 65816 processor was pretty easy too, I just had to copy over the data/languages folder from the (archived) <gh repo=achan1989/ghidra-65816>ghidra-65816</gh> repository, fix a minor bug, add some specialization to the spec, so it would be tied to my extension, and call it a day.
 This feels easy because it…sorta is. If all you want your loader to do is “recognize that file type, assign it that language/arch”, you can stop reading here.
 But if you keep reading, the second line of the load method invokes detectRomKind(provider). I knew SNES ROMs had different sizes, but different types… I guess that’s not about RPGs and platformers?
 Banks and rommers
 I’m no stranger to romhacking, ROMs and ISOs, but I never thought too much about it. This changed with the 3DS and PSVita scenes, where translations and mods would be distributed using the LayeredFS and rePatch methods respectively. Those were no mere .ips or .xdelta patches, but an imitation of the game’s filesystem, only containing the modified files, with the system’s open(3) syscall being hooked with a conditional loader:
 class="highlight">1
2
3
4
5
6
7
int open(const char *path, int oflag, ... ) {
  char *patchPath = replace(path, "/game_path/", "/patch_path");
  if (exists(patchPath)) {
    return _open(patchPath, ...)
  }
  return _open(path, ...)
}
 A very smart piece of technology, which makes me wonder why it wasn’t invented before the 3DS era. Could it be because a ROM, being Read-Only Memory has no concept of a file system?
 See, ROMs have always looked like the exact representation of a game’s cart, and nobody running ZSNES or Snes9x would dare challenge that idea, since all the data was there, and there was nothing missing, right? This was all a sweet lie perpetuated by emulator developers because the approach of emulators (at the time) was to use shortcuts in emulation to work in spite of the low specs of emulation machines.
 The statu quo was questioned when byuu/Near first published the bsnes/higan emulator: it required powerful (at the time) machine specs to run, but was the first to entirely emulate the SNES down to each of its circuits, be they inside the console, or inside the game cart. Because yes: there are different chips inside the carts, and the ROM was only one of them!
 This new model completely reset the preservation efforts for the console: GoodSNES² only included the ROMs and there was no way to know which chips came with it. The emulators were just guessing with heuristics, or maintaining their internal database, which was the only way an emulator would know that you could save your game. Thanks to byuu/Near’s preservation efforts, all game carts were identified by boards, chips, ROMs,… and mappings. But those “chips” didn’t explain why the bytes in my debugger didn’t line up with the bytes in my ROM, nor with the bytes in Ghidra!
 See, one thing we didn’t touch on is that the SNES is a 16-bit console, which means it performs calculations over 16 bits, 2 bytes, in the hexadecimal range 0x0000-0xFFFF (so, 0-65,535 in decimal). If it could only address 64KB³, how could it access data over a 1MB⁴? The same way everyone else did: Bank switching. Depending on a single 8-bit register (0x00-0xFF, so 0-255), memory accesses would now be directed to the chosen “page” of the ROM. You’re in page 4 and want to read something in page 6? Easy: just bank-switch!
 I like that “page” analogy. It’s just a shame that the real model is closer to the Necronomicon than to a neatly numbered book.
 O ye’ll take the high ROM and I’ll take the low ROM
 The full system is a bit complicated⁵, but in short, the SNES and the game cart will form a single unit, and share the same memory layout. As shown in the following image (courtesy of SNesDev Wiki), some areas will ALWAYS be dedicated to the SNES I/O and RAM:
 
  In bank $00, from 0x0000 to 0x1FFF are the first 8KB of RAM.
 In bank $00, from 0x2000 to 0x5FFF is the full SNES I/O in multiple registers.
 These two mappings are also accessible from $01 to $3F and from $80 to $BF. It is not a separate mapping but a mirror that functions just the same, like a shortcut.
 In banks $7E to $7F, from 0x0000 to 0xFFFF, the whole SNES WRAM can be accessed.
 
 And then, there is the rest, which the cart is free to dedicate to ROM mappings as it pleases. In practice, there are two (and a half) canonical mapping types:
  LoROM will split the ROM into chunks of 32KB (so 0x8000 bytes), and map them to banks $80⁶ (128) to $FF (255). Since the chunks are half the size of a bank, they will be mapped from 0x8000 to 0xFFFF.
 HiROM will split the ROM into chunks of 64KB (so 0x10000 bytes⁷). Since the lower parts of banks $80 to $BF already contain mirror mappings of the SNES RAM and I/O, these ROM chunks will instead be mapped in banks $C0 to $FF.
 ExHiROM is an extension of HiROM and will map its first 64 banks the same way, then map the next ones from $40 to $7D (and not $7F since it’s used by the RAM!), and can ALSO map two final banks in $3E and $3F, for a total of 128 banks.
 
 This mapping was a bit of a pain to implement, but nothing that a good loop cannot solve, although I must admit that for the mapping of the last two banks of an ExHiROM, I decided this was “a problem for someone skilled enough to be working on the games that would require them”. Don’t thank me, I love providing my peers with pull request opportunities.
 So, now that we know how to map, how do we know whether our ROM is LoROM or HiROM?
 The safest method is to calculate the ROM’s SHA256 digest and compare it to bsnes/higan’s Super Famicom.bml, a COMPLETE database of all identified SNES carts. If we get a result, we can identify the board model, then use the nocash specifications to identify the ROM type. Needless to say, it won’t work on RomHacks, since they are not included in the Super Famicom.bml database.
 The old-school but proven method is the heuristic emulators usually rely on, based on detecting the ROM’s title label:
  In LoROM, this is ROM offset 0x7FC0.
 In HiROM, this is ROM offset 0xFFC0.
 In ExHiROM, this is ROM offset 0x40FFC0.
 
 So in the same file, we read 21 bytes at each of those addresses, and depending on which is the most plausible title, we know the ROM type. Those numbers may seem arbitrary, but it’s because the SNES expects the ROM header contents (which include the title label) in bank $00 at 0xFFC0.
 Wait, I never said this area was mapped, no?
 Mirror, mirror, on the ROM, who’s the fairest of them all?
 By this point, I had already seen the CPU vectors defined in the language specs, and I knew of the VEC_RESET_EMULATION vector, which is where the console looks for the ROM entry point, but I had never noticed its value: $00:FFFC.
 And what the hell, even using Ghidra’s “Go to”, I could never access this address: it just DID NOT exist in my listing, as if there was nothing there, and obviously there was nothing, because we’ve already discussed where the ROM chunks are mapped. So how is the SNES expecting to find ROM code here?
 Well, do you remember how I told you the bank $00’s I/O and RAM mappings were also accessible from $01 to $3F and from $80 to $BF?
    When you stare into the SNesDev documentation for too long, it stares back at you.
 
 
 Yes, the area from $00:8000 to $00:FFFF is a complete mirror of one of the canonical mappings of the ROM. And the best part? It’s not always the same.
  In LoROM, it mirrors $80:8000 to $80:FFFF.
 In HiROM, it mirrors $C0:8000 to $C0:FFFF.
 In ExHiROM, it mirrors $40:8000 to $40:FFFF.
 
 While this is enough pain on its own, it’s at least kind of standard. Would you believe me if I told you there were EVEN MORE mirror mappings, and that all of them are dependent on the physical board in the cart?
 Smashing the banks for fun and profits
 At this point of my research, the first thing that came to my mind was just a question: “How did the emulators of old manage to work that well?”. The truth is that most of them were KINDA duct-taped, and using heuristics to deduce the mirror mappings, and TECHNICALLY, you can just see the memory layout as eight zones, with two taken by the SNES I/O and RAM, and mirror the canonical ROM banks everywhere you can.
 And then, the emulator has to add more duct-tape because “actually, this game does NOT have ROM mapping here, and uses that behavior⁸ to check that it’s running on real hardware”.
 Discovering it made me imagine an emulator-proof board and ROM: a ROM can have its entry point at an unexpected offset, but as long as the board properly maps it to $00:FFFC, a console will run the game, and an emulator will fail, until a specific board mapping rule is added for the emulator.
 But really, don’t I have enough work mapping the mirrors I already know about?
 99 Bottles of Chips on the wall
 So in practice, I now have a loader that currently only exposes the canonical mappings directly, and leaves the weirder board-specific mirrors to future work (and user-defined mappings). Which is honestly fine for a first version: by that point, the goal wasn’t to perfectly emulate every possible SNES board, but to finally have Ghidra display the same addresses as the debugger (though if the code relies on mirror mappings, they aren’t visible yet).
 Finally, for that one ROM I was working on, once I reimported it, suddenly, the bytes lined up. Time to hack.
 Credits and Acknowledgements
  This article and ghidra-snes are dedicated to Near (formerly known as byuu), whose contributions to SNES documentation, emulation, and preservation have had a lasting impact on the community.
 Thanks to achan1989 for ghidra-65816, the original 65816 processor implementation in Ghidra.
 Thanks to the SNesDev Wiki and the very helpful members of the Discord
 Thanks to nocash for the fullsnes documentation.
 Thanks to the NSA guy watching over me if they participated in Ghidra.
 
 
    In Ghidra parlance, a “program” is a file you’re analyzing. ↩︎
 
  The GoodTools are tools to properly rename ROMs according to their hash, and the set of renamed files it produces are called GoodSets, with the one related to the SNES being GoodSNES. ↩︎
 
  Of course, it’s a bit more complicated: the 65C816 processor uses 16-bit registers, but a 24-bit address space split into banks. Most instructions operate within the currently selected bank, which keeps addresses smaller and operations faster. ↩︎
 
  The two largest games released for the console, Tales of Phantasia and Star Ocean, both used 6MB ROMs. ↩︎
 
  But you can read the full specifications by nocash if you’re curious! ↩︎
 
  In SNES development lingo, bank numbers are usually prefixed with a dollar sign, like $00 to $FF. ↩︎
 
  Yes, in hex, the double of 0x08 is 0x10! ↩︎
 
  Technically, you can read/write to non-mapped areas of the memory layout to trigger console-only behavior. ↩︎
 
 
  


Exploring Spira's PS3 binary with Ghidra
2026-05-03T00:00:00+02:00
 I finished Final Fantasy X more than one month ago, and I still haven’t written about how almost half of my quest didn’t happen in front of my PlayStation 3, but in Ghidra, trying to cheat some of the game’s most difficult trophies…
 Cheating is cooler when you learn stuff
 It’s okay: Final Fantasy X isn’t really a HARD game, and neither is the remaster that came out in 2013. Some parts of it are obviously going to take a while, but according to PSNProfiles Trophy Guide, it’s rather tame: 110 hours, and a 5 (out of 10) difficulty. So what’s the hard part? An optional super-boss? A gauntlet of fights without healing items? A low-level run? Worse than that: a chocobo race.
 A Chocobo is a bipedal animal in the Final Fantasy franchise, kind of a bird that can be used like a horse. In this episode, they are used both as a means of transportation, and racing animals. They are also tied to three trophies:
  Chocobo License (Bronze): Pass all chocobo training
 Actually, that one isn’t too hard, let’s not think about it.
 Chocobo Rider (Bronze): Win a race with a catcher chocobo with a total time of 0:0:0
 Okay, just from the description, this one reads like bad news.
 Chocobo Master (Silver): Get 5 treasure chests during the Chocobo Race at Remiem Temple and win the race 
 This one reads easy but of course, there’s a reason it’s Master.
 
 The first one already sounds CRAZY: you must finish a race with a total time of 0:0:0. While it sounds impossible, it can be achieved based on your final time in this race: the race itself takes around 30 seconds, and you can catch balloons that will each subtract 3 seconds from your final time. You can also be hit by hazards that will ADD 3 seconds to your final time. And of course, balloons and hazards are completely random (but will also affect your opponent!).
 The second one requires you to follow a VERY precise path on a race track, collect treasures along the way, and just avoid touching the “walls” of the tracks to avoid losing some seconds there. It’s a test of skills, but not one riddled with randomness.
 As I was making my way through the game, I got Chocobo License easily, and then tried to get Chocobo Rider. Once, twice,… maybe twenty times. Screw that!
 It’s very easy to unlock trophies on the PS3, but I just thought of something else: why not look for a cheat that would set my time to zero, give me a thousand balloons, or make each balloon subtract a thousand seconds?
 The state of cheating on the PS3
 While there is a very good cheat engine available on the PS3, there is no proper (and multi-platform) solution to find new cheats¹. And even though I could find a cheat, it worked only on the US release of the game: I was playing on the Japanese.
 But maybe I could analyze it?
 class="highlight">1
2
3
4
5
6
7
8
9
10
11
12
13
14
Time Chocobo Training
0
Hiei-YYH
0 00769D60 2C090014
0 00769D64 41820008
0 00769D68 48000018
0 00769D6C 3F800C00
0 00769D70 639C00B0
0 00769D74 7C1C2800
0 00769D78 40820008
0 00769D7C 3BE00000
0 00769D80 9BE30000
0 00769D84 4E800020
0 00339A9C 484302C5
 First off, that seems like a lot of instructions, which is strange when you’re mostly used to Action Replay-style cheats: usually, after you’ve found where the value you’re interested in is stored in memory, you hook the cheat engine to it, and overwrite it with your value, and prevent further writes to it.
 I also remember how my PS3 has crashed A LOT when using cheats, because I was using those similar “always-on” cheats, while “once” cheats, would always work. Maybe the PS3 isn’t powerful enough to run “always-on” memory hooks, but accepts writing arbitrary data straight into the game’s code to alter behavior?
 So I asked an LLM for a first pass at what this code was doing. The answer came in the form of ASM, and spat out the following pseudo-C code, with registers:
 class="highlight">1
2
3
4
5
6
7
8
9
10
11
12
void hook() {
  if (r9 == 0x14) {
    r28 = 0x0C0000B0;

    if (r5 == r28) {
      r31 = 0;
    }
  }

  *(uint8_t*)r3 = r31;
  return;
}
 I’m not sure I get it all yet². Those rX thingies are registers and there are at least 31 of them (what? where’s my RAX?), but since I’m here already, I might as well dig into the game’s binary.
 Spira on my laptop
 Two minutes later, I’ve got a 9.4MB file called ffx_phyreApp.self on my laptop. I’ve spent enough time around to understand it’s a “signed ELF” or, well, a SELF. The first part is easy to circumvent, thanks to RPCS3’s “Decrypt PS3 Binaries” to turn it into a ffx_phyreApp.elf file. And guess what? The ELF format is so standard, that any Unix worth its salt can understand it:
 class="highlight">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
$ file ffx_phyreApp.elf
ffx_phyreApp.elf: ELF 64-bit MSB executable, 64-bit PowerPC or cisco 7500, Unspecified or Power ELF V1 ABI, version 1, statically linked, stripped

$ objdump --section-headers ffx_phyreApp.elf
ffx_phyreApp.elf:       file format elf64-powerpc

Sections:
Idx Name          Size     VMA              Type
  0               00000000 0000000000000000
  1               0000002c 0000000000010200 TEXT
  2               007298e0 0000000000010230 TEXT
  3               00000024 0000000000739b10 TEXT
  4               000028e0 0000000000739b34 TEXT
  5               00000188 000000000073c414 DATA
  6               0000051c 000000000073c59c DATA
  7               00000004 000000000073cab8 DATA
  8               00000004 000000000073cabc DATA
  9               00000004 000000000073cac0 DATA
 10               00000420 000000000073cac4 DATA
 11               00000004 000000000073cee4 DATA
 12               00079650 000000000073cef0 DATA
 13               000a4550 00000000007b6580 DATA
 14               00000028 000000000085aad0 DATA
 15               00000040 000000000085aaf8 DATA
 16               000003a4 0000000000860000 DATA
 17               00000304 00000000008603a4 DATA
 18               00000004 00000000008606a8 DATA
 19               00003a34 00000000008606b0 DATA
 20               0000051c 00000000008640e4 DATA
 21               0001d9b8 0000000000864600 DATA
 22               00003868 0000000000881fb8 DATA
 23               00000008 0000000000885820 DATA
 24               000001f0 0000000000885828 BSS
 25               0007f73c 0000000000885a80 DATA
 26               01d26d60 0000000000905200 BSS
 27               00003206 0000000000000000
 28               000004e0 0000000000000000
 29               0000012c 0000000000000000
 Oh yeah, it works. My school taught me to use objdump and nm, and it was my first glimpse into the beauty of the System V ABI, so let’s see what else our good old, reliable toolbox can tell…
 class="highlight">1
2
$ nm ffx_phyreApp.elf
ffx_phyreApp.elf: no symbols
 
 Oh, fuck me.
 Time to bring out the big guns and open Ghidra.
 I love this tool and its approach: smart enough to detect straight away what I could have chosen myself. Remember our file output? All there: PowerPC, 64-bit, and MSB Executable³ so Big Endian, which maps to PowerPC:BE:64:default:default.
 A few seconds later, the file is imported and I’m faced with a wall of hex. I run “Analyze” on the code a few times, and try to look up stuff that could be my code, like instances of 3 (remember? seconds to add or subtract to my time). Of course, no luck anywhere.
 I need to start planning a better approach, because there’s no way I’m reading 10 Megs of assembly.
 Pedal to the metal
 Since I’ve acquired a PS Vita in Japan⁴, in 2017, I’ve been hanging around r/vitahacks. Sometimes, I would read the source of stuff published there (mainly that one time someone posted a “trophy_unlocker.vpk” app). The name is a mouthful, but I remember it from the community VitaSDK project: sceNpTrophyUnlockTrophy.
 Since both consoles lived around the same time, chances are they both use the same approach. What is the PSL1GHT project telling me? Oh, it's got an FNID definition!
 A “function NID” (sometimes called FNID) is the hashed name of a function. In our case, 0x8CEEDD21 is the NID for sceNpTrophyUnlockTrophy. It seems like Sony was one of the few to do that kind of stuff, and it’s got to do with dynamic linking, size reduction, and a bit of Security through obscurity. All we need to know is that I cannot look up sceNpTrophyUnlockTrophy, but I can look up 0x8CEEDD21. One result, perfect.
 class="highlight">1
2
3
4
5
6
7
8
9
10
11
12
    XREF[1]:     0073cd40(*)  
0073c858 11 97 b5 2c     undefined4   // sceNpTrophyRegisterContext
0073c85c 1C 25 47 0D     undefined4   // sceNpTrophyCreateHandle
0073c860 37 01 36 FE     undefined4   // sceNpTrophyGetRequiredDiskSpace
0073c864 37 41 EC C7     undefined4   // sceNpTrophyDestroyContext
0073c868 39 56 77 81     undefined4   // sceNpTrophyInit
0073c86c 49 D1 82 17     undefined4   // sceNpTrophyGetGameInfo
0073c870 62 3C D2 DC     undefined4   // sceNpTrophyDestroyHandle
0073c874 8C EE DD 21     undefined4   // sceNpTrophyUnlockTrophy
0073c878 A7 FA BF 4D     undefined4   // sceNpTrophyTerm
0073c87c B3 AC 34 78     undefined4   // sceNpTrophyGetTrophyUnlockState
0073c880 E3 BF 9A 28     undefined4   // sceNpTrophyCreateContext
 Can you believe that? I just hit the mother lode! Now let’s check on that XREF⁵ at 0x0073cd40.
 class="highlight">1
2
3
4
5
0073cd38 00 00 00 00     addr
0073cd3c 00 73 c5 00     addr
0073cd40 00 73 c8 58     addr
0073cd44 00 86 44 28     addr
0073cd48 00 00 00 00     addr
 Okay, pretty useless. What’s around at 0x0073c500? Another set of NIDs?
 class="highlight">1
2
3
4
5
6
7
8
9
10
11
12
0073c500 73              ??         73h    s
0073c501 63              ??         63h    c
0073c502 65              ??         65h    e
0073c503 4e              ??         4Eh    N
0073c504 70              ??         70h    p
0073c505 54              ??         54h    T
0073c506 72              ??         72h    r
0073c507 6f              ??         6Fh    o
0073c508 70              ??         70h    p
0073c509 68              ??         68h    h
0073c50a 79              ??         79h    y
0073c50b 00              ??         00h
 Oh. Oh. Oh. That’s a very pretty C String there. Let’s go back to 0x0073cd3c and follow the other path to 0x00864428.
 class="highlight">1
2
3
4
5
6
7
8
9
10
11
12
00864428 00 73 ad 34     addr      XREF[2]:     0073ad3C(R), 0073cd44(*)  
0086442c 00 73 ad 54     addr      XREF[1]:     0073ad5c(R)
00864430 00 73 ad 74     addr      XREF[1]:     0073ad7c(R)
00864434 00 73 ad 94     addr      XREF[1]:     0073ad9c(R)
00864438 00 73 ad b4     addr      XREF[1]:     0073adbc(R)  
0086443c 00 73 ad d4     addr      XREF[1]:     0073addc(R)
00864440 00 73 ad f4     addr      XREF[1]:     0073adfc(R)
00864444 00 73 ae 14     addr      XREF[1]:     0073ae1c(R)
00864448 00 73 ae 34     addr      XREF[1]:     0073ae3c(R)  
0086444c 00 73 ae 54     addr      XREF[1]:     0073ae5c(R)
00864450 00 73 ae 74     addr      XREF[1]:     0073ae7c(R)
 Now that is..interesting? They all point to something at 0x0073Axx4 and are XREF at 0x0073AxxC?
 Let’s check that, plus along the way, Ghidra (and ChatGPT) have been nice enough to help me to spot functions boundaries in assembly.
 class="highlight">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
0073adf4 39 80 00 00     li         r12,0x0
0073adf8 65 8c 00 86     oris       r12,r12,0x86
0073adfc 81 8c 44 40     lwz        r12,offset sceNpTrophy_func_address_table[6](r)
0073ae00 f8 41 00 28     std        r2,local_res28(r1)
0073ae04 80 0c 00 00     lwz        r0,0x0(r12)=>LAB_0073adf4
0073ae08 80 4c 00 04     lwz        r2,0x4(r12)=>LAB_0073adf8
0073ae0c 7c 09 03 a6     mtspr      CTR,r0
0073ae10 4e 80 04 20     bctr

0073ae14 39 80 00 00     li         r12,0x0
0073ae18 65 8c 00 86     oris       r12,r12,0x86
0073ae1c 81 8c 44 44     lwz        r12,offset sceNpTrophy_func_address_table[7](r)
0073ae20 f8 41 00 28     std        r2=>TOC_BASE,local_res28(r1)
0073ae24 80 0c 00 00     lwz        r0,0x0(r12)=>LAB_0073ae14
0073ae28 80 4c 00 04     lwz        r2,0x4(r12)=>LAB_0073ae18
0073ae2c 7c 09 03 a6     mtspr      CTR,r0
0073ae30 4e 80 04 20     bctr
 I check once, twice, three times, yell “WTF” a few times, think of a meme like “this code duplication could have been a function argument”, but I’m looking at assembly, and furthermore, it’s game console assembly, so maybe there’s a logic here that eludes me.
 I ask my LLM coworker for its opinion: these are import stub / trampoline functions. They are inserted at link time to provide a jump from the game code to the real function implementation. I’m still not entirely sure why this is required: I’ve heard a ton about linking, but zilch about stubs.
 Until I remember something else: on the PSP⁶, the homebrew plugins are often distributed as .prx files, and the PS3 homebrew scene often provides .sprx files. Maybe this isn’t just a homebrew thing, but a console thing, the local equivalents of .dll and .so for dynamically loaded code. Wouldn’t that explain why I’ve only found eleven NIDs related to trophies, and not the full seventeen referenced in the SDK?
 I dig a bit further and find the answer: there are indeed a few np_trophy_*.sprx files.
 So that explains everything: the binary contains a list of imported NIDs, and a function table. In the static file, those table entries point back to tiny trampoline functions (yes, it’s a loop, I know, it makes me crazy too). At runtime, the loader resolves the requested NIDs from the relevant SPRX modules and replaces those table entries with pointers to the real function descriptors. The stubs then load the real address from there and jump to it.
    Me, when I finally understand what the fuck that mess was.
 
 
 Okay, but seriously, even if I found a trail for trophy unlocking, I’m not really in the mood to annotate all the NIDs one by one, and my limited knowledge of the PS3 might lead me to some very questionable interpretations.
 Time to pull more guns.
 From the shoulders of giants
 I had actually stumbled on the Ps3GhidraScripts repository earlier, but a part of me felt too lazy to even look at it. Which, in hindsight, is a good thing: it saved me from skipping all the discoveries above, and it saved you from missing out on my brilliant prose.
 That said.
  It will then parse the information sections and define imports/exports and name the ones it can from the nids file, and then set the TOC.
 
 Rinse, repeat. I removed the file from the project, and reimported it. At this point, I realized Ghidra actually had a more fitting language spec for the PS3: PowerPC:BE:64:A2ALT, meaning 64-bit PowerPC, big-endian, with AltiVec support. There is actually an EVEN MORE fitting one, but I won’t understand that it is needed⁷ before reversing at least a dozen structs. But that will be a problem for future me, let’s run those scripts and see the results.
 
 True to its documentation, the script set up the function stubs for the trophies, and for other dynamically-linked modules like sceNpManager and sceNpTus. Completely useless to me. The syscalls are a bit funnier to look at, since most of them got the prototype you would expect if you spent enough time coding in Unix C.
 Oh well, who cares? We’re hunting trophies now!
 There’s only one function calling sceNpTrophyUnlockTrophy, so let’s follow it and call it UnlockTrophyWrapper. Thanks to the Ghidra disassembly pseudo-C, we can extract a few interesting informations:
 class="highlight">1
2
3
4
5
6
7
8
9
10
11
12
13
s32 FFXTrophyPS3Backend::UnlockWrapper(FFXTrophyPS3Backend *t, s32 trophy_id) {
  s32 result;
  sysTrophyId platinum_id = -1;
  
  sys_lwmutex_lock(trophy_mutex, 0);

  result = sceNpTrophyUnlockTrophy(t->context, t->handle, trophy_id,&platinum_id);
  if (result < 0) {
    error_printf(s_Error:_sceNpTrophyUnlockTrophy()_00755a98,result);
  }
  sys_lwmutex_unlock(trophy_mutex);
  return result;
}
  While sceNpTrophyUnlockTrophy needs a sysTrophyContext, sysTrophyHandle, and a trophy ID, this function takes a single structure, and a trophy ID. So we can know the game passes around a struct of its own (let’s call it FFXTrophyPS3Backend). This also confirms that we’ve moved out of “OS code” and are in “game code”.
 We’ve found the fprintf equivalent!
 There are mutexes surrounding the trophy unlock, so the game is handling it and not relying on the system? Funny.
 Omitted from the sample, the FFXTrophyPS3Backend struct contains a bit map of the already-unlocked trophies? Can’t the system handle that on its own?
 
 That was very interesting. Now let’s see who’s calling that, I’m sure I’ll find a call passing the trophy ID I’m looking for down the chain!
 Or not:
 class="highlight">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
void ReceiverThread(FFXTrophyPS3Backend *t) {
  while (true) {
    FFXTrophyEvent event;

    int ret = sys_event_queue_receive(t->eventQueue, &event);
    if (ret != 0) {
      sys_ppu_thread_exit(ret);
    }

    switch (event.command) {
      case UNLOCK_TROPHY:
        UnlockWrapper(t, event.trophy_id);
        break;

      case EXIT:
        sys_ppu_thread_exit(0);
        break;

      default:
        error_printf("Unknown event command");
        break;
    }
  }
}
 While I do want to cry (at carefully synchronized intervals, one eye at a time, to be thread-safe), this gives me two new leads:
  There’s something somewhere sending events to this thread, telling it to unlock trophies.
 There’s a setup somewhere that sets up this thread loop.
 
 Following the setup isn’t really complicated, except that it spans a lot of functions, since it has to set up the sysTrophyContext and the sysTrophyHandle, plus handle all the related error cases. After around eight functions of setup, I get to the function that spawns the trophy initialization (and would you believe it? It’s another thread!).
 That gives me two new insights:
  The struct FFXTrophyPS3Backend contains a boolean that indicates whether the trophy setup thread was started, let’s call it flagTrophyThreadRegistered.
 The struct also contains a boolean that, if set to false, will just not launch the trophy setup thread⁸.
 
 Wait, what.
 I check Wikipedia: the PS3 got trophies in 2008, and that remaster came out in 2013. The Switch version, the only achievement-less platform, came out in 2019.
 Could this mean that, from the start, the remaster wasn’t a PlayStation-centric port, but was built to support multiple achievement backends? Since the Windows/Steam version came out in 2016, and the Xbox One version in 2019, that feels like both overengineering and careful planning.
 Anyway, going further up that path gets me in game engine setup territory, even finding out some localization backend stuff. I don’t need many languages to find out that I suck at chocobo riding, I need my trophies!
 Finding our first trophies
 Actually, a bit around that time, I had advanced in the game, and while I was running some stuff from my laptop⁹, I managed to legitimately obtain Chocobo Rider. But things were too much fun to stop now, plus maybe I could leverage all the things I found at some other point.
 
 So, let’s get back: we know there’s an event queue somewhere, as sys_event_queue_receive told us. Next step is easy, just look up sys_event_queue_send, right? Well, as you can see from the screenshot, not exactly, and that’s where I’m glad that a script had already mapped the syscalls for me, so I could easily find sys_event_port_send.
 While it is smart, in hindsight, to send events from multiple sources on a port to receive them from a queue, that’s the kind of API design that makes you understand why Mark Cerny spent the PS4 generation trying to win back the developers.
 But still, there are only seven uses of sys_event_port_send, and right at the second one:
 class="highlight">1
2
3
4
5
6
7
8
9
10
s32 LAB_001af1e0(undefined4 *param_1, long param_2)
{
  s32 result;
  
  result = sys_event_port_send((char *)param_1 + 0x2c, 0, param_2, 0);
  if (result < 0) {
    error_printf(s_Error_:_sys_event_port_send()_fa_00755884,(long)result);
  }
  return result;
}
 This looks very unassuming, and it would be easy to dismiss it, but there’s just something about it that screams at me: “You got it bro!”, and it’s because of assembly and structs.
 See, while I’ve written some assembly¹⁰ for fun, I’ve never done more than small functions, and never used anything beyond basic types. So I had always wondered: how does assembly deal with struct members?
 This reverse-engineering endeavor is where I got the answer: it doesn’t. It just passes the struct’s base address, plus the member’s offset¹¹. And that + 0x2c? I’ve seen it before, when I first defined the FFXTrophyPS3Backend struct (which got split with FFXTrophyContext as its base)!
 Of course, it could be another structure with a similar layout, but the five last candidates all use very different offsets (as for the first candidate, it’s a trophy context teardown function). So let’s assume this one is our FFXTrophyPS3Backend::PostEvent function, and move up the two XREFs. For funsies, let’s take the second one, since it’s the one I went with, and my first foray into the game code.
 The prototype looks promising: param_1 is obviously our FFXTrophyContext, and param_2 is an ID, then there’s a loop. A loop that only runs five times. The game got 34 trophies, so that cannot be here, but let’s keep digging what happens during those five iterations: first, param_2 is compared to a global g_0075291c + loop variable, and if equal, a trophy event is sent with g_0075291c + loop + 0x4 as the trophy ID. That’s crazy, what the hell is in g_0075291c?
 class="highlight">1
2
3
4
5
6
7
8
9
10
0075291c 00 00 01 14
00752920 00 00 00 1f
00752924 00 00 01 58
00752928 00 00 00 1e
0075292c 00 00 00 82
00752930 00 00 00 0a
00752934 00 00 00 b0
00752938 00 00 00 20
0075293c 00 00 00 83
00752940 00 00 00 0b
 So, first value is the check value, and the second would be the trophy being unlocked. Let’s see, if I convert these to decimal, I get: 31, 30, 10, 32, 11. And assuming the Platinum trophy is zero, those would map to trophies…
  31: Overcoming the Nemesis: Defeat Nemesis
 30: Perseverance: Defeat Penance
 10: Overcoming the Past: Defeat Yunalesca
 32: The Eternal Calm: Defeat Yu Yevon
 11: The Destination of Hatred: Defeat Seymour Omnis
 
 I quickly make a new struct FFXTrophyForBoss { s32 enemyId; sysTrophyId trophyId; };, map it to the variable, and check the code. The disassembly is a bit wonky on indexes, but the result is telling:
 class="highlight">1
2
3
4
5
6
7
8
9
10
11
12
s32 ffxTrophyUnlock::DefeatEnemy(FFXTrophyContext *gameObject,int enemyId)
{
  for (int i = 0; i < 5; i++) {
    if (enemyId == g_ffxTrophyForBosses[i].enemyId) {
      s32 trophyId = g_ffxTrophyForBosses[i].trophyId;
      FFXTrophyPS3Backend::PostEvent(
        &gameObject->trophyObject,
        trophyId);
      break;
    }
  }
}
 Even after a second and a third look at the trophy list, indeed, only those five are related to a “Defeat X” goal. Which means I have found an entry into the combat system. In retrospect, calling it enemyId feels wrong, and I’m 99% sure this ID is closer to being an encounterID.
 Okay, I don’t feel like going all the way up the combat system, so let’s go back a step and check the other routes that set those trophies!
 At this point, there are two separate functions acting as gates, one checking that a trophy backend is enabled, and the other checking whether the target trophy has been unlocked, which feels sub-optimal, since those are checked in a million places already. Give me back those cycles!
 After a few more misdirections, we get to a function being XREF eighteen times. Now THAT looks promising, since we only have twenty-eight more trophies to find!
 More functions, more options
 Along the way, I’ve looked into the existing cheats for my version of the game to map more stuff. For instance, I now know where items are located. Could that be useful? Yes!
 ffxCombatEffect::Steal
 The first function is 157 instructions long, full of conditions, but at some point, even with Ghidra’s strange disassembly, it’s easy to understand:
 class="highlight">1
2
3
4
5
6
7
g_02059010 = g_02059010 + 1;
if ((*(param_2 + 0xded) == '\x06') && (199 < g_02059010)) {
  ffxTrophyTryPostIfGateOpen(4);
}
FFX::InventoryItem::ModifyInventoryRow(
  *(FFX::InventoryItem::InventoryRowItem *)(param_4 + 10),
  (int)*(short *)(param_4 + 8));
 There are two trophies where the number 200 is relevant, but there’s only one that modifies your inventory: A Talent for Acquisition: Steal successfully with Rikku 200 times. So we can map that function, and we know where g_ffxTrophyUnlockRikkuStealCount is located. We can also assume very easily that param_2 is a kind of struct PlayerCharacter with 0xded being the character ID, which will become very interesting on the next function.
 ffxCombatEffect::Lancet
 Next up is 632 instructions, 28 local variables, and yet…
 class="highlight">1
2
3
4
if ((g_02058fcc >> 8 == 0xff) &&
    ((g_02058fce & 0xf) == 0xf)) {
  ffxTrophyTryPostIfGateOpen(0x19);
}
 Found it: Learning!: Learn to use all enemy abilities. So if we check the code, we can see the first global check covers 8 bits, and the second covers 4 more, for a total of 12 flags, which matches the 12 Ronso Rages. Perfect.
 This leads to an interesting bit of trophy design. By the end of Final Fantasy X, characters can learn nearly every ability, and very few things remain exclusive¹². You can use Steal with Kimahri, and Lancet with Rikku. Yet the stealing trophy specifically checks for Rikku, while the Lancet trophy only cares that all enemy abilities were learned, regardless of who triggered the final check.
 ffxCombatEffect::Bribe
 Let’s go: 1029 instructions, 54 local variables, and the answer is right in front of our eyes again:
 class="highlight">1
2
3
4
g_02059014 = g_02059014 + g_ffxEncounterBaseStruct.field11_0x118._5288_4_;
if ((DAT_0205e748 == 0x302a) && (99999 < g_02059014)) {
  ffxTrophyTryPostIfGateOpen(0x13);
}
 Since that trophy is Under the Table: Spend 100,000 gil or more in bribes, it makes sense to call this global g_ffxTrophyUnlockBribeTotal and not bother about those strange structures we cannot make sense of. Who would have thought it would be this easy?
 ffxTrophyUnlock::JechtSpheres
 By this point, I don’t even bother to read the rest of the code anymore
 class="highlight">1
2
3
4
5
6
7
8
if (g_ffxTrophyUnlockJechtSpheresCount == '\b') {
  g_ffxTrophyUnlockJechtSpheresIsObtained = false;
}
else if ((g_ffxTrophyUnlockJechtSpheresCount == '\t') &&
        (g_ffxTrophyUnlockJechtSpheresIsObtained == false)) {
  g_ffxTrophyUnlockJechtSpheresIsObtained = true;
  ffxTrophyTryPostIfGateOpen(0x14);
}
 Yes, the hardest part is to count in hexadecimal to map 0x14 to the correct trophy: Messenger from the Past: Obtain all Jecht Spheres. At this point, I’ll just remember that '\t' is 9 in the ASCII table and call it a day. What tedious work!
 FUN_00365344
 However, some parts remain cryptic:
 class="highlight">1
2
3
4
5
6
7
8
if (g_ffxTrophyUnlockDamage9999NotObtained != 0) {
  ffxTrophyTryPostIfGateOpen(0x12);
  g_ffxTrophyUnlockDamage9999NotObtained = 0;
}
if (g_ffxTrophyUnlockDamage99999NotObtained != 0) {
  ffxTrophyTryPostIfGateOpen(0x15);
  g_ffxTrophyUnlockDamage99999NotObtained = 0;
}
 Two more: Power Strike: Do 9999 damage or more in a single attack and Mega Strike: Deal 99999 damage with one attack. The logic is obvious, but the only XREFs to these globals are right here. So clearly, something else is setting those two flags to true, but I haven’t mapped that part yet.
 ffxTrophyUnlock::Aeons
 At that point, I had made good progress in the game, but despite following a walkthrough, I had trouble obtaining the Magus Sisters Aeons, because I was trying to obtain them before Yojimbo. It took another guide to find out I had to obtain all other Aeons before.
 class="highlight">1
2
3
4
5
6
7
8
9
10
11
12
13
14
if (uVar1 == 0x11) {
  /* Trophy 15: Delta Attack! (Obtain Magus Sisters) */
  ffxTrophyTryPostIfGateOpen(0xf);
  /* Trophy 26: Summon Master (Obtain all Aeons) */
  ffxTrophyTryPostIfGateOpen(0x1a);
}
else if (uVar1 == 0xd) {
  /* Trophy 13: Feel the Pain (Obtain Anima) */
  ffxTrophyTryPostIfGateOpen(0xd);
}
else if (uVar1 == 0xe) {
  /* Trophy 14: It's All About the Money (Obtain Yojimbo) */
  ffxTrophyTryPostIfGateOpen(0xe);
}
 The code confirmed what my guide wasn’t telling: Unlocking the Magus Sisters implicitly meant all Aeons had been obtained.
 ffxTrophyUnlock::TheaterSphere
 No idea what that g_FFXGlobalState even is, but I looked at it a few times.
 class="highlight">1
2
3
4
5
6
7
if ((((g_FFXGlobalState == 0x9e) && (g_ffxTrophyUnlockTheaterSpheresMovies == '2')) &&
    (g_ffxTrophyUnlockTheaterSpheresMusic == 'G')) &&
    (g_ffxTrophyUnlockTheaterSpheresNotObtained == true)) {
  g_ffxTrophyUnlockTheaterSpheresNotObtained = false;
  /* 16: Theater Enthusiast (Buy every sphere at the Luca Theater) */
  ffxTrophyTryPostIfGateOpen(0x10);
}
 And once again, the ASCII table comes to the rescue: '2' is 50, and 'G' is 71, matching the number of movie and music spheres listed in every guide.
 ffxTrophyUnlock::EternalCalmMovie
 An easy one:
 class="highlight">1
2
3
4
5
if (DAT_008f5a18 == 0x3a) {
  /* Trophy 33: A Journey's Catalyst (View "Eternal Calm") */
  ffxTrophyTryPostIfGateOpen(0x21);
  DAT_008f5a18 = -1;
}
 No idea what those globals are.
 ffxTrophyUnlock::AlBhedPrimers
 This one was actually the first one I mapped out and it was a LOT of fun:
 class="highlight">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
s32 ffxTrophyUnlock::AlBhedPrimers(int alBhedPrimerIndex) {
  if (alBhedPrimerIndex == -1) {
    g_ffxAlBhedBitmap = 0;
    alBhedPrimerIndex = 0;
  }
  else if ((alBhedPrimerIndex < 0) || (0x19 < alBhedPrimerIndex)) {
    if (alBhedPrimerIndex == 0xff) {
      g_ffxAlBhedBitmap = 0x3ffffff;
      alBhedPrimerIndex = ffxTrophyTryPostIfGateOpen(0x1c);
    }
  }
  else {
    g_ffxAlBhedBitmap = g_ffxAlBhedBitmap | 1 << (alBhedPrimerIndex & 0x3fU);
    /* Trophy 01: Speaking in Tongues (Find 1 Al Bhed Primer) */
    ffxTrophyTryPostIfGateOpen(1);
    alBhedPrimerIndex = g_ffxAlBhedBitmap;
    if (g_ffxAlBhedBitmap == 0x3ffffff) {
      /* Trophy 28: Master Linguist (Find all 26 Al Bhed Primers) */
      alBhedPrimerIndex = ffxTrophyTryPostIfGateOpen(0x1c);
    }
  }
  return alBhedPrimerIndex;
}
 There are 26 “Al Bhed Primer” items in the game, which maps perfectly to a 26-bit bitmap.
 Everything in this function is crystal clear:
  There is a “reset” path.
 The normal path is triggered only if the index is below 26 (indexes start at 0): the bitmap is updated, the Speaking in Tongues trophy is always triggered, and then the bitmap is checked against a full 26-bit mask for the Master Linguist trophy.
 There is a debug path that gives out Master Linguist, but not the Speaking in Tongues, which could lead to a very funny outcome of getting Master Linguist before Speaking in Tongues, which should be impossible.
 
 This is one of the rare places where the game logic is both simple and perfectly aligned with the data model, which isn’t the case everywhere…
 ffxTrophyUnlock::SphereGrid
 While this function is short, it’s full of if and other strange control flow I’d rather not touch, so let’s only look at the relevant parts:
 class="highlight">1
2
3
4
5
6
7
if ((int)unaff_r28 < 1) {
  /* Trophy 23: Sphere Master (Complete a Sphere Grid for one character) */
  ffxTrophyTryPostIfGateOpen(0x17);
}
// ...
/* Trophy 29: Perfect Sphere Master (Complete the Sphere Grids for all main characters) */
sVar9 = ffxTrophyTryPostIfGateOpen(0x1d);
 That was easy to map, and easier to discard.
 FUN_003103f8
 Strangely, this one is one level higher in the call chain, and calls ffxTrophyTryPostById while all others above used ffxTrophyTryPostIfGateOpen. The end result is the same, since both paths will call ffxTrophyGateCheck, but it’s still surprising that the code would be structured that way:
 class="highlight">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
if (DAT_008e55cc != DAT_008e55c8) {
  DAT_008e55cc = DAT_008e55c8;
  if (DAT_008e55c8 == 0x933) {
    sVar7 = ffxTrophyGateCheck();
    if (sVar7 != 0) {
      /* Trophy 06: Heartstrings (View the "Underwater Date" scene) */
      ffxTrophyTryPostById(6);
    }
  }
  else if (DAT_008e55c8 == 0x43d) {
    sVar7 = ffxTrophyGateCheck();
    if (sVar7 != 0) {
      /* Trophy 05: All Together (All party members come together) */
      ffxTrophyTryPostById(5);
    }
  }
  else if ((DAT_008e55c8 == 0xa4) && (sVar7 = ffxTrophyGateCheck(), sVar7 != 0)) {
    /* Trophy 03: The Right Thing (Clear the Besaid Cloister of Trials) */
    ffxTrophyTryPostById(3);
  }
}
// ...
else if ((DAT_02055e99 == 0x7f) && (g_obtainedAllCelestialWeaponsTrophy == 0)) {
  g_obtainedAllCelestialWeaponsTrophy = 1;
  sVar7 = ffxTrophyGateCheck();
  if (sVar7 != 0) {
    /* Trophy 27: Weapon Master (Obtain all Celestial Weapons) */
    ffxTrophyTryPostById(0x1b);
  }
}
 The first three are story-related beats, while the last one is clearly a bitmask check against the set of obtained Celestial Weapons.
 Dude, where’s my Chocobo?
 Out of 33 trophies, I have now mapped out how to obtain 25 of them, and still no chocobo in sight. Let’s recap the eight remaining trophies:
  02 Teamwork!: Win a blitzball match
 07 Show Off!: Win a blitzball tournament
 08 Striker: Learn the Jecht shot
 09 Chocobo License: Pass all chocobo training
 10 Lightning Dancer Dodge 200 lightning strikes and obtain the reward
 17 Chocobo Rider: Win a race with a catcher chocobo with a total time of 0:0:0
 22 Chocobo Master: Get 5 treasure chests during the Chocobo Race at Remiem Temple and win the race
 24 Blitzball Master: Unlock all slot reels
 
 None of those seems to match any of the previous trophy categories, and we have three new categories: blitzball minigame, chocobo minigames, and that 200 lightning strikes nightmare. And only one call to ffxTrophyTryPostIfGateOpen still unmapped:
 class="highlight">1
2
3
4
5
6
7
8
9
10
11
12
13
14
s32 ffxTrophyRemapAndTryPost(void *param_1, uint32_t param_2, void *param_3, uint32_t param_4) {
  s32 trophyId;
  trophyId = FUN_00337f30(param_1, param_2, param_3, param_4);
  if (trophyId != 0x16) {
    if (trophyId == 0x17) {
      trophyId = 0x16;
    }
    else if (trophyId == 0x19) {
      trophyId = 0x18;
    }
    ffxTrophyTryPostIfGateOpen(trophyId);
  }
  return trophyId;
}
 That is… a very strange bit of code. If the computed trophy ID is 22, it does nothing. If it is 23, it remaps it to 22. If it is 25, it remaps it to 24. Every other value goes through unchanged, and the final result is passed to ffxTrophyTryPostIfGateOpen.
 By this point, I’m hoping I can just go up the chain, straight to my minigames, and my hopes are crushed: I end up in a TOC pointer, with a single XREF, and no way to find out who’s calling it without mapping more of the code. And looking at the pseudo-code sends a chill down my spine: I may be looking at a virtual machine.
 In spite of everything, I give it one more try and decide to go for even bigger guns: other humans! I find a repository dedicated to a FFX game randomizer, and even a Discord dedicated to reverse-engineering the full game, who nicely give me access to their work and findings on the Steam and Switch versions of the game.
 Unfortunately, my worst fears are confirmed: there is a scripting language in there, and a script engine to run it.
 Learning to give up
 By this point, I’m starting to realize it’s the second day in a row I’m going to bed at 6 AM. I can recognize the prologue and epilogue of a function at a glance. I even had a weird dream about assembly, where I made assumptions about code that I then confirmed when I woke up. And I haven’t actually played the game in almost a week.
 I’m at a crossroads.
 Going through the game’s code has been a ton of fun. I’ve learned how C++ vtables actually work with structs, assembly, and inheritance. I’ve even discovered the game uses intrusive linked lists, just like the Linux kernel.
 On the other hand, we’re talking about a 9.4 MB binary: that’s 2,352,212 instructions. Even ignoring the .bss segment and system stubs, it’s still a massive undertaking. And at this point, I’m not even sure why I’m doing it anymore. Or rather: is cheating my way to one last trophy really worth all that much work?
 As I ponder the question, I notice the rules to the Discord I just joined:
   Don’t stress yourself about contributing to modding/reverse engineering. 
 If your current situation doesn’t allow it it’s no big deal. Always put your well being first.
 
 
 I’m not stressing myself, but I know a part of me could keep going on and on over this binary, down to its last byte, until I burn myself out.
 Which would have been stupid, since I got the trophy the regular way less than two days later. Turns out, the answer is in the classics: winning is knowing when not to play.
 Thanks
 Huge thanks to:
   The Ghidra team for making reverse-engineering actually fun
 
  The RPCS3 team for making this whole mess possible
 
  The Ps3GhidraScripts contributors for saving me hours of setting up NIDs
 
  The FFX reverse-engineering Discord, Cid’s Salvage Ship, for giving me access to their knowledge, and telling me to touch grass
 
  My step-brother for gifting me his PS3 on a whim twelve years ago
 
  My sister for dealing with my live commentary over WhatsApp for three weeks
 
 
 
    A problem I am actively trying to solve. ↩︎
 
  And isn’t that actually for the better, since it means I’m gonna learn stuff? ↩︎
 
  Most Significant Byte, meaning Big Endian. ↩︎
 
  Obviously, I bought the Cosmic Red version. ↩︎
 
  In Ghidra, an XREF is a reference from another part of the binary. ↩︎
 
  Actually my first console ever, which I acquired in 2008 in its Crisis Core limited edition. ↩︎
 
  And I will only discover it exists while writing this article: the processor is 64-bit, but the PS3 uses 32-bit addressing. ↩︎
 
  Therefore sparing me one of the sources of my ADHD. ↩︎
 
  The reason it’s a laptop is because it’s ALWAYS on my lap, even when I’m in front of my TV playing console games. ↩︎
 
  My greatest hit was writing a recursive strlen(3), and it took me until this year to learn how this was a stupid idea. ↩︎
 
  Struct size arithmetics being another nice C subject, but, like everything C and beautiful: deterministic! ↩︎
 
  Except “Summon” which is exclusive to Yuna. ↩︎
 
 
  


macOS Document Icons: a composition system exists (good luck finding it!)
2026-04-16T00:00:00+02:00
 Still building stuff for Spark-Club, and after way too many rewrites of my FFI¹, the Spark-Flasher desktop app was coming together, with every new commit bringing either an important feature, or some small UX polish.
 
 While I did dabble in image manipulation², and had a long Illustrator phase, I am not a designer, not even close. So when I designed the app icon, I just took the Spark-Club logo, slapped a free “flash” icon from Flaticon over it, and I called it a day.
 A few commits and features later, as I was learning how to associate the .spark-pi bundles to my application, I noticed they had no icon³. Right, obviously, I needed an icon (or as Apple calls them: “Document Icons”) for the app, and an icon for the documents. If I wanted my .spark-pi files to stand out, I needed to give them identity: this time, I would slap a free “raspberry” icon from Flaticon on top of the logo. Oh well, how hard can that be?
 The best-kept secret of macOS app development
 Things you can hear in love stories and macOS app development: “How do you turn the page?”
 Sure, I can stack layers, but while browsing my Downloads folder to see how other icons worked, I noticed it: they all had the page curl. Yes, you KNOW which one. Every document got it, and it’s almost ALWAYS the same, as if there were a single tool generating them.
 But, no, whichever tool I was looking at, even in open-source, and even the official one, I could find this insidious page curl anywhere! No matter what keywords I used, I would only get results about “Here is how you add the icon you already made like the wonderful person that you are into your app”.
 
 The closest suggestion was to make an icon off Apple’s GenericDocumentIcon.icns⁴, which…was actually something I could do, but even then, this one had a rounded curl, while most of the icons on my machine used a square curl? The mystery deepened: there were two curls: one was for “real things”, and the other was “generic”, and I didn’t want that for my app!
 I was clearly missing something, and the best next step was to gaze at the situation from atop the shoulders of giants. In this case: Transmission. The project was mature, open-source, perfectly integrated, and they had a beautiful document icon, so there should have been something in there, helping me, no? No. Their document icon was a perfect .icns file with raster graphics, and trying to edit my half-assed logo over it was beyond my capabilities.
 I was getting nowhere that wasn’t in a frustrated state.
 Going to the source
 Something great about Apple⁵, is the rule that “an app made for Apple must work just like any other app on Apple”. That’s how you do proper UX, that’s how you don’t end up with a crazy designer trying to sell you their “framework on design” to justify a huge bill. So clearly, somewhere in their design guidelines, there must have been a guide about that. Luckily for me, there was.
 
  To create a custom document icon, you can supply any combination of background fill, center image, and text. The system layers, positions, and masks these elements as needed and composites them onto the familiar folded-corner icon shape. 
 (…) 
 macOS composites the elements you supply to produce your custom document icon. 
 — Platform considerations - macOS: Document icons
 
 The example was mind-blowing. So that’s how it was working! I scrolled down: a link to the Apple Design Resources promised “a template you can use to create a custom background fill and center image for a document icon”. I clicked, opened it both in Figma and Sketch, and looked up “icon” then “document” in both. Nothing relevant. What the hell!?
 I went back to the guidelines and kept reading. It seemed like each of the two elements needed to be in the app as a .iconset. Fine, but that was still not telling me anything about the API in use there!
 I kept scrolling until I saw the document icon for the .xcodeproj file type. Yes, I know you, I’m seeing you every day, each time I’m opening you in XCo… Wait a minute.
 Dragging the answer out of the corpse
 So far, every time I had been opening a FLOSS project’s GitHub⁶, I was always going straight for the Info.plist file, hoping to find out what was different, and not finding anything. But this time, I knew an app that was doing it, no more prodding in the dark.
 I went into XCode’s .app bundle, and looked at Info.plist:
 class="highlight">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
$ cd /Applications/Xcode.app/Contents/
$ plutil -p Info.plist | grep xcodeproj -A5 -B15
      }
    }
    39 => {
      "UTTypeConformsTo" => [
        0 => "public.composite-content"
        1 => "com.apple.package"
      ]
      "UTTypeDescription" => "Xcode Project"
      "UTTypeIcons" => {
        "UTTypeIconBackgroundName" => "xcode-project-fill"
      }
      "UTTypeIdentifier" => "com.apple.xcode.project"
      "UTTypeReferenceURL" => "http://developer.apple.com/tools/xcode/"
      "UTTypeTagSpecification" => {
        "public.filename-extension" => [
          0 => "xcodeproj"
          1 => "xcode"
          2 => "pbproj"
        ]
      }
    }
 Once again, the answer was right in my face. Obviously, it was the UTTypeIconBackgroundName property. I opened a search engine tab, clicked on it. Only two pages, no official API from Apple.
 And the first one was a StackOverflow question. I read it with shock: the question described the full API:
 class="highlight">1
2
3
4
5
6
7
8
9
<key>UTTypeIcons</key>
<dict>
    <key>UTTypeIconBackgroundName</key>
    <string>Icon Fill</string>
    <key>UTTypeIconBadgeName</key>
    <string>Icon Image</string>
    <key>UTTypeIconText</key>
    <string>Dapka</string>
</dict>
 To my horror, the most upvoted answer was a complete dismissal of this API, instead telling the user to use the CFBundleTypeIconFile configuration key to provide a full icon… I looked at the other links: one open-source app
 Wrapping
 At that point, I had everything I needed: a barely documented API, a couple of scattered examples, and a system that clearly worked. Just not in any way that was discoverable. I wired up my “badge” (attention: it must be a .iconset, using a .imageset won’t work!), killed the Finder, waited for it to restart, and the Spark-Raspberry logo showed up, with a glorious page curl.
 What still puzzles me isn’t about how it works, but about how to find it. Looking up CFBundleTypeIconFile on Github gives thousands of results, while the UTTypeIcon* keys barely give more than five hundred each, and while Apple’s own guidelines clearly describe a compositing system, the actual keys (UTTypeIconBackgroundName, etc.) are barely documented and you have to read their app’s Info.plist to get clues, while the internet as a whole will just point you toward “make your own icon and use CFBundleTypeIconFile!”.
 In the end, macOS does have a proper document icon composition system, just not one you’re supposed to find easily (or not one anyone bothered to explain properly?), that almost sounds like it’s a private Apple API.
 Either way, if you were also wondering where that page curl came from:
 Now you too can turn the page.
 
    A story for another post, promise. ↩︎
 
  In this place, we worship Paint Shop Pro! ↩︎
 
  Restarting my Finder would have fixed the association, and they would have gotten the default icon, which is something I admit I did not think of at the time. ↩︎
 
  Located in ` /System/Library/CoreServices/CoreTypes.bundle/Contents/Resources/GenericDocumentIcon.icns`. ↩︎
 
  Midwits would call it annoying. ↩︎
 
  For brevity’s sake, I did not describe ALL the repositories I went to search into… ↩︎
 
 
  


How to write to /dev/rdiskX without root: a look at macOS authopen
2026-04-15T00:00:00+02:00
 Hot off the heels of writing a C API to manipulate Raspbian images, I started writing the Swift app, which led to more API rewrites, until finally, I had a final image file and a /dev/rdiskX device as the destination. All that was left was a simple copy from one to the other.
 And obviously, no sane operating system would let a user arbitrarily write onto a raw disk device, right?
 Privilege escalations on MacOS
 If you look this up online, you would get a lot of very nice security writeups, and a looming sense of despair, as you don’t have a 0-day handy to give your (perfectly legitimate!) app the same capabilities.
 The truth is that beyond compromising your app, it’s generally not possible to securely elevate an application’s privileges once it’s already running. If you look at the countless deprecated solutions Apple gave to developers, you’ll see it’s all the same principle: “We’ll allow you to launch a helper with privileged access and communicate with it”.
 Which is actually quite a sensible, if somewhat cumbersome, approach: I was writing a Swift GUI app, the last thing I wanted was to spin up a separate privileged helper (and end up back in Rust).
 Thankfully, there exists a tool for that.
 The wisdom hiding in plain sight
 Among all the links you’ll find when searching for “macOS privilege escalation” (welcome to the watchlist!), one stands out, even if it doesn’t look like it. Unassuming at first, it turned out to be one of the smartest pieces of developer documentation I’ve read this year: Apple’s very own Elevating Privileges Safely - Secure Coding Guide.
 Compared to the rest of Apple’s developer documentation, this page feels very raw, almost empty, there’s no code at all, and yet everything it says is something I’ve repeated countless times to any junior developer asking for advice:
  Any program can come under attack, and probably will. 
 (…) 
 The principle of least privilege states: 
 “Every program and every user of the system should operate using the least set of privileges necessary to complete the job.” 
 —Saltzer, J.H. AND Schroeder, M.D., “The Protection of Information in Computer Systems,” Proceedings of the IEEE, vol. 63, no. 9, Sept 1975. 
 — The Hostile Environment and the Principle of Least Privilege
 
 While every line feels like a repeat of stuff that was hammered in my brain in school¹, down to the simplest of “Never trust user input parameters”, it feels good, especially in this emerging era of vibe-coded slop to still say those things out loud.
 That said, the document was long, light on examples, and I wasn’t convinced it would actually help in my case. I didn’t feel like experimenting with seteuid, so I dismissed it, perhaps too quickly, and went back to the real source of truth: rpi-imager.
 Upon the shoulders of people who solved the problem before me
 Searching for “privileges” in the code didn’t immediately help, but it was nudging me in the right direction: both Windows and Linux required elevated privileges, while macOS had a "sensible permissions model that operates as expected". Well, I know iOS does, and I always yell at apps asking me access to my contact list, but macOS too, really?
 I was nowhere close to an answer, so I continued through the MacOS parts of the code, marveling at all the tricks it used, like writing the result image straight to the device. Meanwhile, my Swift tool was writing the image to a temporary file and then copying it to /dev/rdiskX.
 And then, I found FileError MacOSFileOperations::OpenDevice(const std::string& path):
 class="highlight">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
FileError MacOSFileOperations::OpenDevice(const std::string& path) {
  std::cout << "Opening macOS device: " << path << std::endl;
  
  bool isBlockDevice = IsBlockDevicePath(path);
  
  // For raw device access on macOS, we need to use the authorization mechanism
  // Similar to how MacFile::authOpen() works
  if (isBlockDevice) {
    std::cout << "Device path detected, using macOS authorization..." << std::endl;
    
    // Create a MacFile instance to handle authorization
    MacFile macfile;
    QByteArray devicePath = QByteArray::fromStdString(path);
    
    auto authResult = macfile.authOpen(devicePath);
    if (authResult == MacFile::authOpenCancelled) {
      std::cout << "Authorization cancelled by user" << std::endl;
      return FileError::kOpenError;
    } else if (authResult == MacFile::authOpenError) {
      std::cout << "Authorization failed" << std::endl;
      return FileError::kOpenError;
    }
 Of course. I had flashed my own microSD cards a dozen times and every time, this MacOS authentification window would appear and ask me for my password because “Raspberry Pi Imager needs to access the disk to write the image.”. That was it, that was the gold mine: a 112-line function, all centered around something I had already heard of: const char *cmd = "/usr/libexec/authopen";.
 And just like that, the answer was right there in the Apple documentation:
  If you do need to run code with elevated privileges, there are several approaches you can take: 
 You can use the authopen command to read, create, or update a file (see authopen). 
 — Running with Elevated Privileges
 
 What’s an authopen anyway?
 As the online documentation was sparse, I turned to the other documentation source: man(1).
 Running man authopen proved to be informative enough:
  authopen provides authorization-based file opening services. In its simplest form, authopen verifies that it is allowed to open ~filename~ (using an appropriate sys.openfile.* authorization right) and then writes the file to stdout. (…) 
 authopen is designed to be used both from the command line and programmatically. The -stdoutpipe flag allows a parent process to receive an open file descriptor pointing to the file in question. 
 — man authopen(1)
 
 That was exactly what I needed: a temporary authorization to write to /dev/rdiskX!
 And then something dawned upon me: no matter how much I searched online, it seemed no one had ever documented how authopen worked. Sure, I was looking at this code, and I even found a few other implementations in Java or Rust, but otherwise, no one seemed to have documented how it actually worked.
 Well then. Time to reverse-engineer some C++.
 The missing authopen documentation
 All eating functions start with a fork()
 The first useful part of the code is a standard fork() setup:
 class="highlight">1
2
3
4
5
6
7
8
9
10
11
// Prepare command to run
const char *cmd = "/usr/libexec/authopen";
QByteArray mode = QByteArray::number(O_RDWR);
int pipe[2];
int stdinpipe[2];
// Set up a two-directional socket
::socketpair(AF_UNIX, SOCK_STREAM, 0, pipe);
// set up a pipe for stdin (unidirectional)
::pipe(stdinpipe);
// Let's fork
pid_t pid = ::fork();
 While they feel like a lost art in a world of GUI, Unix Pipes still power a huge part of systems, and their simplicity makes them usable in a ton of situations. In my case, having already written a shell² in C, there was no dark magic here, yet.
  
 Pipes and sockets Please note that in this case, we are creating a pipe for STDIN and a bi-directional socket for communication. There is a small difference in treatment between the two, and it will influence what happens later.
 
 Getting to authopen
 As the original code comment says: we are in “child” territory. In Unix, a fork(2) separates the process in two branches, with the return value indicating in which you are, a 0 indicating the child, and any other result, the parent.
 class="highlight">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
if (pid == 0)
{
  // child
  // Close the end of the socket we won't use
  ::close(pipe[0]);
  // Close the writing side of the pipe
  ::close(stdinpipe[1]);
  // Duplicate our side of the socket to capture STDOUT
  ::dup2(pipe[1], STDOUT_FILENO);
  // Duplicate the reading side of the pipe to write to STDIN
  ::dup2(stdinpipe[0], STDIN_FILENO);
  // Execute authopen
  ::execl(cmd, cmd, "-stdoutpipe", "-extauth", "-o", mode.data(), filename.data(), NULL);
  // In case we couldn't run authopen, return an error
  ::exit(-1);
}
 So, we close the parts of the pipes we won’t need, and reuse the remaining ones as authopen’s STDIN and STDOUT to communicate with the parent. At this point, it’s just plumbing.
  
 Exit minus one While it may seem strange to exit with a status of -1 here, it is actually normal: on successful execution, execl(3) (and the whole exec family) replaces the current process. Therefore, this line is only reached if launching authopen failed.
 
 Local news: parent closes door on child
 On the parent's side, nothing too unusual happens at first:
 class="highlight">1
2
3
4
5
6
7
8
9
10
11
// pid != 0, so we're still in the parent
else
{
  // Close the end of the socket we won't use
  ::close(pipe[1]);
  // Close the reading side of the pipe
  ::close(stdinpipe[0]);
  // Write on the pipe
  ::write(stdinpipe[1], externalForm.bytes, sizeof(externalForm.bytes));
  // Close the pipe
  ::close(stdinpipe[1]);
 Writing and closing the STDIN pipe isn’t something you’d typically do when using authopen on its own, but in this case, since the -extauth argument is passed, it’s required, as per the manual:
  -extauth specifies that authopen should read one AuthorizationExternalForm structure from stdin, convert it to an AuthorizationRef, and attempt to use it to authorize the open(2) operation. The authorization should refer to the sys.openfile right corresponding to the requested operation. 
 — man authopen(1)
 
 In other words, while you can use authopen to simply open a file, you can also pass additional options, like a custom prompt, for a better user experience.
 As for closing the pipe, it signals EOF, letting authopen know that input is complete, and that it can now kick things into gear!
 Local news: parent steals file descriptor from child
 The following part is where I actually began to stumble. And I still had no idea: how could a file descriptor be transferred from one process to another? So far, every time I had used one, it was an integer, and I knew its value was scoped to the process, and I couldn’t make sense of the model, so I looked up the first term that seemed promising: struct iovec.
  iovec - data storage structure for I/O using uio — man iovec(9s)
 
 Like a good episode of LOST, this actually provides me with a clean answer, and invites me to more questions. Unlike even the best episodes of LOST, these are actually questions I believe I can find answers to!
 While a lot of these terms sound intimidating, looking them up one by one will orient us towards very helpful documentation, and even a very informative Cloudflare blog post³.
 class="highlight">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
// Set up an iovec buffer, big enough for one integer
const size_t bufSize = CMSG_SPACE(sizeof(int));
char buf[bufSize];
struct iovec io_vec[1];
io_vec[0].iov_base = buf;
io_vec[0].iov_len = bufSize;
// Set up a control message buffer, big enough for one integer
const size_t cmsgSize = CMSG_SPACE(sizeof(int));
char cmsg[cmsgSize];

// Set up a header structure, referencing our two previous buffer
struct msghdr msg = {0};
msg.msg_iov = io_vec;
msg.msg_iovlen = 1;
msg.msg_control = cmsg;
msg.msg_controllen = cmsgSize;

// Read from the socket until we get something
ssize_t size;
do {
    size = recvmsg(pipe[0], &msg, 0);
} while (size == -1 && errno == EINTR);

// If we get some data
if (size > 0) {
  // Extract the first control message header
  struct cmsghdr *chdr = CMSG_FIRSTHDR(&msg);
  // Check that we received SCM_RIGHTS
  if (chdr && chdr->cmsg_type == SCM_RIGHTS) {
    qDebug() << "SCMRIGHTS";
    // Extract the FD from the data part
    fd = *( (int*) (CMSG_DATA(chdr)) );
  }
}
 Okay, that was strange!
 The Unix socket API includes a lesser-known feature called control messages: a way to attach metadata to a message, alongside the regular byte stream. One such metadata type is SCM_RIGHTS, which allows a process to transfer an open file descriptor to another process.
 In our case, this is the “secret sauce” behind authopen: instead of returning a file path or raw data, it sends back a fully opened (and privileged) file descriptor through that mechanism.
 And just like a good LOST episode, my initial assumptions got trumped, as the iovec structure wasn’t the solution to the mystery.
  
 Reading for zero While you may be accustomed to the form read(stream, buf, size), in that case, the third argument to recvmsg is flags, with the msghdr describing the buffers used to receive both data and metadata.
 
 Wrapping up
 What started as a simple question (“how to write to /dev/rdiskX?”) turned into a small journey through macOS internals, Unix IPC, and a surprisingly elegant piece of engineering.
 Instead of elevating the entire process, authopen takes a much more surgical approach: it opens the file with the right privileges, and hands you back a file descriptor. No daemon, no persistent helper, no global privilege escalation. Just a capability, passed from one process to another.
 Along the way, I also got a reminder of how much power still lies in the old Unix primitives. Pipes, sockets, fork, exec, and a few lesser-known features like control messages are enough to build surprisingly robust systems… if you know where to look.
 And, perhaps more importantly, that sometimes the best documentation isn’t documentation at all, but the code of someone who already solved the problem.
 
    My school is famous for piping /dev/urandom into student binaries to test how much they clean up user input. ↩︎
 
  A very humbling experience… ↩︎
 
  Another reason I love their service! ↩︎
 
 
  


Making a Raspberry Pi image builder
2026-04-11T00:00:00+02:00
 My work with Spark-Club has led me to implement an IoT fleet. While the process to build a common system on each device came with its own set of complications, the most complicated was to write a guide to ensure anyone with a Mac could set up IoTs. After encountering one too many PEBKAC¹ issues, I decided that having a single-step process was the new priority.
 The old way: rpi-imager
 I cannot overstate how much of rpi-imager is a work of art, passion, and dedication. When I set up the first Pi for Spark-Club, it just felt so easy, my Wi-Fi info was already there, I could SSH into the Pi without relying on Ethernet, I could take out the microSD card and restart the process to try new things,…
 And, it’s great, really. I’m pretty sure anyone who gets their hand on a Pi would be delighted about how it just works, the GUI wizard is easy, image download, light customization,… It’s perfect, too perfect actually, because it’s only perfect for humans.
 While the app can be started in CLI mode to control it from a script it’s unfortunately very limited, and the best you can do is attach a firstrun.sh file to the Linux image that will include some basic setup that will run on the device’s first boot.
 Our requirements
 When I finalized the setup required to run our Pis, we had the following files:
  firstrun.sh sets up the hostname, current user, wi-fi passwords, .authorized_keys, sound devices, services, and copies files to the user $HOME directory.
 setup.sh is ran on the user’s first boot, and sets up user-level configuration we don’t have access to during the first run process.
 {UUID}.spark-club is the ENV file the device uses to uniquely identify itself.
 And some more secret ingredients…
 
 Since rpi-imager would only copy the first one to the image, there would be a multi-step process:
  Get the configuration files for the device being set up
 Flash the microSD with the Raspbian image, customized with firstrun.sh.
 Re-insert the microSD in the mac, and copy the rest of files on the device.
 
 With too any steps, it was VERY EASY to for things to break, so I quickly looked into alternatives.
 The alternatives: How much HEREDOC do you want?
 The first thing I looked into was the --cloudinit-userdata <cloudinit-userdata> flag.
 Touting itself as “The standard for customising cloud instances”, cloud-init could have solved my problem. As a Rails user, I’m not intimidated by YAML files, but writing huge files using YAML, while possible felt very counter-intuitive to my usage: I wanted to add multiple files at need, and here I was, writing their content and packing them all as strings in a dictionary.
 The solution could have been there, but not for my needs.
 The alternatives: “are you a sysadmin enough dude?”
 Therefore, how else do you customize a Raspbian image?
 Easily: you build your own.
 I won’t even go there: there’s a point where “customizing your image” turns into “Spark-Club becomes the first sports company with its own Linux distribution”, and it’s a line I’m not crossing. But if you’ve got the team for it, if your business relies on owning the tech, and requires the distro tailored to your needs, it’s the perfect answer!
 In-between: an image with arbitrary files
 A Raspberry Pi image is actually a MBR² defining two partitions: a 512MB FAT32 boot partition, and a 5-and-some-gigs ext4 Linux partition. While there are ways to play with an ext4 partition without mounting it, since my previous process involved mounting the microSD’s FAT32 boot partition to copy setup files, I thought playing with the boot partition would be the best (and easiest!) solution.
 And it was!
 Enter rpi-imgpatcher
 Using mbrman to parse the MBR records and identify the FAT32 partition location on the drive, then a rust implementation of fatfs to manipulate the FAT32 partition filesystem, a quick Rust script managed to write a “Hello World” inside my image, which felt really great!
 The next step was devising an API and turning it into a proper library, which led to some design decisions.
 For instance, fatfs writes straight to the FAT buffer it’s working on, which means a careless approach would modify an image in place. If you want to generate multiple images, keeping the original untouched is a must³.
 First design decision: we never modify the original image in place, the FAT partition is always extracted so it can be worked on, and then saved to a new file, keeping the original MBR record and ext4 partition intact (they are copied straight from the original image).
 Second design decision: how to interact with the FAT filesystem. The simplest idea was an obvious add_file(fat_path: String, local_file: PathBuf), but I quickly realized I also needed to write arbitrary bytes, and even append to existing files, would be good, since it’s (almost!) what rpi-imager does when editing cmdline.txt.
 After a few hours of work, I had a small but useful API I was very proud of:
 class="highlight">1
2
3
4
5
6
7
8
impl RpiImage {
  pub fn new(image_path: impl AsRef<Path>) -> Result<Self, Error>
  pub fn read_file(&self, fat_path: &str) -> Result<Vec<u8>, Error>
  pub fn write_file(&mut self, fat_path: &str, file: impl AsRef<Path>) -> Result<u64, Error>
  pub fn write_bytes(&mut self, fat_path: &str, bytes: &[u8]) -> Result<u64, Error>
  pub fn append_bytes(&mut self, fat_path: &str, bytes: &[u8]) -> Result<u64, Error>
  pub fn save_to_file(self, out_file: impl AsRef<Path>) -> Result<(), Error>
}
 Simple enough to use in a script, but flexible enough to cover most use cases I could think of.
 I felt so good, that I decided against going to sleep (I mean, it was Sunday already morning, there was no point anymore), and kept adding features.
 FFI: Foreign Function Interface
 A huge selling point of rpi-imager, that can’t be overstated , is that it runs on all OSes, with a level of abstraction that’s close to artistry. Whatever your desktop OS is, it WORKS.
 In my case, this wasn’t a concern: the whole team uses MacBooks. I did consider using a Rust GUI library for a while, but I felt like going the extra mile and learning some more. Plus I had already written an FFI implementation (for a WASM library) a few days ago, so things felt fresh, and it’s always fun to use nm in CLI to look at a library’s contents.
 Just as much as I love C for the beauty of its design, I love the Foreign function interface. Just imagine: by using a common calling method, you can compile your Rust code into a library, and then that library can be called by a script language,…
 It does, however, come with caveats: everyone must adhere to C is the common denominator, which can sometimes be a hurdle. While ::new() was easy to write (we just return a pointer to an opaque struct, could be worse!), ::write_file came with some complications: how to return a value OR an error? Rust’s Result<Ok, Err> makes this trivial, but in C…?
 I’m not gonna blow your brains here, the answer is as old as C: one goes in the return value, the other in an outparam, a function argument that can be modified and checked by the caller. By convention, the return value is the Ok() value and the outparam is the Err() value, if there is one.
 A simple macro made quick work of it:
 class="highlight">1
2
3
4
5
6
7
8
macro_rules! return_out {
  ($out:expr, $err:expr) => {
    if !$out.is_null() {
      unsafe { *$out = $err };
    }
    return -1;
  };
}
 A “writing” function would return the number of bytes written, or -1 in the case of an error, and the error value would be the out param. Perfect!
 Which got me to my first real defeat: ::read().
 While C is beautiful, it’s also a harsh mistress, and won’t let you casually move string buffers around. The cleanest way to get a String from rust to C is to know its size, allocate a buffer, and give it to Rust to write into.
 Unfortunately, the fatfs library doesn’t implement an easy ::size() function on a file, which means my C API has no way to know the file size before reading it. The only alternative would be to have Rust allocate a buffer, lose ownership of it, and give it to C, which is the best way to leak memory (and nobody wants that).
 A similar issue came when implementing the C API for ::read_bytes(). While the argument is just an array of bytes, C has no native idea of arrays⁴. Therefore, from C, you must pass both a pointer to the array of bytes AND its length. Nothing too hard, but always a surprise.
 In the end, with a good C API, I could write a full implementation test in C, and the library was ready to integrate into anything, even a Swift app, which was the initial goal!
 Everyone wants their Thingfile
 I wasn’t ready to sleep yet. While I was coding, one idea kept coming back: “Why not have a DSL for the tool?”. I mean, sure, you could use the library from any language, but really…would you want to?
 That’s why I needed a binary, and not something with a bloated args parsing library. Since docker got the Dockerfile, having a similar DSL felt like a no-brainer:
 class="highlight">1
2
3
4
5
6
7
8
9
10
FROM 2025-12-04-raspios-trixie-arm64.img

EXEC cat firstrun.sh | sed "s|spark-base|spark-$NAME|g" > firstrun-$NAME.sh

ADD FILE firstrun.sh firstrun-$NAME.sh
(...more instructions)

APPEND FILE cmdline.txt "systemd.run=/boot/firstrun.sh systemd.run_success_action=reboot systemd.unit=kernel-command-line.target"

SAVE trixie-$NAME.img
 I love writing parsers for DSLs, especially in Rust. Every time I write a small VM for an AdventOfCode exercise, Rust’s match operator makes it a delight to do, and the implementation of traits trivialized turning error enums into helpful error messages.
 So it comes as no surprise that my problem wasn’t with the code, but with the image itself: Raspbian follows instructions from cmdline.txt to boot up. When you set up a firstrun.sh in rpi-imager, it appends a configuration telling it to run it on first boot.
 As the name implies, it’s a single line.
 Which comes back to bite you the moment you realize the original file ends with a newline, preventing any meaningful APPEND operation on it.
 Maybe because of the lack of sleep, I decided to add an APPEND CMDLINE instruction to the DSL, and treat this file as a special case. Though saner people may prefer overwriting the whole file with a clean one with ADD FILE.
 And because there’s no such thing as “enough work”, I also added $VARIABLE substitution using environment variables, turning what started as a script into a proper build pipeline.
 One last thing
 Anytime you download a Raspbian image from the Raspberry Pi image server, it comes as a .xz file, a compression format derived from LZMA. There is a Rust library that can handle that, of course, but the question was: how much changes would I have to implement if I wanted to support reading from (or even writing to!) a .xz archive?
 The answer was…not much.
 My original implementation was based on File structs, and passing data from input to output using std::io::copy. Do you notice the arguments to that function? Those are generic traits: std::io::Read and std::io::Write.
 As long as I could provide a type implementing those traits, everything became plug-and-play⁵, and could even be extended to any archive type exposing a Read stream.
 That said, my enthusiasm had limits: reading from a .img.xz slowed things down by a factor of 10: from ~6 seconds to nearly a minute. And my patience ran out before even trying to save an .img.xz archive.
 Nonetheless, discovering how Rust traits can improve internal API compatibility was yet another moment of appreciation for the language’s design, just like when I first discovered how Ruby’s Enumerable works.
 Coming next
 Despite a few rough edges, the API already feels very close to production-ready. The real test now is integrating it into my Swift desktop application.
 
    Problem Exists Between Keyboard And Chair ↩︎
 
  A Master Boot Record is the boot sector of a hard drive, defining its partitions. ↩︎
 
  It feels shameful to admit, but I don’t have fiber at home, and my download speed is limited to 2MB/s. ↩︎
 
  While C strings are “char arrays”, and by convention end on \0, and while it’s often a convention to indicate the end of a pointer array with a NULL, it’s not a convention you can rely on for arbitrary data! ↩︎
 
  I’m not gonna sugar-coat it, type-checker’s ability to guide you through any change you make is a huge reason to love Rust. ↩︎
 
 
  


Cloudflare's Gateway Drug
2025-06-06T00:00:00+02:00
 When you’re not yet huge enough to run your own datacenter, or rent a 2U server colocation, hosting web applications can be a pain, and as DHH eloquently wrote it: the merchants of complexity made sure all developers found themselves faced with overpriced technologies they wouldn’t actually need.
 But it turns out there’s a much simpler (and often free!) path to getting projects online, if you know where to look.
 The myth of the million visitors
 Three years ago, I was writing the API for a chat app where the higher-ups suddenly decided we required a Kubernetes cluster to handle the influx of users that would be coming our way on launch day. It took three months and a team of two freelancers. In all fairness, those freelancers built exactly what was asked of them: our very own K8S cluster, with production/staging support, CD on git deploy, and even auto-rent of short-burst Amazon EC2 instances to get them for a lower price. Seriously, what could go wrong?
 The infrastructure was bulletproof. No traffic ever bothered to shoot at us.
 Before
 As mainly a web developer, each project always comes with a pain: where am I gonna host it? I’m not a viral developer most of the time, my projects could actually be run and served off my own laptop¹. What then?
  Spin up a t3.nano on AWS? No, I don’t want to set up a server every time I want to put something online?
 One giant server for everything²? That’s too much work, too expensive upfront, and often, too many app environments to handle together. And on AWS, do I really want to live in fear of the EGRESS FEES!?
 So-called “serverless” ³ platforms like Heroku or Fly? They are actually nice, but some of them (looking at you Vercel!) have a tendency to produce bills in the six-figure ranges⁴.
 Github Pages? It’s free, comes with GitHub actions for deployments, and…there’s messing with gh-pages.
 
 So Heroku ended up as my default: cheap, easy, and if my project ever uses too much resources, they’ll gladly put it down. Kind of reassuring, actually.
 But then I stumbled onto Cloudflare, and everything changed.
 Hosting my own blog
 In May of 2024, I got back into the habit of reading books⁵, and a bad one got me to write about them⁶. So I went for a static blog with Jekyll⁷, and my first instinct was to look at Github Pages, but again, gh-pages: I didn’t want to deal with “artifacts” or any of those strange branch-switching shenanigans.
 At that point, I still thought Cloudflare was only about those annoying “Are you human?” pages on direct-download sites, CDN, or protecting sites from DDoS and all, which are not problems I’ve ever come into contact with. So I had no idea how I stumbled onto Cloudflare Pages. Surely the pricing for that kind of around-the-world service would be expensive, right?
  On both free and paid plans, requests to static assets are free and unlimited. A request is considered static when it does not invoke Functions. 
 ― Cloudflare Pages Pricing
 
 In Ruby on Rails terms, “static assets” means stuff like the CSS and the JavaScript that the server would create once at build time, and serve as-is to visitors. But if I’m using Jekyll, then that would mean the HTML files generated are ALSO static assets, right? Is it saying Cloudflare would host my blog FOR FREE!?
 Yes. Yes, it is.
 Hosting sites for other people (and feeling good about it)
 Chances are, if you’re reading this, you’re the most tech-aware person of your family or friend group, and you’re often asked: “Can you make me a website for my (small company/association/shop idea)?”. It’s okay, we’ve all been there, and I’m pretty sure we’ve all gone through the same thought process:
  Sure, I can make a website. Do they have a domain name or should I set it up? Do they have hosting? Should I set it up all by myself? I guess they won’t need something as powerful as a full Rails engine, or a machine with 4TB of RAM. Should I go for something static then? But where? How will I pay? How do I make sure the domain/website is under their name even if I’m doing all the legwork? And what if they want to update some text, should I add an admin panel, but then it would require auth,… Do I need to set up anything LetsEncrypt again? 
 ― You, me, and everybody else, before going to Wordpress.com.
 
 So when my aunt came to me for help, I was relieved: her association already had the domain and hosting. You know the kind: shared hosting, with its own FTP server. As I’m writing this, I’m trying to remember: did the admin panel indicate they were running PHP 4 or 5? Because I know it was neither 7 nor 8⁸ and it’s making me shiver.
 Thankfully, she just wanted me to rework an existing landing page, and it was static: no strange PHP scripts, no MySQL injections⁹ to sanitize,… Since static page means static assets, I put everything on GitHub, connected the repository to Cloudflare, changed the DNS records in their domain’s admin panel, and voila!, now if I (or a future developer) wanted to update the contents of their website, a simple git push would suffice, and the site would automatically deploy, and with a working SSL certificate!
 The next person who needed my help for a “website” was a friend who wanted to host an image online, so a QR code would lead to it, as part of a birthday treasure hunt. I could have sent her to Imgur, or said no. But now, what was once the daunting task of hosting was so painless that I set it up while half-watching a TV show: I deployed the image under a subdomain of my main site, and it just worked.
 That’s when I realized how happy I was. Hosting had become simple: not a bill to stress over, or a sysadmin side quest. Now, I could just put a file online, instantly, for anyone, with zero drama about pricing, or the egress fees of an AWS S3 bucket. And I had Cloudflare to thank for it.
 Code everywhere
 A few months later, one of my brothers was the one to ask for my help. With crypto. Oh boy.
 There was no trading involved, no degens to cater to,… But my brother had found a few wallets doing extraordinary calls, and he didn’t trust copy bot services. So while he could manually check the wallets online, having code to automate the process would make his “work”¹⁰ easier.
 I hadn’t touched crypto in a while, and my latest crypto project¹¹ was a lot of fun to code for. So I went along with it, looked up the documentation, wrote a JS script, and…now what? My brother did have a computer, an old Windows laptop from who-knows-when, and I wasn’t sure he could even run the script. Would you believe who came to the rescue again?
 For another project, I had been looking at whether Cloudflare supported CRON triggers. Well, if it could run mail jobs at 2am, surely it could also run my bot’s “check-balances-and-send-message” loops. But for what price, would that be free too?
 Yes. Yes, it is.
 The craziness of this pricing can’t be overstated: for my brother’s crypto fun, I had a CRON job running and sending him regular updates. I was even using a D1 Database to store the balances (which was also free for that level of usage), and best of all, I didn’t have to worry about any kind of intrusion, which I would have with a regular server.
 I don’t know how much my brother made, as I don’t really care about trading, but a few months later, he bought himself a MacBook Air, and later offered to buy me a PS5 Pro when I mentioned I was waiting for Death Stranding 2 coming out in June.
 Which is very nice from him, since, so far in this story, I still hadn’t paid a cent to Cloudflare.
 Finally paying for the product
 I finally paid for Cloudflare.
 I still had a professional project running on Heroku, both Rails server and PostgreSQL database, and I wanted to move some parts of the server API to faster NodeJS servers. Unfortunately, the provided database was VERY conservative with the number of allowed client connections. So Hyperdrive felt like an awesome solution, both from a technological standpoint and as an answer to my needs. There was no free tier¹², but just subscribing to the “Workers Plan” enabled unlimited usage, plus it would even increase the (very generous) limits of all other products.
 After a lot of tomfoolery regarding client connection limits and worldwide latency¹³, I ended up with a NodeJS API available worldwide, connecting to my database through Hyperdrive, with a performance measured in the hundreds of milliseconds, all in one night’s work, and a commitment to a monthly payment of only 5 DOLLARS per month. Take that, Kubernetes!
 Honestly, it didn’t even feel strange to finally “give in” and punch in my credit card digits: after so much free utility, it was a no-brainer. For the price of a fancy coffee¹⁴, I got performance and global reach my old infra couldn’t touch!
 The future
 Hey, if someone at Cloudflare is reading this, do you have any opening for a developer evangelist in France?
 When I look at the Node APIs I had started writing to decouple some parts of my Rails monolith, they are hosted on either Heroku, or Fly, and these products are great, but getting one too many “{application_name} ran out of memory and crashed” emails from them was the nail in the coffin. Sure, I’m not leaving Heroku anytime soon, because ActiveAdmin is still a wonderful backoffice solution and requires Rails, but for anything that only needs JavaScript, I can’t imagine going anywhere other than the Workers.
 Which is even funnier when I consider that I’ve barely touched all the available services: I set up Email routing because I don’t need a full-priced tool just to redirect domain emails to my Gmail, I’m still scratching the surface of Workflows (integrating that into my CRONs), I don’t need R2 yet, and I haven’t tried Queues but they look interesting. And there are a bunch more I haven’t even looked at yet!
 But hey, if you’d rather spend time on Twitter circlejerking about the size of the Kubernetes cluster serving five users, or complaining about the six-figure bill that crept up on you¹⁵, you do you.
 Me? I’m building on Cloudflare.
 
    I could, if I had something better than 2MB/s on my home line. ↩︎
 
  Most notably, “indie” maker @levelsio does that. ↩︎
 
  Kudos to whoever coined this term that’s so far away from reality! ↩︎
 
  If you don’t believe me, check out Serverless Horrors! ↩︎
 
  The book to kickstart it was Jake Adelstein’s Tokyo Vice. ↩︎
 
  This book shall live in infamy: Aux Origines de Castlevania Symphony of the Night. ↩︎
 
  Why choose Jekyll? I’m a ruby boy at heart, and I liked the theme I had seen. ↩︎
 
  Can you believe there never was a PHP 6 major version? ↩︎
 
  A staple of all developers who ever touched PHP is learning about SQL injections, sometimes the hard way. ↩︎
 
  I don’t believe day-trading to be a real job, more like an addiction. ↩︎
 
  For Valentine’s Day 2018, I launched The Valentine Coin with a friend. ↩︎
 
  Quite funnily, Hyperdrive now has a free tier, but it’s too low for my requirements. ↩︎
 
  Which will be covered in a further post. ↩︎
 
  And to think that ten years ago, I was buying myself one of those “fancy coffees” every day, if not two per day… ↩︎
 
  Don’t forget to tweet “VICTORY!” when the company discounts it to four in “a commercial gesture”. ↩︎
 
 
  


Adding Giscus comments to a static blog
2024-10-31T00:00:00+01:00
 For my personal blog, which I started a few months ago already, I don’t really need comments. It’s more of a “keep-a-useful-calendar-of-medias-I-consume” list, I write it in French, and it’s not meant to open discussion, so I wasn’t looking at comments. At all.
 However, for this tech-related blog, posting the first article on Reddit gave way to very interesting discussions and commentaries. So I figured: “Heh, why not?”.
 The state of Internet comments in 2024
 My very first personal blog, back in 2004, was written in PHP and hosted on a Free server¹. The (anonymous) comments were saved in own my database, and when I went back to check on it twenty years later, the place was littered with spam, and what I’m pretty sure were various attempts at HTML injections.
 Clearly, nobody wants that. And nobody wants to spend time away from writing in order to play warden and build an user-authentification service just for a few comments². Plus, this blog’s HTML is only rendered once I deploy it, so there’ no way I can add comments anywhere, I guess.
 And then there is Disqus.
 Disqus looks nice. It’s a testament to its beauty in design that every solution tries to imitate Disqus today. I remember over time seeing a LOT of competitors³, but only Disqus survived the test of time, and grew old enough to drink some alcohol in France.
 However, Disqus is also a proprietary solution, I’m not entirely sure of where they stand on GDPR, I’ve seen enough Disqus conversations get derailed by trolls,… This is not the right choice for a tech blog.
 Enter Giscus
 On the paper, Giscus fills what I’m missing:
  Open-source.
 Commenting requires a GitHub account.
 Comments are hosted on GitHub.
 
 Therefore, only developers will comment. Maybe it’s gonna end up being a fate worse than trolls.
 Setting up Giscus
 Setting up Giscus is actually a breeze and their homepage is a very nice setup wizard. My only issues were:
  Which repository is best suited for Giscus?
 
 Giscus comment sections are tied to a repository, which left me puzzled, as I wasn’t sure WHICH repository I should tie them to: this blog’s repository? Or another and completely unrelated one?
 The answer is…it’s up to you, but it must be public. So if your blog is in a private repository, you’ll need a specific repo for the comments. I chose to have everything together to have the full “static-page-blog-on-github” experience, but it’s nice to have a choice.
  What are those pathname/url/<title> options?
 
 Just pick one and forget about it. Having this option feels like too much hassle for what it is, and if you ever change an article’s title/URL, you’ll be screwed anyway. I’d have like an option to map the conversation to something like <meta property="giscus:uuid" value=>, and maybe this will be available in the future, BUT this would break the nice display of the discussions on Github.
 For my part, I started with pathname but just having an URL as the title of the GitHub Discussion was irking me, so I went the og:title route and customised it a little to add the date.
  Which theme should I use on my blog?
 
 I’m used to Dark Mode on Github, but this blog is very LIGHT. Starting with a dark theme was a wrong (and very ugly) move, and trying one of the “no-border” themes rendered the comment section almost invisible.
 The preview on the setup wizard isn’t very effective in guessing how it’s gonna fit on your blog, so apart from manually trying each theme, just decide on light_protanopia or dark_protanopia and roll on with it.
 Adding Giscus
 You’ll get a nice <script> tag to copy-paste into your template. The whole process is so simple, it’s almost ridiculous.
 And then?
 An input will be available when your visitors reach the end of your posts, and they’ll be able to post comments.
 Whenever a comment is posted:
  you’ll get notification in GitHub (nice!).
 you’ll be able to follow (and moderate?) the discussion in your repository’s Discussions tab.
 
 Happy commenting.
 
    “Free” as in both “free of charge”, and “Free the French ISP”. ↩︎
 
  But I’m keeping hope I will become famous in the future and get tons of comments! ↩︎
 
  I remember being quite fond of moot back then, just for its name, but the product is dead as nails today. ↩︎
 
 
  


Trimming down a rust binary in half
2024-10-27T00:00:00+02:00
  
 I posted this on reddit! The https://www.reddit.com/r/rust/comments/1gdd0md/trimming_down_a_rust_binary_in_half/ may interest you, but the best parts have been added to this post.
 
 Lately, I’ve stumbled on a blog post about Rust binary sizes. I haven’t done much compilation¹ since I last touched C in school, but I was intrigued by the subject and decided to look at how binary size reduction could impact my own Rust project: advent-rs, a simple binary taking year/day/part/filename parameters and solving exercises from the AdventOfCode online contest.
 Starting point
 There’s already some stuff I know and set up properly, so I already got the correct defaults for my Cargo.toml.
 class="highlight">1
2
[profile.release]
opt-level = 3
 Let’s look at the resulting file through size² and ls:
 class="highlight">1
2
3
4
5
6
$ size target/release/advent-rs
__TEXT	__DATA	__OBJC	others	dec	hex
1409024	16384	  0	      4295393280	4296818688	1001c4000

$ ls -lah  target/release/advent-rs
 -rwxr-xr-x@ 1 red  staff   1.8M Oct 27 08:49 target/release/advent-rs
 So we are standing at 1.8M, for a binary that’s solving around 150 exercices. That doesn’t sound so bad, but again, I haven’t compiled code in a long time, so I have no idea where I stand. Let’s check the first binary I can think of.
 class="highlight">1
2
$ ls -lah /bin/ls
-rwxr-xr-x  1 root  wheel   151K Sep  5 11:17 /bin/ls
 Well, I definitely need to do some trimming.
 Optimizing compilation
 First things first, let’s check compilation.
 Start with stripping
 As I remember from my C days, and as the blog clearly states: not stripping symbols from a release is the root of all evil.
  
 This is not an absolute! As fredbrancz rightly said, if you release a binary to the public at large and later require debugging it, missing the symbols will come back at you fast!
 
 class="highlight">1
2
3
[profile.release]
opt-level =  3
strip = true
 Build, check file.
 class="highlight">1
2
$ /bin/ls -lah  target/release/advent-rs
-rwxr-xr-x@ 1 red  staff   1.5M Oct 27 09:03 target/release/advent-rs
 Already 300kb trimmed off!
 Halt and catch fire
 A backtrace is a very useful to have when debugging, all interpreted languages come with them, and for a while, I wasn’t even surprised to see them pop up in Rust, in spite of C never printing one, only going as far as telling me “Segmentation fault” when my carefully-crafted binary would have the politeness to tell me on its own that I was doing something wrong.
 Well, Rust should not have them.
 Wait, let me explain myself here: backtraces are not a normal feature of a compiled language, since the backtraces you are used to actually come courtesy of the interpreter. So in a compiled language with no interpreter, where do they come from?
 Yes, from your binary.
 class="highlight">1
2
3
4
[profile.release]
opt-level =  3
strip = true
panic = "abort"
 Build, check file.
 class="highlight">1
2
$ /bin/ls -lah  target/release/advent-rs
-rwxr-xr-x@ 1 red  staff   1.2M Oct 27 09:05 target/release/advent-rs
 Can you believe this shaved off another 300kb?
 Insane stuff I don’t want
 The Cargo documentation gives us more options we can experiment with for fun. They don’t work for me but your mileage may vary.
 Optimize for size
 After 3, there are opt-level values that target smaller binaries. I can try with sor z and they will respectively shave off 100 and 200kb, BUT the tradeoff here is execution speed, which is not a value I want to allow myself to play with³.
 Don’t check integer overflow
 You can disable checking for integer overflow with overflow-checks = false from your binary. Obviously, this is NOT recommended when you’re playing with user input, and in my case, it won’t even register any change.
  
 Wrong! As GamerCounter pointed out, these checks are removed by default in release mode.
 
 Link-time optimization
 It seems that other than compiling, you can also optimise linking⁴ with lto = true. I don’t recommend it since it doubled my build time AND didn’t give me a good size reduction…
  
 Attention! I wasn’t clear enough on this, but as VorpalWay and hubbamybubba correctly pointed out, the benefits of this parameter depends on your project.
 
 Cleaning up crates
 The other thing that will take up space is…code. More specifically, code you wouldn’t need. Let’s check my dependencies:
 class="highlight">1
2
3
4
5
6
7
8
9
10
[dependencies]
clap = { version = "4.5.1", features = ["derive"] }
itertools = "0.12.1"
md-5 = "0.10.6"
mutants = "0.0.3"

[dev-dependencies]
assert_cmd = "2.0.13"
predicates = "3.1.0"
criterion = { version = "0.5.1", features = ["html_reports"] }
 Nothing too barbaric here, clap is the de-facto crate to parse command-line arguments, itertools is required because a ton of exercises require iterating over data in strange ways, I require md-5 for four exercices, and mutants is allowed here only to skip some tests when running mutations.
 As for the development dependencies, I know I got nothing to fear from them.
 Removing a crate
 You may feel like the above dependencies are sane, that I need all of them, that there is no way my project could function without any of them,… But what if it could?
 Past experience
 When I started writing all of these parsers I required for handling input data, I often used the regex crate. While it was very useful, I knew from my benchmarks that using old-school “split-on-this-char” approach was faster³, so I slowly started phasing it out.
 As of Year 2017: Day 15, I removed the crate from my Cargo.toml.
 Future ideas
 While I know that “rolling out your own crypto” is a bad idea, the crate md-5can clearly be replaced. Plus the fact that its input and outputs are of different types mean one exercise could actually be way faster⁵ if I rolled out my own!
 Okay, now where’s the bloat?
 I found out the appropriately-named tool cargo-bloat on GitHub.
 Let’s install and try it, it’s just a command⁶ after all.
 class="highlight">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
$ cargo bloat --release
    Finished `release` profile [optimized] target(s) in 0.01s
    Analyzing target/release/advent-rs

 File  .text     Size        Crate Name
 1.1%   1.6%  16.0KiB clap_builder clap_builder::parser::parser::Parser::get_matches_with
 1.1%   1.6%  15.8KiB    [Unknown] __mh_execute_header
 0.7%   1.1%  10.5KiB clap_builder clap_builder::builder::command::Command::_build_self
 0.6%   0.9%   8.9KiB          std std::backtrace_rs::symbolize::gimli::resolve
 0.6%   0.9%   8.7KiB          std std::backtrace_rs::symbolize::gimli::Context::new
 0.5%   0.7%   7.1KiB          std gimli::read::dwarf::Unit<R>::new
 0.4%   0.7%   6.5KiB clap_builder <clap_builder::error::format::RichFormatter as clap_builder::error::format::ErrorFormatter>::format_error
 0.4%   0.6%   6.1KiB clap_builder clap_builder::parser::validator::Validator::validate
 0.4%   0.6%   5.8KiB    advent_rs core::slice::sort::stable::quicksort::quicksort
 0.4%   0.6%   5.6KiB          std addr2line::ResUnit<R>::find_function_or_location::
 0.4%   0.5%   5.2KiB          std addr2line::Lines::parse
 0.4%   0.5%   5.2KiB clap_builder <alloc::vec::Vec<T,A> as core::clone::Clone>::clone
 0.3%   0.5%   5.0KiB clap_builder clap_builder::output::help_template::HelpTemplate::write_all_args
 0.3%   0.5%   5.0KiB clap_builder clap_builder::output::usage::Usage::write_arg_usage
 0.3%   0.5%   4.9KiB    advent_rs advent_rs::year_2016::day_10::solve
 0.3%   0.4%   4.3KiB clap_builder clap_builder::parser::parser::Parser::react
 0.3%   0.4%   4.2KiB clap_builder clap_builder::output::usage::Usage::write_usage_no_title
 0.3%   0.4%   4.2KiB    advent_rs advent_rs::year_2017::day_21::Image::mutate
 0.3%   0.4%   4.1KiB          std addr2line::function::Function<R>::parse_children
 0.3%   0.4%   3.9KiB clap_builder clap_builder::output::help_template::HelpTemplate::write_templated_help
58.8%  87.9% 872.3KiB              And 1925 smaller methods. Use -n N to show more.
66.9% 100.0% 992.5KiB              .text section size, the file size is 1.4MiB
    Never going to that fortune-teller ever again
 
 
 Something must be wrong, let’s try another command.
 class="highlight">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
$ cargo bloat --release --crates
    Finished `release` profile [optimized] target(s) in 0.01s
    Analyzing target/release/advent-rs

 File  .text     Size Crate
25.9%  38.8% 384.8KiB std
22.3%  33.4% 331.1KiB advent_rs
15.8%  23.5% 233.6KiB clap_builder
 1.2%   1.7%  17.1KiB itertools
 1.1%   1.6%  15.9KiB [Unknown]
 0.7%   1.0%  10.0KiB md5
 0.3%   0.5%   5.0KiB digest
 0.3%   0.5%   4.6KiB strsim
 0.2%   0.3%   3.3KiB clap_lex
 0.1%   0.2%   2.1KiB anstyle
 0.1%   0.2%   1.8KiB anstream
 0.0%   0.0%      16B colorchoice
66.9% 100.0% 992.5KiB .text section size, the file size is 1.4MiB
 First off, I’m surprised that md5 is only taking around 10Kb, so maybe I won’t need to replace it until I really want to shave a few milliseconds off that one exercise⁵. But for the rest…
 What the fuck Clap?
 It seems I’m not the first person to complain about clap’s binary size. I’m even surprised it can even get to this size because its only use is to parse an array of strings.
 I mean, seriously:
 class="highlight">1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
#[derive(Parser, Debug)]
#[command(version, about, long_about = None)]
struct Args {
  /// The year of the exercise, from 2015 to today
  #[arg(short, long, value_parser = clap::value_parser!(u16))]
  year:  u16,
  
  /// The day of the exercise, from 1 to 25
  #[arg(short, long, value_parser = clap::value_parser!(u8))]
  day:  u8,

  /// The part of the exercise, 1 or 2
  #[arg(short, long, default_value_t = 1, value_parser = clap::value_parser!(u8))]
  part:  u8,

  /// File name
  #[arg(help =  "Input file path (will read from STDIN if empty)", value_parser = clap::value_parser!(PathBuf))]
  input: Option<PathBuf>,
}
fn main() {
  let  args  = Args::parse();
}
 And I’m not even using ALL the features!
  
 By the way... I wasn’t very clear on this, but as hubbamybubba specified, you can select features from a crate import in your Cargo.toml.
 
 Picking a sane alternative to Clap
 Am I in the mood to look up for a new crate, and completely rewrite the input part of my code? Not really, but I’ve got a blog to write!
 Let’s do a quick Google, and thankfully, among the first results is a recap of various arg-parsing libraries:
  null Obviously, I’m not in the mood to parse them by name.
 argh Looks very similar to Clap.
 bpaf If I wanted a DSL, I’d be using Ruby.
 
  
 That's not a DSL! As manpacket corrected me, bpaf’s API is not a DSL (Domain-Specific Language), but a Fluent Interface.
 
  gumdrop That looks like A LOT of code.
 lexop I kinda like this one’s approach!
 pico-args No examples, that’s too simple for my taste.
 xflags Looks a bit too complicated.
 
 I can easily start with Argh then, it looks similar enough that I could maybe just drop it in and have it work.
  
 Attention! While many other libraries are just as small, I chose argh because it was simpler to adapt my code, but it comes with (depending on your setup) a huge caveat: it’s got issues with invalid UTF-8.
 
 The beauty of Rust unit tests
 As a developper, I’ve written a lot of untested code.
 With advent-rb, I used unit tests for each exercice, using the examples as inputs and result values in RSpec tests, which was very fun.
 With advent-rs, and Rust in general, I discovered a new paradigm. While all the languages I ever used made it “easy” to forget to add another file with the correct name, asked me to remember a complicated syntax, made me think of ways to cheat on my code to access private members from another file,… Rust just made away with that: testing a function is done from the same file where it’s defined, and I don’t even have to care about visibility.
    It’s all there!
 
 
 The only thing not handled out-of-the-box is testing a binary (well, that’s not really a unit test), but since everything else was so easy, it felt okay to take some time to write a test specifically for that in tests/cli.rs.
 In that case, rewriting is very easy:
  remove clap from dependencies.
 add argh to dependencies.
 rewrite my #[derive()]
 some syntax differences
 
 Quite frankly, the whole diff is nothing short of hilarious given how simple it is.
 But the best part?
 I can just run cargo test and my tests will tell me everything works the same!
 Cleaning up
 Let’s check up on that bloat one more time, shall we?
 class="highlight">1
2
3
4
5
6
7
8
9
10
11
12
13
$ cargo bloat --release --crates
    Finished `release` profile [optimized] target(s) in 0.01s
    Analyzing target/release/advent-rs

 File  .text     Size Crate
32.8%  49.3% 360.3KiB std
30.3%  45.5% 332.6KiB advent_rs
 1.6%   2.4%  17.2KiB itertools
 1.0%   1.5%  11.2KiB [Unknown]
 0.9%   1.4%  10.0KiB md5
 0.6%   0.8%   6.2KiB argh
 0.5%   0.7%   5.0KiB digest
66.5% 100.0% 730.3KiB .text section size, the file size is 1.1MiB
 As for the file size:
 class="highlight">1
2
$ ls -lah  target/release/advent-rs
-rwxr-xr-x@ 1 red  staff   898K Oct 27 11:02 target/release/advent-rs
 Woah, not bad, that shaved off 300kb more, and we reduced our total binary size by almost 1MB.
 Going just a bit further
 Once again, I am standing on the shoulders of giants, and while I am quite happy with the trimming I did, I found way more experimentations I could perform on johnthagen’s min-sized-rust repository.
 However, those solutions either require unstable features (we don’t do nightly in this house), or stripping off Rust’s standard library, which is a whole other can of worms…
 
    Blame NodeJS and Ruby! ↩︎
 
  As its man page will tell you, size prints the “sections” in a binary, helping you visualise the separation between code and data for instance. It’s not very needed here, but it’s a useful command to know about. ↩︎
 
  This project was also built to compete with advent-rb, the same project written in Ruby, to marvel at the speed of execution I was missing from compiled languages. ↩︎ ↩︎²
 
  Linking is the second part of building a project: in layman’s terms, it will bind all code parts together into a single executable. ↩︎
 
  Part two of Year 2016: Day 14 requires for a string to be hashed on itself 2016 times. Since the input must be a string, and the output is a buffer of hexadecimal data, my code needs to re-translate that hex buffer to a string 2016 times… ↩︎ ↩︎²
 
  The cargo-bloat command itself will actually bloat your code. I haven’t checked out how, but I’m 99% sure it’s keeping in some symbols in so it can return the name of the fonction taking space. In my case, it’s adding around 200kb. ↩︎
Red Leaves

RomHacking with Ghidra, Part 1: Loading SNES ROMs

Straight from the shoulders of giants

Banks and rommers

O ye’ll take the high ROM and I’ll take the low ROM

Mirror, mirror, on the ROM, who’s the fairest of them all?

Smashing the banks for fun and profits

99 Bottles of Chips on the wall

Credits and Acknowledgements

Exploring Spira's PS3 binary with Ghidra

Cheating is cooler when you learn stuff

The state of cheating on the PS3

Spira on my laptop

Pedal to the metal

From the shoulders of giants

Finding our first trophies

More functions, more options

ffxCombatEffect::Steal

ffxCombatEffect::Lancet

ffxCombatEffect::Bribe

ffxTrophyUnlock::JechtSpheres

FUN_00365344

ffxTrophyUnlock::Aeons

ffxTrophyUnlock::TheaterSphere

ffxTrophyUnlock::EternalCalmMovie

ffxTrophyUnlock::AlBhedPrimers

ffxTrophyUnlock::SphereGrid

FUN_003103f8

Dude, where’s my Chocobo?

Learning to give up

Thanks

macOS Document Icons: a composition system exists (good luck finding it!)

The best-kept secret of macOS app development

Going to the source

Dragging the answer out of the corpse

Wrapping

How to write to /dev/rdiskX without root: a look at macOS authopen

Privilege escalations on MacOS

The wisdom hiding in plain sight

Upon the shoulders of people who solved the problem before me

What’s an authopen anyway?

The missing authopen documentation

All eating functions start with a fork()

Getting to authopen

Local news: parent closes door on child

Local news: parent steals file descriptor from child

Wrapping up

Making a Raspberry Pi image builder

The old way: rpi-imager

Our requirements

The alternatives: How much HEREDOC do you want?

The alternatives: “are you a sysadmin enough dude?”

In-between: an image with arbitrary files

Enter rpi-imgpatcher

FFI: Foreign Function Interface

Everyone wants their Thingfile

One last thing

Coming next

Cloudflare's Gateway Drug

The myth of the million visitors

Before

Hosting my own blog

Hosting sites for other people (and feeling good about it)

Code everywhere

Finally paying for the product

The future

Adding Giscus comments to a static blog

The state of Internet comments in 2024

Enter Giscus

Setting up Giscus

Adding Giscus

And then?

Trimming down a rust binary in half

Starting point

Optimizing compilation

Start with stripping

Halt and catch fire

Insane stuff I don’t want

Optimize for size

Don’t check integer overflow
