< achow101> I'll try an updated native_cctools first
< sipa> cool
< sipa> it's also possible that jonasschnelli's codesign tool is too old!
< sipa> (as native_cctools was updated for 0.21)
< achow101> Well it's worked for the previous rc
< sipa> it may be that some alignment thing changed that doesn't matter in every situation
< sipa> maybe we should just make some arbitrary binary-changing commits for rc4, and hope that that fixes it
< sipa> *hides*
< sipa> jonasschnelli: what version is your cctools? if you have the ability to change it, use 949.0.1
< sipa> achow101: what version is your codesigning tool on macos?
< achow101> no idea
< achow101> there's no version command for codesign or codesign_allocate
< sipa> this line in codesign_allocate changed in the latest cctools:
< sipa> object->seg_linkedit->vmsize = rnd32(object->seg_linkedit->filesize, get_segalign_from_flag(&arch_signs[i].arch_flag));
< achow101> where do you see this?
< sipa> this is in the version we're using in the gitian build
< sipa> but given that it recently changed (may 2020), perhaps somehow jonasschnelli is using an apple equivalent that is older
< sipa> *march 2020
< sipa> actually, no
< sipa> rnd -> rnd32 doesn't change anything
< achow101> eh.. they don't make tags for new versions.. great
< sipa> got it!
< sipa> cctools v921 uses 4k as default alignment for x86_64
< sipa> oh, no, i misread :(
< sipa> what is the full output from that x86_64-apple-darwin-otool -l Bitcoin-Qt command?
< achow101> of which version of Bitcoin-Qt
< sipa> no matter
< gribble> https://github.com/bitcoin/bitcoin/issues/67 | does not compile with boost_1_45_0 on OS X 10.6 · Issue #67 · bitcoin/bitcoin · GitHub
< achow101> bad bot
< sipa> the __LINKEDIT "filesize" entry is the same for the codesign-output-binary and the gitian-reconstructed one, right?
< sipa> just vmsize is different
< achow101> yes
< sipa> yeah, if it wasnt't, more than 1 byte would differ
< sipa> literally every version of cctools i can find uses 0x1000 as segment alignment for X86_64
< sipa> so the correct rounding of filesize 500144 to vmsize is 0x7b000
< sipa> why is jonas' tool rounding it to 0x7c000 (like he's using a segment alignment of 0x2000 or 0x4000)
< sipa> now, libstuff uses 0x2000 for unknown CPU architectures
< sipa> so maybe somehow the wrong architecture is passed down?
< sipa> see https://opensource.apple.com/source/cctools/cctools-949.0.1/libstuff/arch.c.auto.html look for "{ CPU_TYPE_X86_64, LITTLE_ENDIAN_BYTE_SEX, 0x7fff5fc00000LL, 0x1000 },"
< achow101> updating cctools (specifically codesign_allocate) did not make a difference
< achow101> jonasschnelli: are you using the same signing machine as before?
< achow101> IIRC he has one of the M1 macs for testing stuff, maybe he used that for signing?
< sipa> i suspect that if we add 1024 bytes to the sig size in codesign_allocate it may work
< sipa> if you do:
< sipa> codesign_allocate -i <unsigned Bitcoin-Qt filename> -a x86_64 $((225312+1024)) -o <some temp file>
< sipa> on linux
< sipa> it may just work, with the existing signature
< achow101> I'm wondering why this size makes a difference to how the signature validates
< achow101> istm it shouldn't be covered by the signature itself
< sipa> i believe the code signature covers the entire binary, with the signature itself wiped out (this is a guess...)
< sipa> so its size matters
< sipa> that would explain why a tool like codesign_allocate exists in the first place
< sipa> as its behavior is nominal for the signing process
< achow101> but the size of the signature isn't known until the signature is made
< sipa> i assume that the codesign tool does now the signature size beforehand
< sipa> *know
< sipa> or it pads it to make sure it does
< achow101> i suppose it just needs to fall within the page size
< achow101> and that's probably a big enough range
< sipa> right
< achow101> so apple makes codesign_allocate open source, but not codesign itself?
< sipa> apparently
< sipa> but codesign invokes codesign_allocate
< achow101> supposedly
< sipa> and you can override which codesign_allocate to use... as long as it's signed by apple
< sipa> if you run "codesign_allocate -i <gitian Bitcoin-Qt filename> -a x86_64 $((225312+1024)) -o <newname>" in linux, does the resulting binary work in macos?
< sipa> oh, nvm
< sipa> nope, won't work
< sipa> that'll actually make space in the binary
< achow101> what if I set arch to ARM64 so it uses 0x4000 as the page size?
< achow101> and codesign_allocate doesn't like it
< sipa> yeah i don't expect that'd work
< sipa> achow101: but if you run codesign_allocate with 225312, on macos, you also get the 0x7c000 vmsize?
< achow101> Ah, yes
< sipa> so perhaps... the codesign tool isn't invoking it with size 225312
< sipa> mayne it invokes it twice
< sipa> once with a prospective size, to get the to-be-signed binary, and then again with the real size
< achow101> Now, I think the installation of codesign_allocate is different from the open source one
< sipa> that'd explain the filesize of LINKEDIT being correct, but vmsize isn't
< achow101> Because it matches jonas's, not gitian
< sipa> right
< sipa> if you on linux run codesign_allocate with a larger size first, and then again withnthe correct size, does the __linkedit stuff end up being correct?
< achow101> what's a big enough size?
< sipa> $((225312+1024))
< achow101> nope. also still 7b
< sipa> after both commands, or even after the first one?
< achow101> both
< sipa> oh, i'm wrong
< achow101> well both as in both both commands and first one
< sipa> 0x1000 is not 1024, but 4096
< sipa> so $((225312+4024))
< sipa> so $((225312+4096))
< achow101> oh right
< achow101> that works
< achow101> although I will say that this is not a suitable workaround
< sipa> we could produce in the gitian output the information on what size to resize to
< sipa> it can be deterministically inferred from the macos-signed binary
< achow101> i suppose
< achow101> it feels like there should be a better way
< sipa> yes
< sipa> there is clearly a discrepancy between the macos codesign_allocate tool and the opensource one
< sipa> and i'm confused why that isn't a problem for other projects
< achow101> maybe no one has tried to make a new release recently?
< achow101> we didn't see this a few weeks ago for rc2
< sipa> yeah, baffling
< sipa> lol @ actively {-beating our heads against Apple's walled garden wall-} debugging this issue
< sipa> there is a macports version of cctools, presumably using the open source code
< sipa> if you run that on macos, what behavior does it have?
< achow101> I don't have macports installed
< achow101> I found an old version of codesign source on apple's opensource site: https://opensource.apple.com/source/security_systemkeychain/security_systemkeychain-55191.1/src/codesign.cpp.auto.html
< achow101> it disappears eventually, but that might just mean it got moved to somewhere else
< sipa> that one doesn't invoke codesign_allocate
< sipa> as far i can see, but it may be in another source file
< achow101> sipa: macports cctools uses 0x7b000
< ConsciousFlesh> Are there any devs here who were involved with, or around when the `RPCHelpMan()` function replaced throwing a runtime error to display JSON-RPC help text?
< sipa> achow101: great, so the apple codesign_allocate just behaves differently from their published source code
< achow101> i guess we shouldn't be that surprised that they do different things
< achow101> sipa: adding the run codesign_allocate twice trick to detached-sig-apply.sh does make rc3 pass signature validation
< sipa> great
< sipa> so we just need something to take a size from the code-signatures.tgz
< sipa> and modify jonas' script to compute what the right pre-allocate size is
< achow101> I wonder if maybe it would be easier to codesign_allocate the unsigned dmg with a larger than necessary but fixed sized chunk
< achow101> the gitian signer's codesign allocate shrinks it, but I don't think effects the vmsize
< sipa> looking at the codesign_allocate tool, it just only ever increments the vmsize
< sipa> which is going to be the rounded-up version of the largest value the filesize ever had
< sipa> and whatever we do, we need to replicate the apple tool's behavior
< achow101> right, so we could always overshoot by a lot when we make the unsigned dmg
< sipa> yup
< sipa> that's a possibility
< luke-jr> [20:46:44] <jonasschnelli> is there a way to speeup gitians "Upgrading system, may take a while (log in var/install.log)"? <-- I just run the base VM and update it
< achow101> I'm going to go through history and see what the largest sigs we've had before is
< achow101> any idea if an overly large vmsize has any negative effects?
< sipa> i suspect if we crank up the codesign --pagesize value the sig will be much smaller
< sipa> it signs every page separately
< achow101> oh, is that why the sig is so big and varies a lot
< bitcoin-git> [bitcoin] pox opened pull request #20635: fix misleading comment about call to non-existing function (master...master) https://github.com/bitcoin/bitcoin/pull/20635
< jonasschnelli> sipa, achow101: I'm reading the backlog... for your info,... I haven't changed anything on my signing computer since signing rc2 (which was successful).
< achow101> jonasschnelli: good to know. the backlog is quite a journey..
< jonasschnelli> AFAIK detached-sig-create.sh under the hood, it uses the macOS codesign tool (which I haven't changed)
< jonasschnelli> achow101: yeah.. halfway through.
< jonasschnelli> Very interesting.
< jonasschnelli> 2020-12-11T23:57:47 <sipa> an alternative is making jonasschnelli downgrade his codesign_allocate tool to one that's compatible with the native_ccttols we use, and then tell his codesign tool to use that
< jonasschnelli> ^ as said,... I haven't upgraded anything since rc2
< jonasschnelli> 2020-12-12T00:12:49 <sipa> jonasschnelli: what version is your cctools? if you have the ability to change it, use 949.0.1
< jonasschnelli> I guess sipa wants to know what clang version I have on my signing mac. Why would this be relevant? IMO codesign is not part of the llvm suite
< jonasschnelli> And cctools on the gitian host should be irrelevant
< achow101> cctools is separate from clang
< achow101> it's part of xcode's developer tools
< achow101> jonasschnelli: the tl;dr is that codesign_allocate as distributed by apple is different from codesign_allocate compiled from source
< achow101> codesign uses codesign_allocate under the hood somewhere and that's causing us problems
< jonasschnelli> achow101: thats very interesting...
< sipa> jonasschnelli: can you try whag happens if you pass --pagesize 0 to the codesign tool?
< sipa> that's unrelated, but it may result in smaller ans more predictable signature sizes
< jonasschnelli> sipa: okay... I executed "./detached-sig-create.sh -s "Bitcoin" --pagesize 0"
< sipa> no no, to the codesign tool
< sipa> so inside detached-sig-create.sh
< achow101> sipa: I've figured out what that codesign source I found doesn't have a call to codesign_allocate - the actual signing is way deeper in the stack and isn't open source. the code I found is just the wrapper to expose things as a command line tool
< jonasschnelli> sipa: okay
< sipa> or does it pass everything through?
< achow101> passing extra args to detached-sig-create should pass through to codesign
< sipa> ok
< sipa> achow101: yeah, i tried browsing but i couldn't even find any place where the pagesize argument was used
< jonasschnelli> I changed L26 to ${CODESIGN} --pagesize 0 -f --file-list ${TEMPLIST} "$@" "${BUNDLE}"
< jonasschnelli> but why was rc2 fine and rc3 isn't?
< achow101> beats me
< achow101> I think we've found an edge case
< jonasschnelli> as I suspected
< achow101> it's weird that this is not present in the open soure build though
< sipa> jonasschnelli: it appears that the macos tool uses a different segment alignment
< sipa> as your signed result has an alignment of 8192 or 16384 bytes
< sipa> while the gitian result is 4096 byte aligned
< jonasschnelli> okay
< sipa> but the apple (open source) definitely uses 4096
< sipa> it may be that rc2 just accidentally was already 8192 aligned
< sipa> we could check
< sipa> jonasschnelli: how big is the .sign file now?
< achow101> sipa: I checked earlier, it's 7b0000
< sipa> ok, i have no explanation then :)
< jonasschnelli> sipa: jonasschnelli staff 18496 12 Dez 08:10 Bitcoin-Qt.sign
< sipa> much better!
< sipa> the previous ones were 200 kB or so
< jonasschnelli> I can try to gbuild the macOS signer with that
< sipa> it's worth a try
< sipa> if it was a weird edge case due to alignments, maybe this is enough to fix it
< achow101> --pagesize 0 might actually make it possible for us to hack together a tool for codesigning
< achow101> that doesn't require macOS
< jonasschnelli> pagesize 0 seems also not to work
< achow101> I was going to suggest that we try using the macports codesign_allocate, but the manpage for codesign says "The system will not accept such substitutes unless they are specially signed (by Apple)." on that topic, so I don't think it will work
< sipa> indeed
< sipa> jonasschnelli: same problem?
< sipa> achow101: is it again the vmsize being off?
< achow101> checking
< sipa> jonasschnelli: is that file the signed binary that comes out of the codesign tool?
< jonasschnelli> sipa: no... that file above is the gitian signer result with using the signature.tar.gz that was built with pagesize 0
< achow101> i need to compare against the result of the codesign tool
< sipa> jonasschnelli: we'll also need the codesign output
< jonasschnelli> okay.. one sec
< achow101> yes same problem
< achow101> codesign_allocate really likes 7c000 for some reason
< sipa> achow101: what are the vmsize values now?
< sipa> wait what
< sipa> what is the filesize field and the vmsize field in both?
< achow101> gitian is 0x4800 and codesign is 7c000
< sipa> dafuq
< achow101> fileoff and filesize are the same
< achow101> both are otherwise identical except for vmsize
< sipa> and what is that filesize?
< * jonasschnelli> throws away his mac
< achow101> filesize 293328
< sipa> jonasschnelli: a just decision
< achow101> signature is 18496
< sipa> 0x48000 is the correct vmsize
< sipa> well, "correct", from my understanding of what the opensource allocate tool does
< achow101> yes
< sipa> and that's no surprise, because that's also what it did
< sipa> but so this means it is not a rounding issue at all
< sipa> unless it's rounding up to a multiple of 0x7c000 !
< sipa> if you on macos run codesign_allocate infile -a x86_64 18496 -o outfile on the unsigned binary, does it also produce 0x7c000 as vmsize?
< jonasschnelli> let me try
< sipa> jonasschnelli: did you accidentally codesign an already-signed binary?
< jonasschnelli> sipa:.... hmm.. that could be for the latest just done build
< sipa> vmsize is not expected to go down in that case
< sipa> so you have to start with the unsigned binary
< achow101> vmsize 0x0000000000043188
< jonasschnelli> I try it again with pagesize 0 and make sure it's not signed already
< achow101> that's not a multple of 0x1000, 0x2000, or 0x4000
< sipa> achow101: wut
< achow101> ah wrong file
< achow101> vmsize 0x0000000000048000
< achow101> that's correct and what we expect
< sipa> achow101: with the macos tool?
< achow101> yes
< sipa> and just to be sure... if you pass 225312 as size?
< sipa> (i know we did that before, but perhaps that before we knew that history of sig sizes the binary has been through mattered)
< achow101> as before vmsize 0x000000000007c000
< sipa> ok
< sipa> well, if it's an alignment issue, 0x48000 is 0x4000 aligned, so unlikely to cause problem
< jonasschnelli> https://bitcoin.jonasschnelli.ch/bitcoin-0.21.0rc3-osx-unsigned-again-pagesize0.zip <--- signed with pagesize 0, check that is was not already codesigned
< sipa> so i expect this will be a solution
< jonasschnelli> (gitian output follows)
< achow101> sipa: it's aligning against 0x2000
< sipa> achow101: you found that by trying more sizes?
< achow101> if I use 4096 as the size, I get 46000. same with 8192
< jonasschnelli> and if anyone wants to test 0.20.2rc1, I just created the detached signature (no gitian build yet): https://bitcoin.jonasschnelli.ch/signature-osx_0.20.2rc1.tar.gz
< sipa> ok
< sipa> achow101: we could patch out cctools to do the same
< sipa> *our
< sipa> it's a tiny change
< achow101> I'm not optimistic that that's a lasting solution
< sipa> yeah, agree
< sipa> but it's simpler than trying to hack around the issue with multiple preallocate calls and passing the size through
< achow101> if we combine with pagesize 0, then it'll probably always work
< achow101> the largest signature since 0.12.1 is 407424
< sipa> achow101: if the gitian preallocate takes it to a large *odd* multiple of 0x1000, will the macos sign tool not increase it further to a multiple of 0x2000 ?
< sipa> if it's anything like the opensource version, it won't
< sipa> but who knows what it does
< jonasschnelli> oh!
< jonasschnelli> pagesize 0 result in a valid signatue (through gitian)
< jonasschnelli> I didn't before because I signed an already signed binary
< achow101> I think that is currently the expected result
< sipa> jonasschnelli: yeah, matches our understanding
< achow101> sipa: it does not appear to shrink the vmsize at all
< sipa> achow101: yes, but it may increase it further
< sipa> (i don't expect it to, but we should check i think)
< achow101> i'll try against the macports cctools
< achow101> it preserves it
< sipa> great
< sipa> 500000 is probably overkill if we're going to use pagesize 0
< sipa> but that does like a pretty simple fix
< achow101> probably, but I also have no idea what goes into the signature size
< achow101> i'll open a pr when i wake up
< jonasschnelli> thanks achow101!
< jonasschnelli> and sipa
< bitcoin-git> [bitcoin] MarcoFalke pushed 6 commits to master: https://github.com/bitcoin/bitcoin/compare/6a4806367177...ffc4d0499020
< bitcoin-git> bitcoin/master 5aaeb6c Russell Yanofsky: MOVEONLY: Move IsBDBFile, IsSQLiteFile, and ListWalletDir
< bitcoin-git> bitcoin/master 6ee9cbd Russell Yanofsky: refactor: Replace ListWalletDir() function with ListDatabases()
< bitcoin-git> bitcoin/master 6a7a636 Russell Yanofsky: refactor: Drop call to GetWalletEnv in wallet salvage code
< jonasschnelli> I knew that when I'm going to wake up that you guys will have a solution
< achow101> lol
< bitcoin-git> [bitcoin] MarcoFalke merged pull request #20275: wallet: List all wallets in non-SQLite and non-BDB builds (master...pr/exist) https://github.com/bitcoin/bitcoin/pull/20275
< sipa> achow101: number of pages, and "resources" are counted separately
< sipa> with pagesize=0 the executable code is always 1 page
< sipa> so i expect it to be mostly constant with pagesize=0
< jonasschnelli> I'm also getting an invalid signature for 0.20.2rc1
< jonasschnelli> shall I codesign with --pagesize 0?
< sipa> jonasschnelli: that may work (50% chance)
< sipa> jonasschnelli: basically any pagesize you try (has to be zero or a power of 2) has an independent 50% chance of being succesful with the current tooling
< sipa> with achow's fix it should always work
< sipa> better just wait; it can be backported too
< jonasschnelli> ok
< bitcoin-git> [bitcoin] MarcoFalke pushed 3 commits to master: https://github.com/bitcoin/bitcoin/compare/ffc4d0499020...b18978066d87
< bitcoin-git> bitcoin/master fad68af MarcoFalke: p2p: Ignore non-version msgs before version msg
< bitcoin-git> bitcoin/master faaad1b MarcoFalke: p2p: Ignore version msgs after initial version msg
< bitcoin-git> bitcoin/master b189780 MarcoFalke: Merge #20079: p2p: Treat handshake misbehavior like unknown message
< bitcoin-git> [bitcoin] MarcoFalke merged pull request #20079: p2p: Treat handshake misbehavior like unknown message (master...2010-p2pNoVersion) https://github.com/bitcoin/bitcoin/pull/20079
< prusnak> do you know if https://github.com/shesek is on IRC?
< belcher> prusnak yes hes called shesek
< prusnak> shesek: no such nick
< prusnak> thanks, i was asking because he's offline most probably
< achow101> sipa: I found the codesign source. It's in a library called "Security": https://opensource.apple.com/source/Security/Security-59306.140.5/OSX/libsecurity_codesigning/lib/
< achow101> Would it be worthwhile to write a tool that lets us code sign macOS from linux? Given that we can read the original code signing source, I don't think it would be too difficult to write one.
< bitcoin-git> [bitcoin] achow101 opened pull request #20638: Mac codesign fixed alloc (master...mac-codesign-fixed-alloc) https://github.com/bitcoin/bitcoin/pull/20638
< sipa> achow101: maybe, depending on how hard it is to compile that for linux
< wumpus> achow101: nice!
< achow101> sipa: I don't think we can compile it ourselves. It would have to be a completely separate implementation.
< sipa> achow101: that sounds nontrivial
< wumpus> assuming, of course, it's the same as the binary they ship
< sipa> the benefit is that if someone actually makes that work, we may be able to replace it with multiparty RSA or s
< achow101> it appears to be nontrivial. the code is a bit hard to understand
< sipa> o
< achow101> I just wanted to make it use rfc6979
< wumpus> if it's RSA in the first place
< achow101> the cert is rsa
< achow101> it's amazing how hard it is to actually find the function that does the cryptographic signature
< sipa> i assume the signature contais a certnl chain too
< wumpus> you'd say if you're going to do something like per-page signing you'd use an algorithm with short signatures :)
< sipa> which may complicate thihgs
< sipa> and the code for deciding exactly is being signed may be nontrivial too
< wumpus> that explains why no one else made a portable signer i guess
< achow101> I'm surprised no one has put in the effort to make one given that it's actually open source
< sipa> i wonder how many people even care about doing codesigning macos binaries on a non-macos platform
< wumpus> true, not that many projects are doing deterministic builds
< wumpus> though even for non-deterministic builds it can often be useful to do all builds on a single build host platform
< achow101> even with per-page signing, shouldn't the signature be way larger than 200k?
< achow101> given our binary size
< sipa> what is the actual binary size without resources?
< sipa> i think jonasschnelli's tgz has a file that lists all the resources
< achow101> 26891600 Dec 12 11:40 Bitcoin-Qt
< sipa> do you know how many bits the RSA key is?
< achow101> hmm. the cert is not in contrib/macdeploy.. I thought it was
< sipa> achow101: perhaps you can find in that apple source code what the default page size is
< sipa> perhaps the default page size is 32k or 64k
< achow101> turns out codesign has a lot of --verbose options
< achow101> the page size is 4096 in the rc3 sig
< sipa> hmm
< sipa> that makes little sense
< achow101> here is the full verbose output of codesign -d: https://0bin.net/paste/2gi9BEF9#YOEGkrrhbpToojZbGKJhHvcYp5H1zV3MfO+skMBRotg
< sipa> well that"s 35 bytes per sig
< achow101> it looks like it's actually just a ton of hashes and the signature is over all of them?
< achow101> it says Signature size=8973
< sipa> ah!
< sipa> it's a merkle tree over the binary?
< sipa> that makes sens
< achow101> couldn't say. but 6463 * 32 is in the ballpark of the signature size
< achow101> and there's 6432 hashes. that also fits with the page size and binary size, 6432 * 4096 is near the binary size
< achow101> *6463 hashes
< sipa> yup
< sipa> i suspect that's it exactly
< achow101> making hashes of each page seems excessive though
< sipa> it allows for fast per-page validation
< sipa> without needing to hash all the leaves every time
< bitcoin-git> [bitcoin] jbampton opened pull request #20639: doc: fix case of GitHub (master...fix-case-of-github) https://github.com/bitcoin/bitcoin/pull/20639
< achow101> I don't think it's a merkle tree. I'm pretty sure it's just a flat list of hashes
< achow101> that's what gets hashes again and signed
< wumpus> that makes sense
< wumpus> a hash would be much more efficient to verify per page, after all, and the signature check can be done once at load time for the hashes section
< wumpus> per page (or bunch-of-pages) makes sense as it decreases the granularity of loading, a hash over the entire binary means the entire binary has to be read
< achow101> so our trick to do a single page may not be ideal
< wumpus> well if it works as a workaround
< wumpus> but not in the long term no i guess
< achow101> well we now know mostly how big the signature will be, so we could compute an overestimate instead
< sipa> achow101: why not? do we care about not loading all pages all at once?
< wumpus> if someone wants to reverse engineer this i'd prefer having a signature *verifier* first, so the gitian process can detect if it went wrong
< wumpus> sipa: i guess so? there's always a lot of data in a binary that's not 'hot', might never be accessed at all, especially as we static link qt
< achow101> i dunno
< achow101> it seems like there's a default for a reason
< achow101> and it's clearly architecture optimized
< wumpus> that said, given the memory requiremetns i'm not sure we care about paging much
< achow101> verification of the signature itself should only happen once
< sipa> oh you think it's re-verifying the pages every time they get loaded again from disk
< wumpus> yes, but the hash might be done more often, say if a page is paged out and in again?
< achow101> I would guess this is some kind of runtime integrity check
< sipa> that does make sense
< sipa> but just the hash, not the full sig
< achow101> yeah
< wumpus> yes, the sig verification for the hashes section is likely done only once
< sipa> but that means that the list of hashes must be loaded at all times
< wumpus> yes
< wumpus> so it'd be a compromise--too small page size and the list of hashes is huge, too big page size and it has very large paging granularity
< wumpus> and i'm not sure--if it wouldn't check every time a page is re-loaded from disk it might open up some kinds of exploits, espeically for network file systems if manipulation of the data can happen without the OS knowing
< wumpus> but that's not certain
< sipa> if it was a Merkle tree instead they wouldn't need to keep the hash lish loaded, but it'd be twice the size
< achow101> I think we can produce a good enough overestimate to allow for the default page size
< achow101> (((unsigned_size / 4096) + 1) * 32) + 50000
< sipa> yup
< wumpus> good!
< achow101> the sig is 9k, and there's an additional 10k of overhead. with + 50k, we get 30k of extra
< achow101> i'll update the pr
< phantomcircuit> it seems to be using https://en.wikipedia.org/wiki/Cryptographic_Message_Syntax
< achow101> oh!
< achow101> so openssl can be used?
< phantomcircuit> achow101, well to quote a comment 'Sign a Mach-O binary, using liberal dollops of that special Mach-O magic sauce.'
< phantomcircuit> so
< achow101> when I read that, I was not encouraged
< sipa> how to construct the Xcode extract for the 0.20 and 0.19 builds?
< achow101> sipa: see contrib/macdeploy/README in the 0.20 branch
< achow101> it explains how for both
< wumpus> if the download is still available on the apple site, hopefully
< sipa> i guess it should be updated or deleted?
< sipa> or changed into a link to the macdeploy stuff?
< achow101> I think it should link to macdeploy
< wumpus> yea the information on a branch should probably only describe for that branch
< achow101> for some reason the 0.19 branch doesn't describe the extract process, but 0.20 does as "the old way"
< phantomcircuit> achow101, yeha i cant figure out where it's actually signing either
< phantomcircuit> im sure if i could download that entire directory from git or as a tar ball i could
< phantomcircuit> but no
< phantomcircuit> classic apple
< achow101> phantomcircuit: mirror of it here https://github.com/apple-opensource/Security/
< achow101> slightly older version, but close enough
< achow101> I got to SecCodeSigner::Signer::signCodeDirectory in signer.cpp before I got super lost
< phantomcircuit> achow101, yeah that's definitely the right function
< phantomcircuit> i got to CMSEncoderAddSigners but can't tell what it's actually doing
< phantomcircuit> at least i think that's a decent place to look
< phantomcircuit> also, if(hashDict != NULL) {assert(hashDict != NULL); they must not trust their compiler very much
< achow101> CMSEncoderCopyEncodedContent claims to do the actual signing, but I find no evidence that supports that. it just kinda goes into a while function stack that I can't understand
< achow101> *wild
< achow101> CMSEncoderAddSigners just adds some kind of signer object to a list
< phantomcircuit> achow101, CMSEncoderUpdateContent
< phantomcircuit> wait this looks like it's generating hashes and then encrypting them???
< achow101> I think they use encrypt and sign interchangeably
< phantomcircuit> iono kinda looks like it's actually encrypting things
< phantomcircuit> of course there's about 10 levels of indirection though so who can even tell
< achow101> I think it's just some more generic code that can also handle encryption?
< achow101> honstly it's hard to know
< sipa> for RSA encrypting and signing are kind-of interchangeable (but the devil is in the details)
< phantomcircuit> sipa, just gotta figure out where the ciphcx object is set.. which is uhhh
< sipa> ok, downloading xocde 10.2.1
< sipa> *xcode
< wumpus> can you see what hash algorithm they use? if so it might be most straightforward to check if the hashes are encrypted or otherwise obfuscating by hashing pages and looking if they appear in the signature section of the file
< wumpus> pages are bound to have an address aligned to the start of the file, otherwise it'd be wildly inefficient
< achow101> they use sha256, so it shouldn't be hard to check
< achow101> The hashes are definitely embedded
< achow101> From the 2nd page to the end of the file, the hashes in the signed binary match the unsigned one
< achow101> the 2nd page contains the alloc thing, so it's different, but if the alloc is done, it matches
< achow101> only the first page doesn't match. and then there are 3 "special" hashes that are of... something else
< wumpus> achow101: oh that's great!
< achow101> actually the first page matches, it just doesn't match the unsigned one
< wumpus> maybe it updates something in the header first
< sipa> well, we know it does :)
< sipa> if it didn't, this whole problem wouldn't exist
< sipa> the vmsize/filesize entries on the __LINKEDIT section are changed
< wumpus> yeah then it makes sense for the hash to match the new state, not the original one
< wumpus> right, true :)
< achow101> yep, first 2 pages differ from unsigned because of the addition of the code signing section
< achow101> so the hashes in the sig are 4096 byte pages from beginning of the signed binary to the beginning of the code signature
< achow101> this is something we can verify ourselves
< sipa> ah, neat
< achow101> i figured out 2 of the 3 special hashes
< wumpus> that's fast :)
< sipa> last one may be some metadata with the signature/hashing parameters?
< achow101> looking at the source, I think it's "entitlements"
< achow101> but I don't know what that means
< sipa> the codesign tool has a way to provide something called entitlements to it
< achow101> one of the hashes is of the file Bitcoin-Qt.app/Contents/_CodeSignature/CodeResources, and the other is Bitcoin-Qt.app/Contents/Info.plist
< achow101> but this last hash is not any of the files in the bundle
< wumpus> if we don't pass "entitlements", maybe it's a hash of empty data?
< achow101> I don't think so
< achow101> The hash is Bitcoin-Qt.app/Contents
< achow101> 1761bc7a0f37e9809acd22c53f42ff21622ea02093c14058e7daa3a466c95d97
< sipa> is this hash the same for other releases?
< sipa> if it's just some constant we don't actually care what it's a hash of
< wumpus> right a proto-verifyer could just compare it against the fixed hash and raise alarm if it doesn't match
< achow101> it's the same in 0.20.1, 0.21.0rc2, and 0.21.0rc3
< achow101> afaik we don't have any entitlements so it should just be the hash of an empty something
< sipa> google is of no help
< wumpus> "
< wumpus> The specific entitlements will be embedded in the signature of an application. If you are having trouble, it can help to look at what the signature actually says about the entitlements: $ codesign -d --entitlements - Example.app will show an XML property list similar to the one above.
< achow101> it gives nothing so there are no entitlements
< wumpus> so it's a hash of something that is in the signature itself, that sounds strange but...
< sipa> maybe it's just <xml></xml> or something
< wumpus> I'd expect so
< achow101> the source code has it as a datatype of CFDataRef which I think just means it's a void *
< sipa> wumpus: sounds similar to bitcoin's use of a sighash byte
< sipa> part of the signature itself, and also signed
< wumpus> sipa: ah yes
< achow101> is there such a thing as a generic DER decoder?
< sipa> yes
< sipa> ha, thomas pornin
< sipa> i've met him
< achow101> hmm, openssl should be able to decode this if it was properly CMS DER, but it doesn't like it
< wumpus> it apparently DiskRep::component(cdEntitlementSlot) gets the data that is hashed to verify the entitlement slot, there are different implementations of DispRep though, but in diskrep this is the file "CodeEntitlements"
< wumpus> filediskrep*
< wumpus> in CodeDirectory::slotIsPresent a non-zero digest in the slot means that is is present
< wumpus> i guess i should be looking at BundleDiskRep::component(cdEntitlementSlot) as it can fetch components from the executable itself
< achow101> I found a bunch of ASN.1 templates, this means it should be possible to figure the serialization, right?
< sipa> achow101: can you show me the DER data?
< achow101> sipa: in the bitcoin-detached-sigs repo osx/dist/Bitcoin-Qt.app/Contents/MacOS/Bitcoin-Qt.sign
< achow101> that's the code signature. in theory it's DER
< achow101> (checkout 0.21 branch)
< sipa> you don't actually need the schema to decode DER, it's self-descriptive
< sipa> the scheme just helps with naming stuff
< achow101> I'm not convinced it's actually DER
< wumpus> *i have no clue what i'm doing dog image*
< sipa> it's not
< wumpus> ok
< sipa> $ file Bitcoin-Qt.sign
< sipa> Bitcoin-Qt.sign: Mac OS X Detached Code Signature (non-executable) - 216249 bytes
< achow101> wait file recognizes this???
< sipa> hmm where does file keeps its magic definitions these days?
< sipa> i only find a precompiled magic.mgc file
< achow101> I think that's it
< achow101> gotta find the source for file
< wumpus> from MachORep::compoenent to EmbeddedSignatureBlob::component now
< wumpus> "An EmbeddedSignatureBlob is a SuperBlob indexed by component slot number." of coursee
< wumpus> class EmbeddedSignatureBlob : public SuperBlobCore<EmbeddedSignatureBlob, 0xfade0cc0, uint32_t>
< sipa> _0x0ff: belong0xfade0cc1Mac OS X Detached Code Signature
< sipa> 0 belong 0xfade0cc1 Mac OS X Detached Code Signature
< wumpus> typedef SuperBlob<0xfade0cc1> DetachedSignatureBlob; // indexed by main architecture
< wumpus> superblobs, all of them
< sipa> turtles all the way down
< wumpus> hehe
< achow101> blobloblob
< wumpus> / as a genuine Blob (e.g. for insertion into a SuperBlob).
< wumpus> a superblob is a map of integers to blobs
< sipa> getting deeper
< wumpus> "// echoes from parent BlobCore (the C++ type system is too restrictive here)" uh
< wumpus> tbh seems like it's actually all fairly straightforward C++ code
< wumpus> the comments are way scarier than the code itself
< sipa> wumpus: you're biased
< sipa> :)
< wumpus> i... suppose hehe
< achow101> i've somehow wandered into some code in this libsecurity that's Copyright Mozilla
< sipa> so do we know how to extract the blobs from the EmbeddedSignatureBlob ?
< achow101> nope?
< achow101> That unknown hash earlier is apparently "internal requirements", not entitlements
< wumpus> kind of, it's u32 magic (BE), u32 (BE) size, then u32 (LE) number of blobs, then per blob u32 (LE) blob type, u32 LE offset
< wumpus> then the data
< wumpus> blob type is the slot id, so 5 for the cdEntitlementSlot, but apprantly we're no longer looking for that
< sipa> ok so 3 blobs
< sipa> first blob is type 0, length 0x24
< wumpus> which is good as it doesn't have that one
< sipa> ah no, not length; offset
< sipa> the individual blobs don't have a length?
< wumpus> sipa: that seems correct?
< wumpus> yes, offset
< wumpus> the length is in the blob itself
< sipa> the first blob has magic fa de 0c 02
< wumpus> blobs are all <magic> <length>
< sipa> got it
< achow101> list of magics in OSX/libsecurity_codesigning/lib/CSCommonPriv.h
< sipa> Mac OS X Code Directory is the first one
< wumpus> so yes we have 0 at offset 0x24, 2 at offset 0x2f7a4 and 0x100 at offset 0x2f864
< achow101> sipa: yes. this contains our list of hashes
< wumpus> (all integers are BE, it seemed from the code that they'd be LE but apparently they're swapped somewhere)
< achow101> it looks like the structure is the class members
< achow101> in order as listed
< sipa> second one has magic fa de 0c 01, Mac OS X Code Requirement Set
< wumpus> there's your requirement set to hash i guess
< sipa> last blob has magic fa de 0b 01
< wumpus> class BlobWrapper : public Blob<BlobWrapper, 0xfade0b01>
< sipa> the first blob has magic fa de 0c 02
< achow101> oh nice
< achow101> hmm, it can't be the structure as listed because I run into some strings before I'm supposed to
< achow101> but it's pretty close
< wumpus> euhh the third's slot id is 0x10000 which is cdSignatureSlot = 0x10000,| | | // CMS signature
< achow101> that might be DER encoded
< wumpus> which is wrapped in a BlobWrapper which is simply a blob header with arbitrary binary dat
< sipa> this has so many layers, i'd be surprised if it isn't somehow exploitable...
< achow101> In class CodeDirectory, everything in the class decodes in order up to spare3
< achow101> then we get to the offset for identifier string and the identifer string begins there, as expected
< wumpus> otoh the blob format is super simple, there could be a mistake of course, but yes e.g. DER parsers are notoriously exploitable
< wumpus> the CMS signature is produced in SecCodeSigner::Signer::signCodeDirectory in signer.cpp
< achow101> it seems like the message that is signed is the CodeDirectory blob
< wumpus> that's what the comment says too "Generate the CMS signature for a (finished) CodeDirectory.", in addition it takes a hashDict and hashList
< wumpus> where hashDict is apparently "v2" and hashList for "v1"
< wumpus> the encoding is indeed the Cryptographic Message Syntax what phantomcircuit linked to with apple specific tags
< phantomcircuit> <sipa> this has so many layers, i'd be surprised if it isn't somehow exploitable...
< phantomcircuit> indeed
< achow101> wumpus: that Requirement Set blob is indeed the unknown hash.