diff --git a/Makefile b/Makefile index cf5faed..aac5d7d 100644 --- a/Makefile +++ b/Makefile @@ -33,7 +33,7 @@ include $(DEVKITPRO)/libnx/switch_rules VERSION_MAJOR := 1 VERSION_MINOR := 1 -VERSION_MICRO := 2 +VERSION_MICRO := 3 APP_TITLE := nxdumptool APP_AUTHOR := MCMrARM, DarkMatterCore diff --git a/README.md b/README.md index e8d925c..b930c43 100644 --- a/README.md +++ b/README.md @@ -5,7 +5,8 @@ Main features -------------- * Generates full Cartridge Image dumps (XCI) with optional certificate removal and optional trimming. -* Generates installable Nintendo Submission Packages (NSP) from base applications, updates and DLCs stored in the inserted game card, SD card and eMMC storage devices. +* Generates installable Nintendo Submission Packages (NSP) from base applications, updates and DLCs stored in the inserted gamecard, SD card and eMMC storage devices. + * The generated dumps follow the `AuditingTool` format from Scene releases. * Compatible with multigame carts. * CRC32 checksum calculation for XCI/NSP dumps. * Full XCI dump verification using XML database from NSWDB.COM (NSWreleases.xml). @@ -32,13 +33,16 @@ Thanks to * [RSDuck](https://github.com/RSDuck), for vba-next-switch port. It's UI menu code was taken as a basis for this application. * [foen](https://github.com/foen), for giving me some pretty good hints about how to use the NCM service. * [yellows8](https://github.com/yellows8), for helping me fix a silly bug in my implementation of some NCM service IPC calls. -* [SciresM](https://github.com/SciresM), for [hactool](https://github.com/SciresM/hactool) (licensed under [ISC](https://github.com/SciresM/hactool/blob/master/LICENSE)). It's NCA content handling procedure is reproduced during the NSP dump process. +* [SciresM](https://github.com/SciresM), for [hactool](https://github.com/SciresM/hactool) (licensed under [ISC](https://github.com/SciresM/hactool/blob/master/LICENSE)). It's NCA content handling procedures are reproduced in many parts of the application. * [The-4n](https://github.com/The-4n), for [4NXCI](https://github.com/The-4n/4NXCI) (licensed under [ISC](https://github.com/The-4n/4NXCI/blob/master/LICENSE)) and [hacPack](https://github.com/The-4n/hacPack) (licensed under [GPLv2](https://github.com/The-4n/hacPack/blob/master/LICENSE)). The NCA content patching procedure used in 4NXCI is replicated in the application, as well as the NACP XML generation from hacPack. * [shchmue](https://github.com/shchmue), for [Lockpick](https://github.com/shchmue/Lockpick) (licensed under [GPLv2](https://github.com/shchmue/Lockpick/blob/master/LICENSE)). It is used as a reference for the runtime key-collection algorithm needed for the NSP dump, ExeFS dump/browse and RomFS dump/browse procedures. * Björn Samuelsson, for his [public domain CRC32 checksum calculation C-code](http://home.thep.lu.se/~bjorn/crc). * [Adubbz](https://github.com/Adubbz), for [Tinfoil](https://github.com/Adubbz/Tinfoil) (licensed under [MIT](https://github.com/Adubbz/Tinfoil/blob/master/LICENSE)). Its wrappers for ES service IPC calls are used in the application. +* [WerWolv](https://github.com/WerWolv), for the SX OS detection procedure used in [EdiZon](https://github.com/WerWolv/EdiZon) (licensed under [GPLv2](https://github.com/WerWolv/EdiZon/blob/master/LICENSE)). * ChaN, for the [FatFs module](http://elm-chan.org/fsw/ff/00index_e.html) (licensed under [FatFs license](http://elm-chan.org/fsw/ff/doc/appnote.html#license)). It is used to read ES savedata files from the BIS System partition. -* [AnalogMan](https://github.com/AnalogMan151), for his constant support and ideas. +* [hthh](https://github.com/hthh), for his [switch-reversing](https://github.com/hthh/switch-reversing) repository, which was very helpful to understand how to parse and read information from NSO binaries. +* The [LZ4 project](http://www.lz4.org), for the LZ4 C-code implementation (licensed under [BSD 2-Clause](https://github.com/lz4/lz4/blob/master/lib/LICENSE)). +* [AnalogMan](https://github.com/AnalogMan151) and [0Liam](https://github.com/0Liam), for their constant support and ideas. * [RattletraPM](https://github.com/RattletraPM), for the awesome icon used in the application. * The GNOME project, from which the high contrast directory/file icons for the filebrowser modes were retrieved. * The folks from ReSwitched, for working towards the creation of a good homebrew ecosystem. @@ -54,6 +58,39 @@ If you like my work and you'd like to support me in any way, it's not necessary, Changelog -------------- +**v1.1.3:** +* General changes to the NSP dumping procedure: + * Corrected and updated CNMT XML and NACP XML generation. Thanks to [0Liam](https://github.com/0Liam)! + * Added NACP icon retrieval for each available language. + * Added legalinfo.xml retrieval. + * Added programinfo.xml generation. + * Changed the PFS0 file order to the following: + 1. NCA content files. + 2. CNMT NCA. + 3. CNMT XML. + 4. programinfo.xml (if available). + 5. NACP icons (if available). + 6. NACP XML (if available). + 7. legalinfo.xml (if available). + 8. Ticket + Certificate chain (if available). + * These changes essentially make the NSP dumps generated by the application comparable to Scene releases that follow the `AuditingTool` format (like those from groups like BigBlueBox or JRP), as long as the "Remove console specific data" option is enabled and the "Generate ticket-less dump" option is disabled. Happy dumping! + * Because of this, dumping update NSPs from gamecards will require the keys file at "sdmc:/switch/prod.keys" from now on (but only if the bundled update uses titlekey crypto). Base applications and DLCs can still be dumped from gamecards without needing a keys file. +* Added ExeFS/RomFS browsing/dumping from game updates. + * Upon entering ExeFS/RomFS menus, it is now possible to select which update is going to be used for ExeFS/RomFS procedures. + * In order to dump ExeFS/RomFS content from a installed update for a gamecard title, its respective gamecard must be inserted in the console. + * Likewise, in order to dump ExeFS/RomFS content from a installed update for a SD/eMMC title, its respective base application must be already installed as well. +* Added NSP batch dump mode. Press X while on the SD/eMMC title list to configure the batch dump options and start the process. Supports skipping already dumped titles, dumping selected title types (base applications, updates, DLCs) and dumping titles from a specific source storage (SD, eMMC). +* Added manual directory dumping feature to the RomFS browser. Just enter the directory to be dumped and then press the Y button. +* Added a forced XCI dump option when either the gamecard base application count or their Title IDs can't be retrieved (useful for rare Kiosk gamecards). Press Y at the error message screen to dump the cartridge image to "gamecard.xci". +* Dumped content information is now displayed in the gamecard menu. + * Additionally, if the XCI has already been dumped, information about it will be displayed as well. +* The displayed information about dumped content is now updated after each new dump procedure in both gamecard and SD/eMMC menus. +* The NPDM ACID patching procedure is now performed with Program NCAs from bundled gamecard updates and SD/eMMC titles if the "Generate ticket-less dump" option is enabled. +* Fixed XCI dumping under SX OS. +* Fixed a bug in the DLC NSP dump submenu that made it impossible to change the DLC to be dumped from the selected base application if more than a single DLC is available for it. Thanks to [ckurtz22](https://github.com/ckurtz22)! +* Fixed a bug that made the application get stuck in an endless loop after selecting the SD/eMMC dump option from the main menu if no SD/eMMC titles are available. Thanks to [ckurtz22](https://github.com/ckurtz22)! +* Fixed a bug that made the application return an empty title list if no SD card is inserted or if it contains a "Nintendo" directory from another console (even if there are installed titles in the eMMC). Thanks to [ckurtz22](https://github.com/ckurtz22)! + **v1.1.2:** * Delta fragment NCAs are now included in update NSPs dumped from SD/eMMC if the "Generate ticket-less dump" option is disabled. * It is now possible to generate ticket-less NSP dumps from bundled updates in gamecards. Please bear in mind that this option requires the external "sdmc:/switch/prod.keys" file. diff --git a/source/dumper.c b/source/dumper.c index c072d3e..a7eb39d 100644 --- a/source/dumper.c +++ b/source/dumper.c @@ -17,6 +17,8 @@ /* Extern variables */ +extern bool runningSxOs; + extern FsDeviceOperator fsOperatorInstance; extern nca_keyset_t nca_keyset; @@ -26,7 +28,7 @@ extern u64 freeSpace; extern int breaks; extern int font_height; -extern u64 trimmedCardSize; +extern u64 gameCardSize, trimmedCardSize; extern char trimmedCardSizeStr[32]; extern u8 *hfs0_header; @@ -58,11 +60,19 @@ extern AppletType programAppletType; extern exefs_ctx_t exeFsContext; extern romfs_ctx_t romFsContext; +extern bktr_ctx_t bktrContext; extern char curRomFsPath[NAME_BUF_LEN]; +extern u32 curRomFsDirOffset; extern char strbuf[NAME_BUF_LEN * 4]; +extern char *filenameBuffer; +extern char *filenames[FILENAME_MAX_CNT]; +extern int filenamesCount; + +extern u8 *fileNormalIconBuf; + void workaroundPartitionZeroAccess() { FsGameCardHandle handle; @@ -96,12 +106,19 @@ bool dumpCartridgeImage(bool isFat32, bool setXciArchiveBit, bool dumpCert, bool size_t write_res; - char *dumpName = generateDumpFullName(); + char *dumpName = generateFullDumpName(); if (!dumpName) { - uiDrawString("Error: unable to generate output dump name!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); - breaks += 2; - return false; + // We're probably dealing with a forced XCI dump + dumpName = calloc(16, sizeof(char)); + if (!dumpName) + { + uiDrawString("Error: unable to generate output dump name!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + breaks += 2; + return false; + } + + sprintf(dumpName, "gamecard"); } for(partition = 0; partition < ISTORAGE_PARTITION_CNT; partition++) @@ -110,43 +127,60 @@ bool dumpCartridgeImage(bool isFat32, bool setXciArchiveBit, bool dumpCert, bool uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); breaks++;*/ - workaroundPartitionZeroAccess(); - - if (R_SUCCEEDED(result = fsDeviceOperatorGetGameCardHandle(&fsOperatorInstance, &handle))) + if (partition == (ISTORAGE_PARTITION_CNT - 1) && runningSxOs) { - /*snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "GetGameCardHandle succeeded: 0x%08X", handle.value); - uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); - breaks++;*/ + // Total size for IStorage instances is maxed out under SX OS, so let's manually reduce the size for the last instance - if (R_SUCCEEDED(result = fsOpenGameCardStorage(&gameCardStorage, &handle, partition))) + u64 partitionSizesSum = 0; + for(int i = 0; i < (ISTORAGE_PARTITION_CNT - 1); i++) partitionSizesSum += partitionSizes[i]; + + // Substract the total ECC block size as well as the size for previous IStorage instances + partitionSizes[partition] = ((gameCardSize - ((gameCardSize / GAMECARD_ECC_BLOCK_SIZE) * GAMECARD_ECC_DATA_SIZE)) - partitionSizesSum); + + xciDataSize += partitionSizes[partition]; + convertSize(partitionSizes[partition], partitionSizesStr[partition], sizeof(partitionSizesStr[partition]) / sizeof(partitionSizesStr[partition][0])); + /*snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Partition #%u size: %s (%lu bytes).", partition, partitionSizesStr[partition], partitionSizes[partition]); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); + breaks += 2;*/ + } else { + workaroundPartitionZeroAccess(); + + if (R_SUCCEEDED(result = fsDeviceOperatorGetGameCardHandle(&fsOperatorInstance, &handle))) { - /*snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "OpenGameCardStorage succeeded: 0x%08X", handle.value); + /*snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "GetGameCardHandle succeeded: 0x%08X", handle.value); uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); breaks++;*/ - if (R_SUCCEEDED(result = fsStorageGetSize(&gameCardStorage, &(partitionSizes[partition])))) + if (R_SUCCEEDED(result = fsOpenGameCardStorage(&gameCardStorage, &handle, partition))) { - xciDataSize += partitionSizes[partition]; - convertSize(partitionSizes[partition], partitionSizesStr[partition], sizeof(partitionSizesStr[partition]) / sizeof(partitionSizesStr[partition][0])); - /*snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Partition #%u size: %s (%lu bytes).", partition, partitionSizesStr[partition], partitionSizes[partition]); + /*snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "OpenGameCardStorage succeeded: 0x%08X", handle.value); uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); - breaks += 2;*/ + breaks++;*/ + + if (R_SUCCEEDED(result = fsStorageGetSize(&gameCardStorage, &(partitionSizes[partition])))) + { + xciDataSize += partitionSizes[partition]; + convertSize(partitionSizes[partition], partitionSizesStr[partition], sizeof(partitionSizesStr[partition]) / sizeof(partitionSizesStr[partition][0])); + /*snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Partition #%u size: %s (%lu bytes).", partition, partitionSizesStr[partition], partitionSizes[partition]); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); + breaks += 2;*/ + } else { + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "StorageGetSize failed! (0x%08X)", result); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + proceed = false; + } + + fsStorageClose(&gameCardStorage); } else { - snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "StorageGetSize failed! (0x%08X)", result); + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "OpenGameCardStorage failed! (0x%08X)", result); uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); proceed = false; } - - fsStorageClose(&gameCardStorage); } else { - snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "OpenGameCardStorage failed! (0x%08X)", result); + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "GetGameCardHandle failed! (0x%08X)", result); uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); proceed = false; } - } else { - snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "GetGameCardHandle failed! (0x%08X)", result); - uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); - proceed = false; } uiRefreshDisplay(); @@ -437,7 +471,6 @@ bool dumpCartridgeImage(bool isFat32, bool setXciArchiveBit, bool dumpCert, bool if (!proceed) { setProgressBarError(&progressCtx); - if (fat32_error) breaks += 2; break; } } @@ -445,6 +478,7 @@ bool dumpCartridgeImage(bool isFat32, bool setXciArchiveBit, bool dumpCert, bool free(buf); breaks = (progressCtx.line_offset + 2); + if (fat32_error) breaks += 2; } else { uiDrawString("Failed to allocate memory for the dump process!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); } @@ -541,7 +575,7 @@ bool dumpCartridgeImage(bool isFat32, bool setXciArchiveBit, bool dumpCert, bool return success; } -bool dumpNintendoSubmissionPackage(nspDumpType selectedNspDumpType, u32 titleIndex, bool isFat32, bool calcCrc, bool removeConsoleData, bool tiklessDump) +bool dumpNintendoSubmissionPackage(nspDumpType selectedNspDumpType, u32 titleIndex, bool isFat32, bool calcCrc, bool removeConsoleData, bool tiklessDump, bool batch) { Result result; u32 i = 0, j = 0; @@ -599,9 +633,21 @@ bool dumpNintendoSubmissionPackage(nspDumpType selectedNspDumpType, u32 titleInd bool cnmtFound = false; char *cnmtXml = NULL; + u32 programNcaIndex = 0; + u64 programInfoXmlSize = 0; + char *programInfoXml = NULL; + u32 nacpNcaIndex = 0; + u64 nacpXmlSize = 0; char *nacpXml = NULL; + u8 nacpIconCnt = 0; + nacp_icons_ctx *nacpIcons = NULL; + + u32 legalInfoNcaIndex = 0; + u64 legalInfoXmlSize = 0; + char *legalInfoXml = NULL; + u32 nspFileCount = 0; pfs0_header nspPfs0Header; pfs0_entry_table *nspPfs0EntryTable = NULL; @@ -725,9 +771,12 @@ bool dumpNintendoSubmissionPackage(nspDumpType selectedNspDumpType, u32 titleInd } } - uiDrawString("Retrieving information from encrypted NCA content files...", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); - uiRefreshDisplay(); - breaks++; + if (!batch) + { + uiDrawString("Retrieving information from encrypted NCA content files...", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); + uiRefreshDisplay(); + breaks++; + } titleList = calloc(1, titleListSize); if (!titleList) @@ -896,7 +945,7 @@ bool dumpNintendoSubmissionPackage(nspDumpType selectedNspDumpType, u32 titleInd { if (!has_rights_id) { - // We're could be dealing with a custom XCI mounted through SX OS, so let's change back the content distribution method + // We could be dealing with a custom XCI mounted through SX OS, so let's change back the content distribution method dec_nca_header.distribution = 0; } else { if (!rights_info.retrieved_tik) @@ -910,12 +959,6 @@ bool dumpNintendoSubmissionPackage(nspDumpType selectedNspDumpType, u32 titleInd memcpy(rights_info.enc_titlekey, rights_info.tik_data.titlekey_block, 0x10); - rights_info.retrieved_tik = true; - } - - // Mess with the NCA header if we're dealing with a content with a populated rights ID field and if tiklessDump is true (removeConsoleData is ignored) - if (tiklessDump) - { // Load external keys if (!loadExternalKeys()) { @@ -931,9 +974,15 @@ bool dumpNintendoSubmissionPackage(nspDumpType selectedNspDumpType, u32 titleInd aes128ContextCreate(&titlekey_aes_ctx, nca_keyset.titlekeks[crypto_type], false); aes128DecryptBlock(&titlekey_aes_ctx, rights_info.dec_titlekey, rights_info.enc_titlekey); - memset(xml_content_info[i].decrypted_nca_keys, 0, NCA_KEY_AREA_SIZE); - memcpy(xml_content_info[i].decrypted_nca_keys + (NCA_KEY_AREA_KEY_SIZE * 2), rights_info.dec_titlekey, 0x10); - + rights_info.retrieved_tik = true; + } + + memset(xml_content_info[i].decrypted_nca_keys, 0, NCA_KEY_AREA_SIZE); + memcpy(xml_content_info[i].decrypted_nca_keys + (NCA_KEY_AREA_KEY_SIZE * 2), rights_info.dec_titlekey, 0x10); + + // Mess with the NCA header if we're dealing with a content with a populated rights ID field and if tiklessDump is true (removeConsoleData is ignored) + if (tiklessDump) + { // Generate new encrypted NCA key area using titlekey if (!generateEncryptedNcaKeyAreaWithTitlekey(&dec_nca_header, xml_content_info[i].decrypted_nca_keys)) { @@ -943,6 +992,16 @@ bool dumpNintendoSubmissionPackage(nspDumpType selectedNspDumpType, u32 titleInd // Remove rights ID from NCA memset(dec_nca_header.rights_id, 0, 0x10); + + // Patch ACID pubkey and recreate NCA NPDM signature if we're dealing with the Program NCA + if (xml_content_info[i].type == NcmContentType_Program) + { + if (!processProgramNca(&ncmStorage, &ncaId, &dec_nca_header, &(xml_content_info[i]), &ncaProgramMod)) + { + proceed = false; + break; + } + } } } } @@ -961,15 +1020,49 @@ bool dumpNintendoSubmissionPackage(nspDumpType selectedNspDumpType, u32 titleInd // Remove rights ID from NCA memset(dec_nca_header.rights_id, 0, 0x10); + + // Patch ACID pubkey and recreate NCA NPDM signature if we're dealing with the Program NCA + if (xml_content_info[i].type == NcmContentType_Program) + { + if (!processProgramNca(&ncmStorage, &ncaId, &dec_nca_header, &(xml_content_info[i]), &ncaProgramMod)) + { + proceed = false; + break; + } + } } } - // Generate NACP XML + // Generate programinfo.xml + if (!programInfoXml && xml_content_info[i].type == NcmContentType_Program) + { + programNcaIndex = i; + + if (!generateProgramInfoXml(&ncmStorage, &ncaId, &dec_nca_header, xml_content_info[i].decrypted_nca_keys, &ncaProgramMod, &programInfoXml, &programInfoXmlSize)) + { + proceed = false; + break; + } + } + + // Retrieve NACP data (XML and icons) if (!nacpXml && xml_content_info[i].type == NcmContentType_Icon) { nacpNcaIndex = i; - if (!generateNacpXmlFromNca(&ncmStorage, &ncaId, &dec_nca_header, xml_content_info[i].decrypted_nca_keys, &nacpXml)) + if (!retrieveNacpDataFromNca(&ncmStorage, &ncaId, &dec_nca_header, xml_content_info[i].decrypted_nca_keys, &nacpXml, &nacpXmlSize, &nacpIcons, &nacpIconCnt)) + { + proceed = false; + break; + } + } + + // Retrieve legalinfo.xml + if (!legalInfoXml && xml_content_info[i].type == NcmContentType_Info) + { + legalInfoNcaIndex = i; + + if (!retrieveLegalInfoXmlFromNca(&ncmStorage, &ncaId, &dec_nca_header, xml_content_info[i].decrypted_nca_keys, &legalInfoXml, &legalInfoXmlSize)) { proceed = false; break; @@ -1040,9 +1133,9 @@ bool dumpNintendoSubmissionPackage(nspDumpType selectedNspDumpType, u32 titleInd generateCnmtXml(&xml_program_info, xml_content_info, cnmtXml); - bool includeCertAndTik = (rights_info.has_rights_id && !tiklessDump); + bool includeTikAndCert = (rights_info.has_rights_id && !tiklessDump); - if (includeCertAndTik) + if (includeTikAndCert) { if (curStorageId == FsStorageId_GameCard) { @@ -1084,11 +1177,11 @@ bool dumpNintendoSubmissionPackage(nspDumpType selectedNspDumpType, u32 titleInd // Retrieve cert file if (!retrieveCertData(rights_info.cert_data, (rights_info.tik_data.titlekey_type == ETICKET_TITLEKEY_PERSONALIZED))) goto out; - // File count = NCA count + CNMT XML + cert + tik + // File count = NCA count + CNMT XML + tik + cert nspFileCount = (titleNcaCount + 3); // Calculate PFS0 String Table size - nspPfs0StrTableSize = (((nspFileCount - 4) * NSP_NCA_FILENAME_LENGTH) + (NSP_CNMT_FILENAME_LENGTH * 2) + NSP_CERT_FILENAME_LENGTH + NSP_TIK_FILENAME_LENGTH); + nspPfs0StrTableSize = (((nspFileCount - 4) * NSP_NCA_FILENAME_LENGTH) + (NSP_CNMT_FILENAME_LENGTH * 2) + NSP_TIK_FILENAME_LENGTH + NSP_CERT_FILENAME_LENGTH); } else { // File count = NCA count + CNMT XML nspFileCount = (titleNcaCount + 1); @@ -1097,11 +1190,35 @@ bool dumpNintendoSubmissionPackage(nspDumpType selectedNspDumpType, u32 titleInd nspPfs0StrTableSize = (((nspFileCount - 2) * NSP_NCA_FILENAME_LENGTH) + (NSP_CNMT_FILENAME_LENGTH * 2)); } + // Add our programinfo.xml if we created it + if (programInfoXml) + { + nspFileCount++; + nspPfs0StrTableSize += NSP_PROGRAM_XML_FILENAME_LENGTH; + } + // Add our NACP XML if we created it if (nacpXml) { + // Add icons if we retrieved them + if (nacpIcons && nacpIconCnt) + { + for(i = 0; i < nacpIconCnt; i++) + { + nspFileCount++; + nspPfs0StrTableSize += (strlen(nacpIcons[i].filename) + 1); + } + } + nspFileCount++; - nspPfs0StrTableSize += NSP_NACP_FILENAME_LENGTH; + nspPfs0StrTableSize += NSP_NACP_XML_FILENAME_LENGTH; + } + + // Add our legalinfo.xml if we retrieved it + if (legalInfoXml) + { + nspFileCount++; + nspPfs0StrTableSize += NSP_LEGAL_XML_FILENAME_LENGTH; } // Start NSP creation @@ -1130,25 +1247,48 @@ bool dumpNintendoSubmissionPackage(nspDumpType selectedNspDumpType, u32 titleInd } // Determine our full NSP header size - full_nsp_header_size = (sizeof(pfs0_header) + (nspFileCount * sizeof(pfs0_entry_table)) + nspPfs0StrTableSize); + full_nsp_header_size = (sizeof(pfs0_header) + ((u64)nspFileCount * sizeof(pfs0_entry_table)) + nspPfs0StrTableSize); + + // Round up our full NSP header size to a 0x10-byte boundary + if (!(full_nsp_header_size % 0x10)) full_nsp_header_size++; // If it's already rounded, add more padding full_nsp_header_size = round_up(full_nsp_header_size, 0x10); // Determine our String Table size - nspPfs0Header.str_table_size = (full_nsp_header_size - (sizeof(pfs0_header) + (nspFileCount * sizeof(pfs0_entry_table)))); + nspPfs0Header.str_table_size = (full_nsp_header_size - (sizeof(pfs0_header) + ((u64)nspFileCount * sizeof(pfs0_entry_table)))); // Calculate total dump size progressCtx.totalSize = full_nsp_header_size; - progressCtx.totalSize += strlen(cnmtXml); - if (nacpXml) progressCtx.totalSize += strlen(nacpXml); - if (includeCertAndTik) progressCtx.totalSize += (ETICKET_CERT_FILE_SIZE + ETICKET_TIK_FILE_SIZE); + for(i = 0; i < titleNcaCount; i++) progressCtx.totalSize += xml_content_info[i].size; - breaks++; + progressCtx.totalSize += strlen(cnmtXml); + + if (programInfoXml) progressCtx.totalSize += programInfoXmlSize; + + if (nacpXml) + { + if (nacpIcons && nacpIconCnt) + { + for(i = 0; i < nacpIconCnt; i++) progressCtx.totalSize += nacpIcons[i].icon_size; + } + + progressCtx.totalSize += nacpXmlSize; + } + + if (legalInfoXml) progressCtx.totalSize += legalInfoXmlSize; + + if (includeTikAndCert) progressCtx.totalSize += (ETICKET_TIK_FILE_SIZE + ETICKET_CERT_FILE_SIZE); + convertSize(progressCtx.totalSize, progressCtx.totalSizeStr, sizeof(progressCtx.totalSizeStr) / sizeof(progressCtx.totalSizeStr[0])); - snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Total NSP dump size: %s (%lu bytes).", progressCtx.totalSizeStr, progressCtx.totalSize); - uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); - uiRefreshDisplay(); - breaks++; + + if (!batch) + { + breaks++; + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Total NSP dump size: %s (%lu bytes).", progressCtx.totalSizeStr, progressCtx.totalSize); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); + uiRefreshDisplay(); + breaks++; + } if (progressCtx.totalSize > freeSpace) { @@ -1160,7 +1300,7 @@ bool dumpNintendoSubmissionPackage(nspDumpType selectedNspDumpType, u32 titleInd snprintf(dumpPath, sizeof(dumpPath) / sizeof(dumpPath[0]), "%s%s.nsp", NSP_DUMP_PATH, dumpName); // Check if the dump already exists - if (checkIfFileExists(dumpPath)) + if (!batch && checkIfFileExists(dumpPath)) { // Ask the user if they want to proceed anyway int cur_breaks = breaks; @@ -1207,11 +1347,14 @@ bool dumpNintendoSubmissionPackage(nspDumpType selectedNspDumpType, u32 titleInd goto out; } - breaks++; - snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Dump procedure started. Hold %s to cancel.", NINTENDO_FONT_B); - uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); - uiRefreshDisplay(); - breaks += 2; + if (!batch) + { + breaks++; + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Dump procedure started. Hold %s to cancel.", NINTENDO_FONT_B); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); + uiRefreshDisplay(); + breaks += 2; + } if (programAppletType != AppletType_Application && programAppletType != AppletType_SystemApplication) { @@ -1220,49 +1363,18 @@ bool dumpNintendoSubmissionPackage(nspDumpType selectedNspDumpType, u32 titleInd } // Write placeholder zeroes - write_res = fwrite(buf, 1, full_nsp_header_size + strlen(cnmtXml), outFile); - if (write_res != (full_nsp_header_size + strlen(cnmtXml))) + write_res = fwrite(buf, 1, full_nsp_header_size, outFile); + if (write_res != full_nsp_header_size) { - snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Failed to write %lu bytes placeholder data to file offset 0x%016lX! (wrote %lu bytes)", full_nsp_header_size + strlen(cnmtXml), (u64)0, write_res); + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Failed to write %lu bytes placeholder data to file offset 0x%016lX! (wrote %lu bytes)", full_nsp_header_size, (u64)0, write_res); uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); goto out; } - progressCtx.curOffset = (full_nsp_header_size + strlen(cnmtXml)); - - // Write our NACP XML - if (nacpXml) - { - write_res = fwrite(nacpXml, 1, strlen(nacpXml), outFile); - if (write_res != strlen(nacpXml)) - { - snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Failed to write %lu bytes NACP XML to file offset 0x%016lX! (wrote %lu bytes)", strlen(nacpXml), progressCtx.curOffset, write_res); - uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); - goto out; - } - - progressCtx.curOffset += strlen(nacpXml); - } - - if (includeCertAndTik) - { - memcpy(buf, rights_info.cert_data, ETICKET_CERT_FILE_SIZE); - memcpy(buf + ETICKET_CERT_FILE_SIZE, &(rights_info.tik_data), ETICKET_TIK_FILE_SIZE); - - // Write cert / tik - write_res = fwrite(buf, 1, ETICKET_CERT_FILE_SIZE + ETICKET_TIK_FILE_SIZE, outFile); - if (write_res != (ETICKET_CERT_FILE_SIZE + ETICKET_TIK_FILE_SIZE)) - { - snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Failed to write %u bytes cert + tik to file offset 0x%016lX! (wrote %lu bytes)", ETICKET_CERT_FILE_SIZE + ETICKET_TIK_FILE_SIZE, progressCtx.curOffset, write_res); - uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); - goto out; - } - - progressCtx.curOffset += (ETICKET_CERT_FILE_SIZE + ETICKET_TIK_FILE_SIZE); - } + progressCtx.curOffset = full_nsp_header_size; // Calculate DUMP_BUFFER_SIZE block numbers for the modified Program NCA data blocks - if (selectedNspDumpType != DUMP_PATCH_NSP && ncaProgramMod.block_mod_cnt > 0) + if (ncaProgramMod.block_mod_cnt > 0) { hash_table_dump_buffer_start = ((ncaProgramMod.hash_table_offset / DUMP_BUFFER_SIZE) * DUMP_BUFFER_SIZE); hash_table_dump_buffer_end = (((ncaProgramMod.hash_table_offset + ncaProgramMod.hash_table_size) / DUMP_BUFFER_SIZE) * DUMP_BUFFER_SIZE); @@ -1315,7 +1427,7 @@ bool dumpNintendoSubmissionPackage(nspDumpType selectedNspDumpType, u32 titleInd if (nca_offset == 0) memcpy(buf, xml_content_info[i].encrypted_header_mod, NCA_FULL_HEADER_LENGTH); // Replace modified Program NCA data blocks - if (curStorageId == FsStorageId_GameCard && xml_content_info[i].type == NcmContentType_Program && selectedNspDumpType != DUMP_PATCH_NSP && ncaProgramMod.block_mod_cnt > 0) + if (ncaProgramMod.block_mod_cnt > 0 && xml_content_info[i].type == NcmContentType_Program) { u64 program_nca_prev_write; u64 program_nca_next_write; @@ -1487,16 +1599,12 @@ bool dumpNintendoSubmissionPackage(nspDumpType selectedNspDumpType, u32 titleInd if (!proceed) goto out; - dumping = false; - - breaks = (progressCtx.line_offset + 2); - uiFill(0, ((progressCtx.line_offset - 4) * (font_height + (font_height / 4))) + 8, FB_WIDTH, (font_height + (font_height / 4)) * 4, BG_COLOR_RGB, BG_COLOR_RGB, BG_COLOR_RGB); snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Output file: \"%s\".", strrchr(dumpPath, '/' ) + 1); uiDrawString(strbuf, 8, ((progressCtx.line_offset - 4) * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); - uiDrawString("Writing PFS0 header, CNMT XML and CNMT NCA...", 8, ((progressCtx.line_offset - 2) * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); + uiDrawString("Writing PFS0 header...", 8, ((progressCtx.line_offset - 2) * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); uiRefreshDisplay(); @@ -1515,34 +1623,59 @@ bool dumpNintendoSubmissionPackage(nspDumpType selectedNspDumpType, u32 titleInd for(i = 0; i < nspFileCount; i++) { - // If dealing with a title with rights ID, reserve the first four entries for the CNMT XML, NACP XML (if available), cert and tik - // Otherwise, just reserve the first entry for the CNMT XML - - char ncaFileName[50] = {'\0'}; + char ncaFileName[100] = {'\0'}; u64 cur_file_size = 0; - if (i == 0) + if (i < titleNcaCount) { - // CNMT XML + // Always reserve the first titleNcaCount entries for our NCA contents + sprintf(ncaFileName, "%s.%s", xml_content_info[i].nca_id_str, (i == cnmtNcaIndex ? "cnmt.nca" : "nca")); + cur_file_size = xml_content_info[i].size; + } else + if (i == titleNcaCount) + { + // Reserve the entry right after our NCA contents for the CNMT XML sprintf(ncaFileName, "%s.cnmt.xml", xml_content_info[cnmtNcaIndex].nca_id_str); cur_file_size = strlen(cnmtXml); } else { - if (nacpXml && i == 1) + // Deal with additional files packed into the PFS0, in the following order: + // programinfo.xml (if available) + // NACP icons (if available) + // NACP XML (if available) + // legalinfo.xml (if available) + // Ticket (if available) + // Certificate chain (if available) + + if (programInfoXml && i == (titleNcaCount + 1)) { - // NACP XML + // programinfo.xml entry + sprintf(ncaFileName, "%s.programinfo.xml", xml_content_info[programNcaIndex].nca_id_str); + cur_file_size = programInfoXmlSize; + } else + if (nacpIcons && nacpIconCnt && ((!programInfoXml && i <= (titleNcaCount + nacpIconCnt)) || (programInfoXml && i <= (titleNcaCount + 1 + nacpIconCnt)))) + { + // NACP icon entry + u32 icon_idx = (!programInfoXml ? (i - (titleNcaCount + 1)) : (i - (titleNcaCount + 2))); + sprintf(ncaFileName, nacpIcons[icon_idx].filename); + cur_file_size = nacpIcons[icon_idx].icon_size; + } else + if (nacpXml && ((!programInfoXml && i == (titleNcaCount + nacpIconCnt + 1)) || (programInfoXml && i == (titleNcaCount + 1 + nacpIconCnt + 1)))) + { + // NACP XML entry + // If there are no icons, this will effectively make it the next entry after the CNMT XML sprintf(ncaFileName, "%s.nacp.xml", xml_content_info[nacpNcaIndex].nca_id_str); - cur_file_size = strlen(nacpXml); + cur_file_size = nacpXmlSize; + } else + if (legalInfoXml && ((!includeTikAndCert && i == (nspFileCount - 1)) || (includeTikAndCert && i == (nspFileCount - 3)))) + { + // legalinfo.xml entry + // If none of the previous conditions are met, assume we're dealing with a legalinfo.xml depending on the includeTikAndCert and counter values + sprintf(ncaFileName, "%s.legalinfo.xml", xml_content_info[legalInfoNcaIndex].nca_id_str); + cur_file_size = legalInfoXmlSize; } else { - if (includeCertAndTik && ((!nacpXml && (i == 1 || i == 2)) || (nacpXml && (i == 2 || i == 3)))) - { - // cert / tik - sprintf(ncaFileName, "%s", (((!nacpXml && i == 1) || (nacpXml && i == 2)) ? rights_info.cert_filename : rights_info.tik_filename)); - cur_file_size = (((!nacpXml && i == 1) || (nacpXml && i == 2)) ? ETICKET_CERT_FILE_SIZE : ETICKET_TIK_FILE_SIZE); - } else { - u32 cnt_idx = (i - (includeCertAndTik ? 3 : 1) - (nacpXml ? 1 : 0)); - sprintf(ncaFileName, "%s.%s", xml_content_info[cnt_idx].nca_id_str, (cnt_idx == cnmtNcaIndex ? "cnmt.nca" : "nca")); - cur_file_size = xml_content_info[cnt_idx].size; - } + // tik/cert entry + sprintf(ncaFileName, "%s", (i == (nspFileCount - 2) ? rights_info.tik_filename : rights_info.cert_filename)); + cur_file_size = (i == (nspFileCount - 2) ? ETICKET_TIK_FILE_SIZE : ETICKET_CERT_FILE_SIZE); } } @@ -1556,11 +1689,10 @@ bool dumpNintendoSubmissionPackage(nspDumpType selectedNspDumpType, u32 titleInd filename_offset += (strlen(ncaFileName) + 1); } - // Write our full PFS0 header + CNMT XML + // Write our full PFS0 header memcpy(buf, &nspPfs0Header, sizeof(pfs0_header)); - memcpy(buf + sizeof(pfs0_header), nspPfs0EntryTable, nspFileCount * sizeof(pfs0_entry_table)); - memcpy(buf + sizeof(pfs0_header) + (nspFileCount * sizeof(pfs0_entry_table)), nspPfs0StrTable, nspPfs0Header.str_table_size); - memcpy(buf + full_nsp_header_size, cnmtXml, strlen(cnmtXml)); + memcpy(buf + sizeof(pfs0_header), nspPfs0EntryTable, (u64)nspFileCount * sizeof(pfs0_entry_table)); + memcpy(buf + sizeof(pfs0_header) + ((u64)nspFileCount * sizeof(pfs0_entry_table)), nspPfs0StrTable, nspPfs0Header.str_table_size); if (progressCtx.totalSize > FAT32_FILESIZE_LIMIT && isFat32) { @@ -1576,23 +1708,22 @@ bool dumpNintendoSubmissionPackage(nspDumpType selectedNspDumpType, u32 titleInd if (!outFile) { setProgressBarError(&progressCtx); - uiDrawString("Failed to re-open output file for part #0!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + uiDrawString("Failed to re-open output file for part #0!", 8, ((progressCtx.line_offset + 2) * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); goto out; } } else { rewind(outFile); } - write_res = fwrite(buf, 1, full_nsp_header_size + strlen(cnmtXml), outFile); - if (write_res != (full_nsp_header_size + strlen(cnmtXml))) + write_res = fwrite(buf, 1, full_nsp_header_size, outFile); + if (write_res != full_nsp_header_size) { setProgressBarError(&progressCtx); - snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Failed to write %lu bytes PFS0 header + CNMT XML to file offset 0x%016lX! (wrote %lu bytes)", full_nsp_header_size + strlen(cnmtXml), (u64)0, write_res); - uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Failed to write %lu bytes PFS0 header to file offset 0x%016lX! (wrote %lu bytes)", full_nsp_header_size, (u64)0, write_res); + uiDrawString(strbuf, 8, ((progressCtx.line_offset + 2) * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); goto out; } - // Now let's write our modified CNMT NCA if (progressCtx.totalSize > FAT32_FILESIZE_LIMIT && isFat32) { if (outFile) @@ -1608,95 +1739,226 @@ bool dumpNintendoSubmissionPackage(nspDumpType selectedNspDumpType, u32 titleInd { setProgressBarError(&progressCtx); snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Failed to re-open output file for part #%u!", splitIndex); - uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); - goto out; - } - - fseek(outFile, 0, SEEK_END); - - // This is a pain - u64 cur_file_size = (progressCtx.curOffset - (splitIndex * SPLIT_FILE_NSP_PART_SIZE)); - if ((cur_file_size + xml_content_info[cnmtNcaIndex].size) > SPLIT_FILE_NSP_PART_SIZE) - { - u64 new_file_chunk_size = ((progressCtx.curOffset + xml_content_info[cnmtNcaIndex].size) - ((splitIndex + 1) * SPLIT_FILE_NSP_PART_SIZE)); - u64 old_file_chunk_size = (xml_content_info[cnmtNcaIndex].size - new_file_chunk_size); - - if (old_file_chunk_size > 0) - { - write_res = fwrite(cnmtNcaBuf, 1, old_file_chunk_size, outFile); - if (write_res != old_file_chunk_size) - { - setProgressBarError(&progressCtx); - snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Failed to write %lu bytes CNMT NCA chunk #1 from offset 0x%016lX to part #%02u! (wrote %lu bytes)", old_file_chunk_size, progressCtx.curOffset, splitIndex, write_res); - uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); - goto out; - } - } - - fclose(outFile); - outFile = NULL; - - if (new_file_chunk_size > 0) - { - splitIndex++; - snprintf(dumpPath, sizeof(dumpPath) / sizeof(dumpPath[0]), "%s%s.nsp/%02u", NSP_DUMP_PATH, dumpName, splitIndex); - - outFile = fopen(dumpPath, "wb"); - if (!outFile) - { - setProgressBarError(&progressCtx); - snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Failed to open output file for part #%u!", splitIndex); - uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); - goto out; - } - - uiFill(0, ((progressCtx.line_offset - 4) * (font_height + (font_height / 4))) + 8, FB_WIDTH, (font_height + (font_height / 4)), BG_COLOR_RGB, BG_COLOR_RGB, BG_COLOR_RGB); - - snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Output file: \"%s\".", strrchr(dumpPath, '/' ) + 1); - uiDrawString(strbuf, 8, ((progressCtx.line_offset - 4) * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); - - uiRefreshDisplay(); - - write_res = fwrite(cnmtNcaBuf + old_file_chunk_size, 1, new_file_chunk_size, outFile); - if (write_res != new_file_chunk_size) - { - setProgressBarError(&progressCtx); - snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Failed to write %lu bytes CNMT NCA chunk #2 from offset 0x%016lX to part #%02u! (wrote %lu bytes)", new_file_chunk_size, progressCtx.curOffset + old_file_chunk_size, splitIndex, write_res); - uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); - goto out; - } - } - } else { - write_res = fwrite(cnmtNcaBuf, 1, xml_content_info[cnmtNcaIndex].size, outFile); - if (write_res != xml_content_info[cnmtNcaIndex].size) - { - setProgressBarError(&progressCtx); - snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Failed to write %lu bytes CNMT NCA to file offset 0x%016lX! (wrote %lu bytes)", xml_content_info[cnmtNcaIndex].size, progressCtx.curOffset, write_res); - uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); - goto out; - } - } - } else { - fseek(outFile, 0, SEEK_END); - - write_res = fwrite(cnmtNcaBuf, 1, xml_content_info[cnmtNcaIndex].size, outFile); - if (write_res != xml_content_info[cnmtNcaIndex].size) - { - setProgressBarError(&progressCtx); - snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Failed to write %lu bytes CNMT NCA to file offset 0x%016lX! (wrote %lu bytes)", xml_content_info[cnmtNcaIndex].size, progressCtx.curOffset, write_res); - uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); - - if ((progressCtx.curOffset + xml_content_info[cnmtNcaIndex].size) > FAT32_FILESIZE_LIMIT) - { - breaks += 2; - uiDrawString("You're probably using a FAT32 partition. Make sure to enable the \"Split output dump\" option.", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); - } - + uiDrawString(strbuf, 8, ((progressCtx.line_offset + 2) * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); goto out; } } - progressCtx.curOffset += xml_content_info[cnmtNcaIndex].size; + fseek(outFile, 0, SEEK_END); + + // Now let's write the rest of the data, including our modified CNMT NCA + for(i = (titleNcaCount - 1); i < nspFileCount; i++) + { + n = DUMP_BUFFER_SIZE; + + char ncaFileName[100] = {'\0'}; + u64 cur_file_size = 0; + + if (i == (titleNcaCount - 1)) + { + // CNMT NCA + sprintf(ncaFileName, "%s.cnmt.nca", xml_content_info[i].nca_id_str); + cur_file_size = xml_content_info[cnmtNcaIndex].size; + } else + if (i == titleNcaCount) + { + // CNMT XML + sprintf(ncaFileName, "%s.cnmt.xml", xml_content_info[cnmtNcaIndex].nca_id_str); + cur_file_size = strlen(cnmtXml); + } else { + if (programInfoXml && i == (titleNcaCount + 1)) + { + // programinfo.xml entry + sprintf(ncaFileName, "%s.programinfo.xml", xml_content_info[programNcaIndex].nca_id_str); + cur_file_size = programInfoXmlSize; + } else + if (nacpIcons && nacpIconCnt && ((!programInfoXml && i <= (titleNcaCount + nacpIconCnt)) || (programInfoXml && i <= (titleNcaCount + 1 + nacpIconCnt)))) + { + // NACP icon entry + u32 icon_idx = (!programInfoXml ? (i - (titleNcaCount + 1)) : (i - (titleNcaCount + 2))); + sprintf(ncaFileName, nacpIcons[icon_idx].filename); + cur_file_size = nacpIcons[icon_idx].icon_size; + } else + if (nacpXml && ((!programInfoXml && i == (titleNcaCount + nacpIconCnt + 1)) || (programInfoXml && i == (titleNcaCount + 1 + nacpIconCnt + 1)))) + { + // NACP XML entry + sprintf(ncaFileName, "%s.nacp.xml", xml_content_info[nacpNcaIndex].nca_id_str); + cur_file_size = nacpXmlSize; + } else + if (legalInfoXml && ((!includeTikAndCert && i == (nspFileCount - 1)) || (includeTikAndCert && i == (nspFileCount - 3)))) + { + // legalinfo.xml entry + sprintf(ncaFileName, "%s.legalinfo.xml", xml_content_info[legalInfoNcaIndex].nca_id_str); + cur_file_size = legalInfoXmlSize; + } else { + // tik/cert entry + sprintf(ncaFileName, "%s", (i == (nspFileCount - 2) ? rights_info.tik_filename : rights_info.cert_filename)); + cur_file_size = (i == (nspFileCount - 2) ? ETICKET_TIK_FILE_SIZE : ETICKET_CERT_FILE_SIZE); + } + } + + for(nca_offset = 0; nca_offset < cur_file_size; nca_offset += n, progressCtx.curOffset += n) + { + uiFill(0, ((progressCtx.line_offset - 4) * (font_height + (font_height / 4))) + 8, FB_WIDTH, (font_height + (font_height / 4)) * 4, BG_COLOR_RGB, BG_COLOR_RGB, BG_COLOR_RGB); + + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Output file: \"%s\".", strrchr(dumpPath, '/' ) + 1); + uiDrawString(strbuf, 8, ((progressCtx.line_offset - 4) * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); + + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Writing \"%s\"...", ncaFileName); + uiDrawString(strbuf, 8, ((progressCtx.line_offset - 2) * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); + + uiRefreshDisplay(); + + if (DUMP_BUFFER_SIZE > (cur_file_size - nca_offset)) n = (cur_file_size - nca_offset); + + // Retrieve data from its respective source + if (i == (titleNcaCount - 1)) + { + // CNMT NCA + memcpy(buf, cnmtNcaBuf + nca_offset, n); + } else + if (i == titleNcaCount) + { + // CNMT XML + memcpy(buf, cnmtXml + nca_offset, n); + } else { + if (programInfoXml && i == (titleNcaCount + 1)) + { + // programinfo.xml entry + memcpy(buf, programInfoXml + nca_offset, n); + } else + if (nacpIcons && nacpIconCnt && ((!programInfoXml && i <= (titleNcaCount + nacpIconCnt)) || (programInfoXml && i <= (titleNcaCount + 1 + nacpIconCnt)))) + { + // NACP icon entry + u32 icon_idx = (!programInfoXml ? (i - (titleNcaCount + 1)) : (i - (titleNcaCount + 2))); + memcpy(buf, nacpIcons[icon_idx].icon_data + nca_offset, n); + } else + if (nacpXml && ((!programInfoXml && i == (titleNcaCount + nacpIconCnt + 1)) || (programInfoXml && i == (titleNcaCount + 1 + nacpIconCnt + 1)))) + { + // NACP XML entry + memcpy(buf, nacpXml + nca_offset, n); + } else + if (legalInfoXml && ((!includeTikAndCert && i == (nspFileCount - 1)) || (includeTikAndCert && i == (nspFileCount - 3)))) + { + // legalinfo.xml entry + memcpy(buf, legalInfoXml + nca_offset, n); + } else { + // tik/cert entry + if (i == (nspFileCount - 2)) + { + memcpy(buf, (u8*)(&(rights_info.tik_data)) + nca_offset, n); + } else { + memcpy(buf, rights_info.cert_data + nca_offset, n); + } + } + } + + if (progressCtx.totalSize > FAT32_FILESIZE_LIMIT && isFat32 && (progressCtx.curOffset + n) >= ((splitIndex + 1) * SPLIT_FILE_NSP_PART_SIZE)) + { + u64 new_file_chunk_size = ((progressCtx.curOffset + n) - ((splitIndex + 1) * SPLIT_FILE_NSP_PART_SIZE)); + u64 old_file_chunk_size = (n - new_file_chunk_size); + + if (old_file_chunk_size > 0) + { + write_res = fwrite(buf, 1, old_file_chunk_size, outFile); + if (write_res != old_file_chunk_size) + { + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Failed to write %lu bytes chunk from offset 0x%016lX to part #%02u! (wrote %lu bytes)", old_file_chunk_size, progressCtx.curOffset, splitIndex, write_res); + uiDrawString(strbuf, 8, ((progressCtx.line_offset + 2) * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + proceed = false; + break; + } + } + + fclose(outFile); + outFile = NULL; + + if (new_file_chunk_size > 0 || (progressCtx.curOffset + n) < progressCtx.totalSize) + { + splitIndex++; + snprintf(dumpPath, sizeof(dumpPath) / sizeof(dumpPath[0]), "%s%s.nsp/%02u", NSP_DUMP_PATH, dumpName, splitIndex); + + outFile = fopen(dumpPath, "wb"); + if (!outFile) + { + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Failed to open output file for part #%u!", splitIndex); + uiDrawString(strbuf, 8, ((progressCtx.line_offset + 2) * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + proceed = false; + break; + } + + if (new_file_chunk_size > 0) + { + write_res = fwrite(buf + old_file_chunk_size, 1, new_file_chunk_size, outFile); + if (write_res != new_file_chunk_size) + { + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Failed to write %lu bytes chunk from offset 0x%016lX to part #%02u! (wrote %lu bytes)", new_file_chunk_size, progressCtx.curOffset + old_file_chunk_size, splitIndex, write_res); + uiDrawString(strbuf, 8, ((progressCtx.line_offset + 2) * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + proceed = false; + break; + } + } + } + } else { + write_res = fwrite(buf, 1, n, outFile); + if (write_res != n) + { + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Failed to write %lu bytes chunk from offset 0x%016lX! (wrote %lu bytes)", n, progressCtx.curOffset, write_res); + uiDrawString(strbuf, 8, ((progressCtx.line_offset + 2) * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + + if ((progressCtx.curOffset + n) > FAT32_FILESIZE_LIMIT) + { + uiDrawString("You're probably using a FAT32 partition. Make sure to enable the \"Split output dump\" option.", 8, ((progressCtx.line_offset + 4) * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); + fat32_error = true; + } + + proceed = false; + break; + } + } + + printProgressBar(&progressCtx, true, n); + + if ((progressCtx.curOffset + n) < progressCtx.totalSize && ((progressCtx.curOffset / DUMP_BUFFER_SIZE) % 10) == 0) + { + hidScanInput(); + + u32 keysDown = hidKeysDown(CONTROLLER_P1_AUTO); + if (keysDown & KEY_B) + { + uiDrawString("Process canceled.", 8, ((progressCtx.line_offset + 2) * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + proceed = false; + break; + } + } + } + + if (!proceed) + { + setProgressBarError(&progressCtx); + break; + } + + // Support empty files + if (!cur_file_size) + { + uiFill(0, ((progressCtx.line_offset - 4) * (font_height + (font_height / 4))) + 8, FB_WIDTH, (font_height + (font_height / 4)) * 4, BG_COLOR_RGB, BG_COLOR_RGB, BG_COLOR_RGB); + + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), strrchr(dumpPath, '/' ) + 1); + uiDrawString(strbuf, 8, ((progressCtx.line_offset - 4) * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); + + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Writing \"%s\"...", ncaFileName); + uiDrawString(strbuf, 8, ((progressCtx.line_offset - 2) * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); + + printProgressBar(&progressCtx, false, 0); + } + } + + if (!proceed) goto out; + + dumping = false; + + breaks = (progressCtx.line_offset + 2); if (progressCtx.curOffset < progressCtx.totalSize) { @@ -1709,21 +1971,24 @@ bool dumpNintendoSubmissionPackage(nspDumpType selectedNspDumpType, u32 titleInd success = true; // Finalize dump - timeGetCurrentTime(TimeType_LocalSystemClock, &(progressCtx.now)); - progressCtx.now -= progressCtx.start; + if (!batch) + { + timeGetCurrentTime(TimeType_LocalSystemClock, &(progressCtx.now)); + progressCtx.now -= progressCtx.start; + + progressCtx.progress = 100; + progressCtx.remainingTime = 0; + + printProgressBar(&progressCtx, false, 0); + + formatETAString(progressCtx.now, progressCtx.etaInfo, sizeof(progressCtx.etaInfo) / sizeof(progressCtx.etaInfo[0])); + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Process successfully completed after %s!", progressCtx.etaInfo); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 0, 255, 0); + + uiRefreshDisplay(); + } - progressCtx.progress = 100; - progressCtx.remainingTime = 0; - - printProgressBar(&progressCtx, false, 0); - - formatETAString(progressCtx.now, progressCtx.etaInfo, sizeof(progressCtx.etaInfo) / sizeof(progressCtx.etaInfo[0])); - snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Process successfully completed after %s!", progressCtx.etaInfo); - uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 0, 255, 0); - - uiRefreshDisplay(); - - if (calcCrc) + if (!batch && calcCrc) { breaks += 2; uiDrawString("CRC32 checksum calculation will begin in 5 seconds...", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); @@ -1918,14 +2183,20 @@ out: if (cnmtNcaBuf) free(cnmtNcaBuf); - if (ncaProgramMod.block_mod_cnt == 2 && ncaProgramMod.block_data[1]) free(ncaProgramMod.block_data[1]); + if (ncaProgramMod.block_data[1]) free(ncaProgramMod.block_data[1]); if (ncaProgramMod.block_data[0]) free(ncaProgramMod.block_data[0]); if (ncaProgramMod.hash_table) free(ncaProgramMod.hash_table); + if (legalInfoXml) free(legalInfoXml); + + if (nacpIcons) free(nacpIcons); + if (nacpXml) free(nacpXml); + if (programInfoXml) free(programInfoXml); + serviceClose(&(ncmStorage.s)); if (xml_content_info) free(xml_content_info); @@ -1953,11 +2224,427 @@ out: free(dumpName); - breaks += 2; + if (!batch) breaks += 2; return success; } +bool dumpNintendoSubmissionPackageBatch(bool dumpAppTitles, bool dumpPatchTitles, bool dumpAddOnTitles, bool isFat32, bool removeConsoleData, bool tiklessDump, bool skipDumpedTitles, batchModeSourceStorage batchModeSrc) +{ + if ((!dumpAppTitles && !dumpPatchTitles && !dumpAddOnTitles) || (batchModeSrc == BATCH_SOURCE_ALL && ((dumpAppTitles && !titleAppCount) || (dumpPatchTitles && !titlePatchCount) || (dumpAddOnTitles && !titleAddOnCount))) || (batchModeSrc == BATCH_SOURCE_SDCARD && ((dumpAppTitles && !sdCardTitleAppCount) || (dumpPatchTitles && !sdCardTitlePatchCount) || (dumpAddOnTitles && !sdCardTitleAddOnCount))) || (batchModeSrc == BATCH_SOURCE_EMMC && ((dumpAppTitles && !nandUserTitleAppCount) || (dumpPatchTitles && !nandUserTitlePatchCount) || (dumpAddOnTitles && !nandUserTitleAddOnCount)))) + { + uiDrawString("Error: invalid parameters to perform batch NSP dump!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + breaks += 2; + return false; + } + + u32 i, j; + + u32 totalTitleCount = 0, totalAppCount = 0, totalPatchCount = 0, totalAddOnCount = 0; + + u32 titleCount, titleIndex; + + char *dumpName = NULL; + char dumpPath[NAME_BUF_LEN * 2] = {'\0'}; + char curName[NAME_BUF_LEN * 2] = {'\0'}; + + int initial_breaks = breaks, cur_breaks; + + const u32 maxSummaryFileCount = 6; + u32 summaryPage = 0; + + memset(filenameBuffer, 0, FILENAME_BUFFER_SIZE); + filenamesCount = 0; + + char *nextFilename = filenameBuffer; + + bool proceed = true; + + if (dumpAppTitles) + { + titleCount = (batchModeSrc == BATCH_SOURCE_ALL ? titleAppCount : (batchModeSrc == BATCH_SOURCE_SDCARD ? sdCardTitleAppCount : nandUserTitleAppCount)); + + for(i = 0; i < titleCount; i++) + { + titleIndex = ((batchModeSrc == BATCH_SOURCE_ALL || batchModeSrc == BATCH_SOURCE_SDCARD) ? i : (i + sdCardTitleAppCount)); + + dumpName = generateNSPDumpName(DUMP_APP_NSP, titleIndex); + if (!dumpName) + { + uiDrawString("Error: unable to generate output dump name!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + breaks += 2; + return false; + } + + snprintf(dumpPath, sizeof(dumpPath) / sizeof(dumpPath[0]), "%s%s.nsp", NSP_DUMP_PATH, dumpName); + + free(dumpName); + dumpName = NULL; + + // Check if this title has already been dumped + if (skipDumpedTitles && checkIfFileExists(dumpPath)) continue; + + snprintf(curName, sizeof(curName) / sizeof(curName[0]), strrchr(dumpPath, '/') + 1); + + // Fix entry name length + u32 strWidth = uiGetStrWidth(curName); + + if ((8 + strWidth) >= (FB_WIDTH - (font_height * 5))) + { + while((8 + strWidth) >= (FB_WIDTH - (font_height * 5))) + { + curName[strlen(curName) - 1] = '\0'; + strWidth = uiGetStrWidth(curName); + } + + strcat(curName, "..."); + } + + addStringToFilenameBuffer(curName, &nextFilename); + + totalAppCount++; + } + + totalTitleCount += totalAppCount; + } + + if (dumpPatchTitles) + { + titleCount = (batchModeSrc == BATCH_SOURCE_ALL ? titlePatchCount : (batchModeSrc == BATCH_SOURCE_SDCARD ? sdCardTitlePatchCount : nandUserTitlePatchCount)); + + for(i = 0; i < titleCount; i++) + { + titleIndex = ((batchModeSrc == BATCH_SOURCE_ALL || batchModeSrc == BATCH_SOURCE_SDCARD) ? i : (i + sdCardTitlePatchCount)); + + dumpName = generateNSPDumpName(DUMP_PATCH_NSP, titleIndex); + if (!dumpName) + { + uiDrawString("Error: unable to generate output dump name!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + breaks += 2; + return false; + } + + snprintf(dumpPath, sizeof(dumpPath) / sizeof(dumpPath[0]), "%s%s.nsp", NSP_DUMP_PATH, dumpName); + + free(dumpName); + dumpName = NULL; + + // Check if this title has already been dumped + if (skipDumpedTitles && checkIfFileExists(dumpPath)) continue; + + snprintf(curName, sizeof(curName) / sizeof(curName[0]), strrchr(dumpPath, '/') + 1); + + // Fix entry name length + u32 strWidth = uiGetStrWidth(curName); + + if ((8 + strWidth) >= (FB_WIDTH - (font_height * 5))) + { + while((8 + strWidth) >= (FB_WIDTH - (font_height * 5))) + { + curName[strlen(curName) - 1] = '\0'; + strWidth = uiGetStrWidth(curName); + } + + strcat(curName, "..."); + } + + addStringToFilenameBuffer(curName, &nextFilename); + + totalPatchCount++; + } + + totalTitleCount += totalPatchCount; + } + + if (dumpAddOnTitles) + { + titleCount = (batchModeSrc == BATCH_SOURCE_ALL ? titleAddOnCount : (batchModeSrc == BATCH_SOURCE_SDCARD ? sdCardTitleAddOnCount : nandUserTitleAddOnCount)); + + for(i = 0; i < titleCount; i++) + { + titleIndex = ((batchModeSrc == BATCH_SOURCE_ALL || batchModeSrc == BATCH_SOURCE_SDCARD) ? i : (i + sdCardTitleAddOnCount)); + + dumpName = generateNSPDumpName(DUMP_ADDON_NSP, titleIndex); + if (!dumpName) + { + uiDrawString("Error: unable to generate output dump name!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + breaks += 2; + return false; + } + + snprintf(dumpPath, sizeof(dumpPath) / sizeof(dumpPath[0]), "%s%s.nsp", NSP_DUMP_PATH, dumpName); + + free(dumpName); + dumpName = NULL; + + // Check if this title has already been dumped + if (skipDumpedTitles && checkIfFileExists(dumpPath)) continue; + + snprintf(curName, sizeof(curName) / sizeof(curName[0]), strrchr(dumpPath, '/') + 1); + + // Fix entry name length + u32 strWidth = uiGetStrWidth(curName); + + if ((8 + strWidth) >= (FB_WIDTH - (font_height * 5))) + { + while((8 + strWidth) >= (FB_WIDTH - (font_height * 5))) + { + curName[strlen(curName) - 1] = '\0'; + strWidth = uiGetStrWidth(curName); + } + + strcat(curName, "..."); + } + + addStringToFilenameBuffer(curName, &nextFilename); + + totalAddOnCount++; + } + + totalTitleCount += totalAddOnCount; + } + + if (!totalTitleCount) + { + uiDrawString("You have already dumped all titles matching the selected settings!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); + breaks += 2; + return false; + } + + // Display summary + uiDrawString("Summary:", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); + breaks += 2; + + strbuf[0] = '\0'; + + if (totalAppCount) + { + snprintf(curName, sizeof(curName) / sizeof(curName[0]), "BASE: %u", totalAppCount); + strcat(strbuf, curName); + } + + if (totalPatchCount) + { + if (totalAppCount) strcat(strbuf, " | "); + snprintf(curName, sizeof(curName) / sizeof(curName[0]), "UPD: %u", totalPatchCount); + strcat(strbuf, curName); + } + + if (totalAddOnCount) + { + if (totalAppCount || totalPatchCount) strcat(strbuf, " | "); + snprintf(curName, sizeof(curName) / sizeof(curName[0]), "DLC: %u", totalAddOnCount); + strcat(strbuf, curName); + } + + strcat(strbuf, " | "); + snprintf(curName, sizeof(curName) / sizeof(curName[0]), "Total: %u", totalTitleCount); + strcat(strbuf, curName); + + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); + breaks++; + + while(true) + { + cur_breaks = breaks; + + uiFill(0, 8 + (cur_breaks * (font_height + (font_height / 4))), FB_WIDTH, FB_HEIGHT - (8 + (cur_breaks * (font_height + (font_height / 4)))), BG_COLOR_RGB, BG_COLOR_RGB, BG_COLOR_RGB); + + if (totalTitleCount > maxSummaryFileCount) + { + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Current page: %u", summaryPage + 1); + uiDrawString(strbuf, 8, (cur_breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); + cur_breaks++; + } + + cur_breaks++; + + for(i = (summaryPage * maxSummaryFileCount); i < ((summaryPage * maxSummaryFileCount) + maxSummaryFileCount); i++) + { + if (i >= totalTitleCount) break; + uiDrawIcon(fileNormalIconBuf, BROWSER_ICON_DIMENSION, BROWSER_ICON_DIMENSION, 8, 8 + (cur_breaks * (font_height + (font_height / 4))) + (font_height / 8)); + uiDrawString(filenames[i], BROWSER_ICON_DIMENSION + 8, (cur_breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); + cur_breaks++; + } + + cur_breaks++; + + if (totalTitleCount > maxSummaryFileCount) + { + uiDrawString("[ " NINTENDO_FONT_L " / " NINTENDO_FONT_R " / " NINTENDO_FONT_ZL " / " NINTENDO_FONT_ZR " ] Change page | [ " NINTENDO_FONT_A " ] Proceed | [ " NINTENDO_FONT_B " ] Cancel", 8, (cur_breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); + } else { + uiDrawString("[ " NINTENDO_FONT_A " ] Proceed | [ " NINTENDO_FONT_B " ] Cancel", 8, (cur_breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); + } + + uiRefreshDisplay(); + + hidScanInput(); + + u32 keysDown = hidKeysDown(CONTROLLER_P1_AUTO); + + if (keysDown & KEY_A) + { + proceed = true; + break; + } else + if (keysDown & KEY_B) + { + proceed = false; + break; + } else + if (((keysDown & KEY_L) || (keysDown & KEY_ZL)) && totalTitleCount > maxSummaryFileCount) + { + if (summaryPage > 0) summaryPage--; + } else + if (((keysDown & KEY_R) || (keysDown & KEY_ZR)) && totalTitleCount > maxSummaryFileCount) + { + if (((summaryPage * maxSummaryFileCount) + maxSummaryFileCount) < totalTitleCount) summaryPage++; + } + } + + if (!proceed) + { + breaks = (cur_breaks + 2); + uiDrawString("Process canceled", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + breaks += 2; + return false; + } + + breaks = initial_breaks; + uiFill(0, 8 + (breaks * (font_height + (font_height / 4))), FB_WIDTH, FB_HEIGHT - (8 + (breaks * (font_height + (font_height / 4)))), BG_COLOR_RGB, BG_COLOR_RGB, BG_COLOR_RGB); + + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Dump procedure started. Hold %s to cancel.", NINTENDO_FONT_B); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); + breaks += 2; + + initial_breaks = breaks; + + j = 0; + + if (totalAppCount) + { + titleCount = (batchModeSrc == BATCH_SOURCE_ALL ? titleAppCount : (batchModeSrc == BATCH_SOURCE_SDCARD ? sdCardTitleAppCount : nandUserTitleAppCount)); + + for(i = 0; i < titleCount; i++) + { + breaks = initial_breaks; + uiFill(0, 8 + (breaks * (font_height + (font_height / 4))), FB_WIDTH, FB_HEIGHT - (8 + (breaks * (font_height + (font_height / 4)))), BG_COLOR_RGB, BG_COLOR_RGB, BG_COLOR_RGB); + + titleIndex = ((batchModeSrc == BATCH_SOURCE_ALL || batchModeSrc == BATCH_SOURCE_SDCARD) ? i : (i + sdCardTitleAppCount)); + + dumpName = generateNSPDumpName(DUMP_APP_NSP, titleIndex); + if (!dumpName) + { + uiDrawString("Error: unable to generate output dump name!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + breaks += 2; + return false; + } + + snprintf(dumpPath, sizeof(dumpPath) / sizeof(dumpPath[0]), "%s%s.nsp", NSP_DUMP_PATH, dumpName); + + free(dumpName); + dumpName = NULL; + + // Check if this title has already been dumped + if (skipDumpedTitles && checkIfFileExists(dumpPath)) continue; + + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Title: %u / %u.", j + 1, totalTitleCount); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); + uiRefreshDisplay(); + breaks += 2; + + // Dump title + if (!dumpNintendoSubmissionPackage(DUMP_APP_NSP, titleIndex, isFat32, false, removeConsoleData, tiklessDump, true)) return false; + + j++; + } + } + + if (totalPatchCount) + { + titleCount = (batchModeSrc == BATCH_SOURCE_ALL ? titlePatchCount : (batchModeSrc == BATCH_SOURCE_SDCARD ? sdCardTitlePatchCount : nandUserTitlePatchCount)); + + for(i = 0; i < titleCount; i++) + { + breaks = initial_breaks; + uiFill(0, 8 + (breaks * (font_height + (font_height / 4))), FB_WIDTH, FB_HEIGHT - (8 + (breaks * (font_height + (font_height / 4)))), BG_COLOR_RGB, BG_COLOR_RGB, BG_COLOR_RGB); + + titleIndex = ((batchModeSrc == BATCH_SOURCE_ALL || batchModeSrc == BATCH_SOURCE_SDCARD) ? i : (i + sdCardTitlePatchCount)); + + dumpName = generateNSPDumpName(DUMP_PATCH_NSP, titleIndex); + if (!dumpName) + { + uiDrawString("Error: unable to generate output dump name!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + breaks += 2; + return false; + } + + snprintf(dumpPath, sizeof(dumpPath) / sizeof(dumpPath[0]), "%s%s.nsp", NSP_DUMP_PATH, dumpName); + + free(dumpName); + dumpName = NULL; + + // Check if this title has already been dumped + if (skipDumpedTitles && checkIfFileExists(dumpPath)) continue; + + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Title: %u / %u.", j + 1, totalTitleCount); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); + uiRefreshDisplay(); + breaks += 2; + + // Dump title + if (!dumpNintendoSubmissionPackage(DUMP_PATCH_NSP, titleIndex, isFat32, false, removeConsoleData, tiklessDump, true)) return false; + + j++; + } + } + + if (totalAddOnCount) + { + titleCount = (batchModeSrc == BATCH_SOURCE_ALL ? titleAddOnCount : (batchModeSrc == BATCH_SOURCE_SDCARD ? sdCardTitleAddOnCount : nandUserTitleAddOnCount)); + + for(i = 0; i < titleCount; i++) + { + breaks = initial_breaks; + uiFill(0, 8 + (breaks * (font_height + (font_height / 4))), FB_WIDTH, FB_HEIGHT - (8 + (breaks * (font_height + (font_height / 4)))), BG_COLOR_RGB, BG_COLOR_RGB, BG_COLOR_RGB); + + titleIndex = ((batchModeSrc == BATCH_SOURCE_ALL || batchModeSrc == BATCH_SOURCE_SDCARD) ? i : (i + sdCardTitleAddOnCount)); + + dumpName = generateNSPDumpName(DUMP_ADDON_NSP, titleIndex); + if (!dumpName) + { + uiDrawString("Error: unable to generate output dump name!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + breaks += 2; + return false; + } + + snprintf(dumpPath, sizeof(dumpPath) / sizeof(dumpPath[0]), "%s%s.nsp", NSP_DUMP_PATH, dumpName); + + free(dumpName); + dumpName = NULL; + + // Check if this title has already been dumped + if (skipDumpedTitles && checkIfFileExists(dumpPath)) continue; + + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Title: %u / %u.", j + 1, totalTitleCount); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); + uiRefreshDisplay(); + breaks += 2; + + // Dump title + if (!dumpNintendoSubmissionPackage(DUMP_ADDON_NSP, titleIndex, isFat32, false, removeConsoleData, tiklessDump, true)) return false; + + j++; + } + } + + uiDrawString("Process successfully completed!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 0, 255, 0); + + breaks += 2; + + return true; +} + bool dumpRawHfs0Partition(u32 partition, bool doSplitting) { Result result; @@ -1976,7 +2663,7 @@ bool dumpRawHfs0Partition(u32 partition, bool doSplitting) size_t write_res; - char *dumpName = generateDumpFullName(); + char *dumpName = generateFullDumpName(); if (!dumpName) { uiDrawString("Error: unable to generate output dump name!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); @@ -1998,7 +2685,7 @@ bool dumpRawHfs0Partition(u32 partition, bool doSplitting) // * The "update" (0) partition, the "logo" (1) partition and the "normal" (2) partition (for gamecard type 0x02) // The IStorage instance returned for partition == 1 contains the "secure" partition (which can either be 2 or 3 depending on the gamecard type) // This ugly hack makes sure we just dump the *actual* raw HFS0 partition, without preceding data, padding, etc. - // Oddly enough, IFileSystem instances actually points to the specified partition ID filesystem. I don't understand why it doesn't work like that for IStorage, but whatever + // Oddly enough, IFileSystem instances actually point to the specified partition ID filesystem. I don't understand why it doesn't work like that for IStorage, but whatever // NOTE: Using partition == 2 returns error 0x149002, and using higher values probably do so, too if (R_SUCCEEDED(result = fsOpenGameCardStorage(&gameCardStorage, &handle, HFS0_TO_ISTORAGE_IDX(hfs0_partition_cnt, partition)))) @@ -2392,6 +3079,7 @@ bool copyFileFromHfs0(u32 partition, const char* source, const char* dest, const if (!success) { setProgressBarError(progressCtx); + breaks = (progressCtx->line_offset + 2); if (fat32_error) breaks += 2; } @@ -2495,7 +3183,7 @@ bool copyHfs0Contents(u32 partition, hfs0_entry_table *partitionEntryTable, prog return success; } -bool dumpHfs0PartitionData(u32 partition) +bool dumpHfs0PartitionData(u32 partition, bool doSplitting) { bool success = false; u32 i; @@ -2505,7 +3193,7 @@ bool dumpHfs0PartitionData(u32 partition) progress_ctx_t progressCtx; memset(&progressCtx, 0, sizeof(progress_ctx_t)); - char *dumpName = generateDumpFullName(); + char *dumpName = generateFullDumpName(); if (!dumpName) { uiDrawString("Error: unable to generate output dump name!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); @@ -2550,12 +3238,12 @@ bool dumpHfs0PartitionData(u32 partition) progressCtx.line_offset = (breaks + 4); - success = copyHfs0Contents(partition, entryTable, &progressCtx, dumpPath, true); - - breaks = (progressCtx.line_offset + 2); + success = copyHfs0Contents(partition, entryTable, &progressCtx, dumpPath, doSplitting); if (success) { + breaks = (progressCtx.line_offset + 2); + timeGetCurrentTime(TimeType_LocalSystemClock, &(progressCtx.now)); progressCtx.now -= progressCtx.start; @@ -2592,7 +3280,7 @@ bool dumpHfs0PartitionData(u32 partition) return success; } -bool dumpFileFromHfs0Partition(u32 partition, u32 file, char *filename) +bool dumpFileFromHfs0Partition(u32 partition, u32 file, char *filename, bool doSplitting) { if (!partitionHfs0Header) { @@ -2611,7 +3299,7 @@ bool dumpFileFromHfs0Partition(u32 partition, u32 file, char *filename) progress_ctx_t progressCtx; memset(&progressCtx, 0, sizeof(progress_ctx_t)); - char *dumpName = generateDumpFullName(); + char *dumpName = generateFullDumpName(); if (!dumpName) { uiDrawString("Error: unable to generate output dump name!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); @@ -2685,12 +3373,12 @@ bool dumpFileFromHfs0Partition(u32 partition, u32 file, char *filename) progressCtx.line_offset = (breaks + 4); timeGetCurrentTime(TimeType_LocalSystemClock, &(progressCtx.start)); - success = copyFileFromHfs0(partition, filename, destCopyPath, file_offset, file_size, &progressCtx, true); - - breaks = (progressCtx.line_offset + 2); + success = copyFileFromHfs0(partition, filename, destCopyPath, file_offset, file_size, &progressCtx, doSplitting); if (success) { + breaks = (progressCtx.line_offset + 2); + timeGetCurrentTime(TimeType_LocalSystemClock, &(progressCtx.now)); progressCtx.now -= progressCtx.start; @@ -2718,7 +3406,7 @@ bool dumpFileFromHfs0Partition(u32 partition, u32 file, char *filename) return success; } -bool dumpExeFsSectionData(u32 appIndex, bool doSplitting) +bool dumpExeFsSectionData(u32 titleIndex, bool usePatch, bool doSplitting) { u64 n; FILE *outFile; @@ -2737,21 +3425,21 @@ bool dumpExeFsSectionData(u32 appIndex, bool doSplitting) u32 i; u64 offset; - if (!titleAppCount) + if ((!usePatch && !titleAppCount) || (usePatch && !titlePatchCount)) { - uiDrawString("Error: invalid application count!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + uiDrawString("Error: invalid title count!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); breaks += 2; return false; } - if (appIndex > (titleAppCount - 1)) + if ((!usePatch && titleIndex > (titleAppCount - 1)) || (usePatch && titleIndex > (titlePatchCount - 1))) { - uiDrawString("Error: invalid application index!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + uiDrawString("Error: invalid title index!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); breaks += 2; return false; } - char *dumpName = generateNSPDumpName(DUMP_APP_NSP, appIndex); + char *dumpName = generateNSPDumpName((!usePatch ? DUMP_APP_NSP : DUMP_PATCH_NSP), titleIndex); if (!dumpName) { uiDrawString("Error: unable to generate output dump name!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); @@ -2759,11 +3447,17 @@ bool dumpExeFsSectionData(u32 appIndex, bool doSplitting) return false; } - // Remove " (BASE)" - dumpName[strlen(dumpName) - 7] = '\0'; + if (!usePatch) + { + // Remove " (BASE)" + dumpName[strlen(dumpName) - 7] = '\0'; + } else { + // Remove " (UPD)" + dumpName[strlen(dumpName) - 6] = '\0'; + } // Retrieve ExeFS from Program NCA - if (!readProgramNcaExeFsOrRomFs(appIndex, false)) + if (!readProgramNcaExeFsOrRomFs(titleIndex, usePatch, false)) { free(dumpName); breaks += 2; @@ -2773,7 +3467,6 @@ bool dumpExeFsSectionData(u32 appIndex, bool doSplitting) // Calculate total dump size if (!calculateExeFsExtractedDataSize(&(progressCtx.totalSize))) goto out; - breaks++; convertSize(progressCtx.totalSize, progressCtx.totalSizeStr, sizeof(progressCtx.totalSizeStr) / sizeof(progressCtx.totalSizeStr[0])); snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Extracted ExeFS dump size: %s (%lu bytes).", progressCtx.totalSizeStr, progressCtx.totalSize); uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); @@ -2861,7 +3554,7 @@ bool dumpExeFsSectionData(u32 appIndex, bool doSplitting) if (DUMP_BUFFER_SIZE > (exeFsContext.exefs_entries[i].file_size - offset)) n = (exeFsContext.exefs_entries[i].file_size - offset); breaks = (progressCtx.line_offset + 2); - proceed = processNcaCtrSectionBlock(&(exeFsContext.ncmStorage), &(exeFsContext.ncaId), exeFsContext.exefs_data_offset + exeFsContext.exefs_entries[i].file_offset + offset, buf, n, &(exeFsContext.aes_ctx), false); + proceed = processNcaCtrSectionBlock(&(exeFsContext.ncmStorage), &(exeFsContext.ncaId), &(exeFsContext.aes_ctx), exeFsContext.exefs_data_offset + exeFsContext.exefs_entries[i].file_offset + offset, buf, n, false); breaks = (progressCtx.line_offset - 4); if (!proceed) break; @@ -3009,9 +3702,9 @@ out: return success; } -bool dumpFileFromExeFsSection(u32 appIndex, u32 fileIndex, bool doSplitting) +bool dumpFileFromExeFsSection(u32 titleIndex, u32 fileIndex, bool usePatch, bool doSplitting) { - if (!exeFsContext.exefs_header.file_cnt || fileIndex > (exeFsContext.exefs_header.file_cnt - 1) || !exeFsContext.exefs_entries || !exeFsContext.exefs_str_table || exeFsContext.exefs_data_offset <= exeFsContext.exefs_offset || appIndex > (titleAppCount - 1)) + if (!exeFsContext.exefs_header.file_cnt || fileIndex > (exeFsContext.exefs_header.file_cnt - 1) || !exeFsContext.exefs_entries || !exeFsContext.exefs_str_table || exeFsContext.exefs_data_offset <= exeFsContext.exefs_offset || (!usePatch && titleIndex > (titleAppCount - 1)) || (usePatch && titleIndex > (titlePatchCount - 1))) { uiDrawString("Error: invalid parameters to parse file entry from ExeFS section!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); breaks += 2; @@ -3042,7 +3735,7 @@ bool dumpFileFromExeFsSection(u32 appIndex, u32 fileIndex, bool doSplitting) return false; } - char *dumpName = generateNSPDumpName(DUMP_APP_NSP, appIndex); + char *dumpName = generateNSPDumpName((!usePatch ? DUMP_APP_NSP : DUMP_PATCH_NSP), titleIndex); if (!dumpName) { uiDrawString("Error: unable to generate output dump name!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); @@ -3050,8 +3743,14 @@ bool dumpFileFromExeFsSection(u32 appIndex, u32 fileIndex, bool doSplitting) return false; } - // Remove " (BASE)" - dumpName[strlen(dumpName) - 7] = '\0'; + if (!usePatch) + { + // Remove " (BASE)" + dumpName[strlen(dumpName) - 7] = '\0'; + } else { + // Remove " (UPD)" + dumpName[strlen(dumpName) - 6] = '\0'; + } // Generate output path snprintf(dumpPath, sizeof(dumpPath) / sizeof(dumpPath[0]), "%s%s", EXEFS_DUMP_PATH, dumpName); @@ -3146,7 +3845,7 @@ bool dumpFileFromExeFsSection(u32 appIndex, u32 fileIndex, bool doSplitting) if (DUMP_BUFFER_SIZE > (progressCtx.totalSize - progressCtx.curOffset)) n = (progressCtx.totalSize - progressCtx.curOffset); breaks = (progressCtx.line_offset + 2); - proceed = processNcaCtrSectionBlock(&(exeFsContext.ncmStorage), &(exeFsContext.ncaId), exeFsContext.exefs_data_offset + exeFsContext.exefs_entries[fileIndex].file_offset + progressCtx.curOffset, buf, n, &(exeFsContext.aes_ctx), false); + proceed = processNcaCtrSectionBlock(&(exeFsContext.ncmStorage), &(exeFsContext.ncaId), &(exeFsContext.aes_ctx), exeFsContext.exefs_data_offset + exeFsContext.exefs_entries[fileIndex].file_offset + progressCtx.curOffset, buf, n, false); breaks = (progressCtx.line_offset - 2); if (!proceed) break; @@ -3294,9 +3993,9 @@ out: return success; } -bool recursiveDumpRomFsFile(u32 file_offset, char *romfs_path, char *output_path, progress_ctx_t *progressCtx, bool doSplitting) +bool recursiveDumpRomFsFile(u32 file_offset, char *romfs_path, char *output_path, progress_ctx_t *progressCtx, bool usePatch, bool doSplitting) { - if (!romFsContext.romfs_filetable_size || file_offset > romFsContext.romfs_filetable_size || !romFsContext.romfs_file_entries || !romfs_path || !output_path || !progressCtx) + if ((!usePatch && (!romFsContext.romfs_filetable_size || file_offset > romFsContext.romfs_filetable_size || !romFsContext.romfs_file_entries)) || (usePatch && (!bktrContext.romfs_filetable_size || file_offset > bktrContext.romfs_filetable_size || !bktrContext.romfs_file_entries)) || !romfs_path || !output_path || !progressCtx) { uiDrawString("Error: invalid parameters to parse file entry from RomFS section!", 8, ((progressCtx->line_offset + 2) * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); return false; @@ -3317,7 +4016,7 @@ bool recursiveDumpRomFsFile(u32 file_offset, char *romfs_path, char *output_path char tmp_idx[5]; - romfs_file *entry = (romfs_file*)((u8*)romFsContext.romfs_file_entries + file_offset); + romfs_file *entry = (!usePatch ? (romfs_file*)((u8*)romFsContext.romfs_file_entries + file_offset) : (romfs_file*)((u8*)bktrContext.romfs_file_entries + file_offset)); // Check if we're dealing with a nameless file if (!entry->nameLen) @@ -3378,7 +4077,14 @@ bool recursiveDumpRomFsFile(u32 file_offset, char *romfs_path, char *output_path if (DUMP_BUFFER_SIZE > (entry->dataSize - off)) n = (entry->dataSize - off); breaks = (progressCtx->line_offset + 2); - proceed = processNcaCtrSectionBlock(&(romFsContext.ncmStorage), &(romFsContext.ncaId), romFsContext.romfs_filedata_offset + entry->dataOff + off, buf, n, &(romFsContext.aes_ctx), false); + + if (!usePatch) + { + proceed = processNcaCtrSectionBlock(&(romFsContext.ncmStorage), &(romFsContext.ncaId), &(romFsContext.aes_ctx), romFsContext.romfs_filedata_offset + entry->dataOff + off, buf, n, false); + } else { + proceed = readBktrSectionBlock(bktrContext.romfs_filedata_offset + entry->dataOff + off, buf, n); + } + breaks = (progressCtx->line_offset - 4); if (!proceed) break; @@ -3482,22 +4188,26 @@ out: if (outFile) fclose(outFile); - if (!success && fat32_error) breaks += 2; + if (!success) + { + breaks = (progressCtx->line_offset + 2); + if (fat32_error) breaks += 2; + } romfs_path[orig_romfs_path_len] = '\0'; output_path[orig_output_path_len] = '\0'; if (success) { - if (entry->sibling != ROMFS_ENTRY_EMPTY) success = recursiveDumpRomFsFile(entry->sibling, romfs_path, output_path, progressCtx, true); + if (entry->sibling != ROMFS_ENTRY_EMPTY) success = recursiveDumpRomFsFile(entry->sibling, romfs_path, output_path, progressCtx, usePatch, doSplitting); } return success; } -bool recursiveDumpRomFsDir(u32 dir_offset, char *romfs_path, char *output_path, progress_ctx_t *progressCtx) +bool recursiveDumpRomFsDir(u32 dir_offset, char *romfs_path, char *output_path, progress_ctx_t *progressCtx, bool usePatch, bool dumpSiblingDir, bool doSplitting) { - if (!romFsContext.romfs_dirtable_size || dir_offset > romFsContext.romfs_dirtable_size || !romFsContext.romfs_dir_entries || !romFsContext.romfs_filetable_size || !romFsContext.romfs_file_entries || !romfs_path || !output_path || !progressCtx) + if ((!usePatch && (!romFsContext.romfs_dirtable_size || dir_offset > romFsContext.romfs_dirtable_size || !romFsContext.romfs_dir_entries || !romFsContext.romfs_filetable_size || !romFsContext.romfs_file_entries)) || (usePatch && (!bktrContext.romfs_dirtable_size || dir_offset > bktrContext.romfs_dirtable_size || !bktrContext.romfs_dir_entries || !bktrContext.romfs_filetable_size || !bktrContext.romfs_file_entries)) || !romfs_path || !output_path || !progressCtx) { uiDrawString("Error: invalid parameters to parse directory entry from RomFS section!", 8, ((progressCtx->line_offset + 2) * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); return false; @@ -3506,7 +4216,7 @@ bool recursiveDumpRomFsDir(u32 dir_offset, char *romfs_path, char *output_path, size_t orig_romfs_path_len = strlen(romfs_path); size_t orig_output_path_len = strlen(output_path); - romfs_dir *entry = (romfs_dir*)((u8*)romFsContext.romfs_dir_entries + dir_offset); + romfs_dir *entry = (!usePatch ? (romfs_dir*)((u8*)romFsContext.romfs_dir_entries + dir_offset) : (romfs_dir*)((u8*)bktrContext.romfs_dir_entries + dir_offset)); // Check if we're dealing with a nameless directory that's not the root directory if (!entry->nameLen && dir_offset > 0) @@ -3535,7 +4245,7 @@ bool recursiveDumpRomFsDir(u32 dir_offset, char *romfs_path, char *output_path, if (entry->childFile != ROMFS_ENTRY_EMPTY) { - if (!recursiveDumpRomFsFile(entry->childFile, romfs_path, output_path, progressCtx, true)) + if (!recursiveDumpRomFsFile(entry->childFile, romfs_path, output_path, progressCtx, usePatch, doSplitting)) { romfs_path[orig_romfs_path_len] = '\0'; output_path[orig_output_path_len] = '\0'; @@ -3545,7 +4255,7 @@ bool recursiveDumpRomFsDir(u32 dir_offset, char *romfs_path, char *output_path, if (entry->childDir != ROMFS_ENTRY_EMPTY) { - if (!recursiveDumpRomFsDir(entry->childDir, romfs_path, output_path, progressCtx)) + if (!recursiveDumpRomFsDir(entry->childDir, romfs_path, output_path, progressCtx, usePatch, true, doSplitting)) { romfs_path[orig_romfs_path_len] = '\0'; output_path[orig_output_path_len] = '\0'; @@ -3556,15 +4266,15 @@ bool recursiveDumpRomFsDir(u32 dir_offset, char *romfs_path, char *output_path, romfs_path[orig_romfs_path_len] = '\0'; output_path[orig_output_path_len] = '\0'; - if (entry->sibling != ROMFS_ENTRY_EMPTY) + if (dumpSiblingDir && entry->sibling != ROMFS_ENTRY_EMPTY) { - if (!recursiveDumpRomFsDir(entry->sibling, romfs_path, output_path, progressCtx)) return false; + if (!recursiveDumpRomFsDir(entry->sibling, romfs_path, output_path, progressCtx, usePatch, true, doSplitting)) return false; } return true; } -bool dumpRomFsSectionData(u32 appIndex) +bool dumpRomFsSectionData(u32 titleIndex, bool usePatch, bool doSplitting) { progress_ctx_t progressCtx; memset(&progressCtx, 0, sizeof(progress_ctx_t)); @@ -3573,21 +4283,21 @@ bool dumpRomFsSectionData(u32 appIndex) bool success = false; - if (!titleAppCount) + if ((!usePatch && !titleAppCount) || (usePatch && !titlePatchCount)) { - uiDrawString("Error: invalid application count!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + uiDrawString("Error: invalid title count!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); breaks += 2; return false; } - if (appIndex > (titleAppCount - 1)) + if ((!usePatch && titleIndex > (titleAppCount - 1)) || (usePatch && titleIndex > (titlePatchCount - 1))) { - uiDrawString("Error: invalid application index!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + uiDrawString("Error: invalid title index!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); breaks += 2; return false; } - char *dumpName = generateNSPDumpName(DUMP_APP_NSP, appIndex); + char *dumpName = generateNSPDumpName((!usePatch ? DUMP_APP_NSP : DUMP_PATCH_NSP), titleIndex); if (!dumpName) { uiDrawString("Error: unable to generate output dump name!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); @@ -3595,11 +4305,17 @@ bool dumpRomFsSectionData(u32 appIndex) return false; } - // Remove " (BASE)" - dumpName[strlen(dumpName) - 7] = '\0'; + if (!usePatch) + { + // Remove " (BASE)" + dumpName[strlen(dumpName) - 7] = '\0'; + } else { + // Remove " (UPD)" + dumpName[strlen(dumpName) - 6] = '\0'; + } // Retrieve RomFS from Program NCA - if (!readProgramNcaExeFsOrRomFs(appIndex, true)) + if (!readProgramNcaExeFsOrRomFs(titleIndex, usePatch, true)) { free(dumpName); breaks += 2; @@ -3607,9 +4323,8 @@ bool dumpRomFsSectionData(u32 appIndex) } // Calculate total dump size - if (!calculateRomFsExtractedDataSize(&(progressCtx.totalSize))) goto out; + if (!calculateRomFsFullExtractedSize(usePatch, &(progressCtx.totalSize))) goto out; - breaks++; convertSize(progressCtx.totalSize, progressCtx.totalSizeStr, sizeof(progressCtx.totalSizeStr) / sizeof(progressCtx.totalSizeStr[0])); snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Extracted RomFS dump size: %s (%lu bytes).", progressCtx.totalSizeStr, progressCtx.totalSize); uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); @@ -3642,12 +4357,12 @@ bool dumpRomFsSectionData(u32 appIndex) progressCtx.line_offset = (breaks + 4); timeGetCurrentTime(TimeType_LocalSystemClock, &(progressCtx.start)); - success = recursiveDumpRomFsDir(0, romFsPath, dumpPath, &progressCtx); - - breaks = (progressCtx.line_offset + 2); + success = recursiveDumpRomFsDir(0, romFsPath, dumpPath, &progressCtx, usePatch, true, doSplitting); if (success) { + breaks = (progressCtx.line_offset + 2); + timeGetCurrentTime(TimeType_LocalSystemClock, &(progressCtx.now)); progressCtx.now -= progressCtx.start; @@ -3660,6 +4375,8 @@ bool dumpRomFsSectionData(u32 appIndex) } out: + if (usePatch) freeBktrContext(); + freeRomFsContext(); free(dumpName); @@ -3669,9 +4386,9 @@ out: return success; } -bool dumpFileFromRomFsSection(u32 appIndex, u32 file_offset, bool doSplitting) +bool dumpFileFromRomFsSection(u32 titleIndex, u32 file_offset, bool usePatch, bool doSplitting) { - if (!romFsContext.romfs_filetable_size || file_offset > romFsContext.romfs_filetable_size || !romFsContext.romfs_file_entries || appIndex > (titleAppCount - 1)) + if (!romFsContext.romfs_filetable_size || file_offset > romFsContext.romfs_filetable_size || !romFsContext.romfs_file_entries || (!usePatch && titleIndex > (titleAppCount - 1)) || (usePatch && titleIndex > (titlePatchCount - 1))) { uiDrawString("Error: invalid parameters to parse file entry from RomFS section!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); breaks += 2; @@ -3692,7 +4409,7 @@ bool dumpFileFromRomFsSection(u32 appIndex, u32 file_offset, bool doSplitting) size_t write_res; - romfs_file *entry = (romfs_file*)((u8*)romFsContext.romfs_file_entries + file_offset); + romfs_file *entry = (!usePatch ? (romfs_file*)((u8*)romFsContext.romfs_file_entries + file_offset) : (romfs_file*)((u8*)bktrContext.romfs_file_entries + file_offset)); // Check if we're dealing with a nameless file if (!entry->nameLen) @@ -3702,7 +4419,7 @@ bool dumpFileFromRomFsSection(u32 appIndex, u32 file_offset, bool doSplitting) return false; } - char *dumpName = generateNSPDumpName(DUMP_APP_NSP, appIndex); + char *dumpName = generateNSPDumpName((!usePatch ? DUMP_APP_NSP : DUMP_PATCH_NSP), titleIndex); if (!dumpName) { uiDrawString("Error: unable to generate output dump name!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); @@ -3710,8 +4427,14 @@ bool dumpFileFromRomFsSection(u32 appIndex, u32 file_offset, bool doSplitting) return false; } - // Remove " (BASE)" - dumpName[strlen(dumpName) - 7] = '\0'; + if (!usePatch) + { + // Remove " (BASE)" + dumpName[strlen(dumpName) - 7] = '\0'; + } else { + // Remove " (UPD)" + dumpName[strlen(dumpName) - 6] = '\0'; + } // Generate output path snprintf(dumpPath, sizeof(dumpPath) / sizeof(dumpPath[0]), "%s%s", ROMFS_DUMP_PATH, dumpName); @@ -3840,7 +4563,14 @@ bool dumpFileFromRomFsSection(u32 appIndex, u32 file_offset, bool doSplitting) if (DUMP_BUFFER_SIZE > (progressCtx.totalSize - progressCtx.curOffset)) n = (progressCtx.totalSize - progressCtx.curOffset); breaks = (progressCtx.line_offset + 2); - proceed = processNcaCtrSectionBlock(&(romFsContext.ncmStorage), &(romFsContext.ncaId), romFsContext.romfs_filedata_offset + entry->dataOff + progressCtx.curOffset, buf, n, &(romFsContext.aes_ctx), false); + + if (!usePatch) + { + proceed = processNcaCtrSectionBlock(&(romFsContext.ncmStorage), &(romFsContext.ncaId), &(romFsContext.aes_ctx), romFsContext.romfs_filedata_offset + entry->dataOff + progressCtx.curOffset, buf, n, false); + } else { + proceed = readBktrSectionBlock(bktrContext.romfs_filedata_offset + entry->dataOff + progressCtx.curOffset, buf, n); + } + breaks = (progressCtx.line_offset - 2); if (!proceed) break; @@ -3988,6 +4718,141 @@ out: return success; } +bool dumpCurrentDirFromRomFsSection(u32 titleIndex, bool usePatch, bool doSplitting) +{ + progress_ctx_t progressCtx; + memset(&progressCtx, 0, sizeof(progress_ctx_t)); + + char romFsPath[NAME_BUF_LEN * 2] = {'\0'}, dumpPath[NAME_BUF_LEN * 2] = {'\0'}; + + bool success = false; + + if ((!usePatch && !titleAppCount) || (usePatch && !titlePatchCount)) + { + uiDrawString("Error: invalid title count!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + breaks += 2; + return false; + } + + if ((!usePatch && titleIndex > (titleAppCount - 1)) || (usePatch && titleIndex > (titlePatchCount - 1))) + { + uiDrawString("Error: invalid title index!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + breaks += 2; + return false; + } + + char *dumpName = generateNSPDumpName((!usePatch ? DUMP_APP_NSP : DUMP_PATCH_NSP), titleIndex); + if (!dumpName) + { + uiDrawString("Error: unable to generate output dump name!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + breaks += 2; + return false; + } + + if (!usePatch) + { + // Remove " (BASE)" + dumpName[strlen(dumpName) - 7] = '\0'; + } else { + // Remove " (UPD)" + dumpName[strlen(dumpName) - 6] = '\0'; + } + + // Calculate total dump size + if (!calculateRomFsExtractedDirSize(curRomFsDirOffset, usePatch, &(progressCtx.totalSize))) goto out; + + convertSize(progressCtx.totalSize, progressCtx.totalSizeStr, sizeof(progressCtx.totalSizeStr) / sizeof(progressCtx.totalSizeStr[0])); + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Extracted RomFS directory size: %s (%lu bytes).", progressCtx.totalSizeStr, progressCtx.totalSize); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); + uiRefreshDisplay(); + breaks++; + + if (progressCtx.totalSize > freeSpace) + { + uiDrawString("Error: not enough free space available in the SD card.", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + goto out; + } + + if (strlen(curRomFsPath) > 1) snprintf(romFsPath, sizeof(romFsPath) / sizeof(romFsPath[0]), curRomFsPath); + + // Prepare output dump path + snprintf(dumpPath, sizeof(dumpPath) / sizeof(dumpPath[0]), "%s%s", ROMFS_DUMP_PATH, dumpName); + mkdir(dumpPath, 0744); + + // Create subdirectories + char *tmp1 = NULL; + char *tmp2 = NULL; + size_t cur_len; + + tmp1 = strchr(curRomFsPath, '/'); + + while(tmp1 != NULL) + { + tmp1++; + + if (!strlen(tmp1)) break; + + tmp2 = strchr(tmp1, '/'); + if (tmp2 != NULL) + { + strcat(dumpPath, "/"); + + cur_len = strlen(dumpPath); + + strncat(dumpPath, tmp1, tmp2 - tmp1); + + removeIllegalCharacters(dumpPath + cur_len); + + mkdir(dumpPath, 0744); + + tmp1 = tmp2; + } else { + // Skip last entry + tmp1 = NULL; + } + } + + // Start dump process + breaks++; + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Dump procedure started. Hold %s to cancel.", NINTENDO_FONT_B); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); + uiRefreshDisplay(); + breaks += 2; + + if (programAppletType != AppletType_Application && programAppletType != AppletType_SystemApplication) + { + uiDrawString("Do not press the " NINTENDO_FONT_HOME " button. Doing so could corrupt the SD card filesystem.", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + breaks += 2; + } + + progressCtx.line_offset = (breaks + 4); + timeGetCurrentTime(TimeType_LocalSystemClock, &(progressCtx.start)); + + success = recursiveDumpRomFsDir(curRomFsDirOffset, romFsPath, dumpPath, &progressCtx, usePatch, false, doSplitting); + + if (success) + { + breaks = (progressCtx.line_offset + 2); + + timeGetCurrentTime(TimeType_LocalSystemClock, &(progressCtx.now)); + progressCtx.now -= progressCtx.start; + + formatETAString(progressCtx.now, progressCtx.etaInfo, sizeof(progressCtx.etaInfo) / sizeof(progressCtx.etaInfo[0])); + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Process successfully completed after %s!", progressCtx.etaInfo); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 0, 255, 0); + } else { + setProgressBarError(&progressCtx); + removeDirectory(dumpPath); + } + +out: + free(dumpName); + + breaks += 2; + + return success; +} + bool dumpGameCardCertificate() { u32 crc = 0; @@ -4000,7 +4865,7 @@ bool dumpGameCardCertificate() u8 buf[CERT_SIZE]; size_t write_res; - char *dumpName = generateDumpFullName(); + char *dumpName = generateFullDumpName(); if (!dumpName) { uiDrawString("Error: unable to generate output dump name!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); diff --git a/source/dumper.h b/source/dumper.h index c2bda19..aec3492 100644 --- a/source/dumper.h +++ b/source/dumper.h @@ -20,16 +20,24 @@ #define SMOOTHING_FACTOR (double)0.05 +typedef enum { + BATCH_SOURCE_ALL = 0, + BATCH_SOURCE_SDCARD, + BATCH_SOURCE_EMMC +} batchModeSourceStorage; + void workaroundPartitionZeroAccess(); bool dumpCartridgeImage(bool isFat32, bool setXciArchiveBit, bool dumpCert, bool trimDump, bool calcCrc); -bool dumpNintendoSubmissionPackage(nspDumpType selectedNspDumpType, u32 titleIndex, bool isFat32, bool calcCrc, bool removeConsoleData, bool tiklessDump); +bool dumpNintendoSubmissionPackage(nspDumpType selectedNspDumpType, u32 titleIndex, bool isFat32, bool calcCrc, bool removeConsoleData, bool tiklessDump, bool batch); +bool dumpNintendoSubmissionPackageBatch(bool dumpAppTitles, bool dumpPatchTitles, bool dumpAddOnTitles, bool isFat32, bool removeConsoleData, bool tiklessDump, bool skipDumpedTitles, batchModeSourceStorage batchModeSrc); bool dumpRawHfs0Partition(u32 partition, bool doSplitting); -bool dumpHfs0PartitionData(u32 partition); -bool dumpFileFromHfs0Partition(u32 partition, u32 file, char *filename); -bool dumpExeFsSectionData(u32 appIndex, bool doSplitting); -bool dumpFileFromExeFsSection(u32 appIndex, u32 fileIndex, bool doSplitting); -bool dumpRomFsSectionData(u32 appIndex); -bool dumpFileFromRomFsSection(u32 appIndex, u32 file_offset, bool doSplitting); +bool dumpHfs0PartitionData(u32 partition, bool doSplitting); +bool dumpFileFromHfs0Partition(u32 partition, u32 file, char *filename, bool doSplitting); +bool dumpExeFsSectionData(u32 titleIndex, bool usePatch, bool doSplitting); +bool dumpFileFromExeFsSection(u32 titleIndex, u32 fileIndex, bool usePatch, bool doSplitting); +bool dumpRomFsSectionData(u32 titleIndex, bool usePatch, bool doSplitting); +bool dumpFileFromRomFsSection(u32 titleIndex, u32 file_offset, bool usePatch, bool doSplitting); +bool dumpCurrentDirFromRomFsSection(u32 titleIndex, bool usePatch, bool doSplitting); bool dumpGameCardCertificate(); #endif diff --git a/source/lz4.c b/source/lz4.c new file mode 100644 index 0000000..e614c45 --- /dev/null +++ b/source/lz4.c @@ -0,0 +1,2299 @@ +/* + LZ4 - Fast LZ compression algorithm + Copyright (C) 2011-present, Yann Collet. + + BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php) + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions are + met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above + copyright notice, this list of conditions and the following disclaimer + in the documentation and/or other materials provided with the + distribution. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + You can contact the author at : + - LZ4 homepage : http://www.lz4.org + - LZ4 source repository : https://github.com/lz4/lz4 +*/ + +/*-************************************ +* Tuning parameters +**************************************/ +/* + * LZ4_HEAPMODE : + * Select how default compression functions will allocate memory for their hash table, + * in memory stack (0:default, fastest), or in memory heap (1:requires malloc()). + */ +#ifndef LZ4_HEAPMODE +# define LZ4_HEAPMODE 0 +#endif + +/* + * ACCELERATION_DEFAULT : + * Select "acceleration" for LZ4_compress_fast() when parameter value <= 0 + */ +#define ACCELERATION_DEFAULT 1 + + +/*-************************************ +* CPU Feature Detection +**************************************/ +/* LZ4_FORCE_MEMORY_ACCESS + * By default, access to unaligned memory is controlled by `memcpy()`, which is safe and portable. + * Unfortunately, on some target/compiler combinations, the generated assembly is sub-optimal. + * The below switch allow to select different access method for improved performance. + * Method 0 (default) : use `memcpy()`. Safe and portable. + * Method 1 : `__packed` statement. It depends on compiler extension (ie, not portable). + * This method is safe if your compiler supports it, and *generally* as fast or faster than `memcpy`. + * Method 2 : direct access. This method is portable but violate C standard. + * It can generate buggy code on targets which assembly generation depends on alignment. + * But in some circumstances, it's the only known way to get the most performance (ie GCC + ARMv6) + * See https://fastcompression.blogspot.fr/2015/08/accessing-unaligned-memory.html for details. + * Prefer these methods in priority order (0 > 1 > 2) + */ +#ifndef LZ4_FORCE_MEMORY_ACCESS /* can be defined externally */ +# if defined(__GNUC__) && \ + ( defined(__ARM_ARCH_6__) || defined(__ARM_ARCH_6J__) || defined(__ARM_ARCH_6K__) \ + || defined(__ARM_ARCH_6Z__) || defined(__ARM_ARCH_6ZK__) || defined(__ARM_ARCH_6T2__) ) +# define LZ4_FORCE_MEMORY_ACCESS 2 +# elif (defined(__INTEL_COMPILER) && !defined(_WIN32)) || defined(__GNUC__) +# define LZ4_FORCE_MEMORY_ACCESS 1 +# endif +#endif + +/* + * LZ4_FORCE_SW_BITCOUNT + * Define this parameter if your target system or compiler does not support hardware bit count + */ +#if defined(_MSC_VER) && defined(_WIN32_WCE) /* Visual Studio for WinCE doesn't support Hardware bit count */ +# define LZ4_FORCE_SW_BITCOUNT +#endif + + + +/*-************************************ +* Dependency +**************************************/ +/* + * LZ4_SRC_INCLUDED: + * Amalgamation flag, whether lz4.c is included + */ +#ifndef LZ4_SRC_INCLUDED +# define LZ4_SRC_INCLUDED 1 +#endif + +#ifndef LZ4_STATIC_LINKING_ONLY +#define LZ4_STATIC_LINKING_ONLY +#endif + +#ifndef LZ4_DISABLE_DEPRECATE_WARNINGS +#define LZ4_DISABLE_DEPRECATE_WARNINGS /* due to LZ4_decompress_safe_withPrefix64k */ +#endif + +#include "lz4.h" +/* see also "memory routines" below */ + + +/*-************************************ +* Compiler Options +**************************************/ +#ifdef _MSC_VER /* Visual Studio */ +# include +# pragma warning(disable : 4127) /* disable: C4127: conditional expression is constant */ +# pragma warning(disable : 4293) /* disable: C4293: too large shift (32-bits) */ +#endif /* _MSC_VER */ + +#ifndef LZ4_FORCE_INLINE +# ifdef _MSC_VER /* Visual Studio */ +# define LZ4_FORCE_INLINE static __forceinline +# else +# if defined (__cplusplus) || defined (__STDC_VERSION__) && __STDC_VERSION__ >= 199901L /* C99 */ +# ifdef __GNUC__ +# define LZ4_FORCE_INLINE static inline __attribute__((always_inline)) +# else +# define LZ4_FORCE_INLINE static inline +# endif +# else +# define LZ4_FORCE_INLINE static +# endif /* __STDC_VERSION__ */ +# endif /* _MSC_VER */ +#endif /* LZ4_FORCE_INLINE */ + +/* LZ4_FORCE_O2_GCC_PPC64LE and LZ4_FORCE_O2_INLINE_GCC_PPC64LE + * gcc on ppc64le generates an unrolled SIMDized loop for LZ4_wildCopy8, + * together with a simple 8-byte copy loop as a fall-back path. + * However, this optimization hurts the decompression speed by >30%, + * because the execution does not go to the optimized loop + * for typical compressible data, and all of the preamble checks + * before going to the fall-back path become useless overhead. + * This optimization happens only with the -O3 flag, and -O2 generates + * a simple 8-byte copy loop. + * With gcc on ppc64le, all of the LZ4_decompress_* and LZ4_wildCopy8 + * functions are annotated with __attribute__((optimize("O2"))), + * and also LZ4_wildCopy8 is forcibly inlined, so that the O2 attribute + * of LZ4_wildCopy8 does not affect the compression speed. + */ +#if defined(__PPC64__) && defined(__LITTLE_ENDIAN__) && defined(__GNUC__) && !defined(__clang__) +# define LZ4_FORCE_O2_GCC_PPC64LE __attribute__((optimize("O2"))) +# define LZ4_FORCE_O2_INLINE_GCC_PPC64LE __attribute__((optimize("O2"))) LZ4_FORCE_INLINE +#else +# define LZ4_FORCE_O2_GCC_PPC64LE +# define LZ4_FORCE_O2_INLINE_GCC_PPC64LE static +#endif + +#if (defined(__GNUC__) && (__GNUC__ >= 3)) || (defined(__INTEL_COMPILER) && (__INTEL_COMPILER >= 800)) || defined(__clang__) +# define expect(expr,value) (__builtin_expect ((expr),(value)) ) +#else +# define expect(expr,value) (expr) +#endif + +#ifndef likely +#define likely(expr) expect((expr) != 0, 1) +#endif +#ifndef unlikely +#define unlikely(expr) expect((expr) != 0, 0) +#endif + + +/*-************************************ +* Memory routines +**************************************/ +#include /* malloc, calloc, free */ +#define ALLOC(s) malloc(s) +#define ALLOC_AND_ZERO(s) calloc(1,s) +#define FREEMEM(p) free(p) +#include /* memset, memcpy */ +#define MEM_INIT(p,v,s) memset((p),(v),(s)) + + +/*-************************************ +* Types +**************************************/ +#if defined(__cplusplus) || (defined (__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */) +# include + typedef uint8_t BYTE; + typedef uint16_t U16; + typedef uint32_t U32; + typedef int32_t S32; + typedef uint64_t U64; + typedef uintptr_t uptrval; +#else + typedef unsigned char BYTE; + typedef unsigned short U16; + typedef unsigned int U32; + typedef signed int S32; + typedef unsigned long long U64; + typedef size_t uptrval; /* generally true, except OpenVMS-64 */ +#endif + +#if defined(__x86_64__) + typedef U64 reg_t; /* 64-bits in x32 mode */ +#else + typedef size_t reg_t; /* 32-bits in x32 mode */ +#endif + +typedef enum { + notLimited = 0, + limitedOutput = 1, + fillOutput = 2 +} limitedOutput_directive; + + +/*-************************************ +* Reading and writing into memory +**************************************/ +static unsigned LZ4_isLittleEndian(void) +{ + const union { U32 u; BYTE c[4]; } one = { 1 }; /* don't use static : performance detrimental */ + return one.c[0]; +} + + +#if defined(LZ4_FORCE_MEMORY_ACCESS) && (LZ4_FORCE_MEMORY_ACCESS==2) +/* lie to the compiler about data alignment; use with caution */ + +static U16 LZ4_read16(const void* memPtr) { return *(const U16*) memPtr; } +static U32 LZ4_read32(const void* memPtr) { return *(const U32*) memPtr; } +static reg_t LZ4_read_ARCH(const void* memPtr) { return *(const reg_t*) memPtr; } + +static void LZ4_write16(void* memPtr, U16 value) { *(U16*)memPtr = value; } +static void LZ4_write32(void* memPtr, U32 value) { *(U32*)memPtr = value; } + +#elif defined(LZ4_FORCE_MEMORY_ACCESS) && (LZ4_FORCE_MEMORY_ACCESS==1) + +/* __pack instructions are safer, but compiler specific, hence potentially problematic for some compilers */ +/* currently only defined for gcc and icc */ +typedef union { U16 u16; U32 u32; reg_t uArch; } __attribute__((packed)) unalign; + +static U16 LZ4_read16(const void* ptr) { return ((const unalign*)ptr)->u16; } +static U32 LZ4_read32(const void* ptr) { return ((const unalign*)ptr)->u32; } +static reg_t LZ4_read_ARCH(const void* ptr) { return ((const unalign*)ptr)->uArch; } + +static void LZ4_write16(void* memPtr, U16 value) { ((unalign*)memPtr)->u16 = value; } +static void LZ4_write32(void* memPtr, U32 value) { ((unalign*)memPtr)->u32 = value; } + +#else /* safe and portable access using memcpy() */ + +static U16 LZ4_read16(const void* memPtr) +{ + U16 val; memcpy(&val, memPtr, sizeof(val)); return val; +} + +static U32 LZ4_read32(const void* memPtr) +{ + U32 val; memcpy(&val, memPtr, sizeof(val)); return val; +} + +static reg_t LZ4_read_ARCH(const void* memPtr) +{ + reg_t val; memcpy(&val, memPtr, sizeof(val)); return val; +} + +static void LZ4_write16(void* memPtr, U16 value) +{ + memcpy(memPtr, &value, sizeof(value)); +} + +static void LZ4_write32(void* memPtr, U32 value) +{ + memcpy(memPtr, &value, sizeof(value)); +} + +#endif /* LZ4_FORCE_MEMORY_ACCESS */ + + +static U16 LZ4_readLE16(const void* memPtr) +{ + if (LZ4_isLittleEndian()) { + return LZ4_read16(memPtr); + } else { + const BYTE* p = (const BYTE*)memPtr; + return (U16)((U16)p[0] + (p[1]<<8)); + } +} + +static void LZ4_writeLE16(void* memPtr, U16 value) +{ + if (LZ4_isLittleEndian()) { + LZ4_write16(memPtr, value); + } else { + BYTE* p = (BYTE*)memPtr; + p[0] = (BYTE) value; + p[1] = (BYTE)(value>>8); + } +} + +/* customized variant of memcpy, which can overwrite up to 8 bytes beyond dstEnd */ +LZ4_FORCE_O2_INLINE_GCC_PPC64LE +void LZ4_wildCopy8(void* dstPtr, const void* srcPtr, void* dstEnd) +{ + BYTE* d = (BYTE*)dstPtr; + const BYTE* s = (const BYTE*)srcPtr; + BYTE* const e = (BYTE*)dstEnd; + + do { memcpy(d,s,8); d+=8; s+=8; } while (d= 16. */ +LZ4_FORCE_O2_INLINE_GCC_PPC64LE void +LZ4_wildCopy32(void* dstPtr, const void* srcPtr, void* dstEnd) +{ + BYTE* d = (BYTE*)dstPtr; + const BYTE* s = (const BYTE*)srcPtr; + BYTE* const e = (BYTE*)dstEnd; + + do { memcpy(d,s,16); memcpy(d+16,s+16,16); d+=32; s+=32; } while (d 65535) /* max supported by LZ4 format */ +# error "LZ4_DISTANCE_MAX is too big : must be <= 65535" +#endif + +#define ML_BITS 4 +#define ML_MASK ((1U<=1) +# include +#else +# ifndef assert +# define assert(condition) ((void)0) +# endif +#endif + +#define LZ4_STATIC_ASSERT(c) { enum { LZ4_static_assert = 1/(int)(!!(c)) }; } /* use after variable declarations */ + +#if defined(LZ4_DEBUG) && (LZ4_DEBUG>=2) +# include +static int g_debuglog_enable = 1; +# define DEBUGLOG(l, ...) { \ + if ((g_debuglog_enable) && (l<=LZ4_DEBUG)) { \ + fprintf(stderr, __FILE__ ": "); \ + fprintf(stderr, __VA_ARGS__); \ + fprintf(stderr, " \n"); \ + } } +#else +# define DEBUGLOG(l, ...) {} /* disabled */ +#endif + + +/*-************************************ +* Common functions +**************************************/ +static unsigned LZ4_NbCommonBytes (reg_t val) +{ + if (LZ4_isLittleEndian()) { + if (sizeof(val)==8) { +# if defined(_MSC_VER) && defined(_WIN64) && !defined(LZ4_FORCE_SW_BITCOUNT) + unsigned long r = 0; + _BitScanForward64( &r, (U64)val ); + return (int)(r>>3); +# elif (defined(__clang__) || (defined(__GNUC__) && (__GNUC__>=3))) && !defined(LZ4_FORCE_SW_BITCOUNT) + return (__builtin_ctzll((U64)val) >> 3); +# else + static const int DeBruijnBytePos[64] = { 0, 0, 0, 0, 0, 1, 1, 2, + 0, 3, 1, 3, 1, 4, 2, 7, + 0, 2, 3, 6, 1, 5, 3, 5, + 1, 3, 4, 4, 2, 5, 6, 7, + 7, 0, 1, 2, 3, 3, 4, 6, + 2, 6, 5, 5, 3, 4, 5, 6, + 7, 1, 2, 4, 6, 4, 4, 5, + 7, 2, 6, 5, 7, 6, 7, 7 }; + return DeBruijnBytePos[((U64)((val & -(long long)val) * 0x0218A392CDABBD3FULL)) >> 58]; +# endif + } else /* 32 bits */ { +# if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT) + unsigned long r; + _BitScanForward( &r, (U32)val ); + return (int)(r>>3); +# elif (defined(__clang__) || (defined(__GNUC__) && (__GNUC__>=3))) && !defined(LZ4_FORCE_SW_BITCOUNT) + return (__builtin_ctz((U32)val) >> 3); +# else + static const int DeBruijnBytePos[32] = { 0, 0, 3, 0, 3, 1, 3, 0, + 3, 2, 2, 1, 3, 2, 0, 1, + 3, 3, 1, 2, 2, 2, 2, 0, + 3, 1, 2, 0, 1, 0, 1, 1 }; + return DeBruijnBytePos[((U32)((val & -(S32)val) * 0x077CB531U)) >> 27]; +# endif + } + } else /* Big Endian CPU */ { + if (sizeof(val)==8) { /* 64-bits */ +# if defined(_MSC_VER) && defined(_WIN64) && !defined(LZ4_FORCE_SW_BITCOUNT) + unsigned long r = 0; + _BitScanReverse64( &r, val ); + return (unsigned)(r>>3); +# elif (defined(__clang__) || (defined(__GNUC__) && (__GNUC__>=3))) && !defined(LZ4_FORCE_SW_BITCOUNT) + return (__builtin_clzll((U64)val) >> 3); +# else + static const U32 by32 = sizeof(val)*4; /* 32 on 64 bits (goal), 16 on 32 bits. + Just to avoid some static analyzer complaining about shift by 32 on 32-bits target. + Note that this code path is never triggered in 32-bits mode. */ + unsigned r; + if (!(val>>by32)) { r=4; } else { r=0; val>>=by32; } + if (!(val>>16)) { r+=2; val>>=8; } else { val>>=24; } + r += (!val); + return r; +# endif + } else /* 32 bits */ { +# if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT) + unsigned long r = 0; + _BitScanReverse( &r, (unsigned long)val ); + return (unsigned)(r>>3); +# elif (defined(__clang__) || (defined(__GNUC__) && (__GNUC__>=3))) && !defined(LZ4_FORCE_SW_BITCOUNT) + return (__builtin_clz((U32)val) >> 3); +# else + unsigned r; + if (!(val>>16)) { r=2; val>>=8; } else { r=0; val>>=24; } + r += (!val); + return r; +# endif + } + } +} + +#define STEPSIZE sizeof(reg_t) +LZ4_FORCE_INLINE +unsigned LZ4_count(const BYTE* pIn, const BYTE* pMatch, const BYTE* pInLimit) +{ + const BYTE* const pStart = pIn; + + if (likely(pIn < pInLimit-(STEPSIZE-1))) { + reg_t const diff = LZ4_read_ARCH(pMatch) ^ LZ4_read_ARCH(pIn); + if (!diff) { + pIn+=STEPSIZE; pMatch+=STEPSIZE; + } else { + return LZ4_NbCommonBytes(diff); + } } + + while (likely(pIn < pInLimit-(STEPSIZE-1))) { + reg_t const diff = LZ4_read_ARCH(pMatch) ^ LZ4_read_ARCH(pIn); + if (!diff) { pIn+=STEPSIZE; pMatch+=STEPSIZE; continue; } + pIn += LZ4_NbCommonBytes(diff); + return (unsigned)(pIn - pStart); + } + + if ((STEPSIZE==8) && (pIn<(pInLimit-3)) && (LZ4_read32(pMatch) == LZ4_read32(pIn))) { pIn+=4; pMatch+=4; } + if ((pIn<(pInLimit-1)) && (LZ4_read16(pMatch) == LZ4_read16(pIn))) { pIn+=2; pMatch+=2; } + if ((pIn compression run slower on incompressible data */ + + +/*-************************************ +* Local Structures and types +**************************************/ +typedef enum { clearedTable = 0, byPtr, byU32, byU16 } tableType_t; + +/** + * This enum distinguishes several different modes of accessing previous + * content in the stream. + * + * - noDict : There is no preceding content. + * - withPrefix64k : Table entries up to ctx->dictSize before the current blob + * blob being compressed are valid and refer to the preceding + * content (of length ctx->dictSize), which is available + * contiguously preceding in memory the content currently + * being compressed. + * - usingExtDict : Like withPrefix64k, but the preceding content is somewhere + * else in memory, starting at ctx->dictionary with length + * ctx->dictSize. + * - usingDictCtx : Like usingExtDict, but everything concerning the preceding + * content is in a separate context, pointed to by + * ctx->dictCtx. ctx->dictionary, ctx->dictSize, and table + * entries in the current context that refer to positions + * preceding the beginning of the current compression are + * ignored. Instead, ctx->dictCtx->dictionary and ctx->dictCtx + * ->dictSize describe the location and size of the preceding + * content, and matches are found by looking in the ctx + * ->dictCtx->hashTable. + */ +typedef enum { noDict = 0, withPrefix64k, usingExtDict, usingDictCtx } dict_directive; +typedef enum { noDictIssue = 0, dictSmall } dictIssue_directive; + + +/*-************************************ +* Local Utils +**************************************/ +int LZ4_versionNumber (void) { return LZ4_VERSION_NUMBER; } +const char* LZ4_versionString(void) { return LZ4_VERSION_STRING; } +int LZ4_compressBound(int isize) { return LZ4_COMPRESSBOUND(isize); } +int LZ4_sizeofState() { return LZ4_STREAMSIZE; } + + +/*-************************************ +* Internal Definitions used in Tests +**************************************/ +#if defined (__cplusplus) +extern "C" { +#endif + +int LZ4_compress_forceExtDict (LZ4_stream_t* LZ4_stream, const char* source, char* dest, int inputSize); + +int LZ4_decompress_safe_forceExtDict(const char* in, char* out, int inSize, int outSize, const void* dict, size_t dictSize); + +#if defined (__cplusplus) +} +#endif + +/*-****************************** +* Compression functions +********************************/ +static U32 LZ4_hash4(U32 sequence, tableType_t const tableType) +{ + if (tableType == byU16) + return ((sequence * 2654435761U) >> ((MINMATCH*8)-(LZ4_HASHLOG+1))); + else + return ((sequence * 2654435761U) >> ((MINMATCH*8)-LZ4_HASHLOG)); +} + +static U32 LZ4_hash5(U64 sequence, tableType_t const tableType) +{ + const U32 hashLog = (tableType == byU16) ? LZ4_HASHLOG+1 : LZ4_HASHLOG; + if (LZ4_isLittleEndian()) { + const U64 prime5bytes = 889523592379ULL; + return (U32)(((sequence << 24) * prime5bytes) >> (64 - hashLog)); + } else { + const U64 prime8bytes = 11400714785074694791ULL; + return (U32)(((sequence >> 24) * prime8bytes) >> (64 - hashLog)); + } +} + +LZ4_FORCE_INLINE U32 LZ4_hashPosition(const void* const p, tableType_t const tableType) +{ + if ((sizeof(reg_t)==8) && (tableType != byU16)) return LZ4_hash5(LZ4_read_ARCH(p), tableType); + return LZ4_hash4(LZ4_read32(p), tableType); +} + +static void LZ4_putIndexOnHash(U32 idx, U32 h, void* tableBase, tableType_t const tableType) +{ + switch (tableType) + { + default: /* fallthrough */ + case clearedTable: /* fallthrough */ + case byPtr: { /* illegal! */ assert(0); return; } + case byU32: { U32* hashTable = (U32*) tableBase; hashTable[h] = idx; return; } + case byU16: { U16* hashTable = (U16*) tableBase; assert(idx < 65536); hashTable[h] = (U16)idx; return; } + } +} + +static void LZ4_putPositionOnHash(const BYTE* p, U32 h, + void* tableBase, tableType_t const tableType, + const BYTE* srcBase) +{ + switch (tableType) + { + case clearedTable: { /* illegal! */ assert(0); return; } + case byPtr: { const BYTE** hashTable = (const BYTE**)tableBase; hashTable[h] = p; return; } + case byU32: { U32* hashTable = (U32*) tableBase; hashTable[h] = (U32)(p-srcBase); return; } + case byU16: { U16* hashTable = (U16*) tableBase; hashTable[h] = (U16)(p-srcBase); return; } + } +} + +LZ4_FORCE_INLINE void LZ4_putPosition(const BYTE* p, void* tableBase, tableType_t tableType, const BYTE* srcBase) +{ + U32 const h = LZ4_hashPosition(p, tableType); + LZ4_putPositionOnHash(p, h, tableBase, tableType, srcBase); +} + +/* LZ4_getIndexOnHash() : + * Index of match position registered in hash table. + * hash position must be calculated by using base+index, or dictBase+index. + * Assumption 1 : only valid if tableType == byU32 or byU16. + * Assumption 2 : h is presumed valid (within limits of hash table) + */ +static U32 LZ4_getIndexOnHash(U32 h, const void* tableBase, tableType_t tableType) +{ + LZ4_STATIC_ASSERT(LZ4_MEMORY_USAGE > 2); + if (tableType == byU32) { + const U32* const hashTable = (const U32*) tableBase; + assert(h < (1U << (LZ4_MEMORY_USAGE-2))); + return hashTable[h]; + } + if (tableType == byU16) { + const U16* const hashTable = (const U16*) tableBase; + assert(h < (1U << (LZ4_MEMORY_USAGE-1))); + return hashTable[h]; + } + assert(0); return 0; /* forbidden case */ +} + +static const BYTE* LZ4_getPositionOnHash(U32 h, const void* tableBase, tableType_t tableType, const BYTE* srcBase) +{ + if (tableType == byPtr) { const BYTE* const* hashTable = (const BYTE* const*) tableBase; return hashTable[h]; } + if (tableType == byU32) { const U32* const hashTable = (const U32*) tableBase; return hashTable[h] + srcBase; } + { const U16* const hashTable = (const U16*) tableBase; return hashTable[h] + srcBase; } /* default, to ensure a return */ +} + +LZ4_FORCE_INLINE const BYTE* LZ4_getPosition(const BYTE* p, + const void* tableBase, tableType_t tableType, + const BYTE* srcBase) +{ + U32 const h = LZ4_hashPosition(p, tableType); + return LZ4_getPositionOnHash(h, tableBase, tableType, srcBase); +} + +LZ4_FORCE_INLINE void LZ4_prepareTable( + LZ4_stream_t_internal* const cctx, + const int inputSize, + const tableType_t tableType) { + /* If compression failed during the previous step, then the context + * is marked as dirty, therefore, it has to be fully reset. + */ + if (cctx->dirty) { + DEBUGLOG(5, "LZ4_prepareTable: Full reset for %p", cctx); + MEM_INIT(cctx, 0, sizeof(LZ4_stream_t_internal)); + return; + } + + /* If the table hasn't been used, it's guaranteed to be zeroed out, and is + * therefore safe to use no matter what mode we're in. Otherwise, we figure + * out if it's safe to leave as is or whether it needs to be reset. + */ + if (cctx->tableType != clearedTable) { + if (cctx->tableType != tableType + || (tableType == byU16 && cctx->currentOffset + inputSize >= 0xFFFFU) + || (tableType == byU32 && cctx->currentOffset > 1 GB) + || tableType == byPtr + || inputSize >= 4 KB) + { + DEBUGLOG(4, "LZ4_prepareTable: Resetting table in %p", cctx); + MEM_INIT(cctx->hashTable, 0, LZ4_HASHTABLESIZE); + cctx->currentOffset = 0; + cctx->tableType = clearedTable; + } else { + DEBUGLOG(4, "LZ4_prepareTable: Re-use hash table (no reset)"); + } + } + + /* Adding a gap, so all previous entries are > LZ4_DISTANCE_MAX back, is faster + * than compressing without a gap. However, compressing with + * currentOffset == 0 is faster still, so we preserve that case. + */ + if (cctx->currentOffset != 0 && tableType == byU32) { + DEBUGLOG(5, "LZ4_prepareTable: adding 64KB to currentOffset"); + cctx->currentOffset += 64 KB; + } + + /* Finally, clear history */ + cctx->dictCtx = NULL; + cctx->dictionary = NULL; + cctx->dictSize = 0; +} + +/** LZ4_compress_generic() : + inlined, to ensure branches are decided at compilation time */ +LZ4_FORCE_INLINE int LZ4_compress_generic( + LZ4_stream_t_internal* const cctx, + const char* const source, + char* const dest, + const int inputSize, + int *inputConsumed, /* only written when outputDirective == fillOutput */ + const int maxOutputSize, + const limitedOutput_directive outputDirective, + const tableType_t tableType, + const dict_directive dictDirective, + const dictIssue_directive dictIssue, + const int acceleration) +{ + int result; + const BYTE* ip = (const BYTE*) source; + + U32 const startIndex = cctx->currentOffset; + const BYTE* base = (const BYTE*) source - startIndex; + const BYTE* lowLimit; + + const LZ4_stream_t_internal* dictCtx = (const LZ4_stream_t_internal*) cctx->dictCtx; + const BYTE* const dictionary = + dictDirective == usingDictCtx ? dictCtx->dictionary : cctx->dictionary; + const U32 dictSize = + dictDirective == usingDictCtx ? dictCtx->dictSize : cctx->dictSize; + const U32 dictDelta = (dictDirective == usingDictCtx) ? startIndex - dictCtx->currentOffset : 0; /* make indexes in dictCtx comparable with index in current context */ + + int const maybe_extMem = (dictDirective == usingExtDict) || (dictDirective == usingDictCtx); + U32 const prefixIdxLimit = startIndex - dictSize; /* used when dictDirective == dictSmall */ + const BYTE* const dictEnd = dictionary + dictSize; + const BYTE* anchor = (const BYTE*) source; + const BYTE* const iend = ip + inputSize; + const BYTE* const mflimitPlusOne = iend - MFLIMIT + 1; + const BYTE* const matchlimit = iend - LASTLITERALS; + + /* the dictCtx currentOffset is indexed on the start of the dictionary, + * while a dictionary in the current context precedes the currentOffset */ + const BYTE* dictBase = (dictDirective == usingDictCtx) ? + dictionary + dictSize - dictCtx->currentOffset : + dictionary + dictSize - startIndex; + + BYTE* op = (BYTE*) dest; + BYTE* const olimit = op + maxOutputSize; + + U32 offset = 0; + U32 forwardH; + + DEBUGLOG(5, "LZ4_compress_generic: srcSize=%i, tableType=%u", inputSize, tableType); + /* If init conditions are not met, we don't have to mark stream + * as having dirty context, since no action was taken yet */ + if (outputDirective == fillOutput && maxOutputSize < 1) return 0; /* Impossible to store anything */ + if ((U32)inputSize > (U32)LZ4_MAX_INPUT_SIZE) return 0; /* Unsupported inputSize, too large (or negative) */ + if ((tableType == byU16) && (inputSize>=LZ4_64Klimit)) return 0; /* Size too large (not within 64K limit) */ + if (tableType==byPtr) assert(dictDirective==noDict); /* only supported use case with byPtr */ + assert(acceleration >= 1); + + lowLimit = (const BYTE*)source - (dictDirective == withPrefix64k ? dictSize : 0); + + /* Update context state */ + if (dictDirective == usingDictCtx) { + /* Subsequent linked blocks can't use the dictionary. */ + /* Instead, they use the block we just compressed. */ + cctx->dictCtx = NULL; + cctx->dictSize = (U32)inputSize; + } else { + cctx->dictSize += (U32)inputSize; + } + cctx->currentOffset += (U32)inputSize; + cctx->tableType = (U16)tableType; + + if (inputSizehashTable, tableType, base); + ip++; forwardH = LZ4_hashPosition(ip, tableType); + + /* Main Loop */ + for ( ; ; ) { + const BYTE* match; + BYTE* token; + + /* Find a match */ + if (tableType == byPtr) { + const BYTE* forwardIp = ip; + int step = 1; + int searchMatchNb = acceleration << LZ4_skipTrigger; + do { + U32 const h = forwardH; + ip = forwardIp; + forwardIp += step; + step = (searchMatchNb++ >> LZ4_skipTrigger); + + if (unlikely(forwardIp > mflimitPlusOne)) goto _last_literals; + assert(ip < mflimitPlusOne); + + match = LZ4_getPositionOnHash(h, cctx->hashTable, tableType, base); + forwardH = LZ4_hashPosition(forwardIp, tableType); + LZ4_putPositionOnHash(ip, h, cctx->hashTable, tableType, base); + + } while ( (match+LZ4_DISTANCE_MAX < ip) + || (LZ4_read32(match) != LZ4_read32(ip)) ); + + } else { /* byU32, byU16 */ + + const BYTE* forwardIp = ip; + int step = 1; + int searchMatchNb = acceleration << LZ4_skipTrigger; + do { + U32 const h = forwardH; + U32 const current = (U32)(forwardIp - base); + U32 matchIndex = LZ4_getIndexOnHash(h, cctx->hashTable, tableType); + assert(matchIndex <= current); + assert(forwardIp - base < (ptrdiff_t)(2 GB - 1)); + ip = forwardIp; + forwardIp += step; + step = (searchMatchNb++ >> LZ4_skipTrigger); + + if (unlikely(forwardIp > mflimitPlusOne)) goto _last_literals; + assert(ip < mflimitPlusOne); + + if (dictDirective == usingDictCtx) { + if (matchIndex < startIndex) { + /* there was no match, try the dictionary */ + assert(tableType == byU32); + matchIndex = LZ4_getIndexOnHash(h, dictCtx->hashTable, byU32); + match = dictBase + matchIndex; + matchIndex += dictDelta; /* make dictCtx index comparable with current context */ + lowLimit = dictionary; + } else { + match = base + matchIndex; + lowLimit = (const BYTE*)source; + } + } else if (dictDirective==usingExtDict) { + if (matchIndex < startIndex) { + DEBUGLOG(7, "extDict candidate: matchIndex=%5u < startIndex=%5u", matchIndex, startIndex); + assert(startIndex - matchIndex >= MINMATCH); + match = dictBase + matchIndex; + lowLimit = dictionary; + } else { + match = base + matchIndex; + lowLimit = (const BYTE*)source; + } + } else { /* single continuous memory segment */ + match = base + matchIndex; + } + forwardH = LZ4_hashPosition(forwardIp, tableType); + LZ4_putIndexOnHash(current, h, cctx->hashTable, tableType); + + if ((dictIssue == dictSmall) && (matchIndex < prefixIdxLimit)) continue; /* match outside of valid area */ + assert(matchIndex < current); + if ((tableType != byU16) && (matchIndex+LZ4_DISTANCE_MAX < current)) continue; /* too far */ + if (tableType == byU16) assert((current - matchIndex) <= LZ4_DISTANCE_MAX); /* too_far presumed impossible with byU16 */ + + if (LZ4_read32(match) == LZ4_read32(ip)) { + if (maybe_extMem) offset = current - matchIndex; + break; /* match found */ + } + + } while(1); + } + + /* Catch up */ + while (((ip>anchor) & (match > lowLimit)) && (unlikely(ip[-1]==match[-1]))) { ip--; match--; } + + /* Encode Literals */ + { unsigned const litLength = (unsigned)(ip - anchor); + token = op++; + if ((outputDirective == limitedOutput) && /* Check output buffer overflow */ + (unlikely(op + litLength + (2 + 1 + LASTLITERALS) + (litLength/255) > olimit)) ) + return 0; /* cannot compress within `dst` budget. Stored indexes in hash table are nonetheless fine */ + + if ((outputDirective == fillOutput) && + (unlikely(op + (litLength+240)/255 /* litlen */ + litLength /* literals */ + 2 /* offset */ + 1 /* token */ + MFLIMIT - MINMATCH /* min last literals so last match is <= end - MFLIMIT */ > olimit))) { + op--; + goto _last_literals; + } + if (litLength >= RUN_MASK) { + int len = (int)(litLength - RUN_MASK); + *token = (RUN_MASK<= 255 ; len-=255) *op++ = 255; + *op++ = (BYTE)len; + } + else *token = (BYTE)(litLength< olimit)) { + /* the match was too close to the end, rewind and go to last literals */ + op = token; + goto _last_literals; + } + + /* Encode Offset */ + if (maybe_extMem) { /* static test */ + DEBUGLOG(6, " with offset=%u (ext if > %i)", offset, (int)(ip - (const BYTE*)source)); + assert(offset <= LZ4_DISTANCE_MAX && offset > 0); + LZ4_writeLE16(op, (U16)offset); op+=2; + } else { + DEBUGLOG(6, " with offset=%u (same segment)", (U32)(ip - match)); + assert(ip-match <= LZ4_DISTANCE_MAX); + LZ4_writeLE16(op, (U16)(ip - match)); op+=2; + } + + /* Encode MatchLength */ + { unsigned matchCode; + + if ( (dictDirective==usingExtDict || dictDirective==usingDictCtx) + && (lowLimit==dictionary) /* match within extDict */ ) { + const BYTE* limit = ip + (dictEnd-match); + assert(dictEnd > match); + if (limit > matchlimit) limit = matchlimit; + matchCode = LZ4_count(ip+MINMATCH, match+MINMATCH, limit); + ip += (size_t)matchCode + MINMATCH; + if (ip==limit) { + unsigned const more = LZ4_count(limit, (const BYTE*)source, matchlimit); + matchCode += more; + ip += more; + } + DEBUGLOG(6, " with matchLength=%u starting in extDict", matchCode+MINMATCH); + } else { + matchCode = LZ4_count(ip+MINMATCH, match+MINMATCH, matchlimit); + ip += (size_t)matchCode + MINMATCH; + DEBUGLOG(6, " with matchLength=%u", matchCode+MINMATCH); + } + + if ((outputDirective) && /* Check output buffer overflow */ + (unlikely(op + (1 + LASTLITERALS) + (matchCode>>8) > olimit)) ) { + if (outputDirective == fillOutput) { + /* Match description too long : reduce it */ + U32 newMatchCode = 15 /* in token */ - 1 /* to avoid needing a zero byte */ + ((U32)(olimit - op) - 2 - 1 - LASTLITERALS) * 255; + ip -= matchCode - newMatchCode; + matchCode = newMatchCode; + } else { + assert(outputDirective == limitedOutput); + return 0; /* cannot compress within `dst` budget. Stored indexes in hash table are nonetheless fine */ + } + } + if (matchCode >= ML_MASK) { + *token += ML_MASK; + matchCode -= ML_MASK; + LZ4_write32(op, 0xFFFFFFFF); + while (matchCode >= 4*255) { + op+=4; + LZ4_write32(op, 0xFFFFFFFF); + matchCode -= 4*255; + } + op += matchCode / 255; + *op++ = (BYTE)(matchCode % 255); + } else + *token += (BYTE)(matchCode); + } + + anchor = ip; + + /* Test end of chunk */ + if (ip >= mflimitPlusOne) break; + + /* Fill table */ + LZ4_putPosition(ip-2, cctx->hashTable, tableType, base); + + /* Test next position */ + if (tableType == byPtr) { + + match = LZ4_getPosition(ip, cctx->hashTable, tableType, base); + LZ4_putPosition(ip, cctx->hashTable, tableType, base); + if ( (match+LZ4_DISTANCE_MAX >= ip) + && (LZ4_read32(match) == LZ4_read32(ip)) ) + { token=op++; *token=0; goto _next_match; } + + } else { /* byU32, byU16 */ + + U32 const h = LZ4_hashPosition(ip, tableType); + U32 const current = (U32)(ip-base); + U32 matchIndex = LZ4_getIndexOnHash(h, cctx->hashTable, tableType); + assert(matchIndex < current); + if (dictDirective == usingDictCtx) { + if (matchIndex < startIndex) { + /* there was no match, try the dictionary */ + matchIndex = LZ4_getIndexOnHash(h, dictCtx->hashTable, byU32); + match = dictBase + matchIndex; + lowLimit = dictionary; /* required for match length counter */ + matchIndex += dictDelta; + } else { + match = base + matchIndex; + lowLimit = (const BYTE*)source; /* required for match length counter */ + } + } else if (dictDirective==usingExtDict) { + if (matchIndex < startIndex) { + match = dictBase + matchIndex; + lowLimit = dictionary; /* required for match length counter */ + } else { + match = base + matchIndex; + lowLimit = (const BYTE*)source; /* required for match length counter */ + } + } else { /* single memory segment */ + match = base + matchIndex; + } + LZ4_putIndexOnHash(current, h, cctx->hashTable, tableType); + assert(matchIndex < current); + if ( ((dictIssue==dictSmall) ? (matchIndex >= prefixIdxLimit) : 1) + && ((tableType==byU16) ? 1 : (matchIndex+LZ4_DISTANCE_MAX >= current)) + && (LZ4_read32(match) == LZ4_read32(ip)) ) { + token=op++; + *token=0; + if (maybe_extMem) offset = current - matchIndex; + DEBUGLOG(6, "seq.start:%i, literals=%u, match.start:%i", + (int)(anchor-(const BYTE*)source), 0, (int)(ip-(const BYTE*)source)); + goto _next_match; + } + } + + /* Prepare next loop */ + forwardH = LZ4_hashPosition(++ip, tableType); + + } + +_last_literals: + /* Encode Last Literals */ + { size_t lastRun = (size_t)(iend - anchor); + if ( (outputDirective) && /* Check output buffer overflow */ + (op + lastRun + 1 + ((lastRun+255-RUN_MASK)/255) > olimit)) { + if (outputDirective == fillOutput) { + /* adapt lastRun to fill 'dst' */ + assert(olimit >= op); + lastRun = (size_t)(olimit-op) - 1; + lastRun -= (lastRun+240)/255; + } else { + assert(outputDirective == limitedOutput); + return 0; /* cannot compress within `dst` budget. Stored indexes in hash table are nonetheless fine */ + } + } + if (lastRun >= RUN_MASK) { + size_t accumulator = lastRun - RUN_MASK; + *op++ = RUN_MASK << ML_BITS; + for(; accumulator >= 255 ; accumulator-=255) *op++ = 255; + *op++ = (BYTE) accumulator; + } else { + *op++ = (BYTE)(lastRun< 0); + return result; +} + + +int LZ4_compress_fast_extState(void* state, const char* source, char* dest, int inputSize, int maxOutputSize, int acceleration) +{ + LZ4_stream_t_internal* const ctx = & LZ4_initStream(state, sizeof(LZ4_stream_t)) -> internal_donotuse; + assert(ctx != NULL); + if (acceleration < 1) acceleration = ACCELERATION_DEFAULT; + if (maxOutputSize >= LZ4_compressBound(inputSize)) { + if (inputSize < LZ4_64Klimit) { + return LZ4_compress_generic(ctx, source, dest, inputSize, NULL, 0, notLimited, byU16, noDict, noDictIssue, acceleration); + } else { + const tableType_t tableType = ((sizeof(void*)==4) && ((uptrval)source > LZ4_DISTANCE_MAX)) ? byPtr : byU32; + return LZ4_compress_generic(ctx, source, dest, inputSize, NULL, 0, notLimited, tableType, noDict, noDictIssue, acceleration); + } + } else { + if (inputSize < LZ4_64Klimit) {; + return LZ4_compress_generic(ctx, source, dest, inputSize, NULL, maxOutputSize, limitedOutput, byU16, noDict, noDictIssue, acceleration); + } else { + const tableType_t tableType = ((sizeof(void*)==4) && ((uptrval)source > LZ4_DISTANCE_MAX)) ? byPtr : byU32; + return LZ4_compress_generic(ctx, source, dest, inputSize, NULL, maxOutputSize, limitedOutput, tableType, noDict, noDictIssue, acceleration); + } + } +} + +/** + * LZ4_compress_fast_extState_fastReset() : + * A variant of LZ4_compress_fast_extState(). + * + * Using this variant avoids an expensive initialization step. It is only safe + * to call if the state buffer is known to be correctly initialized already + * (see comment in lz4.h on LZ4_resetStream_fast() for a definition of + * "correctly initialized"). + */ +int LZ4_compress_fast_extState_fastReset(void* state, const char* src, char* dst, int srcSize, int dstCapacity, int acceleration) +{ + LZ4_stream_t_internal* ctx = &((LZ4_stream_t*)state)->internal_donotuse; + if (acceleration < 1) acceleration = ACCELERATION_DEFAULT; + + if (dstCapacity >= LZ4_compressBound(srcSize)) { + if (srcSize < LZ4_64Klimit) { + const tableType_t tableType = byU16; + LZ4_prepareTable(ctx, srcSize, tableType); + if (ctx->currentOffset) { + return LZ4_compress_generic(ctx, src, dst, srcSize, NULL, 0, notLimited, tableType, noDict, dictSmall, acceleration); + } else { + return LZ4_compress_generic(ctx, src, dst, srcSize, NULL, 0, notLimited, tableType, noDict, noDictIssue, acceleration); + } + } else { + const tableType_t tableType = ((sizeof(void*)==4) && ((uptrval)src > LZ4_DISTANCE_MAX)) ? byPtr : byU32; + LZ4_prepareTable(ctx, srcSize, tableType); + return LZ4_compress_generic(ctx, src, dst, srcSize, NULL, 0, notLimited, tableType, noDict, noDictIssue, acceleration); + } + } else { + if (srcSize < LZ4_64Klimit) { + const tableType_t tableType = byU16; + LZ4_prepareTable(ctx, srcSize, tableType); + if (ctx->currentOffset) { + return LZ4_compress_generic(ctx, src, dst, srcSize, NULL, dstCapacity, limitedOutput, tableType, noDict, dictSmall, acceleration); + } else { + return LZ4_compress_generic(ctx, src, dst, srcSize, NULL, dstCapacity, limitedOutput, tableType, noDict, noDictIssue, acceleration); + } + } else { + const tableType_t tableType = ((sizeof(void*)==4) && ((uptrval)src > LZ4_DISTANCE_MAX)) ? byPtr : byU32; + LZ4_prepareTable(ctx, srcSize, tableType); + return LZ4_compress_generic(ctx, src, dst, srcSize, NULL, dstCapacity, limitedOutput, tableType, noDict, noDictIssue, acceleration); + } + } +} + + +int LZ4_compress_fast(const char* source, char* dest, int inputSize, int maxOutputSize, int acceleration) +{ + int result; +#if (LZ4_HEAPMODE) + LZ4_stream_t* ctxPtr = ALLOC(sizeof(LZ4_stream_t)); /* malloc-calloc always properly aligned */ + if (ctxPtr == NULL) return 0; +#else + LZ4_stream_t ctx; + LZ4_stream_t* const ctxPtr = &ctx; +#endif + result = LZ4_compress_fast_extState(ctxPtr, source, dest, inputSize, maxOutputSize, acceleration); + +#if (LZ4_HEAPMODE) + FREEMEM(ctxPtr); +#endif + return result; +} + + +int LZ4_compress_default(const char* src, char* dst, int srcSize, int maxOutputSize) +{ + return LZ4_compress_fast(src, dst, srcSize, maxOutputSize, 1); +} + + +/* hidden debug function */ +/* strangely enough, gcc generates faster code when this function is uncommented, even if unused */ +int LZ4_compress_fast_force(const char* src, char* dst, int srcSize, int dstCapacity, int acceleration) +{ + LZ4_stream_t ctx; + LZ4_initStream(&ctx, sizeof(ctx)); + + if (srcSize < LZ4_64Klimit) { + return LZ4_compress_generic(&ctx.internal_donotuse, src, dst, srcSize, NULL, dstCapacity, limitedOutput, byU16, noDict, noDictIssue, acceleration); + } else { + tableType_t const addrMode = (sizeof(void*) > 4) ? byU32 : byPtr; + return LZ4_compress_generic(&ctx.internal_donotuse, src, dst, srcSize, NULL, dstCapacity, limitedOutput, addrMode, noDict, noDictIssue, acceleration); + } +} + + +/* Note!: This function leaves the stream in an unclean/broken state! + * It is not safe to subsequently use the same state with a _fastReset() or + * _continue() call without resetting it. */ +static int LZ4_compress_destSize_extState (LZ4_stream_t* state, const char* src, char* dst, int* srcSizePtr, int targetDstSize) +{ + void* const s = LZ4_initStream(state, sizeof (*state)); + assert(s != NULL); (void)s; + + if (targetDstSize >= LZ4_compressBound(*srcSizePtr)) { /* compression success is guaranteed */ + return LZ4_compress_fast_extState(state, src, dst, *srcSizePtr, targetDstSize, 1); + } else { + if (*srcSizePtr < LZ4_64Klimit) { + return LZ4_compress_generic(&state->internal_donotuse, src, dst, *srcSizePtr, srcSizePtr, targetDstSize, fillOutput, byU16, noDict, noDictIssue, 1); + } else { + tableType_t const addrMode = ((sizeof(void*)==4) && ((uptrval)src > LZ4_DISTANCE_MAX)) ? byPtr : byU32; + return LZ4_compress_generic(&state->internal_donotuse, src, dst, *srcSizePtr, srcSizePtr, targetDstSize, fillOutput, addrMode, noDict, noDictIssue, 1); + } } +} + + +int LZ4_compress_destSize(const char* src, char* dst, int* srcSizePtr, int targetDstSize) +{ +#if (LZ4_HEAPMODE) + LZ4_stream_t* ctx = (LZ4_stream_t*)ALLOC(sizeof(LZ4_stream_t)); /* malloc-calloc always properly aligned */ + if (ctx == NULL) return 0; +#else + LZ4_stream_t ctxBody; + LZ4_stream_t* ctx = &ctxBody; +#endif + + int result = LZ4_compress_destSize_extState(ctx, src, dst, srcSizePtr, targetDstSize); + +#if (LZ4_HEAPMODE) + FREEMEM(ctx); +#endif + return result; +} + + + +/*-****************************** +* Streaming functions +********************************/ + +LZ4_stream_t* LZ4_createStream(void) +{ + LZ4_stream_t* const lz4s = (LZ4_stream_t*)ALLOC(sizeof(LZ4_stream_t)); + LZ4_STATIC_ASSERT(LZ4_STREAMSIZE >= sizeof(LZ4_stream_t_internal)); /* A compilation error here means LZ4_STREAMSIZE is not large enough */ + DEBUGLOG(4, "LZ4_createStream %p", lz4s); + if (lz4s == NULL) return NULL; + LZ4_initStream(lz4s, sizeof(*lz4s)); + return lz4s; +} + +#ifndef _MSC_VER /* for some reason, Visual fails the aligment test on 32-bit x86 : + it reports an aligment of 8-bytes, + while actually aligning LZ4_stream_t on 4 bytes. */ +static size_t LZ4_stream_t_alignment(void) +{ + struct { char c; LZ4_stream_t t; } t_a; + return sizeof(t_a) - sizeof(t_a.t); +} +#endif + +LZ4_stream_t* LZ4_initStream (void* buffer, size_t size) +{ + DEBUGLOG(5, "LZ4_initStream"); + if (buffer == NULL) return NULL; + if (size < sizeof(LZ4_stream_t)) return NULL; +#ifndef _MSC_VER /* for some reason, Visual fails the aligment test on 32-bit x86 : + it reports an aligment of 8-bytes, + while actually aligning LZ4_stream_t on 4 bytes. */ + if (((size_t)buffer) & (LZ4_stream_t_alignment() - 1)) return NULL; /* alignment check */ +#endif + MEM_INIT(buffer, 0, sizeof(LZ4_stream_t)); + return (LZ4_stream_t*)buffer; +} + +/* resetStream is now deprecated, + * prefer initStream() which is more general */ +void LZ4_resetStream (LZ4_stream_t* LZ4_stream) +{ + DEBUGLOG(5, "LZ4_resetStream (ctx:%p)", LZ4_stream); + MEM_INIT(LZ4_stream, 0, sizeof(LZ4_stream_t)); +} + +void LZ4_resetStream_fast(LZ4_stream_t* ctx) { + LZ4_prepareTable(&(ctx->internal_donotuse), 0, byU32); +} + +int LZ4_freeStream (LZ4_stream_t* LZ4_stream) +{ + if (!LZ4_stream) return 0; /* support free on NULL */ + DEBUGLOG(5, "LZ4_freeStream %p", LZ4_stream); + FREEMEM(LZ4_stream); + return (0); +} + + +#define HASH_UNIT sizeof(reg_t) +int LZ4_loadDict (LZ4_stream_t* LZ4_dict, const char* dictionary, int dictSize) +{ + LZ4_stream_t_internal* dict = &LZ4_dict->internal_donotuse; + const tableType_t tableType = byU32; + const BYTE* p = (const BYTE*)dictionary; + const BYTE* const dictEnd = p + dictSize; + const BYTE* base; + + DEBUGLOG(4, "LZ4_loadDict (%i bytes from %p into %p)", dictSize, dictionary, LZ4_dict); + + /* It's necessary to reset the context, + * and not just continue it with prepareTable() + * to avoid any risk of generating overflowing matchIndex + * when compressing using this dictionary */ + LZ4_resetStream(LZ4_dict); + + /* We always increment the offset by 64 KB, since, if the dict is longer, + * we truncate it to the last 64k, and if it's shorter, we still want to + * advance by a whole window length so we can provide the guarantee that + * there are only valid offsets in the window, which allows an optimization + * in LZ4_compress_fast_continue() where it uses noDictIssue even when the + * dictionary isn't a full 64k. */ + + if ((dictEnd - p) > 64 KB) p = dictEnd - 64 KB; + base = dictEnd - 64 KB - dict->currentOffset; + dict->dictionary = p; + dict->dictSize = (U32)(dictEnd - p); + dict->currentOffset += 64 KB; + dict->tableType = tableType; + + if (dictSize < (int)HASH_UNIT) { + return 0; + } + + while (p <= dictEnd-HASH_UNIT) { + LZ4_putPosition(p, dict->hashTable, tableType, base); + p+=3; + } + + return (int)dict->dictSize; +} + +void LZ4_attach_dictionary(LZ4_stream_t *working_stream, const LZ4_stream_t *dictionary_stream) { + /* Calling LZ4_resetStream_fast() here makes sure that changes will not be + * erased by subsequent calls to LZ4_resetStream_fast() in case stream was + * marked as having dirty context, e.g. requiring full reset. + */ + LZ4_resetStream_fast(working_stream); + + if (dictionary_stream != NULL) { + /* If the current offset is zero, we will never look in the + * external dictionary context, since there is no value a table + * entry can take that indicate a miss. In that case, we need + * to bump the offset to something non-zero. + */ + if (working_stream->internal_donotuse.currentOffset == 0) { + working_stream->internal_donotuse.currentOffset = 64 KB; + } + working_stream->internal_donotuse.dictCtx = &(dictionary_stream->internal_donotuse); + } else { + working_stream->internal_donotuse.dictCtx = NULL; + } +} + + +static void LZ4_renormDictT(LZ4_stream_t_internal* LZ4_dict, int nextSize) +{ + assert(nextSize >= 0); + if (LZ4_dict->currentOffset + (unsigned)nextSize > 0x80000000) { /* potential ptrdiff_t overflow (32-bits mode) */ + /* rescale hash table */ + U32 const delta = LZ4_dict->currentOffset - 64 KB; + const BYTE* dictEnd = LZ4_dict->dictionary + LZ4_dict->dictSize; + int i; + DEBUGLOG(4, "LZ4_renormDictT"); + for (i=0; ihashTable[i] < delta) LZ4_dict->hashTable[i]=0; + else LZ4_dict->hashTable[i] -= delta; + } + LZ4_dict->currentOffset = 64 KB; + if (LZ4_dict->dictSize > 64 KB) LZ4_dict->dictSize = 64 KB; + LZ4_dict->dictionary = dictEnd - LZ4_dict->dictSize; + } +} + + +int LZ4_compress_fast_continue (LZ4_stream_t* LZ4_stream, + const char* source, char* dest, + int inputSize, int maxOutputSize, + int acceleration) +{ + const tableType_t tableType = byU32; + LZ4_stream_t_internal* streamPtr = &LZ4_stream->internal_donotuse; + const BYTE* dictEnd = streamPtr->dictionary + streamPtr->dictSize; + + DEBUGLOG(5, "LZ4_compress_fast_continue (inputSize=%i)", inputSize); + + if (streamPtr->dirty) return 0; /* Uninitialized structure detected */ + LZ4_renormDictT(streamPtr, inputSize); /* avoid index overflow */ + if (acceleration < 1) acceleration = ACCELERATION_DEFAULT; + + /* invalidate tiny dictionaries */ + if ( (streamPtr->dictSize-1 < 4-1) /* intentional underflow */ + && (dictEnd != (const BYTE*)source) ) { + DEBUGLOG(5, "LZ4_compress_fast_continue: dictSize(%u) at addr:%p is too small", streamPtr->dictSize, streamPtr->dictionary); + streamPtr->dictSize = 0; + streamPtr->dictionary = (const BYTE*)source; + dictEnd = (const BYTE*)source; + } + + /* Check overlapping input/dictionary space */ + { const BYTE* sourceEnd = (const BYTE*) source + inputSize; + if ((sourceEnd > streamPtr->dictionary) && (sourceEnd < dictEnd)) { + streamPtr->dictSize = (U32)(dictEnd - sourceEnd); + if (streamPtr->dictSize > 64 KB) streamPtr->dictSize = 64 KB; + if (streamPtr->dictSize < 4) streamPtr->dictSize = 0; + streamPtr->dictionary = dictEnd - streamPtr->dictSize; + } + } + + /* prefix mode : source data follows dictionary */ + if (dictEnd == (const BYTE*)source) { + if ((streamPtr->dictSize < 64 KB) && (streamPtr->dictSize < streamPtr->currentOffset)) + return LZ4_compress_generic(streamPtr, source, dest, inputSize, NULL, maxOutputSize, limitedOutput, tableType, withPrefix64k, dictSmall, acceleration); + else + return LZ4_compress_generic(streamPtr, source, dest, inputSize, NULL, maxOutputSize, limitedOutput, tableType, withPrefix64k, noDictIssue, acceleration); + } + + /* external dictionary mode */ + { int result; + if (streamPtr->dictCtx) { + /* We depend here on the fact that dictCtx'es (produced by + * LZ4_loadDict) guarantee that their tables contain no references + * to offsets between dictCtx->currentOffset - 64 KB and + * dictCtx->currentOffset - dictCtx->dictSize. This makes it safe + * to use noDictIssue even when the dict isn't a full 64 KB. + */ + if (inputSize > 4 KB) { + /* For compressing large blobs, it is faster to pay the setup + * cost to copy the dictionary's tables into the active context, + * so that the compression loop is only looking into one table. + */ + memcpy(streamPtr, streamPtr->dictCtx, sizeof(LZ4_stream_t)); + result = LZ4_compress_generic(streamPtr, source, dest, inputSize, NULL, maxOutputSize, limitedOutput, tableType, usingExtDict, noDictIssue, acceleration); + } else { + result = LZ4_compress_generic(streamPtr, source, dest, inputSize, NULL, maxOutputSize, limitedOutput, tableType, usingDictCtx, noDictIssue, acceleration); + } + } else { + if ((streamPtr->dictSize < 64 KB) && (streamPtr->dictSize < streamPtr->currentOffset)) { + result = LZ4_compress_generic(streamPtr, source, dest, inputSize, NULL, maxOutputSize, limitedOutput, tableType, usingExtDict, dictSmall, acceleration); + } else { + result = LZ4_compress_generic(streamPtr, source, dest, inputSize, NULL, maxOutputSize, limitedOutput, tableType, usingExtDict, noDictIssue, acceleration); + } + } + streamPtr->dictionary = (const BYTE*)source; + streamPtr->dictSize = (U32)inputSize; + return result; + } +} + + +/* Hidden debug function, to force-test external dictionary mode */ +int LZ4_compress_forceExtDict (LZ4_stream_t* LZ4_dict, const char* source, char* dest, int srcSize) +{ + LZ4_stream_t_internal* streamPtr = &LZ4_dict->internal_donotuse; + int result; + + LZ4_renormDictT(streamPtr, srcSize); + + if ((streamPtr->dictSize < 64 KB) && (streamPtr->dictSize < streamPtr->currentOffset)) { + result = LZ4_compress_generic(streamPtr, source, dest, srcSize, NULL, 0, notLimited, byU32, usingExtDict, dictSmall, 1); + } else { + result = LZ4_compress_generic(streamPtr, source, dest, srcSize, NULL, 0, notLimited, byU32, usingExtDict, noDictIssue, 1); + } + + streamPtr->dictionary = (const BYTE*)source; + streamPtr->dictSize = (U32)srcSize; + + return result; +} + + +/*! LZ4_saveDict() : + * If previously compressed data block is not guaranteed to remain available at its memory location, + * save it into a safer place (char* safeBuffer). + * Note : you don't need to call LZ4_loadDict() afterwards, + * dictionary is immediately usable, you can therefore call LZ4_compress_fast_continue(). + * Return : saved dictionary size in bytes (necessarily <= dictSize), or 0 if error. + */ +int LZ4_saveDict (LZ4_stream_t* LZ4_dict, char* safeBuffer, int dictSize) +{ + LZ4_stream_t_internal* const dict = &LZ4_dict->internal_donotuse; + const BYTE* const previousDictEnd = dict->dictionary + dict->dictSize; + + if ((U32)dictSize > 64 KB) dictSize = 64 KB; /* useless to define a dictionary > 64 KB */ + if ((U32)dictSize > dict->dictSize) dictSize = (int)dict->dictSize; + + memmove(safeBuffer, previousDictEnd - dictSize, dictSize); + + dict->dictionary = (const BYTE*)safeBuffer; + dict->dictSize = (U32)dictSize; + + return dictSize; +} + + + +/*-******************************* + * Decompression functions + ********************************/ + +typedef enum { endOnOutputSize = 0, endOnInputSize = 1 } endCondition_directive; +typedef enum { decode_full_block = 0, partial_decode = 1 } earlyEnd_directive; + +#undef MIN +#define MIN(a,b) ( (a) < (b) ? (a) : (b) ) + +/* Read the variable-length literal or match length. + * + * ip - pointer to use as input. + * lencheck - end ip. Return an error if ip advances >= lencheck. + * loop_check - check ip >= lencheck in body of loop. Returns loop_error if so. + * initial_check - check ip >= lencheck before start of loop. Returns initial_error if so. + * error (output) - error code. Should be set to 0 before call. + */ +typedef enum { loop_error = -2, initial_error = -1, ok = 0 } variable_length_error; +LZ4_FORCE_INLINE unsigned +read_variable_length(const BYTE**ip, const BYTE* lencheck, int loop_check, int initial_check, variable_length_error* error) +{ + unsigned length = 0; + unsigned s; + if (initial_check && unlikely((*ip) >= lencheck)) { /* overflow detection */ + *error = initial_error; + return length; + } + do { + s = **ip; + (*ip)++; + length += s; + if (loop_check && unlikely((*ip) >= lencheck)) { /* overflow detection */ + *error = loop_error; + return length; + } + } while (s==255); + + return length; +} + +/*! LZ4_decompress_generic() : + * This generic decompression function covers all use cases. + * It shall be instantiated several times, using different sets of directives. + * Note that it is important for performance that this function really get inlined, + * in order to remove useless branches during compilation optimization. + */ +LZ4_FORCE_INLINE int +LZ4_decompress_generic( + const char* const src, + char* const dst, + int srcSize, + int outputSize, /* If endOnInput==endOnInputSize, this value is `dstCapacity` */ + + endCondition_directive endOnInput, /* endOnOutputSize, endOnInputSize */ + earlyEnd_directive partialDecoding, /* full, partial */ + dict_directive dict, /* noDict, withPrefix64k, usingExtDict */ + const BYTE* const lowPrefix, /* always <= dst, == dst when no prefix */ + const BYTE* const dictStart, /* only if dict==usingExtDict */ + const size_t dictSize /* note : = 0 if noDict */ + ) +{ + if (src == NULL) return -1; + + { const BYTE* ip = (const BYTE*) src; + const BYTE* const iend = ip + srcSize; + + BYTE* op = (BYTE*) dst; + BYTE* const oend = op + outputSize; + BYTE* cpy; + + const BYTE* const dictEnd = (dictStart == NULL) ? NULL : dictStart + dictSize; + + const int safeDecode = (endOnInput==endOnInputSize); + const int checkOffset = ((safeDecode) && (dictSize < (int)(64 KB))); + + + /* Set up the "end" pointers for the shortcut. */ + const BYTE* const shortiend = iend - (endOnInput ? 14 : 8) /*maxLL*/ - 2 /*offset*/; + const BYTE* const shortoend = oend - (endOnInput ? 14 : 8) /*maxLL*/ - 18 /*maxML*/; + + const BYTE* match; + size_t offset; + unsigned token; + size_t length; + + + DEBUGLOG(5, "LZ4_decompress_generic (srcSize:%i, dstSize:%i)", srcSize, outputSize); + + /* Special cases */ + assert(lowPrefix <= op); + if ((endOnInput) && (unlikely(outputSize==0))) return ((srcSize==1) && (*ip==0)) ? 0 : -1; /* Empty output buffer */ + if ((!endOnInput) && (unlikely(outputSize==0))) return (*ip==0 ? 1 : -1); + if ((endOnInput) && unlikely(srcSize==0)) return -1; + + /* Currently the fast loop shows a regression on qualcomm arm chips. */ +#if LZ4_FAST_DEC_LOOP + if ((oend - op) < FASTLOOP_SAFE_DISTANCE) { + DEBUGLOG(6, "skip fast decode loop"); + goto safe_decode; + } + + /* Fast loop : decode sequences as long as output < iend-FASTLOOP_SAFE_DISTANCE */ + while (1) { + /* Main fastloop assertion: We can always wildcopy FASTLOOP_SAFE_DISTANCE */ + assert(oend - op >= FASTLOOP_SAFE_DISTANCE); + if (endOnInput) assert(ip < iend); + token = *ip++; + length = token >> ML_BITS; /* literal length */ + + assert(!endOnInput || ip <= iend); /* ip < iend before the increment */ + + /* decode literal length */ + if (length == RUN_MASK) { + variable_length_error error = ok; + length += read_variable_length(&ip, iend-RUN_MASK, endOnInput, endOnInput, &error); + if (error == initial_error) goto _output_error; + if ((safeDecode) && unlikely((uptrval)(op)+length<(uptrval)(op))) goto _output_error; /* overflow detection */ + if ((safeDecode) && unlikely((uptrval)(ip)+length<(uptrval)(ip))) goto _output_error; /* overflow detection */ + + /* copy literals */ + cpy = op+length; + LZ4_STATIC_ASSERT(MFLIMIT >= WILDCOPYLENGTH); + if (endOnInput) { /* LZ4_decompress_safe() */ + if ((cpy>oend-32) || (ip+length>iend-32)) goto safe_literal_copy; + LZ4_wildCopy32(op, ip, cpy); + } else { /* LZ4_decompress_fast() */ + if (cpy>oend-8) goto safe_literal_copy; + LZ4_wildCopy8(op, ip, cpy); /* LZ4_decompress_fast() cannot copy more than 8 bytes at a time : + * it doesn't know input length, and only relies on end-of-block properties */ + } + ip += length; op = cpy; + } else { + cpy = op+length; + if (endOnInput) { /* LZ4_decompress_safe() */ + DEBUGLOG(7, "copy %u bytes in a 16-bytes stripe", (unsigned)length); + /* We don't need to check oend, since we check it once for each loop below */ + if (ip > iend-(16 + 1/*max lit + offset + nextToken*/)) goto safe_literal_copy; + /* Literals can only be 14, but hope compilers optimize if we copy by a register size */ + memcpy(op, ip, 16); + } else { /* LZ4_decompress_fast() */ + /* LZ4_decompress_fast() cannot copy more than 8 bytes at a time : + * it doesn't know input length, and relies on end-of-block properties */ + memcpy(op, ip, 8); + if (length > 8) memcpy(op+8, ip+8, 8); + } + ip += length; op = cpy; + } + + /* get offset */ + offset = LZ4_readLE16(ip); ip+=2; + match = op - offset; + + /* get matchlength */ + length = token & ML_MASK; + + if ((checkOffset) && (unlikely(match + dictSize < lowPrefix))) goto _output_error; /* Error : offset outside buffers */ + + if (length == ML_MASK) { + variable_length_error error = ok; + length += read_variable_length(&ip, iend - LASTLITERALS + 1, endOnInput, 0, &error); + if (error != ok) goto _output_error; + if ((safeDecode) && unlikely((uptrval)(op)+length<(uptrval)op)) goto _output_error; /* overflow detection */ + length += MINMATCH; + if (op + length >= oend - FASTLOOP_SAFE_DISTANCE) { + goto safe_match_copy; + } + } else { + length += MINMATCH; + if (op + length >= oend - FASTLOOP_SAFE_DISTANCE) { + goto safe_match_copy; + } + + /* Fastpath check: Avoids a branch in LZ4_wildCopy32 if true */ + if (!(dict == usingExtDict) || (match >= lowPrefix)) { + if (offset >= 8) { + memcpy(op, match, 8); + memcpy(op+8, match+8, 8); + memcpy(op+16, match+16, 2); + op += length; + continue; + } } } + + /* match starting within external dictionary */ + if ((dict==usingExtDict) && (match < lowPrefix)) { + if (unlikely(op+length > oend-LASTLITERALS)) { + if (partialDecoding) length = MIN(length, (size_t)(oend-op)); + else goto _output_error; /* doesn't respect parsing restriction */ + } + + if (length <= (size_t)(lowPrefix-match)) { + /* match fits entirely within external dictionary : just copy */ + memmove(op, dictEnd - (lowPrefix-match), length); + op += length; + } else { + /* match stretches into both external dictionary and current block */ + size_t const copySize = (size_t)(lowPrefix - match); + size_t const restSize = length - copySize; + memcpy(op, dictEnd - copySize, copySize); + op += copySize; + if (restSize > (size_t)(op - lowPrefix)) { /* overlap copy */ + BYTE* const endOfMatch = op + restSize; + const BYTE* copyFrom = lowPrefix; + while (op < endOfMatch) *op++ = *copyFrom++; + } else { + memcpy(op, lowPrefix, restSize); + op += restSize; + } } + continue; + } + + /* copy match within block */ + cpy = op + length; + + assert((op <= oend) && (oend-op >= 32)); + if (unlikely(offset<16)) { + LZ4_memcpy_using_offset(op, match, cpy, offset); + } else { + LZ4_wildCopy32(op, match, cpy); + } + + op = cpy; /* wildcopy correction */ + } + safe_decode: +#endif + + /* Main Loop : decode remaining sequences where output < FASTLOOP_SAFE_DISTANCE */ + while (1) { + token = *ip++; + length = token >> ML_BITS; /* literal length */ + + assert(!endOnInput || ip <= iend); /* ip < iend before the increment */ + + /* A two-stage shortcut for the most common case: + * 1) If the literal length is 0..14, and there is enough space, + * enter the shortcut and copy 16 bytes on behalf of the literals + * (in the fast mode, only 8 bytes can be safely copied this way). + * 2) Further if the match length is 4..18, copy 18 bytes in a similar + * manner; but we ensure that there's enough space in the output for + * those 18 bytes earlier, upon entering the shortcut (in other words, + * there is a combined check for both stages). + */ + if ( (endOnInput ? length != RUN_MASK : length <= 8) + /* strictly "less than" on input, to re-enter the loop with at least one byte */ + && likely((endOnInput ? ip < shortiend : 1) & (op <= shortoend)) ) { + /* Copy the literals */ + memcpy(op, ip, endOnInput ? 16 : 8); + op += length; ip += length; + + /* The second stage: prepare for match copying, decode full info. + * If it doesn't work out, the info won't be wasted. */ + length = token & ML_MASK; /* match length */ + offset = LZ4_readLE16(ip); ip += 2; + match = op - offset; + assert(match <= op); /* check overflow */ + + /* Do not deal with overlapping matches. */ + if ( (length != ML_MASK) + && (offset >= 8) + && (dict==withPrefix64k || match >= lowPrefix) ) { + /* Copy the match. */ + memcpy(op + 0, match + 0, 8); + memcpy(op + 8, match + 8, 8); + memcpy(op +16, match +16, 2); + op += length + MINMATCH; + /* Both stages worked, load the next token. */ + continue; + } + + /* The second stage didn't work out, but the info is ready. + * Propel it right to the point of match copying. */ + goto _copy_match; + } + + /* decode literal length */ + if (length == RUN_MASK) { + variable_length_error error = ok; + length += read_variable_length(&ip, iend-RUN_MASK, endOnInput, endOnInput, &error); + if (error == initial_error) goto _output_error; + if ((safeDecode) && unlikely((uptrval)(op)+length<(uptrval)(op))) goto _output_error; /* overflow detection */ + if ((safeDecode) && unlikely((uptrval)(ip)+length<(uptrval)(ip))) goto _output_error; /* overflow detection */ + } + + /* copy literals */ + cpy = op+length; +#if LZ4_FAST_DEC_LOOP + safe_literal_copy: +#endif + LZ4_STATIC_ASSERT(MFLIMIT >= WILDCOPYLENGTH); + if ( ((endOnInput) && ((cpy>oend-MFLIMIT) || (ip+length>iend-(2+1+LASTLITERALS))) ) + || ((!endOnInput) && (cpy>oend-WILDCOPYLENGTH)) ) + { + if (partialDecoding) { + if (cpy > oend) { cpy = oend; assert(op<=oend); length = (size_t)(oend-op); } /* Partial decoding : stop in the middle of literal segment */ + if ((endOnInput) && (ip+length > iend)) goto _output_error; /* Error : read attempt beyond end of input buffer */ + } else { + if ((!endOnInput) && (cpy != oend)) goto _output_error; /* Error : block decoding must stop exactly there */ + if ((endOnInput) && ((ip+length != iend) || (cpy > oend))) goto _output_error; /* Error : input must be consumed */ + } + memcpy(op, ip, length); + ip += length; + op += length; + if (!partialDecoding || (cpy == oend)) { + /* Necessarily EOF, due to parsing restrictions */ + break; + } + + } else { + LZ4_wildCopy8(op, ip, cpy); /* may overwrite up to WILDCOPYLENGTH beyond cpy */ + ip += length; op = cpy; + } + + /* get offset */ + offset = LZ4_readLE16(ip); ip+=2; + match = op - offset; + + /* get matchlength */ + length = token & ML_MASK; + + _copy_match: + if ((checkOffset) && (unlikely(match + dictSize < lowPrefix))) goto _output_error; /* Error : offset outside buffers */ + if (!partialDecoding) { + assert(oend > op); + assert(oend - op >= 4); + LZ4_write32(op, 0); /* silence an msan warning when offset==0; costs <1%; */ + } /* note : when partialDecoding, there is no guarantee that at least 4 bytes remain available in output buffer */ + + if (length == ML_MASK) { + variable_length_error error = ok; + length += read_variable_length(&ip, iend - LASTLITERALS + 1, endOnInput, 0, &error); + if (error != ok) goto _output_error; + if ((safeDecode) && unlikely((uptrval)(op)+length<(uptrval)op)) goto _output_error; /* overflow detection */ + } + length += MINMATCH; + +#if LZ4_FAST_DEC_LOOP + safe_match_copy: +#endif + /* match starting within external dictionary */ + if ((dict==usingExtDict) && (match < lowPrefix)) { + if (unlikely(op+length > oend-LASTLITERALS)) { + if (partialDecoding) length = MIN(length, (size_t)(oend-op)); + else goto _output_error; /* doesn't respect parsing restriction */ + } + + if (length <= (size_t)(lowPrefix-match)) { + /* match fits entirely within external dictionary : just copy */ + memmove(op, dictEnd - (lowPrefix-match), length); + op += length; + } else { + /* match stretches into both external dictionary and current block */ + size_t const copySize = (size_t)(lowPrefix - match); + size_t const restSize = length - copySize; + memcpy(op, dictEnd - copySize, copySize); + op += copySize; + if (restSize > (size_t)(op - lowPrefix)) { /* overlap copy */ + BYTE* const endOfMatch = op + restSize; + const BYTE* copyFrom = lowPrefix; + while (op < endOfMatch) *op++ = *copyFrom++; + } else { + memcpy(op, lowPrefix, restSize); + op += restSize; + } } + continue; + } + + /* copy match within block */ + cpy = op + length; + + /* partialDecoding : may end anywhere within the block */ + assert(op<=oend); + if (partialDecoding && (cpy > oend-MATCH_SAFEGUARD_DISTANCE)) { + size_t const mlen = MIN(length, (size_t)(oend-op)); + const BYTE* const matchEnd = match + mlen; + BYTE* const copyEnd = op + mlen; + if (matchEnd > op) { /* overlap copy */ + while (op < copyEnd) *op++ = *match++; + } else { + memcpy(op, match, mlen); + } + op = copyEnd; + if (op==oend) break; + continue; + } + + if (unlikely(offset<8)) { + op[0] = match[0]; + op[1] = match[1]; + op[2] = match[2]; + op[3] = match[3]; + match += inc32table[offset]; + memcpy(op+4, match, 4); + match -= dec64table[offset]; + } else { + memcpy(op, match, 8); + match += 8; + } + op += 8; + + if (unlikely(cpy > oend-MATCH_SAFEGUARD_DISTANCE)) { + BYTE* const oCopyLimit = oend - (WILDCOPYLENGTH-1); + if (cpy > oend-LASTLITERALS) goto _output_error; /* Error : last LASTLITERALS bytes must be literals (uncompressed) */ + if (op < oCopyLimit) { + LZ4_wildCopy8(op, match, oCopyLimit); + match += oCopyLimit - op; + op = oCopyLimit; + } + while (op < cpy) *op++ = *match++; + } else { + memcpy(op, match, 8); + if (length > 16) LZ4_wildCopy8(op+8, match+8, cpy); + } + op = cpy; /* wildcopy correction */ + } + + /* end of decoding */ + if (endOnInput) + return (int) (((char*)op)-dst); /* Nb of output bytes decoded */ + else + return (int) (((const char*)ip)-src); /* Nb of input bytes read */ + + /* Overflow error detected */ + _output_error: + return (int) (-(((const char*)ip)-src))-1; + } +} + + +/*===== Instantiate the API decoding functions. =====*/ + +LZ4_FORCE_O2_GCC_PPC64LE +int LZ4_decompress_safe(const char* source, char* dest, int compressedSize, int maxDecompressedSize) +{ + return LZ4_decompress_generic(source, dest, compressedSize, maxDecompressedSize, + endOnInputSize, decode_full_block, noDict, + (BYTE*)dest, NULL, 0); +} + +LZ4_FORCE_O2_GCC_PPC64LE +int LZ4_decompress_safe_partial(const char* src, char* dst, int compressedSize, int targetOutputSize, int dstCapacity) +{ + dstCapacity = MIN(targetOutputSize, dstCapacity); + return LZ4_decompress_generic(src, dst, compressedSize, dstCapacity, + endOnInputSize, partial_decode, + noDict, (BYTE*)dst, NULL, 0); +} + +LZ4_FORCE_O2_GCC_PPC64LE +int LZ4_decompress_fast(const char* source, char* dest, int originalSize) +{ + return LZ4_decompress_generic(source, dest, 0, originalSize, + endOnOutputSize, decode_full_block, withPrefix64k, + (BYTE*)dest - 64 KB, NULL, 0); +} + +/*===== Instantiate a few more decoding cases, used more than once. =====*/ + +LZ4_FORCE_O2_GCC_PPC64LE /* Exported, an obsolete API function. */ +int LZ4_decompress_safe_withPrefix64k(const char* source, char* dest, int compressedSize, int maxOutputSize) +{ + return LZ4_decompress_generic(source, dest, compressedSize, maxOutputSize, + endOnInputSize, decode_full_block, withPrefix64k, + (BYTE*)dest - 64 KB, NULL, 0); +} + +/* Another obsolete API function, paired with the previous one. */ +int LZ4_decompress_fast_withPrefix64k(const char* source, char* dest, int originalSize) +{ + /* LZ4_decompress_fast doesn't validate match offsets, + * and thus serves well with any prefixed dictionary. */ + return LZ4_decompress_fast(source, dest, originalSize); +} + +LZ4_FORCE_O2_GCC_PPC64LE +static int LZ4_decompress_safe_withSmallPrefix(const char* source, char* dest, int compressedSize, int maxOutputSize, + size_t prefixSize) +{ + return LZ4_decompress_generic(source, dest, compressedSize, maxOutputSize, + endOnInputSize, decode_full_block, noDict, + (BYTE*)dest-prefixSize, NULL, 0); +} + +LZ4_FORCE_O2_GCC_PPC64LE +int LZ4_decompress_safe_forceExtDict(const char* source, char* dest, + int compressedSize, int maxOutputSize, + const void* dictStart, size_t dictSize) +{ + return LZ4_decompress_generic(source, dest, compressedSize, maxOutputSize, + endOnInputSize, decode_full_block, usingExtDict, + (BYTE*)dest, (const BYTE*)dictStart, dictSize); +} + +LZ4_FORCE_O2_GCC_PPC64LE +static int LZ4_decompress_fast_extDict(const char* source, char* dest, int originalSize, + const void* dictStart, size_t dictSize) +{ + return LZ4_decompress_generic(source, dest, 0, originalSize, + endOnOutputSize, decode_full_block, usingExtDict, + (BYTE*)dest, (const BYTE*)dictStart, dictSize); +} + +/* The "double dictionary" mode, for use with e.g. ring buffers: the first part + * of the dictionary is passed as prefix, and the second via dictStart + dictSize. + * These routines are used only once, in LZ4_decompress_*_continue(). + */ +LZ4_FORCE_INLINE +int LZ4_decompress_safe_doubleDict(const char* source, char* dest, int compressedSize, int maxOutputSize, + size_t prefixSize, const void* dictStart, size_t dictSize) +{ + return LZ4_decompress_generic(source, dest, compressedSize, maxOutputSize, + endOnInputSize, decode_full_block, usingExtDict, + (BYTE*)dest-prefixSize, (const BYTE*)dictStart, dictSize); +} + +LZ4_FORCE_INLINE +int LZ4_decompress_fast_doubleDict(const char* source, char* dest, int originalSize, + size_t prefixSize, const void* dictStart, size_t dictSize) +{ + return LZ4_decompress_generic(source, dest, 0, originalSize, + endOnOutputSize, decode_full_block, usingExtDict, + (BYTE*)dest-prefixSize, (const BYTE*)dictStart, dictSize); +} + +/*===== streaming decompression functions =====*/ + +LZ4_streamDecode_t* LZ4_createStreamDecode(void) +{ + LZ4_streamDecode_t* lz4s = (LZ4_streamDecode_t*) ALLOC_AND_ZERO(sizeof(LZ4_streamDecode_t)); + LZ4_STATIC_ASSERT(LZ4_STREAMDECODESIZE >= sizeof(LZ4_streamDecode_t_internal)); /* A compilation error here means LZ4_STREAMDECODESIZE is not large enough */ + return lz4s; +} + +int LZ4_freeStreamDecode (LZ4_streamDecode_t* LZ4_stream) +{ + if (LZ4_stream == NULL) return 0; /* support free on NULL */ + FREEMEM(LZ4_stream); + return 0; +} + +/*! LZ4_setStreamDecode() : + * Use this function to instruct where to find the dictionary. + * This function is not necessary if previous data is still available where it was decoded. + * Loading a size of 0 is allowed (same effect as no dictionary). + * @return : 1 if OK, 0 if error + */ +int LZ4_setStreamDecode (LZ4_streamDecode_t* LZ4_streamDecode, const char* dictionary, int dictSize) +{ + LZ4_streamDecode_t_internal* lz4sd = &LZ4_streamDecode->internal_donotuse; + lz4sd->prefixSize = (size_t) dictSize; + lz4sd->prefixEnd = (const BYTE*) dictionary + dictSize; + lz4sd->externalDict = NULL; + lz4sd->extDictSize = 0; + return 1; +} + +/*! LZ4_decoderRingBufferSize() : + * when setting a ring buffer for streaming decompression (optional scenario), + * provides the minimum size of this ring buffer + * to be compatible with any source respecting maxBlockSize condition. + * Note : in a ring buffer scenario, + * blocks are presumed decompressed next to each other. + * When not enough space remains for next block (remainingSize < maxBlockSize), + * decoding resumes from beginning of ring buffer. + * @return : minimum ring buffer size, + * or 0 if there is an error (invalid maxBlockSize). + */ +int LZ4_decoderRingBufferSize(int maxBlockSize) +{ + if (maxBlockSize < 0) return 0; + if (maxBlockSize > LZ4_MAX_INPUT_SIZE) return 0; + if (maxBlockSize < 16) maxBlockSize = 16; + return LZ4_DECODER_RING_BUFFER_SIZE(maxBlockSize); +} + +/* +*_continue() : + These decoding functions allow decompression of multiple blocks in "streaming" mode. + Previously decoded blocks must still be available at the memory position where they were decoded. + If it's not possible, save the relevant part of decoded data into a safe buffer, + and indicate where it stands using LZ4_setStreamDecode() +*/ +LZ4_FORCE_O2_GCC_PPC64LE +int LZ4_decompress_safe_continue (LZ4_streamDecode_t* LZ4_streamDecode, const char* source, char* dest, int compressedSize, int maxOutputSize) +{ + LZ4_streamDecode_t_internal* lz4sd = &LZ4_streamDecode->internal_donotuse; + int result; + + if (lz4sd->prefixSize == 0) { + /* The first call, no dictionary yet. */ + assert(lz4sd->extDictSize == 0); + result = LZ4_decompress_safe(source, dest, compressedSize, maxOutputSize); + if (result <= 0) return result; + lz4sd->prefixSize = (size_t)result; + lz4sd->prefixEnd = (BYTE*)dest + result; + } else if (lz4sd->prefixEnd == (BYTE*)dest) { + /* They're rolling the current segment. */ + if (lz4sd->prefixSize >= 64 KB - 1) + result = LZ4_decompress_safe_withPrefix64k(source, dest, compressedSize, maxOutputSize); + else if (lz4sd->extDictSize == 0) + result = LZ4_decompress_safe_withSmallPrefix(source, dest, compressedSize, maxOutputSize, + lz4sd->prefixSize); + else + result = LZ4_decompress_safe_doubleDict(source, dest, compressedSize, maxOutputSize, + lz4sd->prefixSize, lz4sd->externalDict, lz4sd->extDictSize); + if (result <= 0) return result; + lz4sd->prefixSize += (size_t)result; + lz4sd->prefixEnd += result; + } else { + /* The buffer wraps around, or they're switching to another buffer. */ + lz4sd->extDictSize = lz4sd->prefixSize; + lz4sd->externalDict = lz4sd->prefixEnd - lz4sd->extDictSize; + result = LZ4_decompress_safe_forceExtDict(source, dest, compressedSize, maxOutputSize, + lz4sd->externalDict, lz4sd->extDictSize); + if (result <= 0) return result; + lz4sd->prefixSize = (size_t)result; + lz4sd->prefixEnd = (BYTE*)dest + result; + } + + return result; +} + +LZ4_FORCE_O2_GCC_PPC64LE +int LZ4_decompress_fast_continue (LZ4_streamDecode_t* LZ4_streamDecode, const char* source, char* dest, int originalSize) +{ + LZ4_streamDecode_t_internal* lz4sd = &LZ4_streamDecode->internal_donotuse; + int result; + assert(originalSize >= 0); + + if (lz4sd->prefixSize == 0) { + assert(lz4sd->extDictSize == 0); + result = LZ4_decompress_fast(source, dest, originalSize); + if (result <= 0) return result; + lz4sd->prefixSize = (size_t)originalSize; + lz4sd->prefixEnd = (BYTE*)dest + originalSize; + } else if (lz4sd->prefixEnd == (BYTE*)dest) { + if (lz4sd->prefixSize >= 64 KB - 1 || lz4sd->extDictSize == 0) + result = LZ4_decompress_fast(source, dest, originalSize); + else + result = LZ4_decompress_fast_doubleDict(source, dest, originalSize, + lz4sd->prefixSize, lz4sd->externalDict, lz4sd->extDictSize); + if (result <= 0) return result; + lz4sd->prefixSize += (size_t)originalSize; + lz4sd->prefixEnd += originalSize; + } else { + lz4sd->extDictSize = lz4sd->prefixSize; + lz4sd->externalDict = lz4sd->prefixEnd - lz4sd->extDictSize; + result = LZ4_decompress_fast_extDict(source, dest, originalSize, + lz4sd->externalDict, lz4sd->extDictSize); + if (result <= 0) return result; + lz4sd->prefixSize = (size_t)originalSize; + lz4sd->prefixEnd = (BYTE*)dest + originalSize; + } + + return result; +} + + +/* +Advanced decoding functions : +*_usingDict() : + These decoding functions work the same as "_continue" ones, + the dictionary must be explicitly provided within parameters +*/ + +int LZ4_decompress_safe_usingDict(const char* source, char* dest, int compressedSize, int maxOutputSize, const char* dictStart, int dictSize) +{ + if (dictSize==0) + return LZ4_decompress_safe(source, dest, compressedSize, maxOutputSize); + if (dictStart+dictSize == dest) { + if (dictSize >= 64 KB - 1) + return LZ4_decompress_safe_withPrefix64k(source, dest, compressedSize, maxOutputSize); + return LZ4_decompress_safe_withSmallPrefix(source, dest, compressedSize, maxOutputSize, dictSize); + } + return LZ4_decompress_safe_forceExtDict(source, dest, compressedSize, maxOutputSize, dictStart, dictSize); +} + +int LZ4_decompress_fast_usingDict(const char* source, char* dest, int originalSize, const char* dictStart, int dictSize) +{ + if (dictSize==0 || dictStart+dictSize == dest) + return LZ4_decompress_fast(source, dest, originalSize); + return LZ4_decompress_fast_extDict(source, dest, originalSize, dictStart, dictSize); +} + + +/*=************************************************* +* Obsolete Functions +***************************************************/ +/* obsolete compression functions */ +int LZ4_compress_limitedOutput(const char* source, char* dest, int inputSize, int maxOutputSize) +{ + return LZ4_compress_default(source, dest, inputSize, maxOutputSize); +} +int LZ4_compress(const char* source, char* dest, int inputSize) +{ + return LZ4_compress_default(source, dest, inputSize, LZ4_compressBound(inputSize)); +} +int LZ4_compress_limitedOutput_withState (void* state, const char* src, char* dst, int srcSize, int dstSize) +{ + return LZ4_compress_fast_extState(state, src, dst, srcSize, dstSize, 1); +} +int LZ4_compress_withState (void* state, const char* src, char* dst, int srcSize) +{ + return LZ4_compress_fast_extState(state, src, dst, srcSize, LZ4_compressBound(srcSize), 1); +} +int LZ4_compress_limitedOutput_continue (LZ4_stream_t* LZ4_stream, const char* src, char* dst, int srcSize, int dstCapacity) +{ + return LZ4_compress_fast_continue(LZ4_stream, src, dst, srcSize, dstCapacity, 1); +} +int LZ4_compress_continue (LZ4_stream_t* LZ4_stream, const char* source, char* dest, int inputSize) +{ + return LZ4_compress_fast_continue(LZ4_stream, source, dest, inputSize, LZ4_compressBound(inputSize), 1); +} + +/* +These decompression functions are deprecated and should no longer be used. +They are only provided here for compatibility with older user programs. +- LZ4_uncompress is totally equivalent to LZ4_decompress_fast +- LZ4_uncompress_unknownOutputSize is totally equivalent to LZ4_decompress_safe +*/ +int LZ4_uncompress (const char* source, char* dest, int outputSize) +{ + return LZ4_decompress_fast(source, dest, outputSize); +} +int LZ4_uncompress_unknownOutputSize (const char* source, char* dest, int isize, int maxOutputSize) +{ + return LZ4_decompress_safe(source, dest, isize, maxOutputSize); +} + +/* Obsolete Streaming functions */ + +int LZ4_sizeofStreamState() { return LZ4_STREAMSIZE; } + +int LZ4_resetStreamState(void* state, char* inputBuffer) +{ + (void)inputBuffer; + LZ4_resetStream((LZ4_stream_t*)state); + return 0; +} + +void* LZ4_create (char* inputBuffer) +{ + (void)inputBuffer; + return LZ4_createStream(); +} + +char* LZ4_slideInputBuffer (void* state) +{ + /* avoid const char * -> char * conversion warning */ + return (char *)(uptrval)((LZ4_stream_t*)state)->internal_donotuse.dictionary; +} + +#endif /* LZ4_COMMONDEFS_ONLY */ diff --git a/source/lz4.h b/source/lz4.h new file mode 100644 index 0000000..a9c932c --- /dev/null +++ b/source/lz4.h @@ -0,0 +1,682 @@ +/* + * LZ4 - Fast LZ compression algorithm + * Header File + * Copyright (C) 2011-present, Yann Collet. + + BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php) + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions are + met: + + * Redistributions of source code must retain the above copyright + notice, this list of conditions and the following disclaimer. + * Redistributions in binary form must reproduce the above + copyright notice, this list of conditions and the following disclaimer + in the documentation and/or other materials provided with the + distribution. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + + You can contact the author at : + - LZ4 homepage : http://www.lz4.org + - LZ4 source repository : https://github.com/lz4/lz4 +*/ +#if defined (__cplusplus) +extern "C" { +#endif + +#ifndef LZ4_H_2983827168210 +#define LZ4_H_2983827168210 + +/* --- Dependency --- */ +#include /* size_t */ + + +/** + Introduction + + LZ4 is lossless compression algorithm, providing compression speed at 500 MB/s per core, + scalable with multi-cores CPU. It features an extremely fast decoder, with speed in + multiple GB/s per core, typically reaching RAM speed limits on multi-core systems. + + The LZ4 compression library provides in-memory compression and decompression functions. + It gives full buffer control to user. + Compression can be done in: + - a single step (described as Simple Functions) + - a single step, reusing a context (described in Advanced Functions) + - unbounded multiple steps (described as Streaming compression) + + lz4.h generates and decodes LZ4-compressed blocks (doc/lz4_Block_format.md). + Decompressing a block requires additional metadata, such as its compressed size. + Each application is free to encode and pass such metadata in whichever way it wants. + + lz4.h only handle blocks, it can not generate Frames. + + Blocks are different from Frames (doc/lz4_Frame_format.md). + Frames bundle both blocks and metadata in a specified manner. + This are required for compressed data to be self-contained and portable. + Frame format is delivered through a companion API, declared in lz4frame.h. + Note that the `lz4` CLI can only manage frames. +*/ + +/*^*************************************************************** +* Export parameters +*****************************************************************/ +/* +* LZ4_DLL_EXPORT : +* Enable exporting of functions when building a Windows DLL +* LZ4LIB_VISIBILITY : +* Control library symbols visibility. +*/ +#ifndef LZ4LIB_VISIBILITY +# if defined(__GNUC__) && (__GNUC__ >= 4) +# define LZ4LIB_VISIBILITY __attribute__ ((visibility ("default"))) +# else +# define LZ4LIB_VISIBILITY +# endif +#endif +#if defined(LZ4_DLL_EXPORT) && (LZ4_DLL_EXPORT==1) +# define LZ4LIB_API __declspec(dllexport) LZ4LIB_VISIBILITY +#elif defined(LZ4_DLL_IMPORT) && (LZ4_DLL_IMPORT==1) +# define LZ4LIB_API __declspec(dllimport) LZ4LIB_VISIBILITY /* It isn't required but allows to generate better code, saving a function pointer load from the IAT and an indirect jump.*/ +#else +# define LZ4LIB_API LZ4LIB_VISIBILITY +#endif + +/*------ Version ------*/ +#define LZ4_VERSION_MAJOR 1 /* for breaking interface changes */ +#define LZ4_VERSION_MINOR 9 /* for new (non-breaking) interface capabilities */ +#define LZ4_VERSION_RELEASE 1 /* for tweaks, bug-fixes, or development */ + +#define LZ4_VERSION_NUMBER (LZ4_VERSION_MAJOR *100*100 + LZ4_VERSION_MINOR *100 + LZ4_VERSION_RELEASE) + +#define LZ4_LIB_VERSION LZ4_VERSION_MAJOR.LZ4_VERSION_MINOR.LZ4_VERSION_RELEASE +#define LZ4_QUOTE(str) #str +#define LZ4_EXPAND_AND_QUOTE(str) LZ4_QUOTE(str) +#define LZ4_VERSION_STRING LZ4_EXPAND_AND_QUOTE(LZ4_LIB_VERSION) + +LZ4LIB_API int LZ4_versionNumber (void); /**< library version number; useful to check dll version */ +LZ4LIB_API const char* LZ4_versionString (void); /**< library version string; useful to check dll version */ + + +/*-************************************ +* Tuning parameter +**************************************/ +/*! + * LZ4_MEMORY_USAGE : + * Memory usage formula : N->2^N Bytes (examples : 10 -> 1KB; 12 -> 4KB ; 16 -> 64KB; 20 -> 1MB; etc.) + * Increasing memory usage improves compression ratio. + * Reduced memory usage may improve speed, thanks to better cache locality. + * Default value is 14, for 16KB, which nicely fits into Intel x86 L1 cache + */ +#ifndef LZ4_MEMORY_USAGE +# define LZ4_MEMORY_USAGE 14 +#endif + + +/*-************************************ +* Simple Functions +**************************************/ +/*! LZ4_compress_default() : + Compresses 'srcSize' bytes from buffer 'src' + into already allocated 'dst' buffer of size 'dstCapacity'. + Compression is guaranteed to succeed if 'dstCapacity' >= LZ4_compressBound(srcSize). + It also runs faster, so it's a recommended setting. + If the function cannot compress 'src' into a more limited 'dst' budget, + compression stops *immediately*, and the function result is zero. + In which case, 'dst' content is undefined (invalid). + srcSize : max supported value is LZ4_MAX_INPUT_SIZE. + dstCapacity : size of buffer 'dst' (which must be already allocated) + @return : the number of bytes written into buffer 'dst' (necessarily <= dstCapacity) + or 0 if compression fails + Note : This function is protected against buffer overflow scenarios (never writes outside 'dst' buffer, nor read outside 'source' buffer). +*/ +LZ4LIB_API int LZ4_compress_default(const char* src, char* dst, int srcSize, int dstCapacity); + +/*! LZ4_decompress_safe() : + compressedSize : is the exact complete size of the compressed block. + dstCapacity : is the size of destination buffer, which must be already allocated. + @return : the number of bytes decompressed into destination buffer (necessarily <= dstCapacity) + If destination buffer is not large enough, decoding will stop and output an error code (negative value). + If the source stream is detected malformed, the function will stop decoding and return a negative result. + Note : This function is protected against malicious data packets (never writes outside 'dst' buffer, nor read outside 'source' buffer). +*/ +LZ4LIB_API int LZ4_decompress_safe (const char* src, char* dst, int compressedSize, int dstCapacity); + + +/*-************************************ +* Advanced Functions +**************************************/ +#define LZ4_MAX_INPUT_SIZE 0x7E000000 /* 2 113 929 216 bytes */ +#define LZ4_COMPRESSBOUND(isize) ((unsigned)(isize) > (unsigned)LZ4_MAX_INPUT_SIZE ? 0 : (isize) + ((isize)/255) + 16) + +/*! LZ4_compressBound() : + Provides the maximum size that LZ4 compression may output in a "worst case" scenario (input data not compressible) + This function is primarily useful for memory allocation purposes (destination buffer size). + Macro LZ4_COMPRESSBOUND() is also provided for compilation-time evaluation (stack memory allocation for example). + Note that LZ4_compress_default() compresses faster when dstCapacity is >= LZ4_compressBound(srcSize) + inputSize : max supported value is LZ4_MAX_INPUT_SIZE + return : maximum output size in a "worst case" scenario + or 0, if input size is incorrect (too large or negative) +*/ +LZ4LIB_API int LZ4_compressBound(int inputSize); + +/*! LZ4_compress_fast() : + Same as LZ4_compress_default(), but allows selection of "acceleration" factor. + The larger the acceleration value, the faster the algorithm, but also the lesser the compression. + It's a trade-off. It can be fine tuned, with each successive value providing roughly +~3% to speed. + An acceleration value of "1" is the same as regular LZ4_compress_default() + Values <= 0 will be replaced by ACCELERATION_DEFAULT (currently == 1, see lz4.c). +*/ +LZ4LIB_API int LZ4_compress_fast (const char* src, char* dst, int srcSize, int dstCapacity, int acceleration); + + +/*! LZ4_compress_fast_extState() : + * Same as LZ4_compress_fast(), using an externally allocated memory space for its state. + * Use LZ4_sizeofState() to know how much memory must be allocated, + * and allocate it on 8-bytes boundaries (using `malloc()` typically). + * Then, provide this buffer as `void* state` to compression function. + */ +LZ4LIB_API int LZ4_sizeofState(void); +LZ4LIB_API int LZ4_compress_fast_extState (void* state, const char* src, char* dst, int srcSize, int dstCapacity, int acceleration); + + +/*! LZ4_compress_destSize() : + * Reverse the logic : compresses as much data as possible from 'src' buffer + * into already allocated buffer 'dst', of size >= 'targetDestSize'. + * This function either compresses the entire 'src' content into 'dst' if it's large enough, + * or fill 'dst' buffer completely with as much data as possible from 'src'. + * note: acceleration parameter is fixed to "default". + * + * *srcSizePtr : will be modified to indicate how many bytes where read from 'src' to fill 'dst'. + * New value is necessarily <= input value. + * @return : Nb bytes written into 'dst' (necessarily <= targetDestSize) + * or 0 if compression fails. +*/ +LZ4LIB_API int LZ4_compress_destSize (const char* src, char* dst, int* srcSizePtr, int targetDstSize); + + +/*! LZ4_decompress_safe_partial() : + * Decompress an LZ4 compressed block, of size 'srcSize' at position 'src', + * into destination buffer 'dst' of size 'dstCapacity'. + * Up to 'targetOutputSize' bytes will be decoded. + * The function stops decoding on reaching this objective, + * which can boost performance when only the beginning of a block is required. + * + * @return : the number of bytes decoded in `dst` (necessarily <= dstCapacity) + * If source stream is detected malformed, function returns a negative result. + * + * Note : @return can be < targetOutputSize, if compressed block contains less data. + * + * Note 2 : this function features 2 parameters, targetOutputSize and dstCapacity, + * and expects targetOutputSize <= dstCapacity. + * It effectively stops decoding on reaching targetOutputSize, + * so dstCapacity is kind of redundant. + * This is because in a previous version of this function, + * decoding operation would not "break" a sequence in the middle. + * As a consequence, there was no guarantee that decoding would stop at exactly targetOutputSize, + * it could write more bytes, though only up to dstCapacity. + * Some "margin" used to be required for this operation to work properly. + * This is no longer necessary. + * The function nonetheless keeps its signature, in an effort to not break API. + */ +LZ4LIB_API int LZ4_decompress_safe_partial (const char* src, char* dst, int srcSize, int targetOutputSize, int dstCapacity); + + +/*-********************************************* +* Streaming Compression Functions +***********************************************/ +typedef union LZ4_stream_u LZ4_stream_t; /* incomplete type (defined later) */ + +LZ4LIB_API LZ4_stream_t* LZ4_createStream(void); +LZ4LIB_API int LZ4_freeStream (LZ4_stream_t* streamPtr); + +/*! LZ4_resetStream_fast() : v1.9.0+ + * Use this to prepare an LZ4_stream_t for a new chain of dependent blocks + * (e.g., LZ4_compress_fast_continue()). + * + * An LZ4_stream_t must be initialized once before usage. + * This is automatically done when created by LZ4_createStream(). + * However, should the LZ4_stream_t be simply declared on stack (for example), + * it's necessary to initialize it first, using LZ4_initStream(). + * + * After init, start any new stream with LZ4_resetStream_fast(). + * A same LZ4_stream_t can be re-used multiple times consecutively + * and compress multiple streams, + * provided that it starts each new stream with LZ4_resetStream_fast(). + * + * LZ4_resetStream_fast() is much faster than LZ4_initStream(), + * but is not compatible with memory regions containing garbage data. + * + * Note: it's only useful to call LZ4_resetStream_fast() + * in the context of streaming compression. + * The *extState* functions perform their own resets. + * Invoking LZ4_resetStream_fast() before is redundant, and even counterproductive. + */ +LZ4LIB_API void LZ4_resetStream_fast (LZ4_stream_t* streamPtr); + +/*! LZ4_loadDict() : + * Use this function to reference a static dictionary into LZ4_stream_t. + * The dictionary must remain available during compression. + * LZ4_loadDict() triggers a reset, so any previous data will be forgotten. + * The same dictionary will have to be loaded on decompression side for successful decoding. + * Dictionary are useful for better compression of small data (KB range). + * While LZ4 accept any input as dictionary, + * results are generally better when using Zstandard's Dictionary Builder. + * Loading a size of 0 is allowed, and is the same as reset. + * @return : loaded dictionary size, in bytes (necessarily <= 64 KB) + */ +LZ4LIB_API int LZ4_loadDict (LZ4_stream_t* streamPtr, const char* dictionary, int dictSize); + +/*! LZ4_compress_fast_continue() : + * Compress 'src' content using data from previously compressed blocks, for better compression ratio. + * 'dst' buffer must be already allocated. + * If dstCapacity >= LZ4_compressBound(srcSize), compression is guaranteed to succeed, and runs faster. + * + * @return : size of compressed block + * or 0 if there is an error (typically, cannot fit into 'dst'). + * + * Note 1 : Each invocation to LZ4_compress_fast_continue() generates a new block. + * Each block has precise boundaries. + * Each block must be decompressed separately, calling LZ4_decompress_*() with relevant metadata. + * It's not possible to append blocks together and expect a single invocation of LZ4_decompress_*() to decompress them together. + * + * Note 2 : The previous 64KB of source data is __assumed__ to remain present, unmodified, at same address in memory ! + * + * Note 3 : When input is structured as a double-buffer, each buffer can have any size, including < 64 KB. + * Make sure that buffers are separated, by at least one byte. + * This construction ensures that each block only depends on previous block. + * + * Note 4 : If input buffer is a ring-buffer, it can have any size, including < 64 KB. + * + * Note 5 : After an error, the stream status is undefined (invalid), it can only be reset or freed. + */ +LZ4LIB_API int LZ4_compress_fast_continue (LZ4_stream_t* streamPtr, const char* src, char* dst, int srcSize, int dstCapacity, int acceleration); + +/*! LZ4_saveDict() : + * If last 64KB data cannot be guaranteed to remain available at its current memory location, + * save it into a safer place (char* safeBuffer). + * This is schematically equivalent to a memcpy() followed by LZ4_loadDict(), + * but is much faster, because LZ4_saveDict() doesn't need to rebuild tables. + * @return : saved dictionary size in bytes (necessarily <= maxDictSize), or 0 if error. + */ +LZ4LIB_API int LZ4_saveDict (LZ4_stream_t* streamPtr, char* safeBuffer, int maxDictSize); + + +/*-********************************************** +* Streaming Decompression Functions +* Bufferless synchronous API +************************************************/ +typedef union LZ4_streamDecode_u LZ4_streamDecode_t; /* tracking context */ + +/*! LZ4_createStreamDecode() and LZ4_freeStreamDecode() : + * creation / destruction of streaming decompression tracking context. + * A tracking context can be re-used multiple times. + */ +LZ4LIB_API LZ4_streamDecode_t* LZ4_createStreamDecode(void); +LZ4LIB_API int LZ4_freeStreamDecode (LZ4_streamDecode_t* LZ4_stream); + +/*! LZ4_setStreamDecode() : + * An LZ4_streamDecode_t context can be allocated once and re-used multiple times. + * Use this function to start decompression of a new stream of blocks. + * A dictionary can optionally be set. Use NULL or size 0 for a reset order. + * Dictionary is presumed stable : it must remain accessible and unmodified during next decompression. + * @return : 1 if OK, 0 if error + */ +LZ4LIB_API int LZ4_setStreamDecode (LZ4_streamDecode_t* LZ4_streamDecode, const char* dictionary, int dictSize); + +/*! LZ4_decoderRingBufferSize() : v1.8.2+ + * Note : in a ring buffer scenario (optional), + * blocks are presumed decompressed next to each other + * up to the moment there is not enough remaining space for next block (remainingSize < maxBlockSize), + * at which stage it resumes from beginning of ring buffer. + * When setting such a ring buffer for streaming decompression, + * provides the minimum size of this ring buffer + * to be compatible with any source respecting maxBlockSize condition. + * @return : minimum ring buffer size, + * or 0 if there is an error (invalid maxBlockSize). + */ +LZ4LIB_API int LZ4_decoderRingBufferSize(int maxBlockSize); +#define LZ4_DECODER_RING_BUFFER_SIZE(maxBlockSize) (65536 + 14 + (maxBlockSize)) /* for static allocation; maxBlockSize presumed valid */ + +/*! LZ4_decompress_*_continue() : + * These decoding functions allow decompression of consecutive blocks in "streaming" mode. + * A block is an unsplittable entity, it must be presented entirely to a decompression function. + * Decompression functions only accepts one block at a time. + * The last 64KB of previously decoded data *must* remain available and unmodified at the memory position where they were decoded. + * If less than 64KB of data has been decoded, all the data must be present. + * + * Special : if decompression side sets a ring buffer, it must respect one of the following conditions : + * - Decompression buffer size is _at least_ LZ4_decoderRingBufferSize(maxBlockSize). + * maxBlockSize is the maximum size of any single block. It can have any value > 16 bytes. + * In which case, encoding and decoding buffers do not need to be synchronized. + * Actually, data can be produced by any source compliant with LZ4 format specification, and respecting maxBlockSize. + * - Synchronized mode : + * Decompression buffer size is _exactly_ the same as compression buffer size, + * and follows exactly same update rule (block boundaries at same positions), + * and decoding function is provided with exact decompressed size of each block (exception for last block of the stream), + * _then_ decoding & encoding ring buffer can have any size, including small ones ( < 64 KB). + * - Decompression buffer is larger than encoding buffer, by a minimum of maxBlockSize more bytes. + * In which case, encoding and decoding buffers do not need to be synchronized, + * and encoding ring buffer can have any size, including small ones ( < 64 KB). + * + * Whenever these conditions are not possible, + * save the last 64KB of decoded data into a safe buffer where it can't be modified during decompression, + * then indicate where this data is saved using LZ4_setStreamDecode(), before decompressing next block. +*/ +LZ4LIB_API int LZ4_decompress_safe_continue (LZ4_streamDecode_t* LZ4_streamDecode, const char* src, char* dst, int srcSize, int dstCapacity); + + +/*! LZ4_decompress_*_usingDict() : + * These decoding functions work the same as + * a combination of LZ4_setStreamDecode() followed by LZ4_decompress_*_continue() + * They are stand-alone, and don't need an LZ4_streamDecode_t structure. + * Dictionary is presumed stable : it must remain accessible and unmodified during decompression. + * Performance tip : Decompression speed can be substantially increased + * when dst == dictStart + dictSize. + */ +LZ4LIB_API int LZ4_decompress_safe_usingDict (const char* src, char* dst, int srcSize, int dstCapcity, const char* dictStart, int dictSize); + + +/*^************************************* + * !!!!!! STATIC LINKING ONLY !!!!!! + ***************************************/ + +/*-**************************************************************************** + * Experimental section + * + * Symbols declared in this section must be considered unstable. Their + * signatures or semantics may change, or they may be removed altogether in the + * future. They are therefore only safe to depend on when the caller is + * statically linked against the library. + * + * To protect against unsafe usage, not only are the declarations guarded, + * the definitions are hidden by default + * when building LZ4 as a shared/dynamic library. + * + * In order to access these declarations, + * define LZ4_STATIC_LINKING_ONLY in your application + * before including LZ4's headers. + * + * In order to make their implementations accessible dynamically, you must + * define LZ4_PUBLISH_STATIC_FUNCTIONS when building the LZ4 library. + ******************************************************************************/ + +#ifdef LZ4_PUBLISH_STATIC_FUNCTIONS +#define LZ4LIB_STATIC_API LZ4LIB_API +#else +#define LZ4LIB_STATIC_API +#endif + +#ifdef LZ4_STATIC_LINKING_ONLY + + +/*! LZ4_compress_fast_extState_fastReset() : + * A variant of LZ4_compress_fast_extState(). + * + * Using this variant avoids an expensive initialization step. + * It is only safe to call if the state buffer is known to be correctly initialized already + * (see above comment on LZ4_resetStream_fast() for a definition of "correctly initialized"). + * From a high level, the difference is that + * this function initializes the provided state with a call to something like LZ4_resetStream_fast() + * while LZ4_compress_fast_extState() starts with a call to LZ4_resetStream(). + */ +LZ4LIB_STATIC_API int LZ4_compress_fast_extState_fastReset (void* state, const char* src, char* dst, int srcSize, int dstCapacity, int acceleration); + +/*! LZ4_attach_dictionary() : + * This is an experimental API that allows + * efficient use of a static dictionary many times. + * + * Rather than re-loading the dictionary buffer into a working context before + * each compression, or copying a pre-loaded dictionary's LZ4_stream_t into a + * working LZ4_stream_t, this function introduces a no-copy setup mechanism, + * in which the working stream references the dictionary stream in-place. + * + * Several assumptions are made about the state of the dictionary stream. + * Currently, only streams which have been prepared by LZ4_loadDict() should + * be expected to work. + * + * Alternatively, the provided dictionaryStream may be NULL, + * in which case any existing dictionary stream is unset. + * + * If a dictionary is provided, it replaces any pre-existing stream history. + * The dictionary contents are the only history that can be referenced and + * logically immediately precede the data compressed in the first subsequent + * compression call. + * + * The dictionary will only remain attached to the working stream through the + * first compression call, at the end of which it is cleared. The dictionary + * stream (and source buffer) must remain in-place / accessible / unchanged + * through the completion of the first compression call on the stream. + */ +LZ4LIB_STATIC_API void LZ4_attach_dictionary(LZ4_stream_t* workingStream, const LZ4_stream_t* dictionaryStream); + +#endif + + +/*-************************************************************ + * PRIVATE DEFINITIONS + ************************************************************** + * Do not use these definitions directly. + * They are only exposed to allow static allocation of `LZ4_stream_t` and `LZ4_streamDecode_t`. + * Accessing members will expose code to API and/or ABI break in future versions of the library. + **************************************************************/ +#define LZ4_HASHLOG (LZ4_MEMORY_USAGE-2) +#define LZ4_HASHTABLESIZE (1 << LZ4_MEMORY_USAGE) +#define LZ4_HASH_SIZE_U32 (1 << LZ4_HASHLOG) /* required as macro for static allocation */ + +#if defined(__cplusplus) || (defined (__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */) +#include + +typedef struct LZ4_stream_t_internal LZ4_stream_t_internal; +struct LZ4_stream_t_internal { + uint32_t hashTable[LZ4_HASH_SIZE_U32]; + uint32_t currentOffset; + uint16_t dirty; + uint16_t tableType; + const uint8_t* dictionary; + const LZ4_stream_t_internal* dictCtx; + uint32_t dictSize; +}; + +typedef struct { + const uint8_t* externalDict; + size_t extDictSize; + const uint8_t* prefixEnd; + size_t prefixSize; +} LZ4_streamDecode_t_internal; + +#else + +typedef struct LZ4_stream_t_internal LZ4_stream_t_internal; +struct LZ4_stream_t_internal { + unsigned int hashTable[LZ4_HASH_SIZE_U32]; + unsigned int currentOffset; + unsigned short dirty; + unsigned short tableType; + const unsigned char* dictionary; + const LZ4_stream_t_internal* dictCtx; + unsigned int dictSize; +}; + +typedef struct { + const unsigned char* externalDict; + const unsigned char* prefixEnd; + size_t extDictSize; + size_t prefixSize; +} LZ4_streamDecode_t_internal; + +#endif + +/*! LZ4_stream_t : + * information structure to track an LZ4 stream. + * LZ4_stream_t can also be created using LZ4_createStream(), which is recommended. + * The structure definition can be convenient for static allocation + * (on stack, or as part of larger structure). + * Init this structure with LZ4_initStream() before first use. + * note : only use this definition in association with static linking ! + * this definition is not API/ABI safe, and may change in a future version. + */ +#define LZ4_STREAMSIZE_U64 ((1 << (LZ4_MEMORY_USAGE-3)) + 4 + ((sizeof(void*)==16) ? 4 : 0) /*AS-400*/ ) +#define LZ4_STREAMSIZE (LZ4_STREAMSIZE_U64 * sizeof(unsigned long long)) +union LZ4_stream_u { + unsigned long long table[LZ4_STREAMSIZE_U64]; + LZ4_stream_t_internal internal_donotuse; +} ; /* previously typedef'd to LZ4_stream_t */ + +/*! LZ4_initStream() : v1.9.0+ + * An LZ4_stream_t structure must be initialized at least once. + * This is automatically done when invoking LZ4_createStream(), + * but it's not when the structure is simply declared on stack (for example). + * + * Use LZ4_initStream() to properly initialize a newly declared LZ4_stream_t. + * It can also initialize any arbitrary buffer of sufficient size, + * and will @return a pointer of proper type upon initialization. + * + * Note : initialization fails if size and alignment conditions are not respected. + * In which case, the function will @return NULL. + * Note2: An LZ4_stream_t structure guarantees correct alignment and size. + * Note3: Before v1.9.0, use LZ4_resetStream() instead + */ +LZ4LIB_API LZ4_stream_t* LZ4_initStream (void* buffer, size_t size); + + +/*! LZ4_streamDecode_t : + * information structure to track an LZ4 stream during decompression. + * init this structure using LZ4_setStreamDecode() before first use. + * note : only use in association with static linking ! + * this definition is not API/ABI safe, + * and may change in a future version ! + */ +#define LZ4_STREAMDECODESIZE_U64 (4 + ((sizeof(void*)==16) ? 2 : 0) /*AS-400*/ ) +#define LZ4_STREAMDECODESIZE (LZ4_STREAMDECODESIZE_U64 * sizeof(unsigned long long)) +union LZ4_streamDecode_u { + unsigned long long table[LZ4_STREAMDECODESIZE_U64]; + LZ4_streamDecode_t_internal internal_donotuse; +} ; /* previously typedef'd to LZ4_streamDecode_t */ + + +/*-************************************ +* Obsolete Functions +**************************************/ + +/*! Deprecation warnings + * + * Deprecated functions make the compiler generate a warning when invoked. + * This is meant to invite users to update their source code. + * Should deprecation warnings be a problem, it is generally possible to disable them, + * typically with -Wno-deprecated-declarations for gcc + * or _CRT_SECURE_NO_WARNINGS in Visual. + * + * Another method is to define LZ4_DISABLE_DEPRECATE_WARNINGS + * before including the header file. + */ +#ifdef LZ4_DISABLE_DEPRECATE_WARNINGS +# define LZ4_DEPRECATED(message) /* disable deprecation warnings */ +#else +# define LZ4_GCC_VERSION (__GNUC__ * 100 + __GNUC_MINOR__) +# if defined (__cplusplus) && (__cplusplus >= 201402) /* C++14 or greater */ +# define LZ4_DEPRECATED(message) [[deprecated(message)]] +# elif (LZ4_GCC_VERSION >= 405) || defined(__clang__) +# define LZ4_DEPRECATED(message) __attribute__((deprecated(message))) +# elif (LZ4_GCC_VERSION >= 301) +# define LZ4_DEPRECATED(message) __attribute__((deprecated)) +# elif defined(_MSC_VER) +# define LZ4_DEPRECATED(message) __declspec(deprecated(message)) +# else +# pragma message("WARNING: You need to implement LZ4_DEPRECATED for this compiler") +# define LZ4_DEPRECATED(message) +# endif +#endif /* LZ4_DISABLE_DEPRECATE_WARNINGS */ + +/* Obsolete compression functions */ +LZ4_DEPRECATED("use LZ4_compress_default() instead") LZ4LIB_API int LZ4_compress (const char* source, char* dest, int sourceSize); +LZ4_DEPRECATED("use LZ4_compress_default() instead") LZ4LIB_API int LZ4_compress_limitedOutput (const char* source, char* dest, int sourceSize, int maxOutputSize); +LZ4_DEPRECATED("use LZ4_compress_fast_extState() instead") LZ4LIB_API int LZ4_compress_withState (void* state, const char* source, char* dest, int inputSize); +LZ4_DEPRECATED("use LZ4_compress_fast_extState() instead") LZ4LIB_API int LZ4_compress_limitedOutput_withState (void* state, const char* source, char* dest, int inputSize, int maxOutputSize); +LZ4_DEPRECATED("use LZ4_compress_fast_continue() instead") LZ4LIB_API int LZ4_compress_continue (LZ4_stream_t* LZ4_streamPtr, const char* source, char* dest, int inputSize); +LZ4_DEPRECATED("use LZ4_compress_fast_continue() instead") LZ4LIB_API int LZ4_compress_limitedOutput_continue (LZ4_stream_t* LZ4_streamPtr, const char* source, char* dest, int inputSize, int maxOutputSize); + +/* Obsolete decompression functions */ +LZ4_DEPRECATED("use LZ4_decompress_fast() instead") LZ4LIB_API int LZ4_uncompress (const char* source, char* dest, int outputSize); +LZ4_DEPRECATED("use LZ4_decompress_safe() instead") LZ4LIB_API int LZ4_uncompress_unknownOutputSize (const char* source, char* dest, int isize, int maxOutputSize); + +/* Obsolete streaming functions; degraded functionality; do not use! + * + * In order to perform streaming compression, these functions depended on data + * that is no longer tracked in the state. They have been preserved as well as + * possible: using them will still produce a correct output. However, they don't + * actually retain any history between compression calls. The compression ratio + * achieved will therefore be no better than compressing each chunk + * independently. + */ +LZ4_DEPRECATED("Use LZ4_createStream() instead") LZ4LIB_API void* LZ4_create (char* inputBuffer); +LZ4_DEPRECATED("Use LZ4_createStream() instead") LZ4LIB_API int LZ4_sizeofStreamState(void); +LZ4_DEPRECATED("Use LZ4_resetStream() instead") LZ4LIB_API int LZ4_resetStreamState(void* state, char* inputBuffer); +LZ4_DEPRECATED("Use LZ4_saveDict() instead") LZ4LIB_API char* LZ4_slideInputBuffer (void* state); + +/* Obsolete streaming decoding functions */ +LZ4_DEPRECATED("use LZ4_decompress_safe_usingDict() instead") LZ4LIB_API int LZ4_decompress_safe_withPrefix64k (const char* src, char* dst, int compressedSize, int maxDstSize); +LZ4_DEPRECATED("use LZ4_decompress_fast_usingDict() instead") LZ4LIB_API int LZ4_decompress_fast_withPrefix64k (const char* src, char* dst, int originalSize); + +/*! LZ4_decompress_fast() : **unsafe!** + * These functions used to be faster than LZ4_decompress_safe(), + * but it has changed, and they are now slower than LZ4_decompress_safe(). + * This is because LZ4_decompress_fast() doesn't know the input size, + * and therefore must progress more cautiously in the input buffer to not read beyond the end of block. + * On top of that `LZ4_decompress_fast()` is not protected vs malformed or malicious inputs, making it a security liability. + * As a consequence, LZ4_decompress_fast() is strongly discouraged, and deprecated. + * + * The last remaining LZ4_decompress_fast() specificity is that + * it can decompress a block without knowing its compressed size. + * Such functionality could be achieved in a more secure manner, + * by also providing the maximum size of input buffer, + * but it would require new prototypes, and adaptation of the implementation to this new use case. + * + * Parameters: + * originalSize : is the uncompressed size to regenerate. + * `dst` must be already allocated, its size must be >= 'originalSize' bytes. + * @return : number of bytes read from source buffer (== compressed size). + * The function expects to finish at block's end exactly. + * If the source stream is detected malformed, the function stops decoding and returns a negative result. + * note : LZ4_decompress_fast*() requires originalSize. Thanks to this information, it never writes past the output buffer. + * However, since it doesn't know its 'src' size, it may read an unknown amount of input, past input buffer bounds. + * Also, since match offsets are not validated, match reads from 'src' may underflow too. + * These issues never happen if input (compressed) data is correct. + * But they may happen if input data is invalid (error or intentional tampering). + * As a consequence, use these functions in trusted environments with trusted data **only**. + */ + +LZ4_DEPRECATED("This function is deprecated and unsafe. Consider using LZ4_decompress_safe() instead") +LZ4LIB_API int LZ4_decompress_fast (const char* src, char* dst, int originalSize); +LZ4_DEPRECATED("This function is deprecated and unsafe. Consider using LZ4_decompress_safe_continue() instead") +LZ4LIB_API int LZ4_decompress_fast_continue (LZ4_streamDecode_t* LZ4_streamDecode, const char* src, char* dst, int originalSize); +LZ4_DEPRECATED("This function is deprecated and unsafe. Consider using LZ4_decompress_safe_usingDict() instead") +LZ4LIB_API int LZ4_decompress_fast_usingDict (const char* src, char* dst, int originalSize, const char* dictStart, int dictSize); + +/*! LZ4_resetStream() : + * An LZ4_stream_t structure must be initialized at least once. + * This is done with LZ4_initStream(), or LZ4_resetStream(). + * Consider switching to LZ4_initStream(), + * invoking LZ4_resetStream() will trigger deprecation warnings in the future. + */ +LZ4LIB_API void LZ4_resetStream (LZ4_stream_t* streamPtr); + + +#endif /* LZ4_H_2983827168210 */ + + +#if defined (__cplusplus) +} +#endif diff --git a/source/main.c b/source/main.c index 48b9387..c32a601 100644 --- a/source/main.c +++ b/source/main.c @@ -108,6 +108,9 @@ int main(int argc, char *argv[]) /* Init RomFS context */ initRomFsContext(); + /* Init BKTR context */ + initBktrContext(); + /* Make sure output directories exist */ createOutputDirectories(); @@ -217,6 +220,9 @@ int main(int argc, char *argv[]) case resultRomFsSectionBrowserCopyFile: uiSetState(stateRomFsSectionBrowserCopyFile); break; + case resultRomFsSectionBrowserCopyDir: + uiSetState(stateRomFsSectionBrowserCopyDir); + break; case resultDumpGameCardCertificate: uiSetState(stateDumpGameCardCertificate); break; @@ -229,6 +235,12 @@ int main(int argc, char *argv[]) case resultShowSdCardEmmcOrphanPatchAddOnMenu: uiSetState(stateSdCardEmmcOrphanPatchAddOnMenu); break; + case resultShowSdCardEmmcBatchModeMenu: + uiSetState(stateSdCardEmmcBatchModeMenu); + break; + case resultSdCardEmmcBatchDump: + uiSetState(stateSdCardEmmcBatchDump); + break; case resultShowUpdateMenu: uiSetState(stateUpdateMenu); break; diff --git a/source/nca.c b/source/nca.c index 2facf08..70b36fd 100644 --- a/source/nca.c +++ b/source/nca.c @@ -1,11 +1,13 @@ #include #include #include +#include #include "keys.h" #include "util.h" #include "ui.h" #include "rsa.h" +#include "nso.h" /* Extern variables */ @@ -14,6 +16,7 @@ extern int font_height; extern exefs_ctx_t exeFsContext; extern romfs_ctx_t romFsContext; +extern bktr_ctx_t bktrContext; extern char strbuf[NAME_BUF_LEN * 4]; @@ -128,12 +131,12 @@ void generateCnmtXml(cnmt_xml_program_info *xml_program_info, cnmt_xml_content_i u32 i; char tmp[NAME_BUF_LEN] = {'\0'}; - sprintf(out, "\xEF\xBB\xBF\r\n" \ - "\r\n" \ - " %s\r\n" \ - " 0x%016lx\r\n" \ - " %u\r\n" \ - " %u\r\n", \ + sprintf(out, "\n" \ + "\n" \ + " %s\n" \ + " 0x%016lx\n" \ + " %u\n" \ + " %u\n", \ getTitleType(xml_program_info->type), \ xml_program_info->title_id, \ xml_program_info->version, \ @@ -141,13 +144,14 @@ void generateCnmtXml(cnmt_xml_program_info *xml_program_info, cnmt_xml_content_i for(i = 0; i < xml_program_info->nca_cnt; i++) { - sprintf(tmp, " \r\n" \ - " %s\r\n" \ - " %s\r\n" \ - " %lu\r\n" \ - " %s\r\n" \ - " %u\r\n" \ - " \r\n", \ + sprintf(tmp, " \n" \ + " %s\n" \ + " %s\n" \ + " %lu\n" \ + " %s\n" \ + " %u\n" \ + " 0\n" \ + " \n", \ getContentType(xml_content_info[i].type), \ xml_content_info[i].nca_id_str, \ xml_content_info[i].size, \ @@ -157,11 +161,11 @@ void generateCnmtXml(cnmt_xml_program_info *xml_program_info, cnmt_xml_content_i strcat(out, tmp); } - sprintf(tmp, " %s\r\n" \ - " %u\r\n" \ - " <%s>%u\r\n" \ - " <%s>0x%016lx\r\n" \ - "\r\n", \ + sprintf(tmp, " %s\n" \ + " %u\n" \ + " <%s>%u\n" \ + " <%s>0x%016lx\n" \ + "", \ xml_program_info->digest_str, \ xml_program_info->min_keyblob, \ getRequiredMinTitleType(xml_program_info->type), \ @@ -265,7 +269,26 @@ static void nca_update_ctr(unsigned char *ctr, u64 ofs) } } -bool processNcaCtrSectionBlock(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, u64 offset, void *outBuf, size_t bufSize, Aes128CtrContext *ctx, bool encrypt) +/* Updates the CTR for a bktr offset. */ +static void nca_update_bktr_ctr(unsigned char *ctr, u32 ctr_val, u64 ofs) +{ + ofs >>= 4; + unsigned int i; + + for(i = 0; i < 0x8; i++) + { + ctr[0x10 - i - 1] = (unsigned char)(ofs & 0xFF); + ofs >>= 8; + } + + for(i = 0; i < 0x4; i++) + { + ctr[0x8 - i - 1] = (unsigned char)(ctr_val & 0xFF); + ctr_val >>= 8; + } +} + +bool processNcaCtrSectionBlock(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, Aes128CtrContext *ctx, u64 offset, void *outBuf, size_t bufSize, bool encrypt) { if (!ncmStorage || !ncaId || !outBuf || !bufSize || !ctx) { @@ -325,6 +348,246 @@ bool processNcaCtrSectionBlock(NcmContentStorage *ncmStorage, const NcmNcaId *nc return true; } +bktr_relocation_bucket_t *bktr_get_relocation_bucket(bktr_relocation_block_t *block, u32 i) +{ + return (bktr_relocation_bucket_t*)((u8*)block->buckets + ((sizeof(bktr_relocation_bucket_t) + sizeof(bktr_relocation_entry_t)) * (u64)i)); +} + +// Get a relocation entry from offset and relocation block +bktr_relocation_entry_t *bktr_get_relocation(bktr_relocation_block_t *block, u64 offset) +{ + // Weak check for invalid offset + if (offset > block->total_size) + { + uiDrawString("Error: too big offset looked up in BKTR relocation table!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + return NULL; + } + + u32 i, bucket_num = 0; + + for(i = 1; i < block->num_buckets; i++) + { + if (block->bucket_virtual_offsets[i] <= offset) bucket_num++; + } + + bktr_relocation_bucket_t *bucket = bktr_get_relocation_bucket(block, bucket_num); + + // Check for edge case, short circuit + if (bucket->num_entries == 1) return &(bucket->entries[0]); + + // Binary search + u32 low = 0, high = (bucket->num_entries - 1); + + while(low <= high) + { + u32 mid = ((low + high) / 2); + + if (bucket->entries[mid].virt_offset > offset) + { + // Too high + high = (mid - 1); + } else { + // block->entries[mid].offset <= offset + + // Check for success + if (mid == (bucket->num_entries - 1) || bucket->entries[mid + 1].virt_offset > offset) return &(bucket->entries[mid]); + + low = (mid + 1); + } + } + + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Error: failed to find offset 0x%016lX in BKTR relocation table!", offset); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + return NULL; +} + +bktr_subsection_bucket_t *bktr_get_subsection_bucket(bktr_subsection_block_t *block, u32 i) +{ + return (bktr_subsection_bucket_t*)((u8*)block->buckets + ((sizeof(bktr_subsection_bucket_t) + sizeof(bktr_subsection_entry_t)) * (u64)i)); +} + +// Get a subsection entry from offset and subsection block +bktr_subsection_entry_t *bktr_get_subsection(bktr_subsection_block_t *block, u64 offset) +{ + // If offset is past the virtual, we're reading from the BKTR_HEADER subsection + bktr_subsection_bucket_t *last_bucket = bktr_get_subsection_bucket(block, block->num_buckets - 1); + if (offset >= last_bucket->entries[last_bucket->num_entries].offset) return &(last_bucket->entries[last_bucket->num_entries]); + + u32 i, bucket_num = 0; + + for(i = 1; i < block->num_buckets; i++) + { + if (block->bucket_physical_offsets[i] <= offset) bucket_num++; + } + + bktr_subsection_bucket_t *bucket = bktr_get_subsection_bucket(block, bucket_num); + + // Check for edge case, short circuit + if (bucket->num_entries == 1) return &(bucket->entries[0]); + + // Binary search + u32 low = 0, high = (bucket->num_entries - 1); + + while (low <= high) + { + u32 mid = ((low + high) / 2); + + if (bucket->entries[mid].offset > offset) + { + // Too high + high = (mid - 1); + } else { + // block->entries[mid].offset <= offset + + // Check for success + if (mid == (bucket->num_entries - 1) || bucket->entries[mid + 1].offset > offset) return &(bucket->entries[mid]); + + low = (mid + 1); + } + } + + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Error: failed to find offset 0x%016lX in BKTR subsection table!", offset); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + return NULL; +} + +bool bktrSectionSeek(u64 offset) +{ + if (!bktrContext.section_offset || !bktrContext.section_size || !bktrContext.relocation_block || !bktrContext.subsection_block) + { + uiDrawString("Error: invalid parameters to seek within NCA BKTR section!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + return false; + } + + bktr_relocation_entry_t *reloc = bktr_get_relocation(bktrContext.relocation_block, offset); + if (!reloc) return false; + + // No better way to do this than to make all BKTR seeking virtual + bktrContext.virtual_seek = offset; + + u64 section_ofs = (offset - reloc->virt_offset + reloc->phys_offset); + + if (reloc->is_patch) + { + // Seeked within the patch RomFS + bktrContext.bktr_seek = section_ofs; + bktrContext.base_seek = 0; + } else { + // Seeked within the base RomFS + bktrContext.bktr_seek = 0; + bktrContext.base_seek = section_ofs; + } + + return true; +} + +bool bktrSectionPhysicalRead(void *outBuf, size_t bufSize) +{ + if (!bktrContext.section_offset || !bktrContext.section_size || !bktrContext.relocation_block || !bktrContext.subsection_block || !outBuf || !bufSize) + { + uiDrawString("Error: invalid parameters to perform physical block read from NCA BKTR section!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + return false; + } + + Result result; + unsigned char ctr[0x10]; + u8 *tmp_buf = NULL; + + bktr_subsection_entry_t *subsec = bktr_get_subsection(bktrContext.subsection_block, bktrContext.bktr_seek); + if (!subsec) return false; + + bktr_subsection_entry_t *next_subsec = (subsec + 1); + + u64 base_offset = (bktrContext.section_offset + bktrContext.bktr_seek); + + memcpy(ctr, bktrContext.aes_ctx.ctr, 0x10); + nca_update_bktr_ctr(ctr, subsec->ctr_val, base_offset); + aes128CtrContextResetCtr(&(bktrContext.aes_ctx), ctr); + + u64 virt_seek = bktrContext.virtual_seek; + + if ((bktrContext.bktr_seek + bufSize) <= next_subsec->offset) + { + // Easy path, reading *only* within the subsection + u64 block_start_offset = (base_offset - (base_offset % 0x10)); + u64 block_end_offset = (u64)round_up(base_offset + bufSize, 0x10); + u64 block_size = (block_end_offset - block_start_offset); + + tmp_buf = malloc(block_size); + if (!tmp_buf) + { + uiDrawString("Error: unable to allocate memory for the temporary NCA BKTR section block read buffer!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + return false; + } + + if (R_FAILED(result = ncmContentStorageReadContentIdFile(&(bktrContext.ncmStorage), &(bktrContext.ncaId), block_start_offset, tmp_buf, block_size))) + { + free(tmp_buf); + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "BKTR: failed to read encrypted %lu bytes block at offset 0x%016lX! (0x%08X)", block_size, block_start_offset, result); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + return false; + } + + // Decrypt + aes128CtrCrypt(&(bktrContext.aes_ctx), tmp_buf, tmp_buf, block_size); + + memcpy(outBuf, tmp_buf + (base_offset - block_start_offset), bufSize); + + free(tmp_buf); + } else { + // Sad path + u64 within_subsection = (next_subsec->offset - bktrContext.bktr_seek); + + if (!readBktrSectionBlock(virt_seek, outBuf, within_subsection)) return false; + + if (!readBktrSectionBlock(virt_seek + within_subsection, (u8*)outBuf + within_subsection, bufSize - within_subsection)) return false; + } + + return true; +} + +bool readBktrSectionBlock(u64 offset, void *outBuf, size_t bufSize) +{ + if (!bktrContext.section_offset || !bktrContext.section_size || !bktrContext.relocation_block || !bktrContext.subsection_block || !romFsContext.section_offset || !romFsContext.section_size || !outBuf || !bufSize) + { + uiDrawString("Error: invalid parameters to read block from NCA BKTR section!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + return false; + } + + if (!loadNcaKeyset()) return false; + + if (!bktrSectionSeek(offset)) return false; + + bktr_relocation_entry_t *reloc = bktr_get_relocation(bktrContext.relocation_block, bktrContext.virtual_seek); + if (!reloc) return false; + + bktr_relocation_entry_t *next_reloc = (reloc + 1); + + u64 virt_seek = bktrContext.virtual_seek; + + // Perform read operation + if ((bktrContext.virtual_seek + bufSize) <= next_reloc->virt_offset) + { + // Easy path: We're reading *only* within the current relocation + + if (reloc->is_patch) + { + if (!bktrSectionPhysicalRead(outBuf, bufSize)) return false; + } else { + // Nice and easy read from the base RomFS + if (!processNcaCtrSectionBlock(&(romFsContext.ncmStorage), &(romFsContext.ncaId), &(romFsContext.aes_ctx), romFsContext.section_offset + bktrContext.base_seek, outBuf, bufSize, false)) return false; + } + } else { + u64 within_relocation = (next_reloc->virt_offset - bktrContext.virtual_seek); + + if (!readBktrSectionBlock(virt_seek, outBuf, within_relocation)) return false; + + if (!readBktrSectionBlock(virt_seek + within_relocation, (u8*)outBuf + within_relocation, bufSize - within_relocation)) return false; + } + + return true; +} + bool encryptNcaHeader(nca_header_t *input, u8 *outBuf, u64 outBufSize) { if (!input || !outBuf || !outBufSize || outBufSize < NCA_FULL_HEADER_LENGTH || (bswap_32(input->magic) != NCA3_MAGIC && bswap_32(input->magic) != NCA2_MAGIC)) @@ -591,7 +854,7 @@ bool processProgramNca(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, nca memcpy(ctr_key, xml_content_info->decrypted_nca_keys + (NCA_KEY_AREA_KEY_SIZE * 2), NCA_KEY_AREA_KEY_SIZE); aes128CtrContextCreate(&aes_ctx, ctr_key, ctr); - if (!processNcaCtrSectionBlock(ncmStorage, ncaId, nca_pfs0_offset, &nca_pfs0_header, sizeof(pfs0_header), &aes_ctx, false)) + if (!processNcaCtrSectionBlock(ncmStorage, ncaId, &aes_ctx, nca_pfs0_offset, &nca_pfs0_header, sizeof(pfs0_header), false)) { breaks++; uiDrawString("Failed to read Program NCA section #0 PFS0 partition header!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); @@ -618,7 +881,7 @@ bool processProgramNca(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, nca return false; } - if (!processNcaCtrSectionBlock(ncmStorage, ncaId, nca_pfs0_offset + sizeof(pfs0_header), nca_pfs0_entries, (u64)nca_pfs0_header.file_cnt * sizeof(pfs0_entry_table), &aes_ctx, false)) + if (!processNcaCtrSectionBlock(ncmStorage, ncaId, &aes_ctx, nca_pfs0_offset + sizeof(pfs0_header), nca_pfs0_entries, (u64)nca_pfs0_header.file_cnt * sizeof(pfs0_entry_table), false)) { breaks++; uiDrawString("Failed to read Program NCA section #0 PFS0 partition entries!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); @@ -634,7 +897,7 @@ bool processProgramNca(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, nca u64 nca_pfs0_cur_file_offset = (nca_pfs0_data_offset + nca_pfs0_entries[i].file_offset); // Read and decrypt NPDM header - if (!processNcaCtrSectionBlock(ncmStorage, ncaId, nca_pfs0_cur_file_offset, &npdm_header, sizeof(npdm_t), &aes_ctx, false)) + if (!processNcaCtrSectionBlock(ncmStorage, ncaId, &aes_ctx, nca_pfs0_cur_file_offset, &npdm_header, sizeof(npdm_t), false)) { breaks++; snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Failed to read Program NCA section #0 PFS0 entry #%u!", i); @@ -681,7 +944,7 @@ bool processProgramNca(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, nca } // Read and decrypt block - if (!processNcaCtrSectionBlock(ncmStorage, ncaId, block_start_offset[0], block_data[0], block_size[0], &aes_ctx, false)) + if (!processNcaCtrSectionBlock(ncmStorage, ncaId, &aes_ctx, block_start_offset[0], block_data[0], block_size[0], false)) { breaks++; uiDrawString("Failed to read Program NCA section #0 PFS0 NPDM block 0!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); @@ -723,7 +986,7 @@ bool processProgramNca(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, nca return false; } - if (!processNcaCtrSectionBlock(ncmStorage, ncaId, block_start_offset[1], block_data[1], block_size[1], &aes_ctx, false)) + if (!processNcaCtrSectionBlock(ncmStorage, ncaId, &aes_ctx, block_start_offset[1], block_data[1], block_size[1], false)) { breaks++; uiDrawString("Failed to read Program NCA section #0 PFS0 NPDM block 1!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); @@ -746,7 +1009,7 @@ bool processProgramNca(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, nca return false; } - if (!processNcaCtrSectionBlock(ncmStorage, ncaId, hash_table_offset, hash_table, dec_nca_header->fs_headers[0].pfs0_superblock.hash_table_size, &aes_ctx, false)) + if (!processNcaCtrSectionBlock(ncmStorage, ncaId, &aes_ctx, hash_table_offset, hash_table, dec_nca_header->fs_headers[0].pfs0_superblock.hash_table_size, false)) { breaks++; uiDrawString("Failed to read Program NCA section #0 PFS0 hash table!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); @@ -778,7 +1041,7 @@ bool processProgramNca(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, nca } // Reencrypt relevant data blocks - if (!processNcaCtrSectionBlock(ncmStorage, ncaId, block_start_offset[0], block_data[0], block_size[0], &aes_ctx, true)) + if (!processNcaCtrSectionBlock(ncmStorage, ncaId, &aes_ctx, block_start_offset[0], block_data[0], block_size[0], true)) { breaks++; uiDrawString("Failed to encrypt Program NCA section #0 PFS0 NPDM block 0!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); @@ -790,7 +1053,7 @@ bool processProgramNca(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, nca if (block_hash_table_offset != block_hash_table_end_offset) { - if (!processNcaCtrSectionBlock(ncmStorage, ncaId, block_start_offset[1], block_data[1], block_size[1], &aes_ctx, true)) + if (!processNcaCtrSectionBlock(ncmStorage, ncaId, &aes_ctx, block_start_offset[1], block_data[1], block_size[1], true)) { breaks++; uiDrawString("Failed to encrypt Program NCA section #0 PFS0 NPDM block 1!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); @@ -801,7 +1064,7 @@ bool processProgramNca(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, nca } } - if (!processNcaCtrSectionBlock(ncmStorage, ncaId, hash_table_offset, hash_table, dec_nca_header->fs_headers[0].pfs0_superblock.hash_table_size, &aes_ctx, true)) + if (!processNcaCtrSectionBlock(ncmStorage, ncaId, &aes_ctx, hash_table_offset, hash_table, dec_nca_header->fs_headers[0].pfs0_superblock.hash_table_size, true)) { breaks++; uiDrawString("Failed to encrypt Program NCA section #0 PFS0 hash table!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); @@ -830,6 +1093,8 @@ bool processProgramNca(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, nca output->block_size[1] = block_size[1]; } + output->acid_pubkey_offset = (acid_pubkey_offset - block_start_offset[0]); + return true; } @@ -1214,7 +1479,7 @@ bool readExeFsEntryFromNca(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, nca_pfs0_offset = (section_offset + dec_nca_header->fs_headers[exefs_index].pfs0_superblock.pfs0_offset); - if (!processNcaCtrSectionBlock(ncmStorage, ncaId, nca_pfs0_offset, &nca_pfs0_header, sizeof(pfs0_header), &aes_ctx, false)) return false; + if (!processNcaCtrSectionBlock(ncmStorage, ncaId, &aes_ctx, nca_pfs0_offset, &nca_pfs0_header, sizeof(pfs0_header), false)) return false; if (bswap_32(nca_pfs0_header.magic) != PFS0_MAGIC || !nca_pfs0_header.file_cnt || !nca_pfs0_header.str_table_size) continue; @@ -1223,7 +1488,7 @@ bool readExeFsEntryFromNca(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, nca_pfs0_entries = calloc(nca_pfs0_header.file_cnt, sizeof(pfs0_entry_table)); if (!nca_pfs0_entries) continue; - if (!processNcaCtrSectionBlock(ncmStorage, ncaId, nca_pfs0_entries_offset, nca_pfs0_entries, (u64)nca_pfs0_header.file_cnt * sizeof(pfs0_entry_table), &aes_ctx, false)) + if (!processNcaCtrSectionBlock(ncmStorage, ncaId, &aes_ctx, nca_pfs0_entries_offset, nca_pfs0_entries, (u64)nca_pfs0_header.file_cnt * sizeof(pfs0_entry_table), false)) { free(nca_pfs0_entries); return false; @@ -1239,7 +1504,7 @@ bool readExeFsEntryFromNca(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, continue; } - if (!processNcaCtrSectionBlock(ncmStorage, ncaId, nca_pfs0_str_table_offset, nca_pfs0_str_table, (u64)nca_pfs0_header.str_table_size, &aes_ctx, false)) + if (!processNcaCtrSectionBlock(ncmStorage, ncaId, &aes_ctx, nca_pfs0_str_table_offset, nca_pfs0_str_table, (u64)nca_pfs0_header.str_table_size, false)) { free(nca_pfs0_str_table); free(nca_pfs0_entries); @@ -1368,14 +1633,14 @@ bool readRomFsEntryFromNca(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, if (bswap_32(dec_nca_header->fs_headers[romfs_index].romfs_superblock.ivfc_header.magic) != IVFC_MAGIC) { - snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Error: invalid magic word for NCA RomFS section! Wrong KAEK? (0x%08X)", bswap_32(dec_nca_header->fs_headers[romfs_index].romfs_superblock.ivfc_header.magic)); + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Error: invalid IVFC magic word for NCA RomFS section! Wrong KAEK? (0x%08X)", bswap_32(dec_nca_header->fs_headers[romfs_index].romfs_superblock.ivfc_header.magic)); uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); return false; } if (dec_nca_header->fs_headers[romfs_index].crypt_type != NCA_FS_HEADER_CRYPT_CTR) { - snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Error: invalid AES crypt type for NCA RomFS section! (0x%02X)", dec_nca_header->fs_headers[0].crypt_type); + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Error: invalid AES crypt type for NCA RomFS section! (0x%02X)", dec_nca_header->fs_headers[romfs_index].crypt_type); uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); return false; } @@ -1390,7 +1655,7 @@ bool readRomFsEntryFromNca(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, } // First read the RomFS header - if (!processNcaCtrSectionBlock(ncmStorage, ncaId, romfs_offset, &romFsHeader, sizeof(romfs_header), &aes_ctx, false)) + if (!processNcaCtrSectionBlock(ncmStorage, ncaId, &aes_ctx, romfs_offset, &romFsHeader, sizeof(romfs_header), false)) { breaks++; uiDrawString("Failed to read NCA RomFS section header!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); @@ -1431,7 +1696,7 @@ bool readRomFsEntryFromNca(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, return false; } - if (!processNcaCtrSectionBlock(ncmStorage, ncaId, romfs_dirtable_offset, romfs_dir_entries, romfs_dirtable_size, &aes_ctx, false)) + if (!processNcaCtrSectionBlock(ncmStorage, ncaId, &aes_ctx, romfs_dirtable_offset, romfs_dir_entries, romfs_dirtable_size, false)) { breaks++; uiDrawString("Failed to read NCA RomFS section directory entries!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); @@ -1447,7 +1712,7 @@ bool readRomFsEntryFromNca(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, return false; } - if (!processNcaCtrSectionBlock(ncmStorage, ncaId, romfs_filetable_offset, romfs_file_entries, romfs_filetable_size, &aes_ctx, false)) + if (!processNcaCtrSectionBlock(ncmStorage, ncaId, &aes_ctx, romfs_filetable_offset, romfs_file_entries, romfs_filetable_size, false)) { breaks++; uiDrawString("Failed to read NCA RomFS section file entries!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); @@ -1461,6 +1726,8 @@ bool readRomFsEntryFromNca(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, memcpy(&(romFsContext.ncmStorage), ncmStorage, sizeof(NcmContentStorage)); memcpy(&(romFsContext.ncaId), ncaId, sizeof(NcmNcaId)); memcpy(&(romFsContext.aes_ctx), &aes_ctx, sizeof(Aes128CtrContext)); + romFsContext.section_offset = section_offset; + romFsContext.section_size = section_size; romFsContext.romfs_offset = romfs_offset; romFsContext.romfs_size = romfs_size; romFsContext.romfs_dirtable_offset = romfs_dirtable_offset; @@ -1474,6 +1741,613 @@ bool readRomFsEntryFromNca(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, return true; } +bool readBktrEntryFromNca(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, nca_header_t *dec_nca_header, u8 *decrypted_nca_keys) +{ + if (!ncmStorage || !ncaId || !dec_nca_header || !decrypted_nca_keys || !romFsContext.section_offset || !romFsContext.section_size || !romFsContext.romfs_dir_entries || !romFsContext.romfs_file_entries) + { + uiDrawString("Error: invalid parameters to read BKTR section from NCA!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + return false; + } + + u32 i; + + u8 bktr_index; + bool found_bktr = false, success = false; + + romfs_header romFsHeader; + + initBktrContext(); + + memcpy(&(bktrContext.ncmStorage), ncmStorage, sizeof(NcmContentStorage)); + memcpy(&(bktrContext.ncaId), ncaId, sizeof(NcmNcaId)); + + for(bktr_index = 0; bktr_index < 4; bktr_index++) + { + if (dec_nca_header->fs_headers[bktr_index].partition_type == NCA_FS_HEADER_PARTITION_ROMFS && dec_nca_header->fs_headers[bktr_index].fs_type == NCA_FS_HEADER_FSTYPE_ROMFS) + { + found_bktr = true; + break; + } + } + + if (!found_bktr) + { + uiDrawString("Error: NCA doesn't hold a BKTR section!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + return false; + } + + bktrContext.section_offset = ((u64)dec_nca_header->section_entries[bktr_index].media_start_offset * (u64)MEDIA_UNIT_SIZE); + bktrContext.section_size = (((u64)dec_nca_header->section_entries[bktr_index].media_end_offset * (u64)MEDIA_UNIT_SIZE) - bktrContext.section_offset); + + if (!bktrContext.section_offset || bktrContext.section_offset < NCA_FULL_HEADER_LENGTH || !bktrContext.section_size) + { + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Error: invalid offset/size for NCA BKTR section! (#%u)", bktr_index); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + return false; + } + + // Generate initial CTR + unsigned char ctr[0x10]; + u64 ofs = (bktrContext.section_offset >> 4); + + for(i = 0; i < 0x8; i++) + { + ctr[i] = dec_nca_header->fs_headers[bktr_index].section_ctr[0x08 - i - 1]; + ctr[0x10 - i - 1] = (unsigned char)(ofs & 0xFF); + ofs >>= 8; + } + + u8 ctr_key[NCA_KEY_AREA_KEY_SIZE]; + memcpy(ctr_key, decrypted_nca_keys + (NCA_KEY_AREA_KEY_SIZE * 2), NCA_KEY_AREA_KEY_SIZE); + aes128CtrContextCreate(&(bktrContext.aes_ctx), ctr_key, ctr); + + memcpy(&(bktrContext.superblock), &(dec_nca_header->fs_headers[bktr_index].bktr_superblock), sizeof(bktr_superblock_t)); + + if (bswap_32(bktrContext.superblock.ivfc_header.magic) != IVFC_MAGIC) + { + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Error: invalid IVFC magic word for NCA BKTR section! Wrong KAEK? (0x%08X)", bswap_32(bktrContext.superblock.ivfc_header.magic)); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + return false; + } + + if (dec_nca_header->fs_headers[bktr_index].crypt_type != NCA_FS_HEADER_CRYPT_BKTR) + { + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Error: invalid AES crypt type for NCA BKTR section! (0x%02X)", dec_nca_header->fs_headers[bktr_index].crypt_type); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + return false; + } + + if (bswap_32(bktrContext.superblock.relocation_header.magic) != BKTR_MAGIC || bswap_32(bktrContext.superblock.subsection_header.magic) != BKTR_MAGIC) + { + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Error: invalid BKTR magic word for NCA BKTR relocation/subsection header! (0x%02X | 0x%02X)", bswap_32(bktrContext.superblock.relocation_header.magic), bswap_32(bktrContext.superblock.subsection_header.magic)); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + return false; + } + + if ((bktrContext.superblock.relocation_header.offset + bktrContext.superblock.relocation_header.size) != bktrContext.superblock.subsection_header.offset || (bktrContext.superblock.subsection_header.offset + bktrContext.superblock.subsection_header.size) != bktrContext.section_size) + { + uiDrawString("Error: invalid layout for NCA BKTR section!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + return false; + } + + // Allocate space for an extra (fake) relocation entry, to simplify our logic + bktrContext.relocation_block = calloc(1, bktrContext.superblock.relocation_header.size + ((0x3FF0 / sizeof(u64)) * sizeof(bktr_relocation_entry_t))); + if (!bktrContext.relocation_block) + { + uiDrawString("Error: unable to allocate memory for NCA BKTR relocation header!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + return false; + } + + // Allocate space for an extra (fake) subsection entry, to simplify our logic + bktrContext.subsection_block = calloc(1, bktrContext.superblock.subsection_header.size + ((0x3FF0 / sizeof(u64)) * sizeof(bktr_subsection_entry_t)) + sizeof(bktr_subsection_entry_t)); + if (!bktrContext.subsection_block) + { + uiDrawString("Error: unable to allocate memory for NCA BKTR subsection header!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + goto out; + } + + // Read the relocation header + if (!processNcaCtrSectionBlock(ncmStorage, ncaId, &(bktrContext.aes_ctx), bktrContext.section_offset + bktrContext.superblock.relocation_header.offset, bktrContext.relocation_block, bktrContext.superblock.relocation_header.size, false)) + { + breaks++; + uiDrawString("Failed to read NCA BKTR relocation header!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + goto out; + } + + // Read the subsection header + if (!processNcaCtrSectionBlock(ncmStorage, ncaId, &(bktrContext.aes_ctx), bktrContext.section_offset + bktrContext.superblock.subsection_header.offset, bktrContext.subsection_block, bktrContext.superblock.subsection_header.size, false)) + { + breaks++; + uiDrawString("Failed to read NCA BKTR subsection header!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + goto out; + } + + if (bktrContext.subsection_block->total_size != bktrContext.superblock.subsection_header.offset) + { + uiDrawString("Error: invalid NCA BKTR subsection header size!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + goto out; + } + + // This simplifies logic greatly... + for(i = (bktrContext.relocation_block->num_buckets - 1); i > 0; i--) + { + bktr_relocation_bucket_t tmp_bucket; + memcpy(&tmp_bucket, &(bktrContext.relocation_block->buckets[i]), sizeof(bktr_relocation_bucket_t)); + memcpy(bktr_get_relocation_bucket(bktrContext.relocation_block, i), &tmp_bucket, sizeof(bktr_relocation_bucket_t)); + } + + for(i = 0; (i + 1) < bktrContext.relocation_block->num_buckets; i++) + { + bktr_relocation_bucket_t *cur_bucket = bktr_get_relocation_bucket(bktrContext.relocation_block, i); + cur_bucket->entries[cur_bucket->num_entries].virt_offset = bktrContext.relocation_block->bucket_virtual_offsets[i + 1]; + } + + for(i = (bktrContext.subsection_block->num_buckets - 1); i > 0; i--) + { + bktr_subsection_bucket_t tmp_bucket; + memcpy(&tmp_bucket, &(bktrContext.subsection_block->buckets[i]), sizeof(bktr_subsection_bucket_t)); + memcpy(bktr_get_subsection_bucket(bktrContext.subsection_block, i), &tmp_bucket, sizeof(bktr_subsection_bucket_t)); + } + + for(i = 0; (i + 1) < bktrContext.subsection_block->num_buckets; i++) + { + bktr_subsection_bucket_t *cur_bucket = bktr_get_subsection_bucket(bktrContext.subsection_block, i); + bktr_subsection_bucket_t *next_bucket = bktr_get_subsection_bucket(bktrContext.subsection_block, i + 1); + cur_bucket->entries[cur_bucket->num_entries].offset = next_bucket->entries[0].offset; + cur_bucket->entries[cur_bucket->num_entries].ctr_val = next_bucket->entries[0].ctr_val; + } + + bktr_relocation_bucket_t *last_reloc_bucket = bktr_get_relocation_bucket(bktrContext.relocation_block, bktrContext.relocation_block->num_buckets - 1); + bktr_subsection_bucket_t *last_subsec_bucket = bktr_get_subsection_bucket(bktrContext.subsection_block, bktrContext.subsection_block->num_buckets - 1); + last_reloc_bucket->entries[last_reloc_bucket->num_entries].virt_offset = bktrContext.relocation_block->total_size; + last_subsec_bucket->entries[last_subsec_bucket->num_entries].offset = bktrContext.superblock.relocation_header.offset; + last_subsec_bucket->entries[last_subsec_bucket->num_entries].ctr_val = dec_nca_header->fs_headers[bktr_index].section_ctr_low; + last_subsec_bucket->entries[last_subsec_bucket->num_entries + 1].offset = bktrContext.section_size; + last_subsec_bucket->entries[last_subsec_bucket->num_entries + 1].ctr_val = 0; + + // Parse RomFS section + bktrContext.romfs_offset = dec_nca_header->fs_headers[bktr_index].bktr_superblock.ivfc_header.level_headers[IVFC_MAX_LEVEL - 1].logical_offset; + bktrContext.romfs_size = dec_nca_header->fs_headers[bktr_index].bktr_superblock.ivfc_header.level_headers[IVFC_MAX_LEVEL - 1].hash_data_size; + + // Do not check the RomFS size, because it reflects the full patched RomFS image + if (!bktrContext.romfs_offset || bktrContext.romfs_size < ROMFS_HEADER_SIZE || (bktrContext.section_offset + bktrContext.romfs_offset) > (bktrContext.section_offset + bktrContext.section_size)) + { + uiDrawString("Error: invalid offset/size for NCA BKTR RomFS section!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + goto out; + } + + if (!readBktrSectionBlock(bktrContext.romfs_offset, &romFsHeader, sizeof(romfs_header))) + { + breaks++; + uiDrawString("Failed to read NCA BKTR RomFS section header!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + goto out; + } + + if (romFsHeader.headerSize != ROMFS_HEADER_SIZE) + { + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Error: invalid header size for NCA BKTR RomFS section! (0x%016lX at 0x%016lX)", romFsHeader.headerSize, bktrContext.section_offset + bktrContext.romfs_offset); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + goto out; + } + + bktrContext.romfs_dirtable_offset = (bktrContext.romfs_offset + romFsHeader.dirTableOff); + bktrContext.romfs_dirtable_size = romFsHeader.dirTableSize; + + bktrContext.romfs_filetable_offset = (bktrContext.romfs_offset + romFsHeader.fileTableOff); + bktrContext.romfs_filetable_size = romFsHeader.fileTableSize; + + // Then again, do not check these offsets/sizes, because they reflect the patched RomFS image + if (!bktrContext.romfs_dirtable_offset || !bktrContext.romfs_dirtable_size || !bktrContext.romfs_filetable_offset || !bktrContext.romfs_filetable_size) + { + uiDrawString("Error: invalid directory/file table for NCA BKTR RomFS section!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + goto out; + } + + bktrContext.romfs_filedata_offset = (bktrContext.romfs_offset + romFsHeader.fileDataOff); + + if (!bktrContext.romfs_filedata_offset) + { + uiDrawString("Error: invalid file data block offset for NCA BKTR RomFS section!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + goto out; + } + + bktrContext.romfs_dir_entries = calloc(1, bktrContext.romfs_dirtable_size); + if (!bktrContext.romfs_dir_entries) + { + uiDrawString("Error: unable to allocate memory for NCA BKTR RomFS section directory entries!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + goto out; + } + + if (!readBktrSectionBlock(bktrContext.romfs_dirtable_offset, bktrContext.romfs_dir_entries, bktrContext.romfs_dirtable_size)) + { + breaks++; + uiDrawString("Failed to read NCA BKTR RomFS section directory entries!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + goto out; + } + + bktrContext.romfs_file_entries = calloc(1, bktrContext.romfs_filetable_size); + if (!bktrContext.romfs_file_entries) + { + uiDrawString("Error: unable to allocate memory for NCA BKTR RomFS section file entries!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + goto out; + } + + if (!readBktrSectionBlock(bktrContext.romfs_filetable_offset, bktrContext.romfs_file_entries, bktrContext.romfs_filetable_size)) + { + breaks++; + uiDrawString("Failed to read NCA RomFS section file entries!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + goto out; + } + + success = true; + +out: + if (!success) + { + if (bktrContext.romfs_file_entries != NULL) + { + free(bktrContext.romfs_file_entries); + bktrContext.romfs_file_entries = NULL; + } + + if (bktrContext.romfs_dir_entries != NULL) + { + free(bktrContext.romfs_dir_entries); + bktrContext.romfs_dir_entries = NULL; + } + + if (bktrContext.subsection_block != NULL) + { + free(bktrContext.subsection_block); + bktrContext.subsection_block = NULL; + } + + if (bktrContext.relocation_block != NULL) + { + free(bktrContext.relocation_block); + bktrContext.relocation_block = NULL; + } + } + + // The caller function must free the data pointers from the bktrContext struct + + return success; +} + +bool generateProgramInfoXml(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, nca_header_t *dec_nca_header, u8 *decrypted_nca_keys, nca_program_mod_data *program_mod_data, char **outBuf, u64 *outBufSize) +{ + if (!ncmStorage || !ncaId || !dec_nca_header || !decrypted_nca_keys || !program_mod_data || !outBuf || !outBufSize) + { + uiDrawString("Error: invalid parameters to generate \"programinfo.xml\"!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + return false; + } + + if (dec_nca_header->fs_headers[0].partition_type != NCA_FS_HEADER_PARTITION_PFS0 || dec_nca_header->fs_headers[0].fs_type != NCA_FS_HEADER_FSTYPE_PFS0) + { + uiDrawString("Error: Program NCA section #0 doesn't hold a PFS0 partition!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + return false; + } + + if (!dec_nca_header->fs_headers[0].pfs0_superblock.pfs0_size) + { + uiDrawString("Error: invalid size for PFS0 partition in Program NCA section #0!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + return false; + } + + if (dec_nca_header->fs_headers[0].crypt_type != NCA_FS_HEADER_CRYPT_CTR) + { + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Error: invalid AES crypt type for Program NCA section #0! (0x%02X)", dec_nca_header->fs_headers[0].crypt_type); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + return false; + } + + u32 i; + + bool proceed = true, success = false; + + u64 section_offset; + u64 nca_pfs0_offset; + + pfs0_header nca_pfs0_header; + pfs0_entry_table *nca_pfs0_entries = NULL; + char *nca_pfs0_str_table = NULL; + + u64 nca_pfs0_str_table_offset; + u64 nca_pfs0_data_offset; + + Aes128CtrContext aes_ctx; + + char *programInfoXml = NULL; + char tmp[NAME_BUF_LEN] = {'\0'}; + + u32 npdmEntry = 0; + npdm_t npdm_header; + u8 *npdm_acid_section = NULL; + + u64 npdm_acid_section_b64_size = 0; + char *npdm_acid_section_b64 = NULL; + + u32 acid_flags = 0; + + section_offset = ((u64)dec_nca_header->section_entries[0].media_start_offset * (u64)MEDIA_UNIT_SIZE); + nca_pfs0_offset = (section_offset + dec_nca_header->fs_headers[0].pfs0_superblock.pfs0_offset); + + if (!section_offset || section_offset < NCA_FULL_HEADER_LENGTH || !nca_pfs0_offset) + { + uiDrawString("Error: invalid offsets for Program NCA section #0!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + return false; + } + + // Generate initial CTR + unsigned char ctr[0x10]; + u64 ofs = (section_offset >> 4); + + for(i = 0; i < 0x8; i++) + { + ctr[i] = dec_nca_header->fs_headers[0].section_ctr[0x08 - i - 1]; + ctr[0x10 - i - 1] = (unsigned char)(ofs & 0xFF); + ofs >>= 8; + } + + u8 ctr_key[NCA_KEY_AREA_KEY_SIZE]; + memcpy(ctr_key, decrypted_nca_keys + (NCA_KEY_AREA_KEY_SIZE * 2), NCA_KEY_AREA_KEY_SIZE); + aes128CtrContextCreate(&aes_ctx, ctr_key, ctr); + + if (!processNcaCtrSectionBlock(ncmStorage, ncaId, &aes_ctx, nca_pfs0_offset, &nca_pfs0_header, sizeof(pfs0_header), false)) + { + breaks++; + uiDrawString("Failed to read Program NCA section #0 PFS0 partition header!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + return false; + } + + if (bswap_32(nca_pfs0_header.magic) != PFS0_MAGIC) + { + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Error: invalid magic word for Program NCA section #0 PFS0 partition! Wrong KAEK? (0x%08X)", bswap_32(nca_pfs0_header.magic)); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + return false; + } + + if (!nca_pfs0_header.file_cnt || !nca_pfs0_header.str_table_size) + { + uiDrawString("Error: Program NCA section #0 PFS0 partition is empty! Wrong KAEK?", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + return false; + } + + nca_pfs0_entries = calloc(nca_pfs0_header.file_cnt, sizeof(pfs0_entry_table)); + if (!nca_pfs0_entries) + { + uiDrawString("Error: unable to allocate memory for Program NCA section #0 PFS0 partition entries!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + return false; + } + + if (!processNcaCtrSectionBlock(ncmStorage, ncaId, &aes_ctx, nca_pfs0_offset + sizeof(pfs0_header), nca_pfs0_entries, (u64)nca_pfs0_header.file_cnt * sizeof(pfs0_entry_table), false)) + { + breaks++; + uiDrawString("Failed to read Program NCA section #0 PFS0 partition entries!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + goto out; + } + + nca_pfs0_str_table_offset = (nca_pfs0_offset + sizeof(pfs0_header) + ((u64)nca_pfs0_header.file_cnt * sizeof(pfs0_entry_table))); + + nca_pfs0_str_table = calloc((u64)nca_pfs0_header.str_table_size, sizeof(char)); + if (!nca_pfs0_str_table) + { + uiDrawString("Error: unable to allocate memory for Program NCA section #0 PFS0 string table!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + goto out; + } + + if (!processNcaCtrSectionBlock(ncmStorage, ncaId, &aes_ctx, nca_pfs0_str_table_offset, nca_pfs0_str_table, (u64)nca_pfs0_header.str_table_size, false)) + { + breaks++; + uiDrawString("Failed to read Program NCA section #0 PFS0 string table!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + goto out; + } + + nca_pfs0_data_offset = (nca_pfs0_str_table_offset + (u64)nca_pfs0_header.str_table_size); + + // Allocate memory for the programinfo.xml contents, making sure there's enough space + programInfoXml = calloc(0xA00000, sizeof(char)); + if (!programInfoXml) + { + uiDrawString("Error: unable to allocate memory for the \"programinfo.xml\" contents!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + goto out; + } + + sprintf(programInfoXml, "\n" \ + "\n" \ + " %u_%u_%u\n", dec_nca_header->sdk_major, dec_nca_header->sdk_minor, dec_nca_header->sdk_micro); + + // Retrieve the main.npdm contents + bool found_npdm = false; + for(i = 0; i < nca_pfs0_header.file_cnt; i++) + { + char *curFilename = (nca_pfs0_str_table + nca_pfs0_entries[i].filename_offset); + + if (strlen(curFilename) == 9 && !strncasecmp(curFilename, "main.npdm", 9) && nca_pfs0_entries[i].file_size > 0) + { + found_npdm = true; + npdmEntry = i; + break; + } + } + + if (!found_npdm) + { + uiDrawString("Error: unable to allocate memory for the \"programinfo.xml\" contents!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + goto out; + } + + // Read the META header from the NPDM + if (!processNcaCtrSectionBlock(ncmStorage, ncaId, &aes_ctx, nca_pfs0_data_offset + nca_pfs0_entries[npdmEntry].file_offset, &npdm_header, sizeof(npdm_t), false)) + { + breaks++; + uiDrawString("Failed to read NPDM entry header from Program NCA section #0 PFS0!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + goto out; + } + + if (bswap_32(npdm_header.magic) != META_MAGIC) + { + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Error: invalid NPDM META magic word! (0x%08X)", bswap_32(npdm_header.magic)); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + goto out; + } + + // Allocate memory for the ACID section + npdm_acid_section = malloc(npdm_header.acid_size); + if (!npdm_acid_section) + { + uiDrawString("Error: unable to allocate memory for the Program NCA NPDM ACID section contents!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + goto out; + } + + if (!processNcaCtrSectionBlock(ncmStorage, ncaId, &aes_ctx, nca_pfs0_data_offset + nca_pfs0_entries[npdmEntry].file_offset + (u64)npdm_header.acid_offset, npdm_acid_section, (u64)npdm_header.acid_size, false)) + { + breaks++; + uiDrawString("Failed to read ACID section from Program NCA NPDM!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + goto out; + } + + // If we're dealing with a gamecard title, replace the ACID public key with the patched one + if (program_mod_data->block_mod_cnt > 0) memcpy(npdm_acid_section + (u64)NPDM_SIGNATURE_SIZE, rsa_get_public_key(), (u64)NPDM_SIGNATURE_SIZE); + + sprintf(tmp, " %u\n", ((npdm_header.mmu_flags & 0x01) ? 64 : 32)); + strcat(programInfoXml, tmp); + + // Default this one to Release + strcat(programInfoXml, " Release\n"); + + // Retrieve the Base64 conversion length for the whole ACID section + mbedtls_base64_encode(NULL, 0, &npdm_acid_section_b64_size, npdm_acid_section, (u64)npdm_header.acid_size); + if (npdm_acid_section_b64_size <= (u64)npdm_header.acid_size) + { + uiDrawString("Error: invalid Base64 conversion length for the ACID section from Program NCA NPDM!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + goto out; + } + + npdm_acid_section_b64 = calloc(npdm_acid_section_b64_size + 1, sizeof(char)); + if (!npdm_acid_section_b64) + { + uiDrawString("Error: unable to allocate memory for the Base64 converted ACID section from Program NCA NPDM!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + goto out; + } + + // Perform the Base64 conversion + if (mbedtls_base64_encode((unsigned char*)npdm_acid_section_b64, npdm_acid_section_b64_size + 1, &npdm_acid_section_b64_size, npdm_acid_section, (u64)npdm_header.acid_size) != 0) + { + uiDrawString("Error: Base64 conversion failed for the ACID section from Program NCA NPDM!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + goto out; + } + + strcat(programInfoXml, " "); + strcat(programInfoXml, npdm_acid_section_b64); + strcat(programInfoXml, "\n"); + + // TO-DO: Add more ACID flags? + + acid_flags = *((u32*)(&(npdm_acid_section[0x20C]))); + + strcat(programInfoXml, " \n"); + + sprintf(tmp, " %s\n", ((acid_flags & 0x01) ? "true" : "false")); + strcat(programInfoXml, tmp); + + sprintf(tmp, " %s\n", ((acid_flags & 0x02) ? "true" : "false")); + strcat(programInfoXml, tmp); + + strcat(programInfoXml, " \n"); + + // Middleware list + strcat(programInfoXml, " \n"); + + for(i = 0; i < nca_pfs0_header.file_cnt; i++) + { + nso_header_t nsoHeader; + char *curFilename = (nca_pfs0_str_table + nca_pfs0_entries[i].filename_offset); + u64 curFileOffset = (nca_pfs0_data_offset + nca_pfs0_entries[i].file_offset); + + if (!processNcaCtrSectionBlock(ncmStorage, ncaId, &aes_ctx, curFileOffset, &nsoHeader, sizeof(nso_header_t), false)) + { + breaks++; + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Failed to read 0x%016lX bytes from \"%s\" in Program NCA section #0 PFS0 partition!", sizeof(nso_header_t), curFilename); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + proceed = false; + break; + } + + // Check if we're dealing with a NSO + if (bswap_32(nsoHeader.magic) != NSO_MAGIC) continue; + + // Retrieve middleware list from this NSO + if (!retrieveMiddlewareListFromNso(ncmStorage, ncaId, &aes_ctx, curFilename, curFileOffset, &nsoHeader, programInfoXml)) + { + proceed = false; + break; + } + } + + if (!proceed) goto out; + + strcat(programInfoXml, " \n"); + + // Leave these fields empty (for now) + strcat(programInfoXml, " \n"); + strcat(programInfoXml, " \n"); + + // Symbols list from main NSO + strcat(programInfoXml, " \n"); + + for(i = 0; i < nca_pfs0_header.file_cnt; i++) + { + nso_header_t nsoHeader; + char *curFilename = (nca_pfs0_str_table + nca_pfs0_entries[i].filename_offset); + u64 curFileOffset = (nca_pfs0_data_offset + nca_pfs0_entries[i].file_offset); + + if (!processNcaCtrSectionBlock(ncmStorage, ncaId, &aes_ctx, curFileOffset, &nsoHeader, sizeof(nso_header_t), false)) + { + breaks++; + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Failed to read 0x%016lX bytes from \"%s\" in Program NCA section #0 PFS0 partition!", sizeof(nso_header_t), curFilename); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + proceed = false; + break; + } + + // Check if we're dealing with the main NSO + if (strlen(curFilename) != 4 || strncmp(curFilename, "main", 4) != 0 || bswap_32(nsoHeader.magic) != NSO_MAGIC) continue; + + // Retrieve symbols list from main NSO + if (!retrieveSymbolsListFromNso(ncmStorage, ncaId, &aes_ctx, curFilename, curFileOffset, &nsoHeader, programInfoXml)) proceed = false; + + break; + } + + if (!proceed) goto out; + + strcat(programInfoXml, " \n"); + + // Leave this field empty (for now) + strcat(programInfoXml, " \n"); + + strcat(programInfoXml, ""); + + *outBuf = programInfoXml; + *outBufSize = strlen(programInfoXml); + + success = true; + +out: + if (npdm_acid_section_b64) free(npdm_acid_section_b64); + + if (npdm_acid_section) free(npdm_acid_section); + + if (!success && programInfoXml) free(programInfoXml); + + if (nca_pfs0_str_table) free(nca_pfs0_str_table); + + if (nca_pfs0_entries) free(nca_pfs0_entries); + + return success; +} + char *getNacpLangName(u8 val) { char *out = NULL; @@ -1959,9 +2833,29 @@ char *getNacpRequiredNetworkServiceLicenseOnLaunchFlag(u8 val) return out; } -bool generateNacpXmlFromNca(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, nca_header_t *dec_nca_header, u8 *decrypted_nca_keys, char **outBuf) +char *getNacpJitConfigurationFlag(u64 flag) { - if (!ncmStorage || !ncaId || !dec_nca_header || !decrypted_nca_keys || !outBuf) + char *out = NULL; + + switch(flag) + { + case 0: + out = "None"; + break; + case 1: + out = "Enabled"; + break; + default: + out = "Unknown"; + break; + } + + return out; +} + +bool retrieveNacpDataFromNca(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, nca_header_t *dec_nca_header, u8 *decrypted_nca_keys, char **out_nacp_xml, u64 *out_nacp_xml_size, nacp_icons_ctx **out_nacp_icons_ctx, u8 *out_nacp_icons_ctx_cnt) +{ + if (!ncmStorage || !ncaId || !dec_nca_header || !decrypted_nca_keys || !out_nacp_xml || !out_nacp_xml_size || !out_nacp_icons_ctx || !out_nacp_icons_ctx_cnt) { uiDrawString("Error: invalid parameters to generate NACP XML!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); return false; @@ -1974,9 +2868,26 @@ bool generateNacpXmlFromNca(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId nacp_t controlNacp; char *nacpXml = NULL; - u8 i; + u8 i = 0, j = 0; char tmp[NAME_BUF_LEN] = {'\0'}; + u8 nacpIconCnt = 0; + nacp_icons_ctx *nacpIcons = NULL; + + bool found_icon = false; + u8 languageIconHash[0x20]; + char languageIconHashStr[0x21]; + + char ncaIdStr[0x21] = {'\0'}; + convertDataToHexString(ncaId->c, 0x10, ncaIdStr, 0x21); + + char dataStr[100] = {'\0'}; + + u8 null_key[0x10]; + memset(null_key, 0, 0x10); + + bool availableSDC = false, availableRDC = false; + if (!readRomFsEntryFromNca(ncmStorage, ncaId, dec_nca_header, decrypted_nca_keys)) return false; // Look for the control.nacp file @@ -1995,14 +2906,14 @@ bool generateNacpXmlFromNca(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId if (!found_nacp) { - uiDrawString("Error: unable to find control.nacp file in Control NCA RomFS section!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + uiDrawString("Error: unable to find \"control.nacp\" file in Control NCA RomFS section!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); goto out; } - if (!processNcaCtrSectionBlock(ncmStorage, ncaId, romFsContext.romfs_filedata_offset + entry->dataOff, &controlNacp, sizeof(nacp_t), &(romFsContext.aes_ctx), false)) + if (!processNcaCtrSectionBlock(ncmStorage, ncaId, &(romFsContext.aes_ctx), romFsContext.romfs_filedata_offset + entry->dataOff, &controlNacp, sizeof(nacp_t), false)) { breaks++; - uiDrawString("Failed to read Control.nacp from RomFS section in Control NCA!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + uiDrawString("Failed to read \"control.nacp\" from RomFS section in Control NCA!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); goto out; } @@ -2014,18 +2925,18 @@ bool generateNacpXmlFromNca(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId goto out; } - sprintf(nacpXml, "\xEF\xBB\xBF\r\n" \ - "\r\n"); + sprintf(nacpXml, "\n" \ + "\n"); for(i = 0; i < 16; i++) { if (strlen(controlNacp.lang[i].name) || strlen(controlNacp.lang[i].author)) { - sprintf(tmp, " \r\n" \ - " <Language>%s</Language>\r\n" \ - " <Name>%s</Name>\r\n" \ - " <Publisher>%s</Publisher>\r\n" \ - " \r\n", \ + sprintf(tmp, " \n" \ + " <Language>%s</Language>\n" \ + " <Name>%s</Name>\n" \ + " <Publisher>%s</Publisher>\n" \ + " \n", \ getNacpLangName(i), \ controlNacp.lang[i].name, \ controlNacp.lang[i].author); @@ -2036,19 +2947,19 @@ bool generateNacpXmlFromNca(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId if (strlen(controlNacp.Isbn)) { - sprintf(tmp, " %s\r\n", controlNacp.Isbn); + sprintf(tmp, " %s\n", controlNacp.Isbn); strcat(nacpXml, tmp); } else { - strcat(nacpXml, " \r\n"); + strcat(nacpXml, " \n"); } - sprintf(tmp, " %s\r\n", getNacpStartupUserAccount(controlNacp.StartupUserAccount)); + sprintf(tmp, " %s\n", getNacpStartupUserAccount(controlNacp.StartupUserAccount)); strcat(nacpXml, tmp); - sprintf(tmp, " %s\r\n", getNacpUserAccountSwitchLock(controlNacp.UserAccountSwitchLock)); + sprintf(tmp, " %s\n", getNacpUserAccountSwitchLock(controlNacp.UserAccountSwitchLock)); strcat(nacpXml, tmp); - sprintf(tmp, " %s\r\n", getNacpParentalControlFlag(controlNacp.ParentalControlFlag)); + sprintf(tmp, " %s\n", getNacpParentalControlFlag(controlNacp.ParentalControlFlag)); strcat(nacpXml, tmp); for(i = 0; i < 16; i++) @@ -2056,31 +2967,32 @@ bool generateNacpXmlFromNca(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId u8 bit = (u8)((controlNacp.SupportedLanguageFlag >> i) & 0x1); if (bit) { - sprintf(tmp, " %s\r\n", getNacpLangName(i)); + sprintf(tmp, " %s\n", getNacpLangName(i)); strcat(nacpXml, tmp); + nacpIconCnt++; } } - sprintf(tmp, " %s\r\n", getNacpScreenshot(controlNacp.Screenshot)); + sprintf(tmp, " %s\n", getNacpScreenshot(controlNacp.Screenshot)); strcat(nacpXml, tmp); - sprintf(tmp, " %s\r\n", getNacpVideoCapture(controlNacp.VideoCapture)); + sprintf(tmp, " %s\n", getNacpVideoCapture(controlNacp.VideoCapture)); strcat(nacpXml, tmp); - sprintf(tmp, " 0x%016lx\r\n", controlNacp.PresenceGroupId); + sprintf(tmp, " 0x%016lx\n", controlNacp.PresenceGroupId); strcat(nacpXml, tmp); - sprintf(tmp, " %s\r\n", controlNacp.DisplayVersion); + sprintf(tmp, " %s\n", controlNacp.DisplayVersion); strcat(nacpXml, tmp); for(i = 0; i < 32; i++) { if (controlNacp.RatingAge[i] != 0xFF) { - sprintf(tmp, " \r\n" \ - " %s\r\n" \ - " %u\r\n" \ - " \r\n", \ + sprintf(tmp, " \n" \ + " %s\n" \ + " %u\n" \ + " \n", \ getNacpRatingAgeOrganizationName(i), \ controlNacp.RatingAge[i]); @@ -2088,130 +3000,359 @@ bool generateNacpXmlFromNca(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId } } - sprintf(tmp, " %s\r\n", getNacpDataLossConfirmation(controlNacp.DataLossConfirmation)); + sprintf(tmp, " %s\n", getNacpDataLossConfirmation(controlNacp.DataLossConfirmation)); strcat(nacpXml, tmp); - sprintf(tmp, " %s\r\n", getNacpPlayLogPolicy(controlNacp.PlayLogPolicy)); + sprintf(tmp, " %s\n", getNacpPlayLogPolicy(controlNacp.PlayLogPolicy)); strcat(nacpXml, tmp); - sprintf(tmp, " 0x%016lx\r\n", controlNacp.SaveDataOwnerId); + sprintf(tmp, " 0x%016lx\n", controlNacp.SaveDataOwnerId); strcat(nacpXml, tmp); - sprintf(tmp, " 0x%016lx\r\n", controlNacp.UserAccountSaveDataSize); + sprintf(tmp, " 0x%016lx\n", controlNacp.UserAccountSaveDataSize); strcat(nacpXml, tmp); - sprintf(tmp, " 0x%016lx\r\n", controlNacp.UserAccountSaveDataJournalSize); + sprintf(tmp, " 0x%016lx\n", controlNacp.UserAccountSaveDataJournalSize); strcat(nacpXml, tmp); - sprintf(tmp, " 0x%016lx\r\n", controlNacp.DeviceSaveDataSize); + sprintf(tmp, " 0x%016lx\n", controlNacp.DeviceSaveDataSize); strcat(nacpXml, tmp); - sprintf(tmp, " 0x%016lx\r\n", controlNacp.DeviceSaveDataJournalSize); + sprintf(tmp, " 0x%016lx\n", controlNacp.DeviceSaveDataJournalSize); strcat(nacpXml, tmp); - sprintf(tmp, " 0x%016lx\r\n", controlNacp.BcatDeliveryCacheStorageSize); + sprintf(tmp, " 0x%016lx\n", controlNacp.BcatDeliveryCacheStorageSize); strcat(nacpXml, tmp); if (strlen(controlNacp.ApplicationErrorCodeCategory)) { - sprintf(tmp, " %s\r\n", controlNacp.ApplicationErrorCodeCategory); + sprintf(tmp, " %s\n", controlNacp.ApplicationErrorCodeCategory); strcat(nacpXml, tmp); } else { - strcat(nacpXml, " \r\n"); + strcat(nacpXml, " \n"); } - sprintf(tmp, " 0x%016lx\r\n", controlNacp.AddOnContentBaseId); + sprintf(tmp, " 0x%016lx\n", controlNacp.AddOnContentBaseId); strcat(nacpXml, tmp); - sprintf(tmp, " %s\r\n", getNacpLogoType(controlNacp.LogoType)); + sprintf(tmp, " %s\n", getNacpLogoType(controlNacp.LogoType)); strcat(nacpXml, tmp); - sprintf(tmp, " 0x%016lx\r\n", controlNacp.LocalCommunicationId[0]); + for(i = 0; i < 8; i++) + { + if (controlNacp.LocalCommunicationId[i] != 0) + { + sprintf(tmp, " 0x%016lx\n", controlNacp.LocalCommunicationId[i]); + strcat(nacpXml, tmp); + } + } + + sprintf(tmp, " %s\n", getNacpLogoHandling(controlNacp.LogoHandling)); strcat(nacpXml, tmp); - sprintf(tmp, " %s\r\n", getNacpLogoHandling(controlNacp.LogoHandling)); - strcat(nacpXml, tmp); + if (nacpIconCnt) + { + nacpIcons = calloc(nacpIconCnt, sizeof(nacp_icons_ctx)); + if (!nacpIcons) + { + uiDrawString("Error: unable to allocate memory for the NACP icons!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + goto out; + } + + for(i = 0; i < 16; i++) + { + u8 bit = (u8)((controlNacp.SupportedLanguageFlag >> i) & 0x1); + if (bit) + { + // Retrieve the icon file for this language and calculate its SHA-256 checksum + found_icon = false; + + memset(languageIconHash, 0, 0x20); + memset(languageIconHashStr, 0, 0x21); + + entryOffset = 0; + sprintf(tmp, "icon_%s.dat", getNacpLangName(i)); + + while(entryOffset < romFsContext.romfs_filetable_size) + { + entry = (romfs_file*)((u8*)romFsContext.romfs_file_entries + entryOffset); + + if (entry->parent == 0 && entry->nameLen == strlen(tmp) && !strncasecmp((char*)entry->name, tmp, strlen(tmp)) && entry->dataSize <= 0x20000) + { + found_icon = true; + break; + } + + entryOffset += round_up(ROMFS_NONAME_FILEENTRY_SIZE + entry->nameLen, 4); + } + + if (!found_icon) + { + nacpIconCnt--; + continue; + } + + strcat(nacpXml, " \n"); + + sprintf(tmp, " %s\n", getNacpLangName(i)); + strcat(nacpXml, tmp); + + // Fill details for our NACP icon context + sprintf(nacpIcons[j].filename, "%s.nx.%s.jpg", ncaIdStr, getNacpLangName(i)); + nacpIcons[j].icon_size = entry->dataSize; + + if (!processNcaCtrSectionBlock(ncmStorage, ncaId, &(romFsContext.aes_ctx), romFsContext.romfs_filedata_offset + entry->dataOff, nacpIcons[j].icon_data, nacpIcons[j].icon_size, false)) + { + breaks++; + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Failed to read \"%s\" from RomFS section in Control NCA!", tmp); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + goto out; + } + + sha256CalculateHash(languageIconHash, nacpIcons[j].icon_data, nacpIcons[j].icon_size); + + // Only retrieve the first half from the SHA-256 checksum + convertDataToHexString(languageIconHash, 0x10, languageIconHashStr, 0x21); + + // Now print the hash + sprintf(tmp, " %s\n", languageIconHashStr); + strcat(nacpXml, tmp); + + strcat(nacpXml, " \n"); + + j++; + } + } + } - sprintf(tmp, " 0x%016lx\r\n", controlNacp.SeedForPseudoDeviceId); + sprintf(tmp, " 0x%016lx\n", controlNacp.SeedForPseudoDeviceId); strcat(nacpXml, tmp); if (strlen(controlNacp.BcatPassphrase)) { - sprintf(tmp, " %s\r\n", controlNacp.BcatPassphrase); + sprintf(tmp, " %s\n", controlNacp.BcatPassphrase); strcat(nacpXml, tmp); } else { - strcat(nacpXml, " \r\n"); + strcat(nacpXml, " \n"); } - sprintf(tmp, " %s\r\n", getNacpStartupUserAccountOptionFlag(controlNacp.StartupUserAccountOptionFlag)); + sprintf(tmp, " %s\n", getNacpStartupUserAccountOptionFlag(controlNacp.StartupUserAccountOptionFlag)); strcat(nacpXml, tmp); - sprintf(tmp, " %s\r\n", getNacpAddOnContentRegistrationType(controlNacp.AddOnContentRegistrationType)); + sprintf(tmp, " %s\n", getNacpAddOnContentRegistrationType(controlNacp.AddOnContentRegistrationType)); strcat(nacpXml, tmp); - sprintf(tmp, " 0x%016lx\r\n", controlNacp.UserAccountSaveDataSizeMax); + sprintf(tmp, " 0x%016lx\n", controlNacp.UserAccountSaveDataSizeMax); strcat(nacpXml, tmp); - sprintf(tmp, " 0x%016lx\r\n", controlNacp.UserAccountSaveDataJournalSizeMax); + sprintf(tmp, " 0x%016lx\n", controlNacp.UserAccountSaveDataJournalSizeMax); strcat(nacpXml, tmp); - sprintf(tmp, " 0x%016lx\r\n", controlNacp.DeviceSaveDataSizeMax); + sprintf(tmp, " 0x%016lx\n", controlNacp.DeviceSaveDataSizeMax); strcat(nacpXml, tmp); - sprintf(tmp, " 0x%016lx\r\n", controlNacp.DeviceSaveDataJournalSizeMax); + sprintf(tmp, " 0x%016lx\n", controlNacp.DeviceSaveDataJournalSizeMax); strcat(nacpXml, tmp); - sprintf(tmp, " 0x%016lx\r\n", controlNacp.TemporaryStorageSize); + sprintf(tmp, " 0x%016lx\n", controlNacp.TemporaryStorageSize); strcat(nacpXml, tmp); - sprintf(tmp, " 0x%016lx\r\n", controlNacp.CacheStorageSize); + sprintf(tmp, " 0x%016lx\n", controlNacp.CacheStorageSize); strcat(nacpXml, tmp); - sprintf(tmp, " 0x%016lx\r\n", controlNacp.CacheStorageJournalSize); + sprintf(tmp, " 0x%016lx\n", controlNacp.CacheStorageJournalSize); strcat(nacpXml, tmp); - sprintf(tmp, " 0x%016lx\r\n", controlNacp.CacheStorageDataAndJournalSizeMax); + sprintf(tmp, " 0x%016lx\n", controlNacp.CacheStorageDataAndJournalSizeMax); strcat(nacpXml, tmp); - sprintf(tmp, " 0x%04x\r\n", controlNacp.CacheStorageIndexMax); + sprintf(tmp, " 0x%04x\n", controlNacp.CacheStorageIndexMax); strcat(nacpXml, tmp); - sprintf(tmp, " %s\r\n", getNacpHdcp(controlNacp.Hdcp)); + sprintf(tmp, " %s\n", getNacpHdcp(controlNacp.Hdcp)); strcat(nacpXml, tmp); - sprintf(tmp, " %s\r\n", getNacpCrashReport(controlNacp.CrashReport)); + sprintf(tmp, " %s\n", getNacpCrashReport(controlNacp.CrashReport)); strcat(nacpXml, tmp); - sprintf(tmp, " %s\r\n", getNacpRuntimeAddOnContentInstall(controlNacp.RuntimeAddOnContentInstall)); + sprintf(tmp, " %s\n", getNacpRuntimeAddOnContentInstall(controlNacp.RuntimeAddOnContentInstall)); strcat(nacpXml, tmp); - sprintf(tmp, " 0x%016lx\r\n", controlNacp.PlayLogQueryableApplicationId[0]); + sprintf(tmp, " 0x%016lx\n", controlNacp.PlayLogQueryableApplicationId[0]); strcat(nacpXml, tmp); - sprintf(tmp, " %s\r\n", getNacpPlayLogQueryCapability(controlNacp.PlayLogQueryCapability)); + sprintf(tmp, " %s\n", getNacpPlayLogQueryCapability(controlNacp.PlayLogQueryCapability)); strcat(nacpXml, tmp); - sprintf(tmp, " %s\r\n", getNacpRepairFlag(controlNacp.RepairFlag)); + sprintf(tmp, " %s\n", getNacpRepairFlag(controlNacp.RepairFlag)); strcat(nacpXml, tmp); - sprintf(tmp, " %s\r\n", getNacpAttributeFlag(controlNacp.AttributeFlag)); + sprintf(tmp, " %s\n", getNacpAttributeFlag(controlNacp.AttributeFlag)); strcat(nacpXml, tmp); - sprintf(tmp, " %u\r\n", controlNacp.ProgramIndex); + sprintf(tmp, " %u\n", controlNacp.ProgramIndex); strcat(nacpXml, tmp); - sprintf(tmp, " %s\r\n", getNacpRequiredNetworkServiceLicenseOnLaunchFlag(controlNacp.RequiredNetworkServiceLicenseOnLaunchFlag)); + sprintf(tmp, " %s\n", getNacpRequiredNetworkServiceLicenseOnLaunchFlag(controlNacp.RequiredNetworkServiceLicenseOnLaunchFlag)); strcat(nacpXml, tmp); - strcat(nacpXml, "\r\n"); + // Check if we actually have valid NeighborDetectionClientConfiguration values + availableSDC = (controlNacp.SendDataConfiguration.id != 0 && memcmp(controlNacp.SendDataConfiguration.key, null_key, 0x10) != 0); - *outBuf = nacpXml; + for(i = 0; i < 16; i++) + { + if (controlNacp.ReceivableDataConfiguration[i].id != 0 && memcmp(controlNacp.ReceivableDataConfiguration[i].key, null_key, 0x10) != 0) + { + availableRDC = true; + break; + } + } + + if (availableSDC || availableRDC) + { + strcat(nacpXml, " \n"); + + if (availableSDC) + { + convertDataToHexString(controlNacp.SendDataConfiguration.key, 0x10, dataStr, 100); + + sprintf(tmp, " \n" \ + " 0x%016lx\n" \ + " %s\n" \ + " \n", \ + controlNacp.SendDataConfiguration.id, \ + dataStr); + + strcat(nacpXml, tmp); + } + + if (availableRDC) + { + for(i = 0; i < 16; i++) + { + if (controlNacp.ReceivableDataConfiguration[i].id != 0 && memcmp(controlNacp.ReceivableDataConfiguration[i].key, null_key, 0x10) != 0) + { + convertDataToHexString(controlNacp.ReceivableDataConfiguration[i].key, 0x10, dataStr, 100); + + sprintf(tmp, " \n" \ + " 0x%016lx\n" \ + " %s\n" \ + " \n", \ + controlNacp.ReceivableDataConfiguration[i].id, \ + dataStr); + + strcat(nacpXml, tmp); + } + } + } + + strcat(nacpXml, " \n"); + } + + sprintf(tmp, " \n" \ + " %s\n" \ + " 0x%016lx\n" \ + " \n", \ + getNacpJitConfigurationFlag(controlNacp.JitConfigurationFlag), \ + controlNacp.JitMemorySize); + + strcat(nacpXml, tmp); + + strcat(nacpXml, ""); + + *out_nacp_xml = nacpXml; + *out_nacp_xml_size = strlen(nacpXml); + + if (nacpIconCnt) + { + *out_nacp_icons_ctx = nacpIcons; + *out_nacp_icons_ctx_cnt = nacpIconCnt; + } success = true; out: + if (!success) + { + if (nacpIcons != NULL) free(nacpIcons); + + if (nacpXml != NULL) free(nacpXml); + } + // Manually free these pointers - // Calling freeRomFsContext() would also close the ncmStorage handle and the gamecard IStorage instance (if available) + // Calling freeRomFsContext() would also close the ncmStorage handle + free(romFsContext.romfs_dir_entries); + romFsContext.romfs_dir_entries = NULL; + + free(romFsContext.romfs_file_entries); + romFsContext.romfs_file_entries = NULL; + + return success; +} + +bool retrieveLegalInfoXmlFromNca(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, nca_header_t *dec_nca_header, u8 *decrypted_nca_keys, char **outBuf, u64 *outBufSize) +{ + if (!ncmStorage || !ncaId || !dec_nca_header || !decrypted_nca_keys || !outBuf) + { + uiDrawString("Error: invalid parameters to retrieve \"legalinfo.xml\"!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + return false; + } + + u64 entryOffset = 0; + romfs_file *entry = NULL; + bool found_legalinfo = false, success = false; + + u64 legalInfoXmlSize = 0; + char *legalInfoXml = NULL; + + if (!readRomFsEntryFromNca(ncmStorage, ncaId, dec_nca_header, decrypted_nca_keys)) return false; + + // Look for the legalinfo.xml file + while(entryOffset < romFsContext.romfs_filetable_size) + { + entry = (romfs_file*)((u8*)romFsContext.romfs_file_entries + entryOffset); + + if (entry->parent == 0 && entry->nameLen == 13 && !strncasecmp((char*)entry->name, "legalinfo.xml", 13)) + { + found_legalinfo = true; + break; + } + + entryOffset += round_up(ROMFS_NONAME_FILEENTRY_SIZE + entry->nameLen, 4); + } + + if (!found_legalinfo) + { + uiDrawString("Error: unable to find \"legalinfo.xml\" file in Manual NCA RomFS section!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + goto out; + } + + // Allocate memory for the legalinfo.xml contents + legalInfoXmlSize = entry->dataSize; + legalInfoXml = calloc(legalInfoXmlSize, sizeof(char)); + if (!legalInfoXml) + { + uiDrawString("Error: unable to allocate memory for the \"legalinfo.xml\" contents!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + goto out; + } + + if (!processNcaCtrSectionBlock(ncmStorage, ncaId, &(romFsContext.aes_ctx), romFsContext.romfs_filedata_offset + entry->dataOff, legalInfoXml, legalInfoXmlSize, false)) + { + breaks++; + uiDrawString("Failed to read \"legalinfo.xml\" from RomFS section in Manual NCA!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + goto out; + } + + *outBuf = legalInfoXml; + *outBufSize = legalInfoXmlSize; + + success = true; + +out: + if (!success && legalInfoXml != NULL) free(legalInfoXml); + + // Manually free these pointers + // Calling freeRomFsContext() would also close the ncmStorage handle free(romFsContext.romfs_dir_entries); romFsContext.romfs_dir_entries = NULL; diff --git a/source/nca.h b/source/nca.h index 8e30495..ef08b5e 100644 --- a/source/nca.h +++ b/source/nca.h @@ -37,6 +37,8 @@ #define IVFC_MAGIC (u32)0x49564643 // "IVFC" #define IVFC_MAX_LEVEL 6 +#define BKTR_MAGIC (u32)0x424B5452 // "BKTR" + #define ROMFS_HEADER_SIZE 0x50 #define ROMFS_ENTRY_EMPTY (u32)0xFFFFFFFF @@ -53,7 +55,9 @@ #define NSP_NCA_FILENAME_LENGTH 0x25 // NCA ID + ".nca" + NULL terminator #define NSP_CNMT_FILENAME_LENGTH 0x2A // NCA ID + ".cnmt.nca" / ".cnmt.xml" + NULL terminator -#define NSP_NACP_FILENAME_LENGTH 0x2A // NCA ID + ".nacp.xml" + NULL terminator +#define NSP_PROGRAM_XML_FILENAME_LENGTH 0x31 // NCA ID + ".programinfo.xml" + NULL terminator +#define NSP_NACP_XML_FILENAME_LENGTH 0x2A // NCA ID + ".nacp.xml" + NULL terminator +#define NSP_LEGAL_XML_FILENAME_LENGTH 0x2F // NCA ID + ".legalinfo.xml" + NULL terminator #define NSP_TIK_FILENAME_LENGTH 0x25 // Rights ID + ".tik" + NULL terminator #define NSP_CERT_FILENAME_LENGTH 0x26 // Rights ID + ".cert" + NULL terminator @@ -131,6 +135,22 @@ typedef struct { u8 _0xE0[0x58]; } PACKED romfs_superblock_t; +typedef struct { + u64 offset; + u64 size; + u32 magic; /* "BKTR" */ + u32 _0x14; /* Version? */ + u32 num_entries; + u32 _0x1C; /* Reserved? */ +} PACKED bktr_header_t; + +typedef struct { + ivfc_hdr_t ivfc_header; + u8 _0xE0[0x18]; + bktr_header_t relocation_header; + bktr_header_t subsection_header; +} PACKED bktr_superblock_t; + /* NCA FS header. */ typedef struct { u8 _0x0; @@ -142,6 +162,7 @@ typedef struct { union { /* FS-specific superblock. Size = 0x138. */ pfs0_superblock_t pfs0_superblock; romfs_superblock_t romfs_superblock; + bktr_superblock_t bktr_superblock; }; union { u8 section_ctr[0x8]; @@ -259,6 +280,7 @@ typedef struct { u8 *block_data[2]; u64 block_offset[2]; // Relative to NCA start u64 block_size[2]; + u64 acid_pubkey_offset; // Relative to block_data[0] start } PACKED nca_program_mod_data; typedef struct { @@ -320,6 +342,8 @@ typedef struct { NcmContentStorage ncmStorage; NcmNcaId ncaId; Aes128CtrContext aes_ctx; + u64 section_offset; // Relative to NCA start + u64 section_size; u64 romfs_offset; // Relative to NCA start u64 romfs_size; u64 romfs_dirtable_offset; // Relative to NCA start @@ -331,11 +355,87 @@ typedef struct { u64 romfs_filedata_offset; // Relative to NCA start } PACKED romfs_ctx_t; +typedef struct { + u64 virt_offset; + u64 phys_offset; + u32 is_patch; +} PACKED bktr_relocation_entry_t; + +typedef struct { + u32 _0x0; + u32 num_entries; + u64 virtual_offset_end; + bktr_relocation_entry_t entries[0x3FF0 / sizeof(bktr_relocation_entry_t)]; + u8 padding[0x3FF0 % sizeof(bktr_relocation_entry_t)]; +} PACKED bktr_relocation_bucket_t; + +typedef struct { + u32 _0x0; + u32 num_buckets; + u64 total_size; + u64 bucket_virtual_offsets[0x3FF0 / sizeof(u64)]; + bktr_relocation_bucket_t buckets[]; +} PACKED bktr_relocation_block_t; + +typedef struct { + u64 offset; + u32 _0x8; + u32 ctr_val; +} PACKED bktr_subsection_entry_t; + +typedef struct { + u32 _0x0; + u32 num_entries; + u64 physical_offset_end; + bktr_subsection_entry_t entries[0x3FF]; +} PACKED bktr_subsection_bucket_t; + +typedef struct { + u32 _0x0; + u32 num_buckets; + u64 total_size; + u64 bucket_physical_offsets[0x3FF0 / sizeof(u64)]; + bktr_subsection_bucket_t buckets[]; +} PACKED bktr_subsection_block_t; + +typedef struct { + NcmContentStorage ncmStorage; + NcmNcaId ncaId; + Aes128CtrContext aes_ctx; + u64 section_offset; // Relative to NCA start + u64 section_size; + bktr_superblock_t superblock; + bktr_relocation_block_t *relocation_block; + bktr_subsection_block_t *subsection_block; + u64 virtual_seek; // Relative to section start + u64 bktr_seek; // Relative to section start (patch BKTR section) + u64 base_seek; // Relative to section start (base application RomFS section) + u64 romfs_offset; // Relative to section start + u64 romfs_size; + u64 romfs_dirtable_offset; // Relative to section start + u64 romfs_dirtable_size; + romfs_dir *romfs_dir_entries; + u64 romfs_filetable_offset; // Relative to section start + u64 romfs_filetable_size; + romfs_file *romfs_file_entries; + u64 romfs_filedata_offset; // Relative to section start +} PACKED bktr_ctx_t; + typedef struct { u8 type; // 1 = Dir, 2 = File u64 offset; // Relative to directory/file table, depending on type } PACKED romfs_browser_entry; +typedef struct { + u64 id; + u8 key[0x10]; +} PACKED send_data_configuration; + +typedef struct { + u64 id; + u8 key[0x10]; +} PACKED receivable_data_configurations; + typedef struct { NacpLanguageEntry lang[16]; char Isbn[0x25]; @@ -364,7 +464,7 @@ typedef struct { u8 LogoType; u8 LogoHandling; u8 RuntimeAddOnContentInstall; - u8 _0x30F3[0x3]; + u8 Reserved_0x30F3[0x3]; u8 CrashReport; u8 Hdcp; u64 SeedForPseudoDeviceId; @@ -380,22 +480,35 @@ typedef struct { u64 CacheStorageJournalSize; u64 CacheStorageDataAndJournalSizeMax; u16 CacheStorageIndexMax; - u8 reserved_0x318a[0x6]; + u8 Reserved_0x318A[0x6]; u64 PlayLogQueryableApplicationId[0x10]; u8 PlayLogQueryCapability; u8 RepairFlag; u8 ProgramIndex; u8 RequiredNetworkServiceLicenseOnLaunchFlag; - u8 Reserved[0xDEC]; + u8 Reserved_0x3214[0x4]; + send_data_configuration SendDataConfiguration; + receivable_data_configurations ReceivableDataConfiguration[0x10]; + u64 JitConfigurationFlag; + u64 JitMemorySize; + u8 Reserved[0xC40]; } PACKED nacp_t; +typedef struct { + char filename[100]; + u64 icon_size; + u8 icon_data[0x20000]; +} PACKED nacp_icons_ctx; + void generateCnmtXml(cnmt_xml_program_info *xml_program_info, cnmt_xml_content_info *xml_content_info, char *out); void convertNcaSizeToU64(const u8 size[0x6], u64 *out); void convertU64ToNcaSize(const u64 size, u8 out[0x6]); -bool processNcaCtrSectionBlock(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, u64 offset, void *outBuf, size_t bufSize, Aes128CtrContext *ctx, bool encrypt); +bool processNcaCtrSectionBlock(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, Aes128CtrContext *ctx, u64 offset, void *outBuf, size_t bufSize, bool encrypt); + +bool readBktrSectionBlock(u64 offset, void *outBuf, size_t bufSize); bool encryptNcaHeader(nca_header_t *input, u8 *outBuf, u64 outBufSize); @@ -411,6 +524,12 @@ bool readExeFsEntryFromNca(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, bool readRomFsEntryFromNca(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, nca_header_t *dec_nca_header, u8 *decrypted_nca_keys); -bool generateNacpXmlFromNca(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, nca_header_t *dec_nca_header, u8 *decrypted_nca_keys, char **outBuf); +bool readBktrEntryFromNca(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, nca_header_t *dec_nca_header, u8 *decrypted_nca_keys); + +bool generateProgramInfoXml(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, nca_header_t *dec_nca_header, u8 *decrypted_nca_keys, nca_program_mod_data *program_mod_data, char **outBuf, u64 *outBufSize); + +bool retrieveNacpDataFromNca(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, nca_header_t *dec_nca_header, u8 *decrypted_nca_keys, char **out_nacp_xml, u64 *out_nacp_xml_size, nacp_icons_ctx **out_nacp_icons_ctx, u8 *out_nacp_icons_ctx_cnt); + +bool retrieveLegalInfoXmlFromNca(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, nca_header_t *dec_nca_header, u8 *decrypted_nca_keys, char **outBuf, u64 *outBufSize); #endif diff --git a/source/nso.c b/source/nso.c new file mode 100644 index 0000000..77d7bc4 --- /dev/null +++ b/source/nso.c @@ -0,0 +1,438 @@ +#include +#include +#include +#include + +#include "nso.h" +#include "lz4.h" +#include "util.h" +#include "ui.h" + +/* Extern variables */ + +extern int breaks; +extern int font_height; + +extern char strbuf[NAME_BUF_LEN * 4]; + +/* Statically allocated variables */ + +static u8 *nsoBinaryData = NULL; +static u64 nsoBinaryDataSize = 0; + +static u64 nsoBinaryTextSectionOffset = 0; +static u64 nsoBinaryTextSectionSize = 0; + +static u64 nsoBinaryRodataSectionOffset = 0; +static u64 nsoBinaryRodataSectionSize = 0; + +static u64 nsoBinaryDataSectionOffset = 0; +static u64 nsoBinaryDataSectionSize = 0; + +void freeNsoBinaryData() +{ + if (nsoBinaryData) + { + free(nsoBinaryData); + nsoBinaryData = NULL; + } + + nsoBinaryDataSize = 0; + + nsoBinaryTextSectionOffset = 0; + nsoBinaryTextSectionSize = 0; + + nsoBinaryRodataSectionOffset = 0; + nsoBinaryRodataSectionSize = 0; + + nsoBinaryDataSectionOffset = 0; + nsoBinaryDataSectionSize = 0; +} + +bool loadNsoBinaryData(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, Aes128CtrContext *aes_ctx, u64 nso_base_offset, nso_header_t *nsoHeader) +{ + if (!ncmStorage || !ncaId || !aes_ctx || !nso_base_offset || !nsoHeader) + { + uiDrawString("Error: invalid parameters to load .text, .rodata and .data sections from NSO in Program NCA!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + return false; + } + + u8 i; + + u8 *nsoTextSection = NULL; + u64 nsoTextSectionSize = 0; + + u8 *nsoRodataSection = NULL; + u64 nsoRodataSectionSize = 0; + + u8 *nsoDataSection = NULL; + u64 nsoDataSectionSize = 0; + + u8 *curCompressedSection; + u64 curCompressedSectionSize; + u64 curCompressedSectionOffset; + u8 curSectionFlag; + + u8 *curDecompressedSection; + u64 curDecompressedSectionSize; + + bool success = true; + + freeNsoBinaryData(); + + for(i = 0; i < 3; i++) + { + curCompressedSection = NULL; + curCompressedSectionSize = (i == 0 ? (u64)nsoHeader->text_compressed_size : (i == 1 ? (u64)nsoHeader->rodata_compressed_size : (u64)nsoHeader->data_compressed_size)); + curCompressedSectionOffset = (nso_base_offset + (i == 0 ? (u64)nsoHeader->text_segment_header.file_offset : (i == 1 ? (u64)nsoHeader->rodata_segment_header.file_offset : (u64)nsoHeader->data_segment_header.file_offset))); + curSectionFlag = (1 << i); + + curDecompressedSection = NULL; + curDecompressedSectionSize = (i == 0 ? (u64)nsoHeader->text_segment_header.decompressed_size : (i == 1 ? (u64)nsoHeader->rodata_segment_header.decompressed_size : (u64)nsoHeader->data_segment_header.decompressed_size)); + + // Load section + curCompressedSection = malloc(curCompressedSectionSize); + if (!curCompressedSection) + { + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Error: unable to allocate memory for the compressed %s section from NSO in Program NCA!", (i == 0 ? ".text" : (i == 1 ? ".rodata" : ".data"))); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + success = false; + break; + } + + if (!processNcaCtrSectionBlock(ncmStorage, ncaId, aes_ctx, curCompressedSectionOffset, curCompressedSection, curCompressedSectionSize, false)) + { + breaks++; + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Failed to read 0x%016lX bytes %s section from NSO in Program NCA!", curCompressedSectionSize, (i == 0 ? ".text" : (i == 1 ? ".rodata" : ".data"))); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + free(curCompressedSection); + success = false; + break; + } + + if (nsoHeader->flags & curSectionFlag) + { + if (curDecompressedSectionSize <= curCompressedSectionSize) + { + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Error: invalid decompressed size for %s section from NSO in Program NCA!", (i == 0 ? ".text" : (i == 1 ? ".rodata" : ".data"))); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + free(curCompressedSection); + success = false; + break; + } + + // Uncompress section + curDecompressedSection = malloc(curDecompressedSectionSize); + if (!curDecompressedSection) + { + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Error: unable to allocate memory for the decompressed %s section from NSO in Program NCA!", (i == 0 ? ".text" : (i == 1 ? ".rodata" : ".data"))); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + free(curCompressedSection); + success = false; + break; + } + + if (LZ4_decompress_safe((const char*)curCompressedSection, (char*)curDecompressedSection, (int)curCompressedSectionSize, (int)curDecompressedSectionSize) != (int)curDecompressedSectionSize) + { + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Error: unable to allocate memory for the decompressed %s section from NSO in Program NCA!", (i == 0 ? ".text" : (i == 1 ? ".rodata" : ".data"))); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + free(curDecompressedSection); + free(curCompressedSection); + success = false; + break; + } + + free(curCompressedSection); + + switch(i) + { + case 0: + nsoTextSection = curDecompressedSection; + nsoTextSectionSize = curDecompressedSectionSize; + break; + case 1: + nsoRodataSection = curDecompressedSection; + nsoRodataSectionSize = curDecompressedSectionSize; + break; + case 2: + nsoDataSection = curDecompressedSection; + nsoDataSectionSize = curDecompressedSectionSize; + break; + default: + break; + } + } else { + switch(i) + { + case 0: + nsoTextSection = curCompressedSection; + nsoTextSectionSize = curCompressedSectionSize; + break; + case 1: + nsoRodataSection = curCompressedSection; + nsoRodataSectionSize = curCompressedSectionSize; + break; + case 2: + nsoDataSection = curCompressedSection; + nsoDataSectionSize = curCompressedSectionSize; + break; + default: + break; + } + } + } + + curCompressedSection = curDecompressedSection = NULL; + + if (success) + { + // Calculate full binary size + u64 finalTextSectionSize = nsoTextSectionSize; + u64 finalRodataSectionSize = nsoRodataSectionSize; + + nsoBinaryDataSize = finalTextSectionSize = nsoTextSectionSize; + + if ((u64)nsoHeader->rodata_segment_header.memory_offset > nsoBinaryDataSize) + { + nsoBinaryDataSize += ((u64)nsoHeader->rodata_segment_header.memory_offset - nsoBinaryDataSize); + } else + if ((u64)nsoHeader->rodata_segment_header.memory_offset < nsoBinaryDataSize) + { + finalTextSectionSize -= (nsoBinaryDataSize - (u64)nsoHeader->rodata_segment_header.memory_offset); + nsoBinaryDataSize -= (nsoBinaryDataSize - (u64)nsoHeader->rodata_segment_header.memory_offset); + } + + nsoBinaryDataSize += nsoRodataSectionSize; + + if ((u64)nsoHeader->data_segment_header.memory_offset > nsoBinaryDataSize) + { + nsoBinaryDataSize += ((u64)nsoHeader->data_segment_header.memory_offset - nsoBinaryDataSize); + } else + if ((u64)nsoHeader->data_segment_header.memory_offset < nsoBinaryDataSize) + { + finalRodataSectionSize -= (nsoBinaryDataSize - (u64)nsoHeader->data_segment_header.memory_offset); + nsoBinaryDataSize -= (nsoBinaryDataSize - (u64)nsoHeader->data_segment_header.memory_offset); + } + + nsoBinaryDataSize += nsoDataSectionSize; + + nsoBinaryData = calloc(nsoBinaryDataSize, sizeof(u8)); + if (nsoBinaryData) + { + memcpy(nsoBinaryData, nsoTextSection, finalTextSectionSize); + memcpy(nsoBinaryData + (u64)nsoHeader->rodata_segment_header.memory_offset, nsoRodataSection, finalRodataSectionSize); + memcpy(nsoBinaryData + (u64)nsoHeader->data_segment_header.memory_offset, nsoDataSection, nsoDataSectionSize); + + nsoBinaryTextSectionOffset = 0; + nsoBinaryTextSectionSize = finalTextSectionSize; + + nsoBinaryRodataSectionOffset = (u64)nsoHeader->rodata_segment_header.memory_offset; + nsoBinaryRodataSectionSize = finalRodataSectionSize; + + nsoBinaryDataSectionOffset = (u64)nsoHeader->data_segment_header.memory_offset; + nsoBinaryDataSectionSize = nsoDataSectionSize; + } else { + uiDrawString("Error: unable to allocate memory for full decompressed NSO in Program NCA!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + nsoBinaryDataSize = 0; + success = false; + } + } + + if (nsoTextSection) free(nsoTextSection); + if (nsoRodataSection) free(nsoRodataSection); + if (nsoDataSection) free(nsoDataSection); + + return success; +} + +bool retrieveMiddlewareListFromNso(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, Aes128CtrContext *aes_ctx, const char *nso_filename, u64 nso_base_offset, nso_header_t *nsoHeader, char *programInfoXml) +{ + if (!ncmStorage || !ncaId || !aes_ctx || !nso_filename || !strlen(nso_filename) || !nso_base_offset || !nsoHeader || !programInfoXml) + { + uiDrawString("Error: invalid parameters to retrieve middleware list from NSO in Program NCA!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + return false; + } + + u64 i; + + char tmp[NAME_BUF_LEN] = {'\0'}; + + if (!loadNsoBinaryData(ncmStorage, ncaId, aes_ctx, nso_base_offset, nsoHeader)) return false; + + for(i = 0; i < nsoBinaryRodataSectionSize; i++) + { + char *curStr = ((char*)nsoBinaryData + nsoBinaryRodataSectionOffset + i); + + if (strncmp(curStr, "SDK MW+", 7) != 0) continue; + + // Found a match + char *mwDev = (curStr + 7); + char *mwName = (strchr(mwDev, '+') + 1); + + // Filter nnSdk entries + if (!strncasecmp(mwName, "NintendoSdk_nnSdk", 17)) + { + i += strlen(curStr); + continue; + } + + sprintf(tmp, " \n" \ + " %s\n" \ + " %.*s\n" \ + " %s\n" \ + " \n", \ + mwName, \ + (int)(mwName - mwDev - 1), mwDev, \ + nso_filename); + + strcat(programInfoXml, tmp); + + // Update counter + i += strlen(curStr); + } + + freeNsoBinaryData(); + + return true; +} + +bool retrieveSymbolsListFromNso(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, Aes128CtrContext *aes_ctx, const char *nso_filename, u64 nso_base_offset, nso_header_t *nsoHeader, char *programInfoXml) +{ + if (!ncmStorage || !ncaId || !aes_ctx || !nso_filename || !strlen(nso_filename) || !nso_base_offset || !nsoHeader || !programInfoXml) + { + uiDrawString("Error: invalid parameters to retrieve symbols list from NSO in Program NCA!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + return false; + } + + u64 i; + + bool success = false; + + char tmp[NAME_BUF_LEN] = {'\0'}; + + u32 mod_magic_offset; + u32 mod_magic; + s32 dynamic_section_offset; + + bool armv7; + + u64 dynamic_block_size; + u64 dynamic_block_cnt; + + bool found_strtab = false; + u64 symbol_str_table_offset = 0; + + bool found_symtab = false; + u64 symbol_table_offset = 0; + + bool found_strsz = false; + u64 symbol_str_table_size = 0; + + char *symbol_str_table = NULL; + + u64 cur_symbol_table_offset = 0; + + if (!loadNsoBinaryData(ncmStorage, ncaId, aes_ctx, nso_base_offset, nsoHeader)) return false; + + mod_magic_offset = *((u32*)(&(nsoBinaryData[0x04]))); + mod_magic = *((u32*)(&(nsoBinaryData[mod_magic_offset]))); + dynamic_section_offset = ((s32)mod_magic_offset + *((s32*)(&(nsoBinaryData[mod_magic_offset + 0x04])))); + + if (bswap_32(mod_magic) != MOD_MAGIC) + { + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Error: invalid MOD0 magic word in decompressed NSO from Program NCA! (0x%08X)", bswap_32(mod_magic)); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + goto out; + } + + armv7 = (*((u64*)(&(nsoBinaryData[dynamic_section_offset]))) > (u64)0xFFFFFFFF || *((u64*)(&(nsoBinaryData[dynamic_section_offset + 0x10]))) > (u64)0xFFFFFFFF); + + // Read dynamic section + dynamic_block_size = (armv7 ? 0x08 : 0x10); + dynamic_block_cnt = ((nsoBinaryDataSize - dynamic_section_offset) / dynamic_block_size); + + for(i = 0; i < dynamic_block_cnt; i++) + { + if ((nsoBinaryDataSize - dynamic_section_offset - (i * dynamic_block_size)) < dynamic_block_size) break; + + u64 tag = (armv7 ? (u64)(*((u32*)(&(nsoBinaryData[dynamic_section_offset + (i * dynamic_block_size)])))) : *((u64*)(&(nsoBinaryData[dynamic_section_offset + (i * dynamic_block_size)])))); + u64 val = (armv7 ? (u64)(*((u32*)(&(nsoBinaryData[dynamic_section_offset + (i * dynamic_block_size) + 0x04])))) : *((u64*)(&(nsoBinaryData[dynamic_section_offset + (i * dynamic_block_size) + 0x08])))); + + if (!tag) break; + + if (tag == DT_STRTAB && !found_strtab) + { + // Retrieve symbol string table offset + symbol_str_table_offset = val; + found_strtab = true; + } + + if (tag == DT_SYMTAB && !found_symtab) + { + // Retrieve symbol table offset + symbol_table_offset = val; + found_symtab = true; + } + + if (tag == DT_STRSZ && !found_strsz) + { + // Retrieve symbol string table size + symbol_str_table_size = val; + found_strsz = true; + } + + if (found_strtab && found_symtab && found_strsz) break; + } + + if (!found_strtab || !found_symtab || !found_strsz) + { + // Nothing to do here if we can't find what we need + success = true; + goto out; + } + + // Point to the symbol string table + symbol_str_table = ((char*)nsoBinaryData + symbol_str_table_offset); + + // Retrieve symbol list + cur_symbol_table_offset = symbol_table_offset; + while(true) + { + if (symbol_table_offset < symbol_str_table_offset && cur_symbol_table_offset >= symbol_str_table_offset) break; + + u32 st_name = *((u32*)(&(nsoBinaryData[cur_symbol_table_offset]))); + u8 st_info = (armv7 ? nsoBinaryData[cur_symbol_table_offset + 0x0C] : nsoBinaryData[cur_symbol_table_offset + 0x04]); + //u8 st_other = (armv7 ? nsoBinaryData[cur_symbol_table_offset + 0x0D] : nsoBinaryData[cur_symbol_table_offset + 0x05]); + u16 st_shndx = (armv7 ? *((u16*)(&(nsoBinaryData[cur_symbol_table_offset + 0x0E]))) : *((u16*)(&(nsoBinaryData[cur_symbol_table_offset + 0x06])))); + u64 st_value = (armv7 ? (u64)(*((u32*)(&(nsoBinaryData[cur_symbol_table_offset + 0x04])))) : *((u64*)(&(nsoBinaryData[cur_symbol_table_offset + 0x08])))); + //u64 st_size = (armv7 ? (u64)(*((u32*)(&(nsoBinaryData[cur_symbol_table_offset + 0x08])))) : *((u64*)(&(nsoBinaryData[cur_symbol_table_offset + 0x10])))); + + //u8 st_vis = (st_other & 0x03); + u8 st_type = (st_info & 0x0F); + //u8 st_bind = (st_info >> 0x04); + + if (st_name >= symbol_str_table_size) break; + + cur_symbol_table_offset += (armv7 ? 0x10 : 0x18); + + // TO-DO: Add more filters? + if (!st_shndx && !st_value && st_type != ST_OBJECT) + { + sprintf(tmp, " \n" \ + " %s\n" \ + " %s\n" \ + " \n", \ + symbol_str_table + st_name, \ + nso_filename); + + strcat(programInfoXml, tmp); + } + } + + success = true; + +out: + freeNsoBinaryData(); + + return success; +} diff --git a/source/nso.h b/source/nso.h new file mode 100644 index 0000000..0237469 --- /dev/null +++ b/source/nso.h @@ -0,0 +1,58 @@ +#pragma once + +#ifndef __NSO_H__ +#define __NSO_H__ + +#include + +#define NSO_MAGIC (u32)0x4E534F30 // "NSO0" +#define MOD_MAGIC (u32)0x4D4F4430 // "MOD0" + +#define DT_STRTAB 0x05 +#define DT_SYMTAB 0x06 +#define DT_STRSZ 0x0A + +#define ST_OBJECT 0x01 + +typedef struct { + u32 file_offset; + u32 memory_offset; + u32 decompressed_size; +} PACKED segment_header_t; + +typedef struct { + u32 region_offset; + u32 region_size; +} PACKED rodata_extent_t; + +typedef struct { + u32 magic; + u32 version; + u32 reserved1; + u32 flags; + segment_header_t text_segment_header; + u32 module_offset; + segment_header_t rodata_segment_header; + u32 module_file_size; + segment_header_t data_segment_header; + u32 bss_size; + u8 elf_note_build_id[0x20]; + u32 text_compressed_size; + u32 rodata_compressed_size; + u32 data_compressed_size; + u8 reserved2[0x1C]; + rodata_extent_t rodata_api_info; + rodata_extent_t rodata_dynstr; + rodata_extent_t rodata_dynsym; + u8 text_decompressed_hash[0x20]; + u8 rodata_decompressed_hash[0x20]; + u8 data_decompressed_hash[0x20]; +} PACKED nso_header_t; + +// Retrieves the middleware list from a NSO stored in a partition from a NCA file +bool retrieveMiddlewareListFromNso(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, Aes128CtrContext *aes_ctx, const char *nso_filename, u64 nso_base_offset, nso_header_t *nsoHeader, char *programInfoXml); + +// Retrieves the symbols list from a NSO stored in a partition from a NCA file +bool retrieveSymbolsListFromNso(NcmContentStorage *ncmStorage, const NcmNcaId *ncaId, Aes128CtrContext *aes_ctx, const char *nso_filename, u64 nso_base_offset, nso_header_t *nsoHeader, char *programInfoXml); + +#endif diff --git a/source/ui.c b/source/ui.c index 5a3c038..b6e672a 100644 --- a/source/ui.c +++ b/source/ui.c @@ -23,6 +23,8 @@ extern AppletType programAppletType; +extern bool runningSxOs; + extern bool gameCardInserted; extern char gameCardSizeStr[32], trimmedCardSizeStr[32]; @@ -43,18 +45,26 @@ extern FsStorageId *titleAppStorageId; extern u32 titlePatchCount; extern u64 *titlePatchTitleID; extern u32 *titlePatchVersion; -extern FsStorageId *titleAppStorageId; +extern FsStorageId *titlePatchStorageId; extern u32 titleAddOnCount; extern u64 *titleAddOnTitleID; extern u32 *titleAddOnVersion; -extern FsStorageId *titleAppStorageId; +extern FsStorageId *titleAddOnStorageId; extern char **titleName; extern char **titleAuthor; extern char **titleAppVersionStr; extern u8 **titleIcon; +extern u32 sdCardTitleAppCount; +extern u32 sdCardTitlePatchCount; +extern u32 sdCardTitleAddOnCount; + +extern u32 nandUserTitleAppCount; +extern u32 nandUserTitlePatchCount; +extern u32 nandUserTitleAddOnCount; + extern char gameCardUpdateVersionStr[128]; extern char *filenameBuffer; @@ -95,7 +105,8 @@ curMenuType menuType; static bool orphanMode = false; static char titleSelectorStr[NAME_BUF_LEN] = {'\0'}; -static char dumpedNspInfoStr[NAME_BUF_LEN] = {'\0'}; +static char exeFsAndRomFsPatchSelectorStr[NAME_BUF_LEN] = {'\0'}; +static char dumpedContentInfoStr[NAME_BUF_LEN] = {'\0'}; static u32 selectedAppInfoIndex = 0; static u32 selectedAppIndex; @@ -105,11 +116,16 @@ static u32 selectedPartitionIndex; static u32 selectedFileIndex; static nspDumpType selectedNspDumpType; +static batchModeSourceStorage batchModeSrc; + +static bool exeFsAndRomFsUpdateFlag = false; static bool highlight = false; static bool isFat32 = false, dumpCert = false, trimDump = false, calcCrc = true, setXciArchiveBit = false, removeConsoleData = false, tiklessDump = false; +static bool dumpAppTitles = false, dumpPatchTitles = false, dumpAddOnTitles = false, skipDumpedTitles = false; + static char statusMessage[2048] = {'\0'}; static int statusMessageFadeout = 0; @@ -125,7 +141,7 @@ static const char *dirHighlightIconPath = "romfs:/browser/dir_highlight.jpg"; static u8 *dirHighlightIconBuf = NULL; static const char *fileNormalIconPath = "romfs:/browser/file_normal.jpg"; -static u8 *fileNormalIconBuf = NULL; +u8 *fileNormalIconBuf = NULL; static const char *fileHighlightIconPath = "romfs:/browser/file_highlight.jpg"; static u8 *fileHighlightIconBuf = NULL; @@ -134,8 +150,9 @@ static const char *appHeadline = "NXDumpTool v" APP_VERSION ".\nOriginal codebas static const char *appControlsCommon = "[ " NINTENDO_FONT_DPAD " / " NINTENDO_FONT_LSTICK " / " NINTENDO_FONT_RSTICK " ] Move | [ " NINTENDO_FONT_A " ] Select | [ " NINTENDO_FONT_B " ] Back | [ " NINTENDO_FONT_PLUS " ] Exit"; static const char *appControlsGameCardMultiApp = "[ " NINTENDO_FONT_DPAD " / " NINTENDO_FONT_LSTICK " / " NINTENDO_FONT_RSTICK " ] Move | [ " NINTENDO_FONT_A " ] Select | [ " NINTENDO_FONT_B " ] Back | [ " NINTENDO_FONT_L " / " NINTENDO_FONT_R " / " NINTENDO_FONT_ZL " / " NINTENDO_FONT_ZR " ] Change displayed base application info | [ " NINTENDO_FONT_PLUS " ] Exit"; static const char *appControlsNoContent = "[ " NINTENDO_FONT_B " ] Back | [ " NINTENDO_FONT_PLUS " ] Exit"; -static const char *appControlsSdCardEmmcFull = "[ " NINTENDO_FONT_DPAD " / " NINTENDO_FONT_LSTICK " / " NINTENDO_FONT_RSTICK " ] Move | [ " NINTENDO_FONT_A " ] Select | [ " NINTENDO_FONT_B " ] Back | [ " NINTENDO_FONT_Y " ] Dump installed content with missing base application | [ " NINTENDO_FONT_PLUS " ] Exit"; -static const char *appControlsSdCardEmmcNoApp = "[ " NINTENDO_FONT_B " ] Back | [ " NINTENDO_FONT_Y " ] Dump installed content with missing base application | [ " NINTENDO_FONT_PLUS " ] Exit"; +static const char *appControlsSdCardEmmcFull = "[ " NINTENDO_FONT_DPAD " / " NINTENDO_FONT_LSTICK " / " NINTENDO_FONT_RSTICK " ] Move | [ " NINTENDO_FONT_A " ] Select | [ " NINTENDO_FONT_B " ] Back | [ " NINTENDO_FONT_X " ] Batch mode | [ " NINTENDO_FONT_Y " ] Dump installed content with missing base application | [ " NINTENDO_FONT_PLUS " ] Exit"; +static const char *appControlsSdCardEmmcNoApp = "[ " NINTENDO_FONT_B " ] Back | [ " NINTENDO_FONT_X " ] Batch mode | [ " NINTENDO_FONT_Y " ] Dump installed content with missing base application | [ " NINTENDO_FONT_PLUS " ] Exit"; +static const char *appControlsRomFs = "[ " NINTENDO_FONT_DPAD " / " NINTENDO_FONT_LSTICK " / " NINTENDO_FONT_RSTICK " ] Move | [ " NINTENDO_FONT_A " ] Select | [ " NINTENDO_FONT_B " ] Back | [ " NINTENDO_FONT_Y " ] Dump current directory | [ " NINTENDO_FONT_PLUS " ] Exit"; static const char *mainMenuItems[] = { "Dump gamecard content", "Dump SD card / eMMC (NANDUSER) content", "Update options" }; static const char *gameCardMenuItems[] = { "Cartridge Image (XCI) dump", "Nintendo Submission Package (NSP) dump", "HFS0 options", "ExeFS options", "RomFS options", "Dump gamecard certificate" }; @@ -150,13 +167,14 @@ static const char *hfs0PartitionDumpType1MenuItems[] = { "Dump HFS0 partition 0 static const char *hfs0PartitionDumpType2MenuItems[] = { "Dump HFS0 partition 0 (Update)", "Dump HFS0 partition 1 (Logo)", "Dump HFS0 partition 2 (Normal)", "Dump HFS0 partition 3 (Secure)" }; static const char *hfs0BrowserType1MenuItems[] = { "Browse HFS0 partition 0 (Update)", "Browse HFS0 partition 1 (Normal)", "Browse HFS0 partition 2 (Secure)" }; static const char *hfs0BrowserType2MenuItems[] = { "Browse HFS0 partition 0 (Update)", "Browse HFS0 partition 1 (Logo)", "Browse HFS0 partition 2 (Normal)", "Browse HFS0 partition 3 (Secure)" }; -static const char *exeFsMenuItems[] = { "ExeFS section data dump", "Browse ExeFS section" }; -static const char *exeFsSectionDumpMenuItems[] = { "Start ExeFS data dump process", "Bundled application to dump: " }; -static const char *exeFsSectionBrowserMenuItems[] = { "Browse ExeFS section", "Bundled application to browse: " }; -static const char *romFsMenuItems[] = { "RomFS section data dump", "Browse RomFS section" }; -static const char *romFsSectionDumpMenuItems[] = { "Start RomFS data dump process", "Bundled application to dump: " }; -static const char *romFsSectionBrowserMenuItems[] = { "Browse RomFS section", "Bundled application to browse: " }; +static const char *exeFsMenuItems[] = { "ExeFS section data dump", "Browse ExeFS section", "Use update: " }; +static const char *exeFsSectionDumpMenuItems[] = { "Start ExeFS data dump process", "Base application to dump: ", "Use update: " }; +static const char *exeFsSectionBrowserMenuItems[] = { "Browse ExeFS section", "Base application to browse: ", "Use update: " }; +static const char *romFsMenuItems[] = { "RomFS section data dump", "Browse RomFS section", "Use update: " }; +static const char *romFsSectionDumpMenuItems[] = { "Start RomFS data dump process", "Base application to dump: ", "Use update: " }; +static const char *romFsSectionBrowserMenuItems[] = { "Browse RomFS section", "Base application to browse: ", "Use update: " }; static const char *sdCardEmmcMenuItems[] = { "Nintendo Submission Package (NSP) dump", "ExeFS options", "RomFS options" }; +static const char *batchModeMenuItems[] = { "Start batch dump process", "Dump base applications: ", "Dump updates: ", "Dump DLCs: ", "Split output dumps (FAT32 support): ", "Remove console specific data: ", "Generate ticket-less dumps: ", "Skip already dumped titles: ", "Source storage: " }; static const char *updateMenuItems[] = { "Update NSWDB.COM XML database", "Update application" }; void uiFill(int x, int y, int width, int height, u8 r, u8 g, u8 b) @@ -702,7 +720,7 @@ void uiPrintHeadline() void uiPrintOption(int x, int y, int endPosition, bool leftArrow, bool rightArrow, int r, int g, int b) { - if (!strlen(strbuf) || endPosition < OPTIONS_X_END_POS || endPosition >= (FB_WIDTH - 8)) return; + if (!strlen(strbuf) || x < 8 || x >= OPTIONS_X_END_POS || y < 8 || y >= (FB_HEIGHT - 8 - font_height) || endPosition < OPTIONS_X_END_POS || endPosition >= (FB_WIDTH - 8)) return; int xpos = x; char *option = strbuf; @@ -712,21 +730,6 @@ void uiPrintOption(int x, int y, int endPosition, bool leftArrow, bool rightArro xpos += uiGetStrWidth("<"); - // Check if we're dealing with a long title selector string - if (strlen(option) > 3 && optionStrWidth >= (endPosition - xpos - (font_height * 2))) - { - while(optionStrWidth >= (endPosition - xpos - (font_height * 2))) - { - option++; - optionStrWidth = uiGetStrWidth(option); - } - - option[0] = option[1] = option[2] = '.'; - optionStrWidth = uiGetStrWidth(option); - - snprintf(titleSelectorStr, sizeof(titleSelectorStr) / sizeof(titleSelectorStr[0]), option); - } - xpos += (((endPosition - xpos) / 2) - (optionStrWidth / 2)); uiDrawString(option, xpos, y, r, g, b); @@ -739,6 +742,31 @@ void uiPrintOption(int x, int y, int endPosition, bool leftArrow, bool rightArro } } +void uiTruncateOptionStr(char *str, int x, int y, int endPosition) +{ + if (!str || !strlen(str) || x < 8 || x >= OPTIONS_X_END_POS || y < 8 || y >= (FB_HEIGHT - 8 - font_height) || endPosition < OPTIONS_X_END_POS || endPosition >= (FB_WIDTH - 8)) return; + + int xpos = x; + char *option = str; + u32 optionStrWidth = uiGetStrWidth(option); + + // Check if we're dealing with a long title selector string + if (optionStrWidth >= (endPosition - xpos - (font_height * 2))) + { + while(optionStrWidth >= (endPosition - xpos - (font_height * 2))) + { + option++; + optionStrWidth = uiGetStrWidth(option); + } + + option[0] = option[1] = option[2] = '.'; + + memmove(str, option, strlen(option)); + + str[strlen(option)] = '\0'; + } +} + void error_screen(const char *fmt, ...) { consoleInit(NULL); @@ -779,6 +807,9 @@ int uiInit() cursor = 0; scroll = 0; + /* Check if we're running under SX OS */ + runningSxOs = checkSxOsServices(); + /* Initialize pl service */ rc = plInitialize(); if (R_FAILED(rc)) @@ -1054,6 +1085,7 @@ void uiSetState(UIState state) } titleSelectorStr[0] = '\0'; + exeFsAndRomFsPatchSelectorStr[0] = '\0'; } UIState uiGetState() @@ -1085,10 +1117,12 @@ UIResult uiProcess() const char *upwardsArrow = UPWARDS_ARROW; const char *downwardsArrow = DOWNWARDS_ARROW; + bool forcedXciDump = false; + uiPrintHeadline(); loadTitleInfo(); - if (uiState == stateMainMenu || uiState == stateGameCardMenu || uiState == stateXciDumpMenu || uiState == stateNspDumpMenu || uiState == stateNspAppDumpMenu || uiState == stateNspPatchDumpMenu || uiState == stateNspAddOnDumpMenu || uiState == stateHfs0Menu || uiState == stateRawHfs0PartitionDumpMenu || uiState == stateHfs0PartitionDataDumpMenu || uiState == stateHfs0BrowserMenu || uiState == stateHfs0Browser || uiState == stateExeFsMenu || uiState == stateExeFsSectionDataDumpMenu || uiState == stateExeFsSectionBrowserMenu || uiState == stateExeFsSectionBrowser || uiState == stateRomFsMenu || uiState == stateRomFsSectionDataDumpMenu || uiState == stateRomFsSectionBrowserMenu || uiState == stateRomFsSectionBrowser || uiState == stateSdCardEmmcMenu || uiState == stateSdCardEmmcTitleMenu || uiState == stateSdCardEmmcOrphanPatchAddOnMenu || uiState == stateUpdateMenu) + if (uiState == stateMainMenu || uiState == stateGameCardMenu || uiState == stateXciDumpMenu || uiState == stateNspDumpMenu || uiState == stateNspAppDumpMenu || uiState == stateNspPatchDumpMenu || uiState == stateNspAddOnDumpMenu || uiState == stateHfs0Menu || uiState == stateRawHfs0PartitionDumpMenu || uiState == stateHfs0PartitionDataDumpMenu || uiState == stateHfs0BrowserMenu || uiState == stateHfs0Browser || uiState == stateExeFsMenu || uiState == stateExeFsSectionDataDumpMenu || uiState == stateExeFsSectionBrowserMenu || uiState == stateExeFsSectionBrowser || uiState == stateRomFsMenu || uiState == stateRomFsSectionDataDumpMenu || uiState == stateRomFsSectionBrowserMenu || uiState == stateRomFsSectionBrowser || uiState == stateSdCardEmmcMenu || uiState == stateSdCardEmmcTitleMenu || uiState == stateSdCardEmmcOrphanPatchAddOnMenu || uiState == stateSdCardEmmcBatchModeMenu || uiState == stateUpdateMenu) { switch(menuType) { @@ -1096,10 +1130,15 @@ UIResult uiProcess() uiDrawString(appControlsCommon, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); break; case MENUTYPE_GAMECARD: - uiDrawString((!gameCardInserted ? appControlsNoContent : (titleAppCount > 1 ? appControlsGameCardMultiApp : appControlsCommon)), 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); + if (uiState == stateRomFsSectionBrowser && strlen(curRomFsPath) > 1) + { + uiDrawString(appControlsRomFs, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); + } else { + uiDrawString((!gameCardInserted ? appControlsNoContent : (titleAppCount > 1 ? appControlsGameCardMultiApp : appControlsCommon)), 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); + } break; case MENUTYPE_SDCARD_EMMC: - if (uiState == stateSdCardEmmcOrphanPatchAddOnMenu) + if (uiState == stateSdCardEmmcOrphanPatchAddOnMenu || uiState == stateSdCardEmmcBatchModeMenu) { uiDrawString(appControlsCommon, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); } else { @@ -1108,6 +1147,10 @@ UIResult uiProcess() if (uiState == stateSdCardEmmcMenu && ((titlePatchCount && checkOrphanPatchOrAddOn(false)) || (titleAddOnCount && checkOrphanPatchOrAddOn(true)))) { uiDrawString(appControlsSdCardEmmcFull, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); + } else + if (uiState == stateRomFsSectionBrowser && strlen(curRomFsPath) > 1) + { + uiDrawString(appControlsRomFs, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); } else { uiDrawString(appControlsCommon, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); } @@ -1140,6 +1183,8 @@ UIResult uiProcess() { if (hfs0_partition_cnt == GAMECARD_TYPE1_PARTITION_CNT || hfs0_partition_cnt == GAMECARD_TYPE2_PARTITION_CNT) { + forcedXciDump = true; + if (titleAppCount > 0) { uiDrawString("Error: unable to retrieve the gamecard Title ID!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); @@ -1167,6 +1212,12 @@ UIResult uiProcess() uiDrawString("Gamecard is not inserted!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); } + if (forcedXciDump) + { + breaks += 2; + uiDrawString("Press " NINTENDO_FONT_Y " to dump the cartridge image to \"gamecard.xci\".", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); + } + uiUpdateStatusMsg(); uiRefreshDisplay(); @@ -1185,6 +1236,30 @@ UIResult uiProcess() menuType = MENUTYPE_MAIN; } + // Forced XCI dump + if ((keysDown & KEY_Y) && forcedXciDump) + { + uiPrintHeadline(); + + uiDrawString(gameCardMenuItems[0], 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 115, 115, 255); + breaks++; + + uiRefreshDisplay(); + + // Set default options + isFat32 = true; + setXciArchiveBit = false; + dumpCert = true; + trimDump = false; + calcCrc = false; + + dumpCartridgeImage(isFat32, setXciArchiveBit, dumpCert, trimDump, calcCrc); + + waitForButtonPress(); + + uiUpdateFreeSpace(); + } + return res; } } else @@ -1231,9 +1306,9 @@ UIResult uiProcess() } } - if (uiState == stateMainMenu || uiState == stateGameCardMenu || uiState == stateXciDumpMenu || uiState == stateNspDumpMenu || uiState == stateNspAppDumpMenu || uiState == stateNspPatchDumpMenu || uiState == stateNspAddOnDumpMenu || uiState == stateHfs0Menu || uiState == stateRawHfs0PartitionDumpMenu || uiState == stateHfs0PartitionDataDumpMenu || uiState == stateHfs0BrowserMenu || uiState == stateHfs0Browser || uiState == stateExeFsMenu || uiState == stateExeFsSectionDataDumpMenu || uiState == stateExeFsSectionBrowserMenu || uiState == stateExeFsSectionBrowser || uiState == stateRomFsMenu || uiState == stateRomFsSectionDataDumpMenu || uiState == stateRomFsSectionBrowserMenu || uiState == stateRomFsSectionBrowser || uiState == stateSdCardEmmcMenu || uiState == stateSdCardEmmcTitleMenu || uiState == stateSdCardEmmcOrphanPatchAddOnMenu || uiState == stateUpdateMenu) + if (uiState == stateMainMenu || uiState == stateGameCardMenu || uiState == stateXciDumpMenu || uiState == stateNspDumpMenu || uiState == stateNspAppDumpMenu || uiState == stateNspPatchDumpMenu || uiState == stateNspAddOnDumpMenu || uiState == stateHfs0Menu || uiState == stateRawHfs0PartitionDumpMenu || uiState == stateHfs0PartitionDataDumpMenu || uiState == stateHfs0BrowserMenu || uiState == stateHfs0Browser || uiState == stateExeFsMenu || uiState == stateExeFsSectionDataDumpMenu || uiState == stateExeFsSectionBrowserMenu || uiState == stateExeFsSectionBrowser || uiState == stateRomFsMenu || uiState == stateRomFsSectionDataDumpMenu || uiState == stateRomFsSectionBrowserMenu || uiState == stateRomFsSectionBrowser || uiState == stateSdCardEmmcMenu || uiState == stateSdCardEmmcTitleMenu || uiState == stateSdCardEmmcOrphanPatchAddOnMenu || uiState == stateSdCardEmmcBatchModeMenu || uiState == stateUpdateMenu) { - if ((menuType == MENUTYPE_GAMECARD && uiState != stateHfs0Browser && uiState != stateExeFsSectionBrowser && uiState != stateRomFsSectionBrowser) || (menuType == MENUTYPE_SDCARD_EMMC && !orphanMode && uiState != stateSdCardEmmcMenu && uiState != stateExeFsSectionBrowser && uiState != stateRomFsSectionBrowser)) + if ((menuType == MENUTYPE_GAMECARD && uiState != stateHfs0Browser && uiState != stateExeFsSectionBrowser && uiState != stateRomFsSectionBrowser) || (menuType == MENUTYPE_SDCARD_EMMC && !orphanMode && uiState != stateSdCardEmmcMenu && uiState != stateSdCardEmmcBatchModeMenu && uiState != stateExeFsSectionBrowser && uiState != stateRomFsSectionBrowser)) { if (menuType == MENUTYPE_GAMECARD) { @@ -1287,7 +1362,7 @@ UIResult uiProcess() for(patch = 0; patch < titlePatchCount; patch++) { - if ((titleAppTitleID[selectedAppInfoIndex] | APPLICATION_PATCH_BITMASK) == titlePatchTitleID[patch]) + if ((titleAppTitleID[selectedAppInfoIndex] | APPLICATION_PATCH_BITMASK) == titlePatchTitleID[patch] && ((menuType == MENUTYPE_GAMECARD && titleAppStorageId[selectedAppInfoIndex] == titlePatchStorageId[patch]) || menuType == MENUTYPE_SDCARD_EMMC)) { if (patchCnt > 0) strcat(strbuf, ", v"); @@ -1340,9 +1415,8 @@ UIResult uiProcess() if (titleAppCount > 1) { - breaks++; snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Base application count: %u | Base application currently displayed: %u", titleAppCount, selectedAppInfoIndex + 1); - uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 0, 255, 0); + uiDrawString(strbuf, (FB_WIDTH / 2) - (FB_WIDTH / 8), (breaks * (font_height + (font_height / 4))) + (font_height / 8), 0, 255, 0); } breaks++; @@ -1352,174 +1426,213 @@ UIResult uiProcess() if (strlen(gameCardUpdateVersionStr)) { - breaks++; snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Bundled FW update: %s", gameCardUpdateVersionStr); - uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 0, 255, 0); + uiDrawString(strbuf, (FB_WIDTH / 2) - (FB_WIDTH / 8), (breaks * (font_height + (font_height / 4))) + (font_height / 8), 0, 255, 0); } - if (titlePatchCount > 0) - { - breaks++; - snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Total bundled update(s): %u", titlePatchCount); - uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 0, 255, 0); - } + breaks++; - if (titleAddOnCount > 0) + if (titleAppCount > 1 && (titlePatchCount > 0 || titleAddOnCount > 0)) { - breaks++; - snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Total bundled DLC(s): %u", titleAddOnCount); - uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 0, 255, 0); - } - } else - if (menuType == MENUTYPE_SDCARD_EMMC) - { - if (!strlen(dumpedNspInfoStr)) - { - // Look for dumped content in the SD card - char *dumpName = NULL; - char dumpPath[NAME_BUF_LEN] = {'\0'}, tmpStr[64] = {'\0'}; - bool dumpedBase = false, dumpedBaseConsoleData = false; - - u32 patchCnt = 0, addOnCnt = 0; - u32 patchCntConsoleData = 0, addOnCntConsoleData = 0; - - snprintf(dumpedNspInfoStr, sizeof(dumpedNspInfoStr) / sizeof(dumpedNspInfoStr[0]), "Content already dumped: "); - - // First look for a dumped base application - dumpName = generateNSPDumpName(DUMP_APP_NSP, selectedAppInfoIndex); - if (dumpName) - { - snprintf(dumpPath, sizeof(dumpPath) / sizeof(dumpPath[0]), "%s%s.nsp", NSP_DUMP_PATH, dumpName); - - free(dumpName); - dumpName = NULL; - - if (checkIfFileExists(dumpPath)) - { - dumpedBase = true; - dumpedBaseConsoleData = checkIfDumpedNspContainsConsoleData(dumpPath); - } - } - - // Look for dumped updates if (titlePatchCount > 0) { - for(patch = 0; patch < titlePatchCount; patch++) - { - if ((titleAppTitleID[selectedAppInfoIndex] | APPLICATION_PATCH_BITMASK) == titlePatchTitleID[patch]) - { - dumpName = generateNSPDumpName(DUMP_PATCH_NSP, patch); - if (dumpName) - { - snprintf(dumpPath, sizeof(dumpPath) / sizeof(dumpPath[0]), "%s%s.nsp", NSP_DUMP_PATH, dumpName); - - free(dumpName); - dumpName = NULL; - - if (checkIfFileExists(dumpPath)) - { - patchCnt++; - if (checkIfDumpedNspContainsConsoleData(dumpPath)) patchCntConsoleData++; - } - } - } - } + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Total bundled update(s): %u", titlePatchCount); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 0, 255, 0); } - // Look for dumped DLCs if (titleAddOnCount > 0) { - for(addon = 0; addon < titleAddOnCount; addon++) - { - if ((titleAppTitleID[selectedAppInfoIndex] & APPLICATION_ADDON_BITMASK) == (titleAddOnTitleID[addon] & APPLICATION_ADDON_BITMASK)) - { - dumpName = generateNSPDumpName(DUMP_ADDON_NSP, addon); - if (dumpName) - { - snprintf(dumpPath, sizeof(dumpPath) / sizeof(dumpPath[0]), "%s%s.nsp", NSP_DUMP_PATH, dumpName); - - free(dumpName); - dumpName = NULL; - - if (checkIfFileExists(dumpPath)) - { - addOnCnt++; - if (checkIfDumpedNspContainsConsoleData(dumpPath)) addOnCntConsoleData++; - } - } - } - } + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Total bundled DLC(s): %u", titleAddOnCount); + uiDrawString(strbuf, (titlePatchCount > 0 ? ((FB_WIDTH / 2) - (FB_WIDTH / 8)) : 8), (breaks * (font_height + (font_height / 4))) + (font_height / 8), 0, 255, 0); } - if (!dumpedBase && !patchCnt && !addOnCnt) + breaks++; + } + } + + if (!strlen(dumpedContentInfoStr)) + { + // Look for dumped content in the SD card + char *dumpName = NULL; + char dumpPath[NAME_BUF_LEN] = {'\0'}, tmpStr[64] = {'\0'}; + bool dumpedXci = false, dumpedXciCertificate = false, dumpedBase = false, dumpedBaseConsoleData = false; + + u32 patchCnt = 0, addOnCnt = 0; + u32 patchCntConsoleData = 0, addOnCntConsoleData = 0; + + snprintf(dumpedContentInfoStr, sizeof(dumpedContentInfoStr) / sizeof(dumpedContentInfoStr[0]), "Content already dumped: "); + + if (menuType == MENUTYPE_GAMECARD) + { + char *xciName = generateFullDumpName(); + if (xciName) { - strcat(dumpedNspInfoStr, "NONE"); - } else { - if (dumpedBase) + // First check if an unsplitted XCI dump is available + snprintf(dumpPath, sizeof(dumpPath) / sizeof(dumpPath[0]), "%s%s.xci", XCI_DUMP_PATH, xciName); + if (!(dumpedXci = checkIfFileExists(dumpPath))) { - strcat(dumpedNspInfoStr, "BASE"); - - if (dumpedBaseConsoleData) - { - strcat(dumpedNspInfoStr, " (with console data)"); - } else { - strcat(dumpedNspInfoStr, " (without console data)"); - } + // Check if a splitted XCI dump is available + snprintf(dumpPath, sizeof(dumpPath) / sizeof(dumpPath[0]), "%s%s.xc0", XCI_DUMP_PATH, xciName); + dumpedXci = checkIfFileExists(dumpPath); } - if (patchCnt) - { - if (patchCntConsoleData) - { - if (patchCntConsoleData == patchCnt) - { - if (patchCnt > 1) - { - snprintf(tmpStr, sizeof(tmpStr) / sizeof(tmpStr[0]), "%u UPD (all with console data)", patchCnt); - } else { - snprintf(tmpStr, sizeof(tmpStr) / sizeof(tmpStr[0]), "UPD (with console data)"); - } - } else { - snprintf(tmpStr, sizeof(tmpStr) / sizeof(tmpStr[0]), "%u UPD (%u with console data)", patchCnt, patchCntConsoleData); - } - } else { - if (patchCnt > 1) - { - snprintf(tmpStr, sizeof(tmpStr) / sizeof(tmpStr[0]), "%u UPD (all without console data)", patchCnt); - } else { - snprintf(tmpStr, sizeof(tmpStr) / sizeof(tmpStr[0]), "UPD (without console data)"); - } - } - - if (dumpedBase) strcat(dumpedNspInfoStr, ", "); - - strcat(dumpedNspInfoStr, tmpStr); - } + free(xciName); + xciName = NULL; - if (addOnCnt) + if (dumpedXci) dumpedXciCertificate = checkIfDumpedXciContainsCertificate(dumpPath); + } + } + + // Now search for dumped NSPs + + // Look for a dumped base application + dumpName = generateNSPDumpName(DUMP_APP_NSP, selectedAppInfoIndex); + if (dumpName) + { + snprintf(dumpPath, sizeof(dumpPath) / sizeof(dumpPath[0]), "%s%s.nsp", NSP_DUMP_PATH, dumpName); + + free(dumpName); + dumpName = NULL; + + if (checkIfFileExists(dumpPath)) + { + dumpedBase = true; + dumpedBaseConsoleData = checkIfDumpedNspContainsConsoleData(dumpPath); + } + } + + // Look for dumped updates + if (titlePatchCount > 0) + { + for(patch = 0; patch < titlePatchCount; patch++) + { + if ((titleAppTitleID[selectedAppInfoIndex] | APPLICATION_PATCH_BITMASK) == titlePatchTitleID[patch]) { - if (addOnCntConsoleData) + dumpName = generateNSPDumpName(DUMP_PATCH_NSP, patch); + if (dumpName) { - if (addOnCntConsoleData == addOnCnt) + snprintf(dumpPath, sizeof(dumpPath) / sizeof(dumpPath[0]), "%s%s.nsp", NSP_DUMP_PATH, dumpName); + + free(dumpName); + dumpName = NULL; + + if (checkIfFileExists(dumpPath)) { - snprintf(tmpStr, sizeof(tmpStr) / sizeof(tmpStr[0]), "%u DLC (%s console data)", addOnCnt, (addOnCnt > 1 ? "all with" : "with")); - } else { - snprintf(tmpStr, sizeof(tmpStr) / sizeof(tmpStr[0]), "%u DLC (%u with console data)", addOnCnt, addOnCntConsoleData); + patchCnt++; + if (checkIfDumpedNspContainsConsoleData(dumpPath)) patchCntConsoleData++; } - } else { - snprintf(tmpStr, sizeof(tmpStr) / sizeof(tmpStr[0]), "%u DLC (%s console data)", addOnCnt, (addOnCnt > 1 ? "all without" : "without")); } - - if (dumpedBase || patchCnt) strcat(dumpedNspInfoStr, ", "); - - strcat(dumpedNspInfoStr, tmpStr); } } } - uiDrawString(dumpedNspInfoStr, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 0, 255, 0); + // Look for dumped DLCs + if (titleAddOnCount > 0) + { + for(addon = 0; addon < titleAddOnCount; addon++) + { + if ((titleAppTitleID[selectedAppInfoIndex] & APPLICATION_ADDON_BITMASK) == (titleAddOnTitleID[addon] & APPLICATION_ADDON_BITMASK)) + { + dumpName = generateNSPDumpName(DUMP_ADDON_NSP, addon); + if (dumpName) + { + snprintf(dumpPath, sizeof(dumpPath) / sizeof(dumpPath[0]), "%s%s.nsp", NSP_DUMP_PATH, dumpName); + + free(dumpName); + dumpName = NULL; + + if (checkIfFileExists(dumpPath)) + { + addOnCnt++; + if (checkIfDumpedNspContainsConsoleData(dumpPath)) addOnCntConsoleData++; + } + } + } + } + } + + if (!dumpedXci && !dumpedBase && !patchCnt && !addOnCnt) + { + strcat(dumpedContentInfoStr, "NONE"); + } else { + if (dumpedXci) + { + strcat(dumpedContentInfoStr, "XCI"); + + if (dumpedXciCertificate) + { + strcat(dumpedContentInfoStr, " (with cert)"); + } else { + strcat(dumpedContentInfoStr, " (without cert)"); + } + } + + if (dumpedBase) + { + if (dumpedXci) strcat(dumpedContentInfoStr, ", "); + + strcat(dumpedContentInfoStr, "BASE"); + + if (dumpedBaseConsoleData) + { + strcat(dumpedContentInfoStr, " (with console data)"); + } else { + strcat(dumpedContentInfoStr, " (without console data)"); + } + } + + if (patchCnt) + { + if (patchCntConsoleData) + { + if (patchCntConsoleData == patchCnt) + { + if (patchCnt > 1) + { + snprintf(tmpStr, sizeof(tmpStr) / sizeof(tmpStr[0]), "%u UPD (all with console data)", patchCnt); + } else { + snprintf(tmpStr, sizeof(tmpStr) / sizeof(tmpStr[0]), "UPD (with console data)"); + } + } else { + snprintf(tmpStr, sizeof(tmpStr) / sizeof(tmpStr[0]), "%u UPD (%u with console data)", patchCnt, patchCntConsoleData); + } + } else { + if (patchCnt > 1) + { + snprintf(tmpStr, sizeof(tmpStr) / sizeof(tmpStr[0]), "%u UPD (all without console data)", patchCnt); + } else { + snprintf(tmpStr, sizeof(tmpStr) / sizeof(tmpStr[0]), "UPD (without console data)"); + } + } + + if (dumpedBase) strcat(dumpedContentInfoStr, ", "); + + strcat(dumpedContentInfoStr, tmpStr); + } + + if (addOnCnt) + { + if (addOnCntConsoleData) + { + if (addOnCntConsoleData == addOnCnt) + { + snprintf(tmpStr, sizeof(tmpStr) / sizeof(tmpStr[0]), "%u DLC (%s console data)", addOnCnt, (addOnCnt > 1 ? "all with" : "with")); + } else { + snprintf(tmpStr, sizeof(tmpStr) / sizeof(tmpStr[0]), "%u DLC (%u with console data)", addOnCnt, addOnCntConsoleData); + } + } else { + snprintf(tmpStr, sizeof(tmpStr) / sizeof(tmpStr[0]), "%u DLC (%s console data)", addOnCnt, (addOnCnt > 1 ? "all without" : "without")); + } + + if (dumpedBase || patchCnt) strcat(dumpedContentInfoStr, ", "); + + strcat(dumpedContentInfoStr, tmpStr); + } + } } + uiDrawString(dumpedContentInfoStr, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 0, 255, 0); + breaks += 2; } else if (menuType == MENUTYPE_SDCARD_EMMC && orphanMode && (uiState == stateNspPatchDumpMenu || uiState == stateNspAddOnDumpMenu)) @@ -1533,13 +1646,13 @@ UIResult uiProcess() uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 0, 255, 0); breaks++; - if (!strlen(dumpedNspInfoStr)) + if (!strlen(dumpedContentInfoStr)) { // Look for dumped content in the SD card char *dumpName = NULL; char dumpPath[NAME_BUF_LEN] = {'\0'}; - snprintf(dumpedNspInfoStr, sizeof(dumpedNspInfoStr) / sizeof(dumpedNspInfoStr[0]), "Title already dumped: "); + snprintf(dumpedContentInfoStr, sizeof(dumpedContentInfoStr) / sizeof(dumpedContentInfoStr[0]), "Title already dumped: "); if (uiState == stateNspPatchDumpMenu) { @@ -1559,26 +1672,26 @@ UIResult uiProcess() if (checkIfFileExists(dumpPath)) { - strcat(dumpedNspInfoStr, "Yes"); + strcat(dumpedContentInfoStr, "Yes"); if (checkIfDumpedNspContainsConsoleData(dumpPath)) { - strcat(dumpedNspInfoStr, " (with console data)"); + strcat(dumpedContentInfoStr, " (with console data)"); } else { - strcat(dumpedNspInfoStr, " (without console data)"); + strcat(dumpedContentInfoStr, " (without console data)"); } } else { - strcat(dumpedNspInfoStr, "No"); + strcat(dumpedContentInfoStr, "No"); } } else { - strcat(dumpedNspInfoStr, "No"); + strcat(dumpedContentInfoStr, "No"); } } - uiDrawString(dumpedNspInfoStr, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 0, 255, 0); + uiDrawString(dumpedContentInfoStr, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 0, 255, 0); breaks += 2; } else { - dumpedNspInfoStr[0] = '\0'; + dumpedContentInfoStr[0] = '\0'; } switch(uiState) @@ -1701,8 +1814,14 @@ UIResult uiProcess() uiDrawString(exeFsMenuItems[1], 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 115, 115, 255); breaks++; - convertTitleVersionToDecimal(titleAppVersion[selectedAppIndex], versionStr, sizeof(versionStr)); - snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "%s%s v%s", exeFsSectionBrowserMenuItems[1], titleName[selectedAppIndex], versionStr); + if (!exeFsAndRomFsUpdateFlag) + { + convertTitleVersionToDecimal(titleAppVersion[selectedAppIndex], versionStr, sizeof(versionStr)); + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "%s%s v%s", exeFsSectionBrowserMenuItems[1], titleName[selectedAppIndex], versionStr); + } else { + retrieveDescriptionForPatchOrAddOn(titlePatchTitleID[selectedPatchIndex], titlePatchVersion[selectedPatchIndex], false, true, "Update to browse: ", strbuf, sizeof(strbuf) / sizeof(strbuf[0])); + } + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 115, 115, 255); breaks += 2; @@ -1738,8 +1857,14 @@ UIResult uiProcess() uiDrawString(romFsMenuItems[1], 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 115, 115, 255); breaks++; - convertTitleVersionToDecimal(titleAppVersion[selectedAppIndex], versionStr, sizeof(versionStr)); - snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "%s%s v%s", romFsSectionBrowserMenuItems[1], titleName[selectedAppIndex], versionStr); + if (!exeFsAndRomFsUpdateFlag) + { + convertTitleVersionToDecimal(titleAppVersion[selectedAppIndex], versionStr, sizeof(versionStr)); + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "%s%s v%s", romFsSectionBrowserMenuItems[1], titleName[selectedAppIndex], versionStr); + } else { + retrieveDescriptionForPatchOrAddOn(titlePatchTitleID[selectedPatchIndex], titlePatchVersion[selectedPatchIndex], false, true, "Update to browse: ", strbuf, sizeof(strbuf) / sizeof(strbuf[0])); + } + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 115, 115, 255); breaks += 2; @@ -1789,7 +1914,7 @@ UIResult uiProcess() uiDrawString("Dump installed content with missing base application", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 115, 115, 255); breaks += 2; - uiDrawString("Hint: gamecard updates can be found in this section.", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); + uiDrawString("Hint: updates for gamecard titles can be found in this section.", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); if (menuItemsCount) { @@ -1798,6 +1923,13 @@ UIResult uiProcess() uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); } + break; + case stateSdCardEmmcBatchModeMenu: + menu = batchModeMenuItems; + menuItemsCount = (sizeof(batchModeMenuItems) / sizeof(batchModeMenuItems[0])); + + uiDrawString("Batch mode", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 115, 115, 255); + break; case stateUpdateMenu: menu = updateMenuItems; @@ -1868,6 +2000,42 @@ UIResult uiProcess() continue; } + // Avoid printing the "Dump base applications", "Dump updates" and/or "Dump DLCs" options in the batch mode menu if we're dealing with a storage source that doesn't hold any title belonging to the current category + if (uiState == stateSdCardEmmcBatchModeMenu && ((batchModeSrc == BATCH_SOURCE_ALL && ((!titleAppCount && i == 1) || (!titlePatchCount && i == 2) || (!titleAddOnCount && i == 3))) || (batchModeSrc == BATCH_SOURCE_SDCARD && ((!sdCardTitleAppCount && i == 1) || (!sdCardTitlePatchCount && i == 2) || (!sdCardTitleAddOnCount && i == 3))) || (batchModeSrc == BATCH_SOURCE_EMMC && ((!nandUserTitleAppCount && i == 1) || (!nandUserTitlePatchCount && i == 2) || (!nandUserTitleAddOnCount && i == 3))))) + { + j--; + continue; + } + + // Avoid printing the "Generate ticket-less dumps" option in the batch mode menu if the "Remove console specific data" option is disabled + if (uiState == stateSdCardEmmcBatchModeMenu && i == 6 && !removeConsoleData) + { + j--; + continue; + } + + // Avoid printing the "Source storage" option in the batch mode menu if we only have titles available in a single source storage device + if (uiState == stateSdCardEmmcBatchModeMenu && i == 8 && ((!sdCardTitleAppCount && !sdCardTitlePatchCount && !sdCardTitleAddOnCount) || (!nandUserTitleAppCount && !nandUserTitlePatchCount && !nandUserTitleAddOnCount))) + { + j--; + continue; + } + + // Avoid printing the "Use update" option in the ExeFS and RomFS menus if we're dealing with a gamecard and either its base application count is greater than 1 or it has no available patches + // Also avoid printing it if we're dealing with a SD/eMMC title and it has no available patches + if ((uiState == stateExeFsMenu || uiState == stateRomFsMenu) && i == 2 && ((menuType == MENUTYPE_GAMECARD && (titleAppCount > 1 || !checkIfBaseApplicationHasPatchOrAddOn(0, false))) || (menuType == MENUTYPE_SDCARD_EMMC && !checkIfBaseApplicationHasPatchOrAddOn(selectedAppInfoIndex, false)))) + { + j--; + continue; + } + + // Avoid printing the "Use update" option in the ExeFS/RomFS data dump and browser menus if we're not dealing with a gamecard, if the base application count is equal to or less than 1, or if the selected base application has no available patches + if ((uiState == stateExeFsSectionDataDumpMenu || uiState == stateExeFsSectionBrowserMenu || uiState == stateRomFsSectionDataDumpMenu || uiState == stateRomFsSectionBrowserMenu) && i == 2 && (menuType != MENUTYPE_GAMECARD || titleAppCount <= 1 || !checkIfBaseApplicationHasPatchOrAddOn(selectedAppIndex, false))) + { + j--; + continue; + } + // Avoid printing the parent directory entry ("..") in the RomFS browser if we're currently at the root directory if (uiState == stateRomFsSectionBrowser && i == 0 && strlen(curRomFsPath) <= 1) { @@ -1977,6 +2145,7 @@ UIResult uiProcess() // Print application name convertTitleVersionToDecimal(titleAppVersion[selectedAppIndex], versionStr, sizeof(versionStr)); snprintf(titleSelectorStr, sizeof(titleSelectorStr) / sizeof(titleSelectorStr[0]), "%s v%s", titleName[selectedAppIndex], versionStr); + uiTruncateOptionStr(titleSelectorStr, xpos, ypos, OPTIONS_X_END_POS_NSP); } leftArrowCondition = (menuType == MENUTYPE_GAMECARD && titleAppCount > 1 && selectedAppIndex > 0); @@ -1989,6 +2158,7 @@ UIResult uiProcess() // Find a matching application to print its name // Otherwise, just print the Title ID retrieveDescriptionForPatchOrAddOn(titlePatchTitleID[selectedPatchIndex], titlePatchVersion[selectedPatchIndex], false, (menuType == MENUTYPE_GAMECARD), NULL, titleSelectorStr, sizeof(titleSelectorStr) / sizeof(titleSelectorStr[0])); + uiTruncateOptionStr(titleSelectorStr, xpos, ypos, OPTIONS_X_END_POS_NSP); } leftArrowCondition = ((menuType == MENUTYPE_GAMECARD && titlePatchCount > 0 && selectedPatchIndex > 0) || (menuType == MENUTYPE_SDCARD_EMMC && !orphanMode && retrievePreviousPatchOrAddOnIndexFromBaseApplication(selectedPatchIndex, selectedAppInfoIndex, false) != selectedPatchIndex)); @@ -2001,6 +2171,7 @@ UIResult uiProcess() // Find a matching application to print its name and Title ID // Otherwise, just print the Title ID retrieveDescriptionForPatchOrAddOn(titleAddOnTitleID[selectedAddOnIndex], titleAddOnVersion[selectedAddOnIndex], true, (menuType == MENUTYPE_GAMECARD), NULL, titleSelectorStr, sizeof(titleSelectorStr) / sizeof(titleSelectorStr[0])); + uiTruncateOptionStr(titleSelectorStr, xpos, ypos, OPTIONS_X_END_POS_NSP); } leftArrowCondition = ((menuType == MENUTYPE_GAMECARD && titleAddOnCount > 0 && selectedAddOnIndex > 0) || (menuType == MENUTYPE_SDCARD_EMMC && !orphanMode && retrievePreviousPatchOrAddOnIndexFromBaseApplication(selectedAddOnIndex, selectedAppInfoIndex, true) != selectedAddOnIndex)); @@ -2026,7 +2197,97 @@ UIResult uiProcess() } } - // Print ExeFS/RomFS menus settings values + // Print settings values for the batch mode menu + if (uiState == stateSdCardEmmcBatchModeMenu && i > 0) + { + switch(i) + { + case 1: // Dump base applications + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "%s", (dumpAppTitles ? "Yes" : "No")); + uiPrintOption(xpos, ypos, OPTIONS_X_END_POS, dumpAppTitles, !dumpAppTitles, (dumpAppTitles ? 0 : 255), (dumpAppTitles ? 255 : 0), 0); + break; + case 2: // Dump updates + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "%s", (dumpPatchTitles ? "Yes" : "No")); + uiPrintOption(xpos, ypos, OPTIONS_X_END_POS, dumpPatchTitles, !dumpPatchTitles, (dumpPatchTitles ? 0 : 255), (dumpPatchTitles ? 255 : 0), 0); + break; + case 3: // Dump DLCs + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "%s", (dumpAddOnTitles ? "Yes" : "No")); + uiPrintOption(xpos, ypos, OPTIONS_X_END_POS, dumpAddOnTitles, !dumpAddOnTitles, (dumpAddOnTitles ? 0 : 255), (dumpAddOnTitles ? 255 : 0), 0); + break; + case 4: // Split output dumps (FAT32 support) + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "%s", (isFat32 ? "Yes" : "No")); + uiPrintOption(xpos, ypos, OPTIONS_X_END_POS, isFat32, !isFat32, (isFat32 ? 0 : 255), (isFat32 ? 255 : 0), 0); + break; + case 5: // Remove console specific data + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "%s", (removeConsoleData ? "Yes" : "No")); + uiPrintOption(xpos, ypos, OPTIONS_X_END_POS, removeConsoleData, !removeConsoleData, (removeConsoleData ? 0 : 255), (removeConsoleData ? 255 : 0), 0); + break; + case 6: // Generate ticket-less dumps + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "%s", (tiklessDump ? "Yes" : "No")); + uiPrintOption(xpos, ypos, OPTIONS_X_END_POS, tiklessDump, !tiklessDump, (tiklessDump ? 0 : 255), (tiklessDump ? 255 : 0), 0); + break; + case 7: // Skip dumped titles + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "%s", (skipDumpedTitles ? "Yes" : "No")); + uiPrintOption(xpos, ypos, OPTIONS_X_END_POS, skipDumpedTitles, !skipDumpedTitles, (skipDumpedTitles ? 0 : 255), (skipDumpedTitles ? 255 : 0), 0); + break; + case 8: // Source storage + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "%s", (batchModeSrc == BATCH_SOURCE_ALL ? "All (SD card + eMMC)" : (batchModeSrc == BATCH_SOURCE_SDCARD ? "SD card" : "eMMC"))); + + leftArrowCondition = (batchModeSrc != BATCH_SOURCE_ALL); + rightArrowCondition = (batchModeSrc != BATCH_SOURCE_EMMC); + + uiPrintOption(xpos, ypos, OPTIONS_X_END_POS_NSP, leftArrowCondition, rightArrowCondition, 255, 255, 255); + + break; + default: + break; + } + } + + // Print settings values for ExeFS/RomFS menus + if ((uiState == stateExeFsMenu || uiState == stateRomFsMenu) && i > 1) + { + u32 appIndex = (menuType == MENUTYPE_GAMECARD ? 0 : selectedAppInfoIndex); + + switch(i) + { + case 2: // Use update + if (exeFsAndRomFsUpdateFlag) + { + if (!strlen(exeFsAndRomFsPatchSelectorStr)) + { + // Find a matching application to print its name + // Otherwise, just print the Title ID + retrieveDescriptionForPatchOrAddOn(titlePatchTitleID[selectedPatchIndex], titlePatchVersion[selectedPatchIndex], false, (menuType == MENUTYPE_GAMECARD && titleAppCount > 1), NULL, exeFsAndRomFsPatchSelectorStr, sizeof(exeFsAndRomFsPatchSelectorStr) / sizeof(exeFsAndRomFsPatchSelectorStr[0])); + + // Concatenate patch source storage + strcat(exeFsAndRomFsPatchSelectorStr, (titlePatchStorageId[selectedPatchIndex] == FsStorageId_GameCard ? " (gamecard)" : (titlePatchStorageId[selectedPatchIndex] == FsStorageId_SdCard ? " (SD card)" : "(eMMC)"))); + + uiTruncateOptionStr(exeFsAndRomFsPatchSelectorStr, xpos, ypos, OPTIONS_X_END_POS_NSP); + } + + leftArrowCondition = true; + rightArrowCondition = (((menuType == MENUTYPE_GAMECARD && titleAppCount == 1) || menuType == MENUTYPE_SDCARD_EMMC) && retrieveNextPatchOrAddOnIndexFromBaseApplication(selectedPatchIndex, appIndex, false) != selectedPatchIndex); + + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), exeFsAndRomFsPatchSelectorStr); + + uiPrintOption(xpos, ypos, OPTIONS_X_END_POS_NSP, leftArrowCondition, rightArrowCondition, 255, 255, 255); + } else { + leftArrowCondition = false; + rightArrowCondition = (((menuType == MENUTYPE_GAMECARD && titleAppCount == 1) || menuType == MENUTYPE_SDCARD_EMMC) && checkIfBaseApplicationHasPatchOrAddOn(appIndex, false)); + + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "No"); + + uiPrintOption(xpos, ypos, OPTIONS_X_END_POS_NSP, leftArrowCondition, rightArrowCondition, 255, 0, 0); + } + + break; + default: + break; + } + } + + // Print settings values for ExeFS/RomFS submenus if ((uiState == stateExeFsSectionDataDumpMenu || uiState == stateExeFsSectionBrowserMenu || uiState == stateRomFsSectionDataDumpMenu || uiState == stateRomFsSectionBrowserMenu) && i > 0) { switch(i) @@ -2037,6 +2298,7 @@ UIResult uiProcess() // Print application name convertTitleVersionToDecimal(titleAppVersion[selectedAppIndex], versionStr, sizeof(versionStr)); snprintf(titleSelectorStr, sizeof(titleSelectorStr) / sizeof(titleSelectorStr[0]), "%s v%s", titleName[selectedAppIndex], versionStr); + uiTruncateOptionStr(titleSelectorStr, xpos, ypos, OPTIONS_X_END_POS_NSP); } leftArrowCondition = (menuType == MENUTYPE_GAMECARD && titleAppCount > 1 && selectedAppIndex > 0); @@ -2045,6 +2307,38 @@ UIResult uiProcess() snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), titleSelectorStr); uiPrintOption(xpos, ypos, OPTIONS_X_END_POS_NSP, leftArrowCondition, rightArrowCondition, 255, 255, 255); + + break; + case 2: // Use update + if (exeFsAndRomFsUpdateFlag) + { + if (!strlen(exeFsAndRomFsPatchSelectorStr)) + { + // Find a matching application to print its name + // Otherwise, just print the Title ID + retrieveDescriptionForPatchOrAddOn(titlePatchTitleID[selectedPatchIndex], titlePatchVersion[selectedPatchIndex], false, (menuType == MENUTYPE_GAMECARD), NULL, exeFsAndRomFsPatchSelectorStr, sizeof(exeFsAndRomFsPatchSelectorStr) / sizeof(exeFsAndRomFsPatchSelectorStr[0])); + + // Concatenate patch source storage + strcat(exeFsAndRomFsPatchSelectorStr, (titlePatchStorageId[selectedPatchIndex] == FsStorageId_GameCard ? " (gamecard)" : (titlePatchStorageId[selectedPatchIndex] == FsStorageId_SdCard ? " (SD card)" : "(eMMC)"))); + + uiTruncateOptionStr(exeFsAndRomFsPatchSelectorStr, xpos, ypos, OPTIONS_X_END_POS_NSP); + } + + leftArrowCondition = true; + rightArrowCondition = (menuType == MENUTYPE_GAMECARD && titleAppCount > 1 && retrieveNextPatchOrAddOnIndexFromBaseApplication(selectedPatchIndex, selectedAppIndex, false) != selectedPatchIndex); + + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), exeFsAndRomFsPatchSelectorStr); + + uiPrintOption(xpos, ypos, OPTIONS_X_END_POS_NSP, leftArrowCondition, rightArrowCondition, 255, 255, 255); + } else { + leftArrowCondition = false; + rightArrowCondition = (menuType == MENUTYPE_GAMECARD && titleAppCount > 1 && checkIfBaseApplicationHasPatchOrAddOn(selectedAppIndex, false)); + + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "No"); + + uiPrintOption(xpos, ypos, OPTIONS_X_END_POS_NSP, leftArrowCondition, rightArrowCondition, 255, 0, 0); + } + break; default: break; @@ -2092,15 +2386,17 @@ UIResult uiProcess() if (selectedAppInfoIndex > 0) { selectedAppInfoIndex--; - } else { - selectedAppInfoIndex = 0; + dumpedContentInfoStr[0] = '\0'; } } if ((keysDown & KEY_R) || (keysDown & KEY_ZR)) { - selectedAppInfoIndex++; - if (selectedAppInfoIndex >= titleAppCount) selectedAppInfoIndex = (titleAppCount - 1); + if ((selectedAppInfoIndex + 1) < titleAppCount) + { + selectedAppInfoIndex++; + dumpedContentInfoStr[0] = '\0'; + } } } @@ -2360,7 +2656,7 @@ UIResult uiProcess() { if (!orphanMode) { - u32 newIndex = retrieveNextPatchOrAddOnIndexFromBaseApplication(selectedAddOnIndex, selectedAppInfoIndex, false); + u32 newIndex = retrieveNextPatchOrAddOnIndexFromBaseApplication(selectedAddOnIndex, selectedAppInfoIndex, true); if (newIndex != selectedAddOnIndex) { selectedAddOnIndex = newIndex; @@ -2389,6 +2685,255 @@ UIResult uiProcess() scrollWithKeysDown = ((keysDown & KEY_DDOWN) || (keysDown & KEY_LSTICK_DOWN)); } } else + if (uiState == stateSdCardEmmcBatchModeMenu) + { + // Select + if ((keysDown & KEY_A) && cursor == 0 && (dumpAppTitles || dumpPatchTitles || dumpAddOnTitles)) res = resultSdCardEmmcBatchDump; + + // Back + if (keysDown & KEY_B) res = resultShowSdCardEmmcMenu; + + // Change option to false + if (keysDown & KEY_LEFT) + { + switch(cursor) + { + case 1: // Dump base applications + dumpAppTitles = false; + break; + case 2: // Dump updates + dumpPatchTitles = false; + break; + case 3: // Dump DLCs + dumpAddOnTitles = false; + break; + case 4: // Split output dumps (FAT32 support) + isFat32 = false; + break; + case 5: // Remove console specific data + removeConsoleData = tiklessDump = false; + break; + case 6: // Generate ticket-less dumps + tiklessDump = false; + break; + case 7: // Skip already dumped titles + skipDumpedTitles = false; + break; + case 8: // Source storage + if (batchModeSrc != BATCH_SOURCE_ALL) + { + batchModeSrc--; + + if (batchModeSrc == BATCH_SOURCE_ALL) + { + dumpAppTitles = (titleAppCount > 0); + dumpPatchTitles = (titlePatchCount > 0); + dumpAddOnTitles = (titleAddOnCount > 0); + } else + if (batchModeSrc == BATCH_SOURCE_SDCARD) + { + dumpAppTitles = (sdCardTitleAppCount > 0); + dumpPatchTitles = (sdCardTitlePatchCount > 0); + dumpAddOnTitles = (sdCardTitleAddOnCount > 0); + } else + if (batchModeSrc == BATCH_SOURCE_EMMC) + { + dumpAppTitles = (nandUserTitleAppCount > 0); + dumpPatchTitles = (nandUserTitlePatchCount > 0); + dumpAddOnTitles = (nandUserTitleAddOnCount > 0); + } + } + break; + default: + break; + } + } + + // Change option to true + if (keysDown & KEY_RIGHT) + { + switch(cursor) + { + case 1: // Dump base applications + dumpAppTitles = true; + break; + case 2: // Dump updates + dumpPatchTitles = true; + break; + case 3: // Dump DLCs + dumpAddOnTitles = true; + break; + case 4: // Split output dumps (FAT32 support) + isFat32 = true; + break; + case 5: // Remove console specific data + removeConsoleData = true; + break; + case 6: // Generate ticket-less dumps + tiklessDump = true; + break; + case 7: // Skip already dumped titles + skipDumpedTitles = true; + break; + case 8: // Source storage + if (batchModeSrc != BATCH_SOURCE_EMMC) + { + batchModeSrc++; + + if (batchModeSrc == BATCH_SOURCE_ALL) + { + dumpAppTitles = (titleAppCount > 0); + dumpPatchTitles = (titlePatchCount > 0); + dumpAddOnTitles = (titleAddOnCount > 0); + } else + if (batchModeSrc == BATCH_SOURCE_SDCARD) + { + dumpAppTitles = (sdCardTitleAppCount > 0); + dumpPatchTitles = (sdCardTitlePatchCount > 0); + dumpAddOnTitles = (sdCardTitleAddOnCount > 0); + } else + if (batchModeSrc == BATCH_SOURCE_EMMC) + { + dumpAppTitles = (nandUserTitleAppCount > 0); + dumpPatchTitles = (nandUserTitlePatchCount > 0); + dumpAddOnTitles = (nandUserTitleAddOnCount > 0); + } + } + break; + default: + break; + } + } + + // Go up + if ((keysDown & KEY_DUP) || (keysDown & KEY_LSTICK_UP) || (keysHeld & KEY_RSTICK_UP)) + { + scrollAmount = -1; + scrollWithKeysDown = ((keysDown & KEY_DUP) || (keysDown & KEY_LSTICK_UP)); + } + + // Go down + if ((keysDown & KEY_DDOWN) || (keysDown & KEY_LSTICK_DOWN) || (keysHeld & KEY_RSTICK_DOWN)) + { + scrollAmount = 1; + scrollWithKeysDown = ((keysDown & KEY_DDOWN) || (keysDown & KEY_LSTICK_DOWN)); + } + } else + if (uiState == stateExeFsMenu || uiState == stateRomFsMenu) + { + // Select + if (keysDown & KEY_A) + { + // Reset option to its default value + selectedAppIndex = (menuType == MENUTYPE_GAMECARD ? 0 : selectedAppInfoIndex); + + switch(cursor) + { + case 0: + if (uiState == stateExeFsMenu) + { + res = ((menuType == MENUTYPE_GAMECARD && titleAppCount > 1) ? resultShowExeFsSectionDataDumpMenu : resultDumpExeFsSectionData); + } else { + res = ((menuType == MENUTYPE_GAMECARD && titleAppCount > 1) ? resultShowRomFsSectionDataDumpMenu : resultDumpRomFsSectionData); + } + break; + case 1: + if (uiState == stateExeFsMenu) + { + res = ((menuType == MENUTYPE_GAMECARD && titleAppCount > 1) ? resultShowExeFsSectionBrowserMenu : resultExeFsSectionBrowserGetList); + } else { + res = ((menuType == MENUTYPE_GAMECARD && titleAppCount > 1) ? resultShowRomFsSectionBrowserMenu : resultRomFsSectionBrowserGetEntries); + } + break; + default: + break; + } + } + + // Back + if (keysDown & KEY_B) + { + if (menuType == MENUTYPE_GAMECARD) + { + freePatchesFromSdCardAndEmmc(); + res = resultShowGameCardMenu; + } else { + res = resultShowSdCardEmmcTitleMenu; + } + } + + // Go left + if (keysDown & KEY_LEFT) + { + switch(cursor) + { + case 2: // Use update + if ((menuType == MENUTYPE_GAMECARD && titleAppCount == 1 && checkIfBaseApplicationHasPatchOrAddOn(0, false)) || (menuType == MENUTYPE_SDCARD_EMMC && checkIfBaseApplicationHasPatchOrAddOn(selectedAppInfoIndex, false))) + { + if (exeFsAndRomFsUpdateFlag) + { + u32 appIndex = (menuType == MENUTYPE_GAMECARD ? 0 : selectedAppInfoIndex); + u32 newIndex = retrievePreviousPatchOrAddOnIndexFromBaseApplication(selectedPatchIndex, appIndex, false); + + if (newIndex != selectedPatchIndex) + { + selectedPatchIndex = newIndex; + exeFsAndRomFsPatchSelectorStr[0] = '\0'; + } else { + exeFsAndRomFsUpdateFlag = false; + } + } + } + break; + default: + break; + } + } + + // Go right + if (keysDown & KEY_RIGHT) + { + switch(cursor) + { + case 2: // Use update + if ((menuType == MENUTYPE_GAMECARD && titleAppCount == 1 && checkIfBaseApplicationHasPatchOrAddOn(0, false)) || (menuType == MENUTYPE_SDCARD_EMMC && checkIfBaseApplicationHasPatchOrAddOn(selectedAppInfoIndex, false))) + { + u32 appIndex = (menuType == MENUTYPE_GAMECARD ? 0 : selectedAppInfoIndex); + + if (exeFsAndRomFsUpdateFlag) + { + u32 newIndex = retrieveNextPatchOrAddOnIndexFromBaseApplication(selectedPatchIndex, appIndex, false); + if (newIndex != selectedPatchIndex) + { + selectedPatchIndex = newIndex; + exeFsAndRomFsPatchSelectorStr[0] = '\0'; + } + } else { + exeFsAndRomFsUpdateFlag = true; + selectedPatchIndex = retrieveFirstPatchOrAddOnIndexFromBaseApplication(appIndex, false); + exeFsAndRomFsPatchSelectorStr[0] = '\0'; + } + } + break; + default: + break; + } + } + + // Go up + if ((keysDown & KEY_DUP) || (keysDown & KEY_LSTICK_UP) || (keysHeld & KEY_RSTICK_UP)) + { + scrollAmount = -1; + scrollWithKeysDown = ((keysDown & KEY_DUP) || (keysDown & KEY_LSTICK_UP)); + } + + // Go down + if ((keysDown & KEY_DDOWN) || (keysDown & KEY_LSTICK_DOWN) || (keysHeld & KEY_RSTICK_DOWN)) + { + scrollAmount = 1; + scrollWithKeysDown = ((keysDown & KEY_DDOWN) || (keysDown & KEY_LSTICK_DOWN)); + } + } else if (uiState == stateExeFsSectionDataDumpMenu || uiState == stateExeFsSectionBrowserMenu || uiState == stateRomFsSectionDataDumpMenu || uiState == stateRomFsSectionBrowserMenu) { // Select @@ -2405,7 +2950,7 @@ UIResult uiProcess() } // Back - if (keysDown & KEY_B) res = (menuType == MENUTYPE_GAMECARD ? resultShowGameCardMenu : resultShowSdCardEmmcTitleMenu); + if (keysDown & KEY_B) res = ((uiState == stateExeFsSectionDataDumpMenu || uiState == stateExeFsSectionBrowserMenu) ? resultShowExeFsMenu : resultShowRomFsMenu); // Change option to false if (keysDown & KEY_LEFT) @@ -2419,6 +2964,24 @@ UIResult uiProcess() { selectedAppIndex--; titleSelectorStr[0] = '\0'; + + exeFsAndRomFsUpdateFlag = false; + } + } + break; + case 2: // Use update + if (menuType == MENUTYPE_GAMECARD && titleAppCount > 1 && checkIfBaseApplicationHasPatchOrAddOn(selectedAppIndex, false)) + { + if (exeFsAndRomFsUpdateFlag) + { + u32 newIndex = retrievePreviousPatchOrAddOnIndexFromBaseApplication(selectedPatchIndex, selectedAppIndex, false); + if (newIndex != selectedPatchIndex) + { + selectedPatchIndex = newIndex; + exeFsAndRomFsPatchSelectorStr[0] = '\0'; + } else { + exeFsAndRomFsUpdateFlag = false; + } } } break; @@ -2439,6 +3002,26 @@ UIResult uiProcess() { selectedAppIndex++; titleSelectorStr[0] = '\0'; + + exeFsAndRomFsUpdateFlag = false; + } + } + break; + case 2: // Use update + if (menuType == MENUTYPE_GAMECARD && titleAppCount > 1 && checkIfBaseApplicationHasPatchOrAddOn(selectedAppIndex, false)) + { + if (exeFsAndRomFsUpdateFlag) + { + u32 newIndex = retrieveNextPatchOrAddOnIndexFromBaseApplication(selectedPatchIndex, selectedAppIndex, false); + if (newIndex != selectedPatchIndex) + { + selectedPatchIndex = newIndex; + exeFsAndRomFsPatchSelectorStr[0] = '\0'; + } + } else { + exeFsAndRomFsUpdateFlag = true; + selectedPatchIndex = retrieveFirstPatchOrAddOnIndexFromBaseApplication(selectedAppIndex, false); + exeFsAndRomFsPatchSelectorStr[0] = '\0'; } } break; @@ -2518,10 +3101,15 @@ UIResult uiProcess() res = resultShowHfs0Menu; break; case 3: - res = resultShowExeFsMenu; - break; case 4: - res = resultShowRomFsMenu; + loadPatchesFromSdCardAndEmmc(); + + res = (cursor == 3 ? resultShowExeFsMenu : resultShowRomFsMenu); + + // Reset options to their default values + exeFsAndRomFsUpdateFlag = false; + selectedPatchIndex = 0; + break; case 5: res = resultDumpGameCardCertificate; @@ -2614,23 +3202,6 @@ UIResult uiProcess() selectedFileIndex = (u32)cursor; res = resultHfs0BrowserCopyFile; } else - if (uiState == stateExeFsMenu) - { - // Reset option to its default value - selectedAppIndex = (menuType == MENUTYPE_GAMECARD ? 0 : selectedAppInfoIndex); - - switch(cursor) - { - case 0: - res = ((menuType == MENUTYPE_GAMECARD && titleAppCount > 1) ? resultShowExeFsSectionDataDumpMenu : resultDumpExeFsSectionData); - break; - case 1: - res = ((menuType == MENUTYPE_GAMECARD && titleAppCount > 1) ? resultShowExeFsSectionBrowserMenu : resultExeFsSectionBrowserGetList); - break; - default: - break; - } - } else if (uiState == stateExeFsSectionBrowser) { if (menu && menuItemsCount) @@ -2640,23 +3211,6 @@ UIResult uiProcess() res = resultExeFsSectionBrowserCopyFile; } } else - if (uiState == stateRomFsMenu) - { - // Reset option to its default value - selectedAppIndex = (menuType == MENUTYPE_GAMECARD ? 0 : selectedAppInfoIndex); - - switch(cursor) - { - case 0: - res = ((menuType == MENUTYPE_GAMECARD && titleAppCount > 1) ? resultShowRomFsSectionDataDumpMenu : resultDumpRomFsSectionData); - break; - case 1: - res = ((menuType == MENUTYPE_GAMECARD && titleAppCount > 1) ? resultShowRomFsSectionBrowserMenu : resultRomFsSectionBrowserGetEntries); - break; - default: - break; - } - } else if (uiState == stateRomFsSectionBrowser) { if (menu && menuItemsCount) @@ -2692,10 +3246,13 @@ UIResult uiProcess() } break; case 1: - res = resultShowExeFsMenu; - break; case 2: - res = resultShowRomFsMenu; + res = (cursor == 1 ? resultShowExeFsMenu : resultShowRomFsMenu); + + // Reset options to their default values + exeFsAndRomFsUpdateFlag = false; + selectedPatchIndex = 0; + break; default: break; @@ -2747,7 +3304,7 @@ UIResult uiProcess() res = resultShowMainMenu; menuType = MENUTYPE_MAIN; } else - if (menuType == MENUTYPE_GAMECARD && (uiState == stateNspDumpMenu || uiState == stateHfs0Menu || uiState == stateExeFsMenu || uiState == stateRomFsMenu)) + if (menuType == MENUTYPE_GAMECARD && (uiState == stateNspDumpMenu || uiState == stateHfs0Menu)) { res = resultShowGameCardMenu; } else @@ -2766,20 +3323,12 @@ UIResult uiProcess() res = resultShowHfs0BrowserMenu; } else - if (uiState == stateExeFsSectionDataDumpMenu || uiState == stateExeFsSectionBrowserMenu) - { - res = resultShowExeFsMenu; - } else if (uiState == stateExeFsSectionBrowser) { freeExeFsContext(); res = ((menuType == MENUTYPE_GAMECARD && titleAppCount > 1) ? resultShowExeFsSectionBrowserMenu : resultShowExeFsMenu); } else - if (uiState == stateRomFsSectionDataDumpMenu || uiState == stateRomFsSectionBrowserMenu) - { - res = resultShowRomFsMenu; - } else if (uiState == stateRomFsSectionBrowser) { if (strlen(curRomFsPath) > 1) @@ -2794,6 +3343,8 @@ UIResult uiProcess() romFsBrowserEntries = NULL; } + if (exeFsAndRomFsUpdateFlag) freeBktrContext(); + freeRomFsContext(); res = ((menuType == MENUTYPE_GAMECARD && titleAppCount > 1) ? resultShowRomFsSectionBrowserMenu : resultShowRomFsMenu); @@ -2824,11 +3375,41 @@ UIResult uiProcess() } } - // SD/eMMC menu: Dump installed content with missing base application - if (uiState == stateSdCardEmmcMenu && ((titleAppCount && ((titlePatchCount && checkOrphanPatchOrAddOn(false)) || (titleAddOnCount && checkOrphanPatchOrAddOn(true)))) || (!titleAppCount && (titlePatchCount || titleAddOnCount))) && (keysDown & KEY_Y)) + // Special action #1 + if (keysDown & KEY_Y) { - res = resultShowSdCardEmmcOrphanPatchAddOnMenu; - orphanMode = true; + if (uiState == stateSdCardEmmcMenu && ((titleAppCount && ((titlePatchCount && checkOrphanPatchOrAddOn(false)) || (titleAddOnCount && checkOrphanPatchOrAddOn(true)))) || (!titleAppCount && (titlePatchCount || titleAddOnCount)))) + { + // SD/eMMC menu: Dump installed content with missing base application + res = resultShowSdCardEmmcOrphanPatchAddOnMenu; + orphanMode = true; + } else + if (uiState == stateRomFsSectionBrowser && strlen(curRomFsPath) > 1) + { + // RomFS section browser: dump current directory + res = resultRomFsSectionBrowserCopyDir; + } + } + + // Special action #2 + if (keysDown & KEY_X) + { + if (uiState == stateSdCardEmmcMenu && (titleAppCount || titlePatchCount || titleAddOnCount)) + { + // Batch mode + res = resultShowSdCardEmmcBatchModeMenu; + + // Reset options to their default values + dumpAppTitles = (titleAppCount > 0); + dumpPatchTitles = (titlePatchCount > 0); + dumpAddOnTitles = (titleAddOnCount > 0); + isFat32 = false; + calcCrc = false; + removeConsoleData = false; + tiklessDump = false; + skipDumpedTitles = true; + batchModeSrc = BATCH_SOURCE_ALL; + } } if (menu && menuItemsCount) @@ -2962,6 +3543,72 @@ UIResult uiProcess() } } + // Avoid placing the cursor on the "Dump base applications", "Dump updates" and/or "Dump DLCs" options in the batch mode menu if we're dealing with a storage source that doesn't hold any title belonging to the current category + if (uiState == stateSdCardEmmcBatchModeMenu && ((batchModeSrc == BATCH_SOURCE_ALL && ((!titleAppCount && cursor == 1) || (!titlePatchCount && cursor == 2) || (!titleAddOnCount && cursor == 3))) || (batchModeSrc == BATCH_SOURCE_SDCARD && ((!sdCardTitleAppCount && cursor == 1) || (!sdCardTitlePatchCount && cursor == 2) || (!sdCardTitleAddOnCount && cursor == 3))) || (batchModeSrc == BATCH_SOURCE_EMMC && ((!nandUserTitleAppCount && cursor == 1) || (!nandUserTitlePatchCount && cursor == 2) || (!nandUserTitleAddOnCount && cursor == 3))))) + { + if (scrollAmount > 0) + { + cursor++; + } else + if (scrollAmount < 0) + { + cursor--; + } + } + + // Avoid placing the cursor on the "Generate ticket-less dumps" option in the batch mode menu if the "Remove console specific data" option is disabled + if (uiState == stateSdCardEmmcBatchModeMenu && cursor == 6 && !removeConsoleData) + { + if (scrollAmount > 0) + { + cursor++; + } else + if (scrollAmount < 0) + { + cursor--; + } + } + + // Avoid placing the cursor on the "Source storage" option in the batch mode menu if we only have titles available in a single source storage device + if (uiState == stateSdCardEmmcBatchModeMenu && cursor == 8 && ((!sdCardTitleAppCount && !sdCardTitlePatchCount && !sdCardTitleAddOnCount) || (!nandUserTitleAppCount && !nandUserTitlePatchCount && !nandUserTitleAddOnCount))) + { + if (scrollAmount > 0) + { + cursor = (scrollWithKeysDown ? 0 : 7); + } else + if (scrollAmount < 0) + { + cursor--; + } + } + + // Avoid placing the cursor on the "Use update" option in the ExeFS and RomFS menus if we're dealing with a gamecard and either its base application count is greater than 1 or it has no available patches + // Also avoid placing the cursor on it if we're dealing with a SD/eMMC title and it has no available patches + if ((uiState == stateExeFsMenu || uiState == stateRomFsMenu) && cursor == 2 && ((menuType == MENUTYPE_GAMECARD && (titleAppCount > 1 || !checkIfBaseApplicationHasPatchOrAddOn(0, false))) || (menuType == MENUTYPE_SDCARD_EMMC && !checkIfBaseApplicationHasPatchOrAddOn(selectedAppInfoIndex, false)))) + { + if (scrollAmount > 0) + { + cursor = (scrollWithKeysDown ? 0 : 1); + } else + if (scrollAmount < 0) + { + cursor = 0; + } + } + + // Avoid placing the cursor on the "Use update" option in the ExeFS/RomFS data dump and browser menus if we're not dealing with a gamecard, if the base application count is equal to or less than 1, or if the selected base application has no available patches + if ((uiState == stateExeFsSectionDataDumpMenu || uiState == stateExeFsSectionBrowserMenu || uiState == stateRomFsSectionDataDumpMenu || uiState == stateRomFsSectionBrowserMenu) && cursor == 2 && (menuType != MENUTYPE_GAMECARD || titleAppCount <= 1 || !checkIfBaseApplicationHasPatchOrAddOn(selectedAppIndex, false))) + { + if (scrollAmount > 0) + { + cursor = (scrollWithKeysDown ? 0 : 1); + } else + if (scrollAmount < 0) + { + cursor = 0; + } + } + // Avoid placing the cursor on the parent directory entry ("..") in the RomFS browser if we're currently at the root directory if (uiState == stateRomFsSectionBrowser && cursor == 0 && strlen(curRomFsPath) <= 1) { @@ -3012,6 +3659,8 @@ UIResult uiProcess() uiUpdateFreeSpace(); res = resultShowXciDumpMenu; + + dumpedContentInfoStr[0] = '\0'; } else if (uiState == stateDumpNsp) { @@ -3067,13 +3716,79 @@ UIResult uiProcess() uiRefreshDisplay(); - dumpNintendoSubmissionPackage(selectedNspDumpType, (selectedNspDumpType == DUMP_APP_NSP ? selectedAppIndex : (selectedNspDumpType == DUMP_PATCH_NSP ? selectedPatchIndex : selectedAddOnIndex)), isFat32, calcCrc, removeConsoleData, tiklessDump); + dumpNintendoSubmissionPackage(selectedNspDumpType, (selectedNspDumpType == DUMP_APP_NSP ? selectedAppIndex : (selectedNspDumpType == DUMP_PATCH_NSP ? selectedPatchIndex : selectedAddOnIndex)), isFat32, calcCrc, removeConsoleData, tiklessDump, false); waitForButtonPress(); uiUpdateFreeSpace(); res = (selectedNspDumpType == DUMP_APP_NSP ? resultShowNspAppDumpMenu : (selectedNspDumpType == DUMP_PATCH_NSP ? resultShowNspPatchDumpMenu : resultShowNspAddOnDumpMenu)); + + dumpedContentInfoStr[0] = '\0'; + } else + if (uiState == stateSdCardEmmcBatchDump) + { + uiDrawString("Batch dump", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 115, 115, 255); + breaks++; + + menu = batchModeMenuItems; + + if ((batchModeSrc == BATCH_SOURCE_ALL && titleAppCount) || (batchModeSrc == BATCH_SOURCE_SDCARD && sdCardTitleAppCount) || (batchModeSrc == BATCH_SOURCE_EMMC && nandUserTitleAppCount)) + { + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "%s%s", menu[1], (dumpAppTitles ? "Yes" : "No")); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 115, 115, 255); + breaks++; + } + + if ((batchModeSrc == BATCH_SOURCE_ALL && titlePatchCount) || (batchModeSrc == BATCH_SOURCE_SDCARD && sdCardTitlePatchCount) || (batchModeSrc == BATCH_SOURCE_EMMC && nandUserTitlePatchCount)) + { + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "%s%s", menu[2], (dumpPatchTitles ? "Yes" : "No")); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 115, 115, 255); + breaks++; + } + + if ((batchModeSrc == BATCH_SOURCE_ALL && titleAddOnCount) || (batchModeSrc == BATCH_SOURCE_SDCARD && sdCardTitleAddOnCount) || (batchModeSrc == BATCH_SOURCE_EMMC && nandUserTitleAddOnCount)) + { + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "%s%s", menu[3], (dumpAddOnTitles ? "Yes" : "No")); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 115, 115, 255); + breaks++; + } + + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "%s%s", menu[4], (isFat32 ? "Yes" : "No")); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 115, 115, 255); + breaks++; + + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "%s%s", menu[5], (removeConsoleData ? "Yes" : "No")); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 115, 115, 255); + breaks++; + + if (removeConsoleData) + { + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "%s%s", menu[6], (tiklessDump ? "Yes" : "No")); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 115, 115, 255); + breaks++; + } + + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "%s%s", menu[7], (skipDumpedTitles ? "Yes" : "No")); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 115, 115, 255); + + if ((sdCardTitleAppCount || sdCardTitlePatchCount || sdCardTitleAddOnCount) && (nandUserTitleAppCount || nandUserTitlePatchCount || nandUserTitleAddOnCount)) + { + breaks++; + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "%s%s", menu[8], (batchModeSrc == BATCH_SOURCE_ALL ? "All (SD card + eMMC)" : (batchModeSrc == BATCH_SOURCE_SDCARD ? "SD card" : "eMMC"))); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 115, 115, 255); + } + + breaks += 2; + uiRefreshDisplay(); + + dumpNintendoSubmissionPackageBatch(dumpAppTitles, dumpPatchTitles, dumpAddOnTitles, isFat32, removeConsoleData, tiklessDump, skipDumpedTitles, batchModeSrc); + + waitForButtonPress(); + + uiUpdateFreeSpace(); + + res = resultShowSdCardEmmcBatchModeMenu; } else if (uiState == stateDumpRawHfs0Partition) { @@ -3098,7 +3813,7 @@ UIResult uiProcess() uiRefreshDisplay(); - dumpHfs0PartitionData(selectedPartitionIndex); + dumpHfs0PartitionData(selectedPartitionIndex, true); waitForButtonPress(); @@ -3129,7 +3844,7 @@ UIResult uiProcess() uiRefreshDisplay(); - dumpFileFromHfs0Partition(selectedPartitionIndex, selectedFileIndex, filenames[selectedFileIndex]); + dumpFileFromHfs0Partition(selectedPartitionIndex, selectedFileIndex, filenames[selectedFileIndex], true); waitForButtonPress(); @@ -3141,14 +3856,20 @@ UIResult uiProcess() uiDrawString(exeFsMenuItems[0], 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 115, 115, 255); breaks++; - convertTitleVersionToDecimal(titleAppVersion[selectedAppIndex], versionStr, sizeof(versionStr)); - snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "%s%s v%s", exeFsSectionDumpMenuItems[1], titleName[selectedAppIndex], versionStr); + if (!exeFsAndRomFsUpdateFlag) + { + convertTitleVersionToDecimal(titleAppVersion[selectedAppIndex], versionStr, sizeof(versionStr)); + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "%s%s v%s", exeFsSectionDumpMenuItems[1], titleName[selectedAppIndex], versionStr); + } else { + retrieveDescriptionForPatchOrAddOn(titlePatchTitleID[selectedPatchIndex], titlePatchVersion[selectedPatchIndex], false, true, "Update to dump: ", strbuf, sizeof(strbuf) / sizeof(strbuf[0])); + } + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 115, 115, 255); breaks += 2; uiRefreshDisplay(); - dumpExeFsSectionData(selectedAppIndex, true); + dumpExeFsSectionData((!exeFsAndRomFsUpdateFlag ? selectedAppIndex : selectedPatchIndex), exeFsAndRomFsUpdateFlag, true); waitForButtonPress(); @@ -3161,8 +3882,14 @@ UIResult uiProcess() uiDrawString(exeFsMenuItems[1], 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 115, 115, 255); breaks++; - convertTitleVersionToDecimal(titleAppVersion[selectedAppIndex], versionStr, sizeof(versionStr)); - snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "%s%s v%s", exeFsSectionBrowserMenuItems[1], titleName[selectedAppIndex], versionStr); + if (!exeFsAndRomFsUpdateFlag) + { + convertTitleVersionToDecimal(titleAppVersion[selectedAppIndex], versionStr, sizeof(versionStr)); + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "%s%s v%s", exeFsSectionBrowserMenuItems[1], titleName[selectedAppIndex], versionStr); + } else { + retrieveDescriptionForPatchOrAddOn(titlePatchTitleID[selectedPatchIndex], titlePatchVersion[selectedPatchIndex], false, true, "Update to browse: ", strbuf, sizeof(strbuf) / sizeof(strbuf[0])); + } + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 115, 115, 255); breaks += 2; @@ -3171,7 +3898,7 @@ UIResult uiProcess() bool exefs_fail = false; - if (readProgramNcaExeFsOrRomFs(selectedAppIndex, false)) + if (readProgramNcaExeFsOrRomFs((!exeFsAndRomFsUpdateFlag ? selectedAppIndex : selectedPatchIndex), exeFsAndRomFsUpdateFlag, false)) { if (getExeFsFileList()) { @@ -3197,14 +3924,20 @@ UIResult uiProcess() uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 115, 115, 255); breaks++; - convertTitleVersionToDecimal(titleAppVersion[selectedAppIndex], versionStr, sizeof(versionStr)); - snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Base application: %s v%s", titleName[selectedAppIndex], versionStr); + if (!exeFsAndRomFsUpdateFlag) + { + convertTitleVersionToDecimal(titleAppVersion[selectedAppIndex], versionStr, sizeof(versionStr)); + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Base application: %s v%s", titleName[selectedAppIndex], versionStr); + } else { + retrieveDescriptionForPatchOrAddOn(titlePatchTitleID[selectedPatchIndex], titlePatchVersion[selectedPatchIndex], false, true, "Update: ", strbuf, sizeof(strbuf) / sizeof(strbuf[0])); + } + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 115, 115, 255); breaks += 2; uiRefreshDisplay(); - dumpFileFromExeFsSection(selectedAppIndex, selectedFileIndex, true); + dumpFileFromExeFsSection((!exeFsAndRomFsUpdateFlag ? selectedAppIndex : selectedPatchIndex), selectedFileIndex, exeFsAndRomFsUpdateFlag, true); waitForButtonPress(); @@ -3218,12 +3951,21 @@ UIResult uiProcess() convertTitleVersionToDecimal(titleAppVersion[selectedAppIndex], versionStr, sizeof(versionStr)); snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "%s%s v%s", romFsSectionDumpMenuItems[1], titleName[selectedAppIndex], versionStr); + + if (!exeFsAndRomFsUpdateFlag) + { + convertTitleVersionToDecimal(titleAppVersion[selectedAppIndex], versionStr, sizeof(versionStr)); + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "%s%s v%s", romFsSectionDumpMenuItems[1], titleName[selectedAppIndex], versionStr); + } else { + retrieveDescriptionForPatchOrAddOn(titlePatchTitleID[selectedPatchIndex], titlePatchVersion[selectedPatchIndex], false, true, "Update to dump: ", strbuf, sizeof(strbuf) / sizeof(strbuf[0])); + } + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 115, 115, 255); breaks += 2; uiRefreshDisplay(); - dumpRomFsSectionData(selectedAppIndex); + dumpRomFsSectionData((!exeFsAndRomFsUpdateFlag ? selectedAppIndex : selectedPatchIndex), exeFsAndRomFsUpdateFlag, true); waitForButtonPress(); @@ -3236,8 +3978,14 @@ UIResult uiProcess() uiDrawString(romFsMenuItems[1], 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 115, 115, 255); breaks++; - convertTitleVersionToDecimal(titleAppVersion[selectedAppIndex], versionStr, sizeof(versionStr)); - snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "%s%s v%s", romFsSectionBrowserMenuItems[1], titleName[selectedAppIndex], versionStr); + if (!exeFsAndRomFsUpdateFlag) + { + convertTitleVersionToDecimal(titleAppVersion[selectedAppIndex], versionStr, sizeof(versionStr)); + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "%s%s v%s", romFsSectionBrowserMenuItems[1], titleName[selectedAppIndex], versionStr); + } else { + retrieveDescriptionForPatchOrAddOn(titlePatchTitleID[selectedPatchIndex], titlePatchVersion[selectedPatchIndex], false, true, "Update to browse: ", strbuf, sizeof(strbuf) / sizeof(strbuf[0])); + } + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 115, 115, 255); breaks += 2; @@ -3246,12 +3994,13 @@ UIResult uiProcess() bool romfs_fail = false; - if (readProgramNcaExeFsOrRomFs(selectedAppIndex, true)) + if (readProgramNcaExeFsOrRomFs((!exeFsAndRomFsUpdateFlag ? selectedAppIndex : selectedPatchIndex), exeFsAndRomFsUpdateFlag, true)) { - if (getRomFsFileList(0)) + if (getRomFsFileList(0, exeFsAndRomFsUpdateFlag)) { res = resultShowRomFsSectionBrowser; } else { + if (exeFsAndRomFsUpdateFlag) freeBktrContext(); freeRomFsContext(); romfs_fail = true; } @@ -3271,8 +4020,14 @@ UIResult uiProcess() uiDrawString(romFsMenuItems[1], 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 115, 115, 255); breaks++; - convertTitleVersionToDecimal(titleAppVersion[selectedAppIndex], versionStr, sizeof(versionStr)); - snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "%s%s v%s", romFsSectionBrowserMenuItems[1], titleName[selectedAppIndex], versionStr); + if (!exeFsAndRomFsUpdateFlag) + { + convertTitleVersionToDecimal(titleAppVersion[selectedAppIndex], versionStr, sizeof(versionStr)); + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "%s%s v%s", romFsSectionBrowserMenuItems[1], titleName[selectedAppIndex], versionStr); + } else { + retrieveDescriptionForPatchOrAddOn(titlePatchTitleID[selectedPatchIndex], titlePatchVersion[selectedPatchIndex], false, true, "Update to browse: ", strbuf, sizeof(strbuf) / sizeof(strbuf[0])); + } + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 115, 115, 255); breaks += 2; @@ -3280,7 +4035,7 @@ UIResult uiProcess() if (romFsBrowserEntries[selectedFileIndex].type == ROMFS_ENTRY_DIR) { - if (getRomFsFileList(romFsBrowserEntries[selectedFileIndex].offset)) + if (getRomFsFileList(romFsBrowserEntries[selectedFileIndex].offset, exeFsAndRomFsUpdateFlag)) { res = resultShowRomFsSectionBrowser; } else { @@ -3300,6 +4055,8 @@ UIResult uiProcess() romFsBrowserEntries = NULL; } + if (exeFsAndRomFsUpdateFlag) freeBktrContext(); + freeRomFsContext(); breaks += 2; @@ -3313,8 +4070,14 @@ UIResult uiProcess() uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 115, 115, 255); breaks++; - convertTitleVersionToDecimal(titleAppVersion[selectedAppIndex], versionStr, sizeof(versionStr)); - snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Base application: %s v%s", titleName[selectedAppIndex], versionStr); + if (!exeFsAndRomFsUpdateFlag) + { + convertTitleVersionToDecimal(titleAppVersion[selectedAppIndex], versionStr, sizeof(versionStr)); + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Base application: %s v%s", titleName[selectedAppIndex], versionStr); + } else { + retrieveDescriptionForPatchOrAddOn(titlePatchTitleID[selectedPatchIndex], titlePatchVersion[selectedPatchIndex], false, true, "Update: ", strbuf, sizeof(strbuf) / sizeof(strbuf[0])); + } + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 115, 115, 255); breaks += 2; @@ -3322,7 +4085,7 @@ UIResult uiProcess() if (romFsBrowserEntries[selectedFileIndex].type == ROMFS_ENTRY_FILE) { - dumpFileFromRomFsSection(selectedAppIndex, romFsBrowserEntries[selectedFileIndex].offset, true); + dumpFileFromRomFsSection((!exeFsAndRomFsUpdateFlag ? selectedAppIndex : selectedPatchIndex), romFsBrowserEntries[selectedFileIndex].offset, exeFsAndRomFsUpdateFlag, true); } else { // Unexpected condition uiDrawString("Error: the selected entry is not a file!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 115, 115, 255); @@ -3333,6 +4096,32 @@ UIResult uiProcess() uiUpdateFreeSpace(); res = resultShowRomFsSectionBrowser; } else + if (uiState == stateRomFsSectionBrowserCopyDir) + { + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Manual Directory Dump: romfs:%s (RomFS)", curRomFsPath); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 115, 115, 255); + breaks++; + + if (!exeFsAndRomFsUpdateFlag) + { + convertTitleVersionToDecimal(titleAppVersion[selectedAppIndex], versionStr, sizeof(versionStr)); + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Base application: %s v%s", titleName[selectedAppIndex], versionStr); + } else { + retrieveDescriptionForPatchOrAddOn(titlePatchTitleID[selectedPatchIndex], titlePatchVersion[selectedPatchIndex], false, true, "Update: ", strbuf, sizeof(strbuf) / sizeof(strbuf[0])); + } + + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 115, 115, 255); + breaks += 2; + + uiRefreshDisplay(); + + dumpCurrentDirFromRomFsSection((!exeFsAndRomFsUpdateFlag ? selectedAppIndex : selectedPatchIndex), exeFsAndRomFsUpdateFlag, true); + + waitForButtonPress(); + + uiUpdateFreeSpace(); + res = resultShowRomFsSectionBrowser; + } else if (uiState == stateDumpGameCardCertificate) { uiDrawString(gameCardMenuItems[4], 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 115, 115, 255); diff --git a/source/ui.h b/source/ui.h index ab521a9..e5bea8b 100644 --- a/source/ui.h +++ b/source/ui.h @@ -23,7 +23,7 @@ #define UPWARDS_ARROW "\xE2\x86\x91" #define DOWNWARDS_ARROW "\xE2\x86\x93" -#define COMMON_MAX_ELEMENTS 8 +#define COMMON_MAX_ELEMENTS 9 #define HFS0_MAX_ELEMENTS 14 #define ROMFS_MAX_ELEMENTS 12 #define SDCARD_MAX_ELEMENTS 3 @@ -40,6 +40,7 @@ // UTF-16 #define NINTENDO_FONT_A "\xE0\xA0" #define NINTENDO_FONT_B "\xE0\xA1" +#define NINTENDO_FONT_X "\xE0\xA2" #define NINTENDO_FONT_Y "\xE0\xA3" #define NINTENDO_FONT_L "\xE0\xA4" #define NINTENDO_FONT_R "\xE0\xA5" @@ -86,10 +87,13 @@ typedef enum { resultShowRomFsSectionBrowser, resultRomFsSectionBrowserChangeDir, resultRomFsSectionBrowserCopyFile, + resultRomFsSectionBrowserCopyDir, resultDumpGameCardCertificate, resultShowSdCardEmmcMenu, resultShowSdCardEmmcTitleMenu, resultShowSdCardEmmcOrphanPatchAddOnMenu, + resultShowSdCardEmmcBatchModeMenu, + resultSdCardEmmcBatchDump, resultShowUpdateMenu, resultUpdateNSWDBXml, resultUpdateApplication, @@ -130,10 +134,13 @@ typedef enum { stateRomFsSectionBrowser, stateRomFsSectionBrowserChangeDir, stateRomFsSectionBrowserCopyFile, + stateRomFsSectionBrowserCopyDir, stateDumpGameCardCertificate, stateSdCardEmmcMenu, stateSdCardEmmcTitleMenu, stateSdCardEmmcOrphanPatchAddOnMenu, + stateSdCardEmmcBatchModeMenu, + stateSdCardEmmcBatchDump, stateUpdateMenu, stateUpdateNSWDBXml, stateUpdateApplication diff --git a/source/util.c b/source/util.c index f763a86..9a2ced7 100644 --- a/source/util.c +++ b/source/util.c @@ -65,6 +65,8 @@ UEvent exitEvent; AppletType programAppletType; +bool runningSxOs = false; + bool gameCardInserted; u64 gameCardSize = 0, trimmedCardSize = 0; @@ -105,6 +107,10 @@ u32 nandUserTitleAppCount = 0; u32 nandUserTitlePatchCount = 0; u32 nandUserTitleAddOnCount = 0; +static bool sdCardAndEmmcTitleInfoLoaded = false; + +u32 gameCardSdCardEmmcPatchCount = 0; + char **titleName = NULL; char **fixedTitleName = NULL; char **titleAuthor = NULL; @@ -113,8 +119,10 @@ u8 **titleIcon = NULL; exefs_ctx_t exeFsContext; romfs_ctx_t romFsContext; +bktr_ctx_t bktrContext; char curRomFsPath[NAME_BUF_LEN] = {'\0'}; +u32 curRomFsDirOffset = 0; romfs_browser_entry *romFsBrowserEntries = NULL; orphan_patch_addon_entry *orphanEntries = NULL; @@ -154,6 +162,25 @@ void fsGameCardDetectionThreadFunc(void *arg) waitMulti(&idx, 0, waiterForEvent(&fsGameCardKernelEvent), waiterForUEvent(&exitEvent)); } +bool isServiceRunning(const char *serviceName) +{ + if (!serviceName || !strlen(serviceName)) return false; + + Handle handle; + bool running = R_FAILED(smRegisterService(&handle, serviceName, false, 1)); + + svcCloseHandle(handle); + + if (!running) smUnregisterService(serviceName); + + return running; +} + +bool checkSxOsServices() +{ + return (isServiceRunning("tx") && !isServiceRunning("rnx")); +} + void delay(u8 seconds) { if (!seconds) return; @@ -237,6 +264,8 @@ void initRomFsContext() memset(&(romFsContext.ncmStorage), 0, sizeof(NcmContentStorage)); memset(&(romFsContext.ncaId), 0, sizeof(NcmNcaId)); memset(&(romFsContext.aes_ctx), 0, sizeof(Aes128CtrContext)); + romFsContext.section_offset = 0; + romFsContext.section_size = 0; romFsContext.romfs_offset = 0; romFsContext.romfs_size = 0; romFsContext.romfs_dirtable_offset = 0; @@ -267,6 +296,61 @@ void freeRomFsContext() } } +void initBktrContext() +{ + memset(&(bktrContext.ncmStorage), 0, sizeof(NcmContentStorage)); + memset(&(bktrContext.ncaId), 0, sizeof(NcmNcaId)); + memset(&(bktrContext.aes_ctx), 0, sizeof(Aes128CtrContext)); + bktrContext.section_offset = 0; + bktrContext.section_size = 0; + memset(&(bktrContext.superblock), 0, sizeof(bktr_superblock_t)); + bktrContext.relocation_block = NULL; + bktrContext.subsection_block = NULL; + bktrContext.virtual_seek = 0; + bktrContext.bktr_seek = 0; + bktrContext.base_seek = 0; + bktrContext.romfs_offset = 0; + bktrContext.romfs_size = 0; + bktrContext.romfs_dirtable_offset = 0; + bktrContext.romfs_dirtable_size = 0; + bktrContext.romfs_dir_entries = NULL; + bktrContext.romfs_filetable_offset = 0; + bktrContext.romfs_filetable_size = 0; + bktrContext.romfs_file_entries = NULL; + bktrContext.romfs_filedata_offset = 0; +} + +void freeBktrContext() +{ + // Remember to close this NCM service resource + serviceClose(&(bktrContext.ncmStorage.s)); + memset(&(bktrContext.ncmStorage), 0, sizeof(NcmContentStorage)); + + if (bktrContext.relocation_block != NULL) + { + free(bktrContext.relocation_block); + bktrContext.relocation_block = NULL; + } + + if (bktrContext.subsection_block != NULL) + { + free(bktrContext.subsection_block); + bktrContext.subsection_block = NULL; + } + + if (bktrContext.romfs_dir_entries != NULL) + { + free(bktrContext.romfs_dir_entries); + bktrContext.romfs_dir_entries = NULL; + } + + if (bktrContext.romfs_file_entries != NULL) + { + free(bktrContext.romfs_file_entries); + bktrContext.romfs_file_entries = NULL; + } +} + void freeGameCardInfo() { if (hfs0_header != NULL) @@ -412,6 +496,8 @@ void freeGlobalData() freeRomFsContext(); + freeBktrContext(); + if (romFsBrowserEntries != NULL) { free(romFsBrowserEntries); @@ -433,7 +519,7 @@ bool listTitlesByType(NcmContentMetaDatabase *ncmDb, u8 filter) return false; } - bool success = false, proceed = true, freeBuf = false; + bool success = false, proceed = true, memError = false; Result result; @@ -441,8 +527,7 @@ bool listTitlesByType(NcmContentMetaDatabase *ncmDb, u8 filter) NcmApplicationContentMetaKey *titleListTmp = NULL; size_t titleListSize = sizeof(NcmApplicationContentMetaKey); - u32 written = 0; - u32 total = 0; + u32 written = 0, total = 0; u64 *titleIDs = NULL, *tmpTIDs = NULL; u32 *versions = NULL, *tmpVersions = NULL; @@ -512,19 +597,11 @@ bool listTitlesByType(NcmContentMetaDatabase *ncmDb, u8 filter) success = true; } else { - if (tmpTIDs != NULL) - { - titleAppTitleID = tmpTIDs; - free(tmpTIDs); - } + if (tmpTIDs != NULL) titleAppTitleID = tmpTIDs; - if (tmpVersions != NULL) - { - titleAppVersion = tmpVersions; - free(tmpVersions); - } + if (tmpVersions != NULL) titleAppVersion = tmpVersions; - freeBuf = true; + memError = true; } } else if (filter == META_DB_PATCH) @@ -545,19 +622,11 @@ bool listTitlesByType(NcmContentMetaDatabase *ncmDb, u8 filter) success = true; } else { - if (tmpTIDs != NULL) - { - titlePatchTitleID = tmpTIDs; - free(tmpTIDs); - } + if (tmpTIDs != NULL) titlePatchTitleID = tmpTIDs; - if (tmpVersions != NULL) - { - titlePatchVersion = tmpVersions; - free(tmpVersions); - } + if (tmpVersions != NULL) titlePatchVersion = tmpVersions; - freeBuf = true; + memError = true; } } else if (filter == META_DB_ADDON) @@ -578,33 +647,22 @@ bool listTitlesByType(NcmContentMetaDatabase *ncmDb, u8 filter) success = true; } else { - if (tmpTIDs != NULL) - { - titleAddOnTitleID = tmpTIDs; - free(tmpTIDs); - } + if (tmpTIDs != NULL) titleAddOnTitleID = tmpTIDs; - if (tmpVersions != NULL) - { - titleAddOnVersion = tmpVersions; - free(tmpVersions); - } + if (tmpVersions != NULL) titleAddOnVersion = tmpVersions; - freeBuf = true; + memError = true; } } } else { - freeBuf = true; + memError = true; } - if (freeBuf) - { - if (titleIDs != NULL) free(titleIDs); - - if (versions != NULL) free(versions); - - uiStatusMsg("listTitlesByType: failed to allocate memory for TID/version buffer! (0x%02X filter).", filter); - } + if (titleIDs != NULL) free(titleIDs); + + if (versions != NULL) free(versions); + + if (memError) uiStatusMsg("listTitlesByType: failed to allocate memory for TID/version buffer! (0x%02X filter).", filter); } } else { // There are no titles that match the provided filter in the opened storage device @@ -630,6 +688,9 @@ bool getTitleIDAndVersionList(FsStorageId curStorageId) return false; } + /* Check if the SD card is really mounted */ + if (curStorageId == FsStorageId_SdCard && fsdevGetDefaultFileSystem() == NULL) return true; + bool listApp = false, listPatch = false, listAddOn = false, success = false; Result result; @@ -696,12 +757,200 @@ bool getTitleIDAndVersionList(FsStorageId curStorageId) serviceClose(&(ncmDb.s)); } else { - uiStatusMsg("getTitleIDAndVersionList: ncmOpenContentMetaDatabase failed! (0x%08X)", result); + if (curStorageId == FsStorageId_SdCard && result == 0x21005) + { + // If the SD card is mounted, but is isn't currently used by HOS because of some weird reason, just filter this particular error and continue + // This can occur when using the "Nintendo" directory from a different console, or when the "sdmc:/Nintendo/Contents/private" file is corrupted + success = true; + } else { + uiStatusMsg("getTitleIDAndVersionList: ncmOpenContentMetaDatabase failed! (0x%08X)", result); + } } return success; } +bool loadPatchesFromSdCardAndEmmc() +{ + if (menuType != MENUTYPE_GAMECARD || !titleAppCount || !titleAppTitleID) return false; + + u32 i, j; + + Result result; + NcmContentMetaDatabase ncmDb; + + NcmApplicationContentMetaKey *titleList; + NcmApplicationContentMetaKey *titleListTmp; + size_t titleListSize = sizeof(NcmApplicationContentMetaKey); + + u32 written, total; + + u64 *titleIDs, *tmpTIDs; + u32 *versions, *tmpVersions; + FsStorageId *tmpStorages; + + bool proceed; + + for(i = 0; i < 2; i++) + { + FsStorageId curStorageId = (i == 0 ? FsStorageId_SdCard : FsStorageId_NandUser); + + /* Check if the SD card is really mounted */ + if (curStorageId == FsStorageId_SdCard && fsdevGetDefaultFileSystem() == NULL) continue; + + memset(&ncmDb, 0, sizeof(NcmContentMetaDatabase)); + + titleList = titleListTmp = NULL; + + written = total = 0; + + titleIDs = tmpTIDs = NULL; + versions = tmpVersions = NULL; + tmpStorages = NULL; + + proceed = true; + + if (R_SUCCEEDED(result = ncmOpenContentMetaDatabase(curStorageId, &ncmDb))) + { + titleList = calloc(1, titleListSize); + if (titleList) + { + if (R_SUCCEEDED(result = ncmContentMetaDatabaseListApplication(&ncmDb, META_DB_PATCH, titleList, titleListSize, &written, &total)) && written && total) + { + if (total > written) + { + titleListSize *= total; + titleListTmp = realloc(titleList, titleListSize); + if (titleListTmp) + { + titleList = titleListTmp; + memset(titleList, 0, titleListSize); + + if (R_SUCCEEDED(result = ncmContentMetaDatabaseListApplication(&ncmDb, META_DB_PATCH, titleList, titleListSize, &written, &total))) + { + if (written != total) proceed = false; + } else { + proceed = false; + } + } else { + proceed = false; + } + } + + if (proceed) + { + titleIDs = calloc(total, sizeof(u64)); + versions = calloc(total, sizeof(u32)); + + if (titleIDs != NULL && versions != NULL) + { + for(j = 0; j < total; j++) + { + titleIDs[j] = titleList[j].metaRecord.titleId; + versions[j] = titleList[j].metaRecord.version; + } + + // If ptr == NULL, realloc will essentially act as a malloc + tmpTIDs = realloc(titlePatchTitleID, (titlePatchCount + total) * sizeof(u64)); + tmpVersions = realloc(titlePatchVersion, (titlePatchCount + total) * sizeof(u32)); + tmpStorages = realloc(titlePatchStorageId, (titlePatchCount + total) * sizeof(FsStorageId)); + + if (tmpTIDs != NULL && tmpVersions != NULL && tmpStorages != NULL) + { + titlePatchTitleID = tmpTIDs; + memcpy(titlePatchTitleID + titlePatchCount, titleIDs, total * sizeof(u64)); + + titlePatchVersion = tmpVersions; + memcpy(titlePatchVersion + titlePatchCount, versions, total * sizeof(u32)); + + titlePatchStorageId = tmpStorages; + for(j = titlePatchCount; j < (titlePatchCount + total); j++) titlePatchStorageId[j] = curStorageId; + + titlePatchCount += total; + + gameCardSdCardEmmcPatchCount += total; + + if (curStorageId == FsStorageId_SdCard) + { + sdCardTitlePatchCount = total; + } else { + nandUserTitlePatchCount = total; + } + } else { + if (tmpTIDs != NULL) titlePatchTitleID = tmpTIDs; + + if (tmpVersions != NULL) titlePatchVersion = tmpVersions; + + if (tmpStorages != NULL) titlePatchStorageId = tmpStorages; + } + } + + if (titleIDs != NULL) free(titleIDs); + + if (versions != NULL) free(versions); + } + } + + free(titleList); + } + + serviceClose(&(ncmDb.s)); + } + } + + if (gameCardSdCardEmmcPatchCount) return true; + + return false; +} + +void freePatchesFromSdCardAndEmmc() +{ + if (menuType != MENUTYPE_GAMECARD || !titleAppCount || !titleAppTitleID || !titlePatchCount || !titlePatchTitleID || !titlePatchVersion || !titlePatchStorageId || !gameCardSdCardEmmcPatchCount) return; + + u64 *tmpTIDs = NULL; + u32 *tmpVersions = NULL; + FsStorageId *tmpStorages = NULL; + + if ((titlePatchCount - gameCardSdCardEmmcPatchCount) > 0) + { + tmpTIDs = realloc(titlePatchTitleID, (titlePatchCount - gameCardSdCardEmmcPatchCount) * sizeof(u64)); + tmpVersions = realloc(titlePatchVersion, (titlePatchCount - gameCardSdCardEmmcPatchCount) * sizeof(u32)); + tmpStorages = realloc(titlePatchStorageId, (titlePatchCount - gameCardSdCardEmmcPatchCount) * sizeof(FsStorageId)); + + if (tmpTIDs != NULL && tmpVersions != NULL && tmpStorages != NULL) + { + titlePatchTitleID = tmpTIDs; + + titlePatchVersion = tmpVersions; + + titlePatchStorageId = tmpStorages; + } else { + if (tmpTIDs != NULL) titlePatchTitleID = tmpTIDs; + + if (tmpVersions != NULL) titlePatchVersion = tmpVersions; + + if (tmpStorages != NULL) titlePatchStorageId = tmpStorages; + } + } else { + free(titlePatchTitleID); + titlePatchTitleID = NULL; + + free(titlePatchVersion); + titlePatchVersion = NULL; + + free(titlePatchStorageId); + titlePatchStorageId = NULL; + } + + titlePatchCount -= gameCardSdCardEmmcPatchCount; + + gameCardSdCardEmmcPatchCount = 0; + + sdCardTitlePatchCount = 0; + + nandUserTitlePatchCount = 0; +} + void convertTitleVersionToDecimal(u32 version, char *versionBuf, size_t versionBufSize) { u8 major = (u8)((version >> 26) & 0x3F); @@ -1043,6 +1292,7 @@ void loadTitleInfo() if (menuType == MENUTYPE_MAIN) { freeGlobalData(); + sdCardAndEmmcTitleInfoLoaded = false; return; } @@ -1075,7 +1325,7 @@ void loadTitleInfo() } else if (menuType == MENUTYPE_SDCARD_EMMC) { - if (titleAppCount || titlePatchCount || titleAddOnCount) return; + if (titleAppCount || titlePatchCount || titleAddOnCount || sdCardAndEmmcTitleInfoLoaded) return; uiPleaseWait(1); @@ -1096,6 +1346,8 @@ void loadTitleInfo() proceed = true; } } + + sdCardAndEmmcTitleInfoLoaded = true; } if (proceed && titleAppCount > 0) @@ -1479,19 +1731,23 @@ bool calculateExeFsExtractedDataSize(u64 *out) return true; } -bool calculateRomFsExtractedDataSize(u64 *out) +bool calculateRomFsFullExtractedSize(bool usePatch, u64 *out) { - if (!romFsContext.romfs_filetable_size || !romFsContext.romfs_file_entries || !out) + if ((!usePatch && (!romFsContext.romfs_filetable_size || !romFsContext.romfs_file_entries)) || (usePatch && (!bktrContext.romfs_filetable_size || !bktrContext.romfs_file_entries)) || !out) { uiDrawString("Error: invalid parameters to calculate extracted data size for the RomFS section!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); return false; } - u64 offset = 0, totalSize = 0; + u64 offset = 0; + u64 totalSize = 0; - while(offset < romFsContext.romfs_filetable_size) + u64 filetableSize = (!usePatch ? romFsContext.romfs_filetable_size : bktrContext.romfs_filetable_size); + romfs_file *fileEntries = (!usePatch ? romFsContext.romfs_file_entries : bktrContext.romfs_file_entries); + + while(offset < filetableSize) { - romfs_file *entry = (romfs_file*)((u8*)romFsContext.romfs_file_entries + offset); + romfs_file *entry = (romfs_file*)((u8*)fileEntries + offset); totalSize += entry->dataSize; @@ -1503,15 +1759,72 @@ bool calculateRomFsExtractedDataSize(u64 *out) return true; } -bool readProgramNcaExeFsOrRomFs(u32 appIndex, bool readRomFs) +bool calculateRomFsExtractedDirSize(u32 dir_offset, bool usePatch, u64 *out) +{ + if ((!usePatch && (!romFsContext.romfs_dirtable_size || !romFsContext.romfs_dir_entries || !romFsContext.romfs_filetable_size || !romFsContext.romfs_file_entries || dir_offset > romFsContext.romfs_dirtable_size)) || (usePatch && (!bktrContext.romfs_dirtable_size || !bktrContext.romfs_dir_entries || !bktrContext.romfs_filetable_size || !bktrContext.romfs_file_entries || dir_offset > bktrContext.romfs_dirtable_size)) || !out) + { + uiDrawString("Error: invalid parameters to calculate extracted size for the current RomFS directory!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + return false; + } + + u64 totalSize = 0, childDirSize = 0; + romfs_file *fileEntry = NULL; + romfs_dir *childDirEntry = NULL; + romfs_dir *dirEntry = (!usePatch ? (romfs_dir*)((u8*)romFsContext.romfs_dir_entries + dir_offset) : (romfs_dir*)((u8*)bktrContext.romfs_dir_entries + dir_offset)); + + // Check if we're dealing with a nameless directory that's not the root directory + if (!dirEntry->nameLen && dir_offset > 0) + { + uiDrawString("Error: directory entry without name in RomFS section!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + return false; + } + + if (dirEntry->childFile != ROMFS_ENTRY_EMPTY) + { + fileEntry = (!usePatch ? (romfs_file*)((u8*)romFsContext.romfs_file_entries + dirEntry->childFile) : (romfs_file*)((u8*)bktrContext.romfs_file_entries + dirEntry->childFile)); + + totalSize += fileEntry->dataSize; + + while(fileEntry->sibling != ROMFS_ENTRY_EMPTY) + { + fileEntry = (!usePatch ? (romfs_file*)((u8*)romFsContext.romfs_file_entries + fileEntry->sibling) : (romfs_file*)((u8*)bktrContext.romfs_file_entries + fileEntry->sibling)); + + totalSize += fileEntry->dataSize; + } + } + + if (dirEntry->childDir != ROMFS_ENTRY_EMPTY) + { + if (!calculateRomFsExtractedDirSize(dirEntry->childDir, usePatch, &childDirSize)) return false; + + totalSize += childDirSize; + + childDirEntry = (!usePatch ? (romfs_dir*)((u8*)romFsContext.romfs_dir_entries + dirEntry->childDir) : (romfs_dir*)((u8*)bktrContext.romfs_dir_entries + dirEntry->childDir)); + + while(childDirEntry->sibling != ROMFS_ENTRY_EMPTY) + { + if (!calculateRomFsExtractedDirSize(childDirEntry->sibling, usePatch, &childDirSize)) return false; + + totalSize += childDirSize; + + childDirEntry = (!usePatch ? (romfs_dir*)((u8*)romFsContext.romfs_dir_entries + childDirEntry->sibling) : (romfs_dir*)((u8*)bktrContext.romfs_dir_entries + childDirEntry->sibling)); + } + } + + *out = totalSize; + + return true; +} + +bool readProgramNcaExeFsOrRomFs(u32 titleIndex, bool usePatch, bool readRomFs) { Result result; u32 i = 0; u32 written = 0; u32 total = 0; - u32 appCount = 0; - u32 ncmAppIndex = 0; - u32 appNcaCount = 0; + u32 titleCount = 0; + u32 ncmTitleIndex = 0; + u32 titleNcaCount = 0; u32 partition = 0; FsStorageId curStorageId; @@ -1527,9 +1840,9 @@ bool readProgramNcaExeFsOrRomFs(u32 appIndex, bool readRomFs) NcmContentStorage ncmStorage; memset(&ncmStorage, 0, sizeof(NcmContentStorage)); - NcmApplicationContentMetaKey *appList = NULL; - NcmContentRecord *appContentRecords = NULL; - size_t appListSize = sizeof(NcmApplicationContentMetaKey); + NcmApplicationContentMetaKey *titleList = NULL; + NcmContentRecord *titleContentRecords = NULL; + size_t titleListSize = sizeof(NcmApplicationContentMetaKey); NcmNcaId ncaId; char ncaIdStr[33] = {'\0'}; @@ -1540,14 +1853,14 @@ bool readProgramNcaExeFsOrRomFs(u32 appIndex, bool readRomFs) bool success = false, foundProgram = false; - if (!titleAppStorageId) + if ((!usePatch && !titleAppStorageId) || (usePatch && !titlePatchStorageId)) { uiDrawString("Error: title storage ID unavailable!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); breaks += 2; return false; } - curStorageId = titleAppStorageId[appIndex]; + curStorageId = (!usePatch ? titleAppStorageId[titleIndex] : titlePatchStorageId[titleIndex]); if (curStorageId != FsStorageId_GameCard && curStorageId != FsStorageId_SdCard && curStorageId != FsStorageId_NandUser) { @@ -1559,39 +1872,55 @@ bool readProgramNcaExeFsOrRomFs(u32 appIndex, bool readRomFs) switch(curStorageId) { case FsStorageId_GameCard: - appCount = titleAppCount; - ncmAppIndex = appIndex; + titleCount = (!usePatch ? titleAppCount : titlePatchCount); + ncmTitleIndex = titleIndex; break; case FsStorageId_SdCard: - appCount = sdCardTitleAppCount; - ncmAppIndex = appIndex; + titleCount = (!usePatch ? sdCardTitleAppCount : sdCardTitlePatchCount); + + if (menuType == MENUTYPE_SDCARD_EMMC) + { + ncmTitleIndex = titleIndex; + } else { + // Patches loaded using loadPatchesFromSdCardAndEmmc() + ncmTitleIndex = (titleIndex - (titlePatchCount - gameCardSdCardEmmcPatchCount)); // Substract gamecard patch count + } + break; case FsStorageId_NandUser: - appCount = nandUserTitleAppCount; - ncmAppIndex = (appIndex - sdCardTitleAppCount); + titleCount = (!usePatch ? nandUserTitleAppCount : nandUserTitlePatchCount); + + if (menuType == MENUTYPE_SDCARD_EMMC) + { + ncmTitleIndex = (titleIndex - (!usePatch ? sdCardTitleAppCount : sdCardTitlePatchCount)); // Substract SD card patch count + } else { + // Patches loaded using loadPatchesFromSdCardAndEmmc() + ncmTitleIndex = (titleIndex - ((titlePatchCount - gameCardSdCardEmmcPatchCount) + sdCardTitlePatchCount)); // Substract gamecard + SD card patch count + } + break; default: break; } - if (!appCount) + if (!titleCount) { - uiDrawString("Error: invalid base application count!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + uiDrawString("Error: invalid title type count!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); breaks += 2; return false; } - if (ncmAppIndex > (appCount - 1)) + if (ncmTitleIndex > (titleCount - 1)) { - uiDrawString("Error: invalid base application index!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + uiDrawString("Error: invalid title index!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); breaks += 2; return false; } - appListSize *= appCount; + titleListSize *= titleCount; // If we're dealing with a gamecard, call workaroundPartitionZeroAccess() and read the secure partition header. Otherwise, ncmContentStorageReadContentIdFile() will fail with error 0x00171002 - if (curStorageId == FsStorageId_GameCard) + if (curStorageId == FsStorageId_GameCard && !partitionHfs0Header) { partition = (hfs0_partition_cnt - 1); // Select the secure partition @@ -1610,12 +1939,13 @@ bool readProgramNcaExeFsOrRomFs(u32 appIndex, bool readRomFs) } } - uiDrawString("Looking for the Program NCA...", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); + snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Looking for the Program NCA (%s)...", (!usePatch ? "base application" : "update")); + uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); uiRefreshDisplay(); breaks++; - appList = calloc(1, appListSize); - if (!appList) + titleList = calloc(1, titleListSize); + if (!titleList) { uiDrawString("Error: unable to allocate memory for the ApplicationContentMetaKey struct!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); goto out; @@ -1628,7 +1958,9 @@ bool readProgramNcaExeFsOrRomFs(u32 appIndex, bool readRomFs) goto out; } - if (R_FAILED(result = ncmContentMetaDatabaseListApplication(&ncmDb, META_DB_REGULAR_APPLICATION, appList, appListSize, &written, &total))) + u8 filter = (!usePatch ? META_DB_REGULAR_APPLICATION : META_DB_PATCH); + + if (R_FAILED(result = ncmContentMetaDatabaseListApplication(&ncmDb, filter, titleList, titleListSize, &written, &total))) { snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Error: ncmContentMetaDatabaseListApplication failed! (0x%08X)", result); uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); @@ -1648,23 +1980,23 @@ bool readProgramNcaExeFsOrRomFs(u32 appIndex, bool readRomFs) goto out; } - if (R_FAILED(result = ncmContentMetaDatabaseGet(&ncmDb, &(appList[ncmAppIndex].metaRecord), sizeof(NcmContentMetaRecordsHeader), &contentRecordsHeader, &contentRecordsHeaderReadSize))) + if (R_FAILED(result = ncmContentMetaDatabaseGet(&ncmDb, &(titleList[ncmTitleIndex].metaRecord), sizeof(NcmContentMetaRecordsHeader), &contentRecordsHeader, &contentRecordsHeaderReadSize))) { snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Error: ncmContentMetaDatabaseGet failed! (0x%08X)", result); uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); goto out; } - appNcaCount = (u32)(contentRecordsHeader.numContentRecords); + titleNcaCount = (u32)(contentRecordsHeader.numContentRecords); - appContentRecords = calloc(appNcaCount, sizeof(NcmContentRecord)); - if (!appContentRecords) + titleContentRecords = calloc(titleNcaCount, sizeof(NcmContentRecord)); + if (!titleContentRecords) { uiDrawString("Error: unable to allocate memory for the ContentRecord struct!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); goto out; } - if (R_FAILED(result = ncmContentMetaDatabaseListContentInfo(&ncmDb, &(appList[ncmAppIndex].metaRecord), 0, appContentRecords, appNcaCount * sizeof(NcmContentRecord), &written))) + if (R_FAILED(result = ncmContentMetaDatabaseListContentInfo(&ncmDb, &(titleList[ncmTitleIndex].metaRecord), 0, titleContentRecords, titleNcaCount * sizeof(NcmContentRecord), &written))) { snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Error: ncmContentMetaDatabaseListContentInfo failed! (0x%08X)", result); uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); @@ -1678,12 +2010,12 @@ bool readProgramNcaExeFsOrRomFs(u32 appIndex, bool readRomFs) goto out; } - for(i = 0; i < appNcaCount; i++) + for(i = 0; i < titleNcaCount; i++) { - if (appContentRecords[i].type == NcmContentType_Program) + if (titleContentRecords[i].type == NcmContentType_Program) { - memcpy(&ncaId, &(appContentRecords[i].ncaId), sizeof(NcmNcaId)); - convertDataToHexString(appContentRecords[i].ncaId.c, 16, ncaIdStr, 33); + memcpy(&ncaId, &(titleContentRecords[i].ncaId), sizeof(NcmNcaId)); + convertDataToHexString(titleContentRecords[i].ncaId.c, 16, ncaIdStr, 33); foundProgram = true; break; } @@ -1700,10 +2032,10 @@ bool readProgramNcaExeFsOrRomFs(u32 appIndex, bool readRomFs) uiRefreshDisplay(); breaks += 2; - snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Retrieving %s...", (!readRomFs ? "ExeFS entries" : "RomFS entry tables")); + /*snprintf(strbuf, sizeof(strbuf) / sizeof(strbuf[0]), "Retrieving %s...", (!readRomFs ? "ExeFS entries" : "RomFS entry tables")); uiDrawString(strbuf, 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 255, 255); uiRefreshDisplay(); - breaks++; + breaks++;*/ if (R_FAILED(result = ncmContentStorageReadContentIdFile(&ncmStorage, &ncaId, 0, ncaHeader, NCA_FULL_HEADER_LENGTH))) { @@ -1713,21 +2045,21 @@ bool readProgramNcaExeFsOrRomFs(u32 appIndex, bool readRomFs) } // Decrypt the NCA header - if (!decryptNcaHeader(ncaHeader, NCA_FULL_HEADER_LENGTH, &dec_nca_header, NULL, decrypted_nca_keys, (curStorageId != FsStorageId_GameCard))) goto out; + if (!decryptNcaHeader(ncaHeader, NCA_FULL_HEADER_LENGTH, &dec_nca_header, NULL, decrypted_nca_keys, (curStorageId != FsStorageId_GameCard || (curStorageId == FsStorageId_GameCard && usePatch)))) goto out; - bool has_rights_id = false; - - for(i = 0; i < 0x10; i++) + if (curStorageId == FsStorageId_GameCard && !usePatch) { - if (dec_nca_header.rights_id[i] != 0) + bool has_rights_id = false; + + for(i = 0; i < 0x10; i++) { - has_rights_id = true; - break; + if (dec_nca_header.rights_id[i] != 0) + { + has_rights_id = true; + break; + } } - } - - if (curStorageId == FsStorageId_GameCard) - { + if (has_rights_id) { uiDrawString("Error: Rights ID field in Program NCA header not empty!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); @@ -1740,20 +2072,47 @@ bool readProgramNcaExeFsOrRomFs(u32 appIndex, bool readRomFs) // Read file entries from the ExeFS section if (!readExeFsEntryFromNca(&ncmStorage, &ncaId, &dec_nca_header, decrypted_nca_keys)) goto out; } else { - // Read directory and file tables from the RomFS section - if (!readRomFsEntryFromNca(&ncmStorage, &ncaId, &dec_nca_header, decrypted_nca_keys)) goto out; + if (!usePatch) + { + // Read directory and file tables from the RomFS section + if (!readRomFsEntryFromNca(&ncmStorage, &ncaId, &dec_nca_header, decrypted_nca_keys)) goto out; + } else { + // Look for the base application title index + u32 appIndex; + + for(i = 0; i < titleAppCount; i++) + { + if (checkIfPatchOrAddOnBelongsToBaseApplication(titleIndex, i, false)) + { + appIndex = i; + break; + } + } + + if (i == titleAppCount) + { + uiDrawString("Error: unable to find base application title index for the selected update!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); + goto out; + } + + // Read directory and file tables from the RomFS section in the Program NCA from the base application + if (!readProgramNcaExeFsOrRomFs(appIndex, false, true)) goto out; + + // Read BKTR entry data in the Program NCA from the update + if (!readBktrEntryFromNca(&ncmStorage, &ncaId, &dec_nca_header, decrypted_nca_keys)) goto out; + } } success = true; out: - if (appContentRecords) free(appContentRecords); + if (titleContentRecords) free(titleContentRecords); if (!success) serviceClose(&(ncmStorage.s)); serviceClose(&(ncmDb.s)); - if (appList) free(appList); + if (titleList) free(titleList); if (curStorageId == FsStorageId_GameCard) { @@ -1802,24 +2161,24 @@ bool getExeFsFileList() return true; } -bool getRomFsParentDir(u32 dir_offset, u32 *out) +bool getRomFsParentDir(u32 dir_offset, bool usePatch, u32 *out) { - if (!romFsContext.romfs_dirtable_size || dir_offset > romFsContext.romfs_dirtable_size || !romFsContext.romfs_dir_entries) + if ((!usePatch && (!romFsContext.romfs_dirtable_size || dir_offset > romFsContext.romfs_dirtable_size || !romFsContext.romfs_dir_entries)) || (usePatch && (!bktrContext.romfs_dirtable_size || dir_offset > bktrContext.romfs_dirtable_size || !bktrContext.romfs_dir_entries))) { uiDrawString("Error: invalid parameters to retrieve parent RomFS section directory!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); return false; } - romfs_dir *entry = (romfs_dir*)((u8*)romFsContext.romfs_dir_entries + dir_offset); + romfs_dir *entry = (!usePatch ? (romfs_dir*)((u8*)romFsContext.romfs_dir_entries + dir_offset) : (romfs_dir*)((u8*)bktrContext.romfs_dir_entries + dir_offset)); *out = entry->parent; return true; } -bool generateCurrentRomFsPath(u32 dir_offset) +bool generateCurrentRomFsPath(u32 dir_offset, bool usePatch) { - if (!romFsContext.romfs_dirtable_size || dir_offset > romFsContext.romfs_dirtable_size || !romFsContext.romfs_dir_entries) + if ((!usePatch && (!romFsContext.romfs_dirtable_size || dir_offset > romFsContext.romfs_dirtable_size || !romFsContext.romfs_dir_entries)) || (usePatch && (!bktrContext.romfs_dirtable_size || dir_offset > bktrContext.romfs_dirtable_size || !bktrContext.romfs_dir_entries))) { uiDrawString("Error: invalid parameters to generate current RomFS section path!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); return false; @@ -1828,7 +2187,7 @@ bool generateCurrentRomFsPath(u32 dir_offset) // Generate current path if we're not dealing with the root directory if (dir_offset) { - romfs_dir *entry = (romfs_dir*)((u8*)romFsContext.romfs_dir_entries + dir_offset); + romfs_dir *entry = (!usePatch ? (romfs_dir*)((u8*)romFsContext.romfs_dir_entries + dir_offset) : (romfs_dir*)((u8*)bktrContext.romfs_dir_entries + dir_offset)); if (!entry->nameLen) { @@ -1839,7 +2198,7 @@ bool generateCurrentRomFsPath(u32 dir_offset) // Check if we're not a root dir child if (entry->parent) { - if (!generateCurrentRomFsPath(entry->parent)) return false; + if (!generateCurrentRomFsPath(entry->parent, usePatch)) return false; } // Concatenate entry name @@ -1852,7 +2211,7 @@ bool generateCurrentRomFsPath(u32 dir_offset) return true; } -bool getRomFsFileList(u32 dir_offset) +bool getRomFsFileList(u32 dir_offset, bool usePatch) { u64 entryOffset = 0; u32 dirEntryCnt = 1; // Always add the parent directory entry ("..") @@ -1861,6 +2220,9 @@ bool getRomFsFileList(u32 dir_offset) u32 i = 1; u32 romFsParentDir = 0; + u64 dirTableSize; + u64 fileTableSize; + if (romFsBrowserEntries != NULL) { free(romFsBrowserEntries); @@ -1869,22 +2231,25 @@ bool getRomFsFileList(u32 dir_offset) memset(curRomFsPath, 0, NAME_BUF_LEN); - if (!romFsContext.romfs_dirtable_size || dir_offset > romFsContext.romfs_dirtable_size || !romFsContext.romfs_dir_entries || !romFsContext.romfs_filetable_size || !romFsContext.romfs_file_entries) + if ((!usePatch && (!romFsContext.romfs_dirtable_size || dir_offset > romFsContext.romfs_dirtable_size || !romFsContext.romfs_dir_entries || !romFsContext.romfs_filetable_size || !romFsContext.romfs_file_entries)) || (usePatch && (!bktrContext.romfs_dirtable_size || dir_offset > bktrContext.romfs_dirtable_size || !bktrContext.romfs_dir_entries || !bktrContext.romfs_filetable_size || !bktrContext.romfs_file_entries))) { uiDrawString("Error: invalid parameters to retrieve RomFS section filelist!", 8, (breaks * (font_height + (font_height / 4))) + (font_height / 8), 255, 0, 0); return false; } - if (!getRomFsParentDir(dir_offset, &romFsParentDir)) return false; + if (!getRomFsParentDir(dir_offset, usePatch, &romFsParentDir)) return false; - if (!generateCurrentRomFsPath(dir_offset)) return false; + if (!generateCurrentRomFsPath(dir_offset, usePatch)) return false; + + dirTableSize = (!usePatch ? romFsContext.romfs_dirtable_size : bktrContext.romfs_dirtable_size); + fileTableSize = (!usePatch ? romFsContext.romfs_filetable_size : bktrContext.romfs_filetable_size); // First count the directory entries entryOffset = ROMFS_NONAME_DIRENTRY_SIZE; // Always skip the first entry (root directory) - while(entryOffset < romFsContext.romfs_dirtable_size) + while(entryOffset < dirTableSize) { - romfs_dir *entry = (romfs_dir*)((u8*)romFsContext.romfs_dir_entries + entryOffset); + romfs_dir *entry = (!usePatch ? (romfs_dir*)((u8*)romFsContext.romfs_dir_entries + entryOffset) : (romfs_dir*)((u8*)bktrContext.romfs_dir_entries + entryOffset)); if (!entry->nameLen) { @@ -1901,9 +2266,9 @@ bool getRomFsFileList(u32 dir_offset) // Now count the file entries entryOffset = 0; - while(entryOffset < romFsContext.romfs_filetable_size) + while(entryOffset < fileTableSize) { - romfs_file *entry = (romfs_file*)((u8*)romFsContext.romfs_file_entries + entryOffset); + romfs_file *entry = (!usePatch ? (romfs_file*)((u8*)romFsContext.romfs_file_entries + entryOffset) : (romfs_file*)((u8*)bktrContext.romfs_file_entries + entryOffset)); if (!entry->nameLen) { @@ -1947,13 +2312,13 @@ bool getRomFsFileList(u32 dir_offset) addStringToFilenameBuffer("..", &nextFilename); // First add the directory entries - if (dirEntryCnt > 1) + if ((!romFsParentDir && dirEntryCnt > 1) || (romFsParentDir && dirEntryCnt > 0)) { entryOffset = ROMFS_NONAME_DIRENTRY_SIZE; // Always skip the first entry (root directory) - while(entryOffset < romFsContext.romfs_dirtable_size) + while(entryOffset < dirTableSize) { - romfs_dir *entry = (romfs_dir*)((u8*)romFsContext.romfs_dir_entries + entryOffset); + romfs_dir *entry = (!usePatch ? (romfs_dir*)((u8*)romFsContext.romfs_dir_entries + entryOffset) : (romfs_dir*)((u8*)bktrContext.romfs_dir_entries + entryOffset)); // Only add entries inside the directory we're looking in if (entry->parent == dir_offset) @@ -1991,9 +2356,9 @@ bool getRomFsFileList(u32 dir_offset) { entryOffset = 0; - while(entryOffset < romFsContext.romfs_filetable_size) + while(entryOffset < fileTableSize) { - romfs_file *entry = (romfs_file*)((u8*)romFsContext.romfs_file_entries + entryOffset); + romfs_file *entry = (!usePatch ? (romfs_file*)((u8*)romFsContext.romfs_file_entries + entryOffset) : (romfs_file*)((u8*)bktrContext.romfs_file_entries + entryOffset)); // Only add entries inside the directory we're looking in if (entry->parent == dir_offset) @@ -2026,6 +2391,9 @@ bool getRomFsFileList(u32 dir_offset) } } + // Update current RomFS directory offset + curRomFsDirOffset = dir_offset; + return true; } @@ -2108,7 +2476,7 @@ void addStringToFilenameBuffer(const char *string, char **nextFilename) *nextFilename += FILENAME_LENGTH; } -char *generateDumpFullName() +char *generateFullDumpName() { if (!titleAppCount || !fixedTitleName || !titleAppTitleID || !titleAppVersion) return NULL; @@ -2624,6 +2992,46 @@ bool yesNoPrompt(const char *message) return ret; } +bool checkIfDumpedXciContainsCertificate(const char *xciPath) +{ + if (!xciPath || !strlen(xciPath)) return false; + + FILE *xciFile = NULL; + u64 xciSize = 0; + + size_t read_bytes; + + u8 xci_cert[CERT_SIZE]; + + u8 xci_cert_wiped[CERT_SIZE]; + memset(xci_cert_wiped, 0xFF, CERT_SIZE); + + xciFile = fopen(xciPath, "rb"); + if (!xciFile) return false; + + fseek(xciFile, 0, SEEK_END); + xciSize = ftell(xciFile); + rewind(xciFile); + + if (xciSize < (size_t)(CERT_OFFSET + CERT_SIZE)) + { + fclose(xciFile); + return false; + } + + fseek(xciFile, CERT_OFFSET, SEEK_SET); + + read_bytes = fread(xci_cert, 1, CERT_SIZE, xciFile); + + fclose(xciFile); + + if (read_bytes != (size_t)CERT_SIZE) return false; + + if (memcmp(xci_cert, xci_cert_wiped, CERT_SIZE) != 0) return true; + + return false; +} + bool checkIfDumpedNspContainsConsoleData(const char *nspPath) { if (!nspPath || !strlen(nspPath)) return false; diff --git a/source/util.h b/source/util.h index 8f7fcd1..7ba0525 100644 --- a/source/util.h +++ b/source/util.h @@ -43,6 +43,9 @@ #define GAMECARD_SIZE_ADDR 0x10D #define GAMECARD_DATAEND_ADDR 0x118 +#define GAMECARD_ECC_BLOCK_SIZE (u64)0x200 // 512 bytes +#define GAMECARD_ECC_DATA_SIZE (u64)0x24 // 36 bytes + #define HFS0_OFFSET_ADDR 0x130 #define HFS0_SIZE_ADDR 0x138 #define HFS0_MAGIC (u32)0x48465330 // "HFS0" @@ -136,6 +139,10 @@ bool isGameCardInserted(); void fsGameCardDetectionThreadFunc(void *arg); +bool isServiceRunning(const char *serviceName); + +bool checkSxOsServices(); + void delay(u8 seconds); void formatETAString(u64 curTime, char *output, u32 outSize); @@ -148,8 +155,16 @@ void initRomFsContext(); void freeRomFsContext(); +void initBktrContext(); + +void freeBktrContext(); + void freeGlobalData(); +bool loadPatchesFromSdCardAndEmmc(); + +void freePatchesFromSdCardAndEmmc(); + void convertTitleVersionToDecimal(u32 version, char *versionBuf, size_t versionBufSize); void removeIllegalCharacters(char *name); @@ -170,13 +185,15 @@ bool getPartitionHfs0FileByName(FsStorage *gameCardStorage, const char *filename bool calculateExeFsExtractedDataSize(u64 *out); -bool calculateRomFsExtractedDataSize(u64 *out); +bool calculateRomFsFullExtractedSize(bool usePatch, u64 *out); -bool readProgramNcaExeFsOrRomFs(u32 appIndex, bool readRomFs); +bool calculateRomFsExtractedDirSize(u32 dir_offset, bool usePatch, u64 *out); + +bool readProgramNcaExeFsOrRomFs(u32 titleIndex, bool usePatch, bool readRomFs); bool getExeFsFileList(); -bool getRomFsFileList(u32 dir_offset); +bool getRomFsFileList(u32 dir_offset, bool usePatch); int getSdCardFreeSpace(u64 *out); @@ -184,7 +201,7 @@ void convertSize(u64 size, char *out, int bufsize); void addStringToFilenameBuffer(const char *string, char **nextFilename); -char *generateDumpFullName(); +char *generateFullDumpName(); char *generateNSPDumpName(nspDumpType selectedNspDumpType, u32 titleIndex); @@ -216,6 +233,8 @@ bool checkIfFileExists(const char *path); bool yesNoPrompt(const char *message); +bool checkIfDumpedXciContainsCertificate(const char *xciPath); + bool checkIfDumpedNspContainsConsoleData(const char *nspPath); void removeDirectory(const char *path);