So, time for the daily development screenshot:

I use hex editors a lot. In particular, I use XVI32 a lot. While it's a very nice hex editor, I feel that some extra functionality in many areas would be very nice. For example:

• Ability to open 'massive' (i.e. between 4 and 100 GB) files (XVI32 is limited to 2GB)
• Faster loading of 'large' (i.e. between 100 MB and 4 GB) files
• Plugin system to extend the editor in various parts
• 'Data transformation' system to facilitate data [de]compression and [de|en]cryption within the editor
• 'Structure analysis' system to facilitate viewing the structure of a known formats (e.g. Relic Chunky files) - extended to new formats using the plugin system
• More options for the data inspector (i.e. "byte: 0xf4, x86 instruction: hlt") - again, extended via aforementioned plugins
• More built-in checksums (XVI32 only calculates CRC16 and CRC32)
• 'Data visualisation' system to facilitate viewing plaintext or image data in a suitable manner

As such, I'm writing my own hex editor, HexEditIX, the first development screenshot of which is shown below. (of course, it's not much of an editor yet, but that's software development for you - you have to start somewhere)

First of all, if you're not aware of DoW2DB, it's my latest website - a database of information about the units in Dawn of War II.

One of the reasons for making the site was to see how well Google AdSense (online adverts provided by Google) works. Despite applying for AdSense 3 weeks ago, I've only just been able to put up ads on DoW2DB.

1. February 14th: Applied for AdSense. Application page stated that it should take a day or two to review the application, help pages state it may take up to one week if they are busy.
2. February 26th (almost 2 weeks later): Got email reply stating that the application was refused due to an invalid or incorrect postal address. This confused me, as the address in my application matched exactly the address on my bank statement. Hence I sent an email to AdSense support with a scan of a bank statement with address on attached, and queried why they decided that my address was invalid. Got an automated reply "We noticed that you may have contacted us without first visiting our Help Center: ... To receive faster responses to your questions, we encourage you to visit our help resources -- they're full of instant, reliable answers to many common questions and issues ...". The Help Center does not contain any entries on Google deciding that postal addresses are incorrect when they are not.
3. March 5th (one week later): Still no human response from AdSense support. Decided to re-apply to AdSense without making any changes to the application. Application page stated that it should take a day or two to review the application.
4. March 7th (two days later): Application approved.

I'm intrigued as to why it took Google much longer than their stated two days, and almost double their busy period stated week. I'm further intrigued as to why they decided that my address was incorrect, why there was no useful response from their support email address within a week, and why they then decided that everything was good when I hadn't changed anything.

Anyway, there are now ads up on DoW2DB, and I'm interested in seeing what happens as a result.

This post addresses a few of the questions which have arisen since part 1:

"can you put file to rapidshare" (and similar requests)
I'm here to point out flaws in Relic's security chain. I'm not here to give everyone quick and easy access to the pre-order content, so I will not be posting the modified files myself.

"Well done there, though the question of how long this will be possible for, how much data the Steam client collects about the programs it runs (DLL hashes are pretty standard, or?) and what Relic's views on this might be still remain."
Hashing of DLLs is fairly standard; for example, if you modify SimEngine.dll (perhaps to add a maphack), then you can't join online games as your hashes won't match those of the other players. However, as wincrypt.dll is not a DLL which comes with the game, it is not hashed by default. Furthermore, as it is a Windows DLL, it will (probably) change with each Windows version and/or service pack and/or patch, meaning that it cannot be reliably hashed.

"How do I find where this function is which I need to replace the first five bytes of?"
There are tools which list all exported functions of a DLL and their virtual positions, i.e. the position in virtual memory (relative to the DLL base) at which you will find the function when the DLL is loaded into memory. Converting this virtual position into a physical position, i.e. the position in the file on disk of the function, can be done by analysing the DLL's PE section table in a suitable tool.

"I don't see anything about UnlockMask in the content.bin, do you have any more specific instructions for this?"
The contents of content.bin can be viewed in two ways; calling dofile("content.bin") in a Lua interpreter and then printing the resulting DLC table, or running content.bin through luac: (lua and luac are both part of the official Lua.org package)

E:\Lua>lua
Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio
> dofile "content.bin"
> print_recursive(DLC) -- implementation left to the reader

E:\Lua>luac -l content.bin

main <e:\dow2-release\assets\config\content.lua:0,0> (197 instructions, 788 bytes at 008B7AC8)
0+ params, 17 slots, 0 upvalues, 0 locals, 100 constants, 0 functions
    1  [10] NEWTABLE  0 1 0
    2  [11] NEWTABLE  1 0 3
    3  [13] SETTABLE  1 -2 -3 ; "ContentName" "Retailer Exclusive DLC"
    4  [14] SETTABLE  1 -4 -5 ; "ContentID" "FC6A678326EAC085ACA365AE99B50A121F98BE53"
    5  [15] NEWTABLE  2 6 0
    6  [16] NEWTABLE  3 0 6
    7  [18] SETTABLE  3 -7 -8 ; "OfferName" "Assault Marine Bonus Pack"
    8  [19] SETTABLE  3 -9 -10 ; "LicenseMask" "1"
    9  [21] NEWTABLE  4 3 0
   10  [23] LOADK     5 -12 ; "wargear\\wargear\\campaign\\playable\\race_marine\\sm_arm_power_armour_ravens_barding_of_flight"
   11  [24] LOADK     6 -13 ; "wargear\\wargear\\campaign\\playable\\race_marine\\sm_wp2_chainsword_blade_of_ulyus"
   12  [26] LOADK     7 -14 ; "wargear\\wargear\\campaign\\playable\\race_marine\\sm_wp1_bolt_pistol_herald_of_the_coming_doom"
   13  [26] SETLIST   4 3 1 ; 1
   14  [26] SETTABLE  3 -11 4 ; "NewCampaignWargear" -
   15  [28] NEWTABLE  4 4 0
   16  [30] LOADK     5 -16 ; "Burnished Gold"
   17  [31] LOADK     6 -17 ; "Abyss Red"
   18  [32] LOADK     7 -18 ; "Metallic Green"
   19  [34] LOADK     8 -19 ; "Crystal Blue"
   20  [34] SETLIST   4 4 1 ; 1
   21  [34] SETTABLE  3 -15 4 ; "Paints" -
   22  [36] NEWTABLE  4 1 0
   23  [39] LOADK     5 -21 ; "SM_Brazen_Claws"
   24  [39] SETLIST   4 1 1 ; 1
   25  [39] SETTABLE  3 -20 4 ; "Badges" -
   26  [41] NEWTABLE  4 1 0
   27  [44] LOADK     5 -21 ; "SM_Brazen_Claws"
   28  [44] SETLIST   4 1 1 ; 1
   29  [44] SETTABLE  3 -22 4 ; "ArmyPainterTemplates" -
   30  [45] NEWTABLE  4 0 6
   31  [48] SETTABLE  4 -7 -23 ; "OfferName" "Devastator Marine Bonus Pack"
   32  [49] SETTABLE  4 -9 -24 ; "LicenseMask" "2"
   33  [51] NEWTABLE  5 3 0
   34  [53] LOADK     6 -25 ; "wargear\\wargear\\campaign\\playable\\race_marine\\sm_arm_power_armour_armor_of_the_destroyer"
   35  [54] LOADK     7 -26 ; "wargear\\wargear\\campaign\\playable\\race_marine\\sm_wp1_heavy_bolter_purge_of_victory_bay"
   36  [56] LOADK     8 -27 ; "wargear\\wargear\\campaign\\playable\\race_marine\\sm_wp1_missile_launcher_unerring_thunderbolt"
   37  [56] SETLIST   5 3 1 ; 1
   38  [56] SETTABLE  4 -11 5 ; "NewCampaignWargear" -
   39  [58] NEWTABLE  5 4 0
   40  [60] LOADK     6 -28 ; "Dwarf Bronze"
   41  [61] LOADK     7 -29 ; "Abyss Blue"
   42  [62] LOADK     8 -30 ; "Abyss Magenta"
   43  [64] LOADK     9 -31 ; "Bubonic Brown"
   44  [64] SETLIST   5 4 1 ; 1
   45  [64] SETTABLE  4 -15 5 ; "Paints" -
   46  [66] NEWTABLE  5 1 0
   47  [69] LOADK     6 -32 ; "SM_Novamarines"
   48  [69] SETLIST   5 1 1 ; 1
   49  [69] SETTABLE  4 -20 5 ; "Badges" -
   50  [71] NEWTABLE  5 1 0
   51  [74] LOADK     6 -32 ; "SM_Novamarines"
   52  [74] SETLIST   5 1 1 ; 1
   53  [74] SETTABLE  4 -22 5 ; "ArmyPainterTemplates" -
   54  [75] NEWTABLE  5 0 6
   55  [78] SETTABLE  5 -7 -33 ; "OfferName" "Force Commander Bonus Pack"
   56  [79] SETTABLE  5 -9 -34 ; "LicenseMask" "4"
   57  [81] NEWTABLE  6 6 0
   58  [83] LOADK     7 -35 ; "wargear\\wargear\\campaign\\playable\\race_marine\\sm_arm_power_armour_cuirass_of_azariah"
   59  [84] LOADK     8 -36 ; "wargear\\wargear\\campaign\\playable\\race_marine\\sm_wp1_bolt_pistol_pistol_of_baal"
   60  [85] LOADK     9 -37 ; "wargear\\wargear\\campaign\\playable\\race_marine\\sm_wp2_powerfist_gauntlet_of_blood"
   61  [86] LOADK     10 -38 ; "wargear\\wargear\\campaign\\playable\\race_marine\\sm_com_iron_halo_laurels_of_hadrian"
   62  [87] LOADK     11 -39 ; "wargear\\wargear\\campaign\\playable\\race_marine\\sm_wp1_plasma_gun_fearsome_light_of_faith"
   63  [89] LOADK     12 -40 ; "wargear\\wargear\\campaign\\playable\\race_marine\\sm_arm_power_armour_mantle_of_the_great_father"
   64  [89] SETLIST   6 6 1 ; 1
   65  [89] SETTABLE  5 -11 6 ; "NewCampaignWargear" -
   66  [91] NEWTABLE  6 8 0
   67  [93] LOADK     7 -41 ; "Shining Gold"
   68  [94] LOADK     8 -42 ; "Mithril Silver"
   69  [95] LOADK     9 -43 ; "Abyss Purple"
   70  [96] LOADK     10 -44 ; "Insect Green"
   71  [97] LOADK     11 -45 ; "Iron"
   72  [98] LOADK     12 -46 ; "Abyss Orange"
   73  [99] LOADK     13 -47 ; "Brazen Brass"
   74 [101] LOADK     14 -48 ; "Metallic Blue"
   75 [101] SETLIST   6 8 1 ; 1
   76 [101] SETTABLE  5 -15 6 ; "Paints" -
   77 [103] NEWTABLE  6 2 0
   78 [105] LOADK     7 -49 ; "SM_Revilers"
   79 [107] LOADK     8 -50 ; "SM_Angels_Sanguine"
   80 [107] SETLIST   6 2 1 ; 1
   81 [107] SETTABLE  5 -20 6 ; "Badges" -
   82 [109] NEWTABLE  6 2 0
   83 [111] LOADK     7 -49 ; "SM_Revilers"
   84 [113] LOADK     8 -50 ; "SM_Angels_Sanguine"
   85 [113] SETLIST   6 2 1 ; 1
   86 [113] SETTABLE  5 -22 6 ; "ArmyPainterTemplates" -
   87 [114] NEWTABLE  6 0 6
   88 [117] SETTABLE  6 -7 -51 ; "OfferName" "Scout Marine Bonus Pack"
   89 [118] SETTABLE  6 -9 -52 ; "LicenseMask" "8"
   90 [120] NEWTABLE  7 3 0
   91 [122] LOADK     8 -53 ; "wargear\\wargear\\campaign\\playable\\race_marine\\sm_arm_scout_armour_grim_silence"
   92 [123] LOADK     9 -54 ; "wargear\\wargear\\campaign\\playable\\race_marine\\sm_wp1_combat_shotgun_initiates_lesson_of_strength"
   93 [125] LOADK     10 -55 ; "wargear\\wargear\\campaign\\playable\\race_marine\\sm_wp1_sniper_rifle_deathtouch_of_the_angel"
   94 [125] SETLIST   7 3 1 ; 1
   95 [125] SETTABLE  6 -11 7 ; "NewCampaignWargear" -
   96 [127] NEWTABLE  7 4 0
   97 [129] LOADK     8 -56 ; "Abyss Yellow"
   98 [130] LOADK     9 -57 ; "Chainmail"
   99 [131] LOADK     10 -58 ; "Abyss Green"
  100 [133] LOADK     11 -59 ; "Rotting Flesh"
  101 [133] SETLIST   7 4 1 ; 1
  102 [133] SETTABLE  6 -15 7 ; "Paints" -
  103 [135] NEWTABLE  7 1 0
  104 [138] LOADK     8 -60 ; "SM_Angels_of_Redemption"
  105 [138] SETLIST   7 1 1 ; 1
  106 [138] SETTABLE  6 -20 7 ; "Badges" -
  107 [140] NEWTABLE  7 1 0
  108 [143] LOADK     8 -60 ; "SM_Angels_of_Redemption"
  109 [143] SETLIST   7 1 1 ; 1
  110 [143] SETTABLE  6 -22 7 ; "ArmyPainterTemplates" -
  111 [144] NEWTABLE  7 0 6
  112 [147] SETTABLE  7 -7 -61 ; "OfferName" "Tactical Marine Bonus Pack"
  113 [148] SETTABLE  7 -9 -62 ; "LicenseMask" "16"
  114 [150] NEWTABLE  8 3 0
  115 [152] LOADK     9 -63 ; "wargear\\wargear\\campaign\\playable\\race_marine\\sm_arm_power_armour_armor_of_vandea"
  116 [153] LOADK     10 -64 ; "wargear\\wargear\\campaign\\playable\\race_marine\\sm_wp1_bolter_unforgiving_truth"
  117 [155] LOADK     11 -65 ; "wargear\\wargear\\campaign\\playable\\race_marine\\sm_acc_frag_grenade_pack_veterans_grenades"
  118 [155] SETLIST   8 3 1 ; 1
  119 [155] SETTABLE  7 -11 8 ; "NewCampaignWargear" -
  120 [157] NEWTABLE  8 4 0
  121 [159] LOADK     9 -66 ; "Bronze"
  122 [160] LOADK     10 -67 ; "Metallic Yellow"
  123 [161] LOADK     11 -68 ; "Metallic Orange"
  124 [163] LOADK     12 -69 ; "Spectral Blue"
  125 [163] SETLIST   8 4 1 ; 1
  126 [163] SETTABLE  7 -15 8 ; "Paints" -
  127 [165] NEWTABLE  8 1 0
  128 [168] LOADK     9 -70 ; "SM_Taurans"
  129 [168] SETLIST   8 1 1 ; 1
  130 [168] SETTABLE  7 -20 8 ; "Badges" -
  131 [170] NEWTABLE  8 1 0
  132 [173] LOADK     9 -70 ; "SM_Taurans"
  133 [173] SETLIST   8 1 1 ; 1
  134 [173] SETTABLE  7 -22 8 ; "ArmyPainterTemplates" -
  135 [174] NEWTABLE  8 0 7
  136 [177] SETTABLE  8 -7 -71 ; "OfferName" "Wargear Bonus Pack"
  137 [178] SETTABLE  8 -9 -72 ; "LicenseMask" "32"
  138 [180] NEWTABLE  9 1 0
  139 [183] LOADK     10 -74 ; "data:maps\\pvp\\2p_legishighstratum"
  140 [183] SETLIST   9 1 1 ; 1
  141 [183] SETTABLE  8 -73 9 ; "Maps" -
  142 [185] NEWTABLE  9 7 0
  143 [187] LOADK     10 -75 ; "wargear\\wargear\\campaign\\playable\\race_marine\\sm_com_battle_standard_chapter_battle_standard"
  144 [188] LOADK     11 -76 ; "wargear\\wargear\\campaign\\playable\\race_marine\\sm_wp2_chainsword_snarl_of_the_wolf"
  145 [189] LOADK     12 -77 ; "wargear\\wargear\\campaign\\playable\\race_marine\\sm_wp1_plasma_gun_purifier_of_tombs"
  146 [190] LOADK     13 -78 ; "wargear\\wargear\\campaign\\playable\\race_marine\\sm_wp1_sniper_rifle_honored_silence"
  147 [191] LOADK     14 -79 ; "wargear\\wargear\\campaign\\playable\\race_marine\\sm_wp1_heavy_bolter_scourge_of_xenos"
  148 [192] LOADK     15 -80 ; "wargear\\wargear\\campaign\\playable\\race_marine\\sm_acc_purity_seal_light_of_the_astronomicon"
  149 [194] LOADK     16 -81 ; "wargear\\wargear\\campaign\\playable\\race_marine\\sm_acc_purity_seal_parable_of_the_lion"
  150 [194] SETLIST   9 7 1 ; 1
  151 [194] SETTABLE  8 -11 9 ; "NewCampaignWargear" -
  152 [196] NEWTABLE  9 4 0
  153 [198] LOADK     10 -82 ; "Scaly Green"
  154 [199] LOADK     11 -83 ; "Platinum"
  155 [200] LOADK     12 -84 ; "Golden Purple"
  156 [202] LOADK     13 -85 ; "Metallic Red"
  157 [202] SETLIST   9 4 1 ; 1
  158 [202] SETTABLE  8 -15 9 ; "Paints" -
  159 [204] NEWTABLE  9 1 0
  160 [207] LOADK     10 -86 ; "SM_Marauders"
  161 [207] SETLIST   9 1 1 ; 1
  162 [207] SETTABLE  8 -20 9 ; "Badges" -
  163 [209] NEWTABLE  9 1 0
  164 [212] LOADK     10 -86 ; "SM_Marauders"
  165 [212] SETLIST   9 1 1 ; 1
  166 [212] SETTABLE  8 -22 9 ; "ArmyPainterTemplates" -
  167 [214] SETLIST   2 6 1 ; 1
  168 [214] SETTABLE  1 -6 2 ; "Offers" -
  169 [216] SETLIST   0 1 1 ; 1
  170 [216] SETGLOBAL 0 -1 ; DLC
  171 [218] NEWTABLE  0 7 0
  172 [219] NEWTABLE  1 0 2
  173 [221] SETTABLE  1 -88 -89 ; "map_name" "6P_Typhon_plateau"
  174 [222] SETTABLE  1 -90 -91 ; "map_mode_type" "3_vs_3"
  175 [223] NEWTABLE  2 0 2
  176 [225] SETTABLE  2 -88 -92 ; "map_name" "6p_Calderis_outskirts"
  177 [226] SETTABLE  2 -90 -91 ; "map_mode_type" "3_vs_3"
  178 [227] NEWTABLE  3 0 2
  179 [229] SETTABLE  3 -88 -93 ; "map_name" "6p_MeridianHighCity"
  180 [230] SETTABLE  3 -90 -91 ; "map_mode_type" "3_vs_3"
  181 [231] NEWTABLE  4 0 2
  182 [233] SETTABLE  4 -88 -94 ; "map_name" "6p_arena"
  183 [234] SETTABLE  4 -90 -91 ; "map_mode_type" "3_vs_3"
  184 [235] NEWTABLE  5 0 2
  185 [237] SETTABLE  5 -88 -95 ; "map_name" "2p_CalderisDunes"
  186 [238] SETTABLE  5 -90 -96 ; "map_mode_type" "1_vs_1"
  187 [239] NEWTABLE  6 0 2
  188 [241] SETTABLE  6 -88 -97 ; "map_name" "2p_TyphonSwamp"
  189 [242] SETTABLE  6 -90 -96 ; "map_mode_type" "1_vs_1"
  190 [243] NEWTABLE  7 0 2
  191 [245] SETTABLE  7 -88 -98 ; "map_name" "1v1typhonlowlands"
  192 [246] SETTABLE  7 -90 -96 ; "map_mode_type" "1_vs_1"
  193 [248] SETLIST   0 7 1 ; 1
  194 [248] SETGLOBAL 0 -87 ; QuickMatchMaps
  195 [250] LOADK     0 -100 ; "200"
  196 [250] SETGLOBAL 0 -99 ; LocalPingAutomatchAdjustment
  197 [250] RETURN    0 1

Once you've looked at the contents of the file, it is trivial to load it into a hex editor and replace the 1, 2, 4, 8, 16 and 32 strings with 0, 0, 0, 0, 00 and 00 respectively.

Dawn of War II has unlockable content; that is certain maps, campaign wargear, army painter colours and so on are locked by default, and have to be unlocked using special codes. People who pre-ordered the game got a code or two, and codes are also available through other promotional sources.

This unlockable content is implemented as follows:

• The content.bin file in the Dawn of War 2 folder lists all of the unlockable content, in the form of a compiled Lua file.
• Each set of unlockable content has an entry in the content.bin file listing the files/colours/wargear/etc. which it unlocks, and the UnlockMask required to unlock it.
• Each GFWL account has an UnlockBits field.
• Registering an unlock code sets the appropriate bit in your UnlockBits field.
• To decide which content to unlock, the game (DOW2.exe) iterates over each set in turn, tests to see if all the bits in the UnlockMask are set in your UnlockBits (e.g. if UnlockMask bitwise-and UnlockBits equals UnlockMask then), and if so, unlocks the things in that set.
• content.bin is cryptographically signed by the content.cat security catalogue file.
• DOW2.exe is cryptographically signed by the DOW2.exe.cat security catalogue file.
• wintrust.dll is used by DOW2.exe, xlive.dll, and some other DLLs to verify the security catalogues.

The standard approaches to unlocking all of the content might include:

• Modify content.bin and set every UnlockMask to zero (zero bitwise-and UnlockBits equals zero). This would fail as then content.cat would fail to verify content.bin and DOW2.exe would abort loading.
• Modify DOW2.exe and patch the code to read zero for every UnlockMask. Again, this would fail as xlive.dll would fail to validate the signature on DOW2.exe and multiplayer wouldn't work.

Neither of the above methods would work, due to the cryptographic checks done on the content.bin and DOW2.exe files. The weakpoint of DoW2's system is how these checks are done. As previously stated, wintrust.dll (a Microsoft DLL which lives in C:\Windows\System32) is used to make sure that the security catalogue files are valid and successfully verify DOW2.exe and content.bin, using the WinVerifyTrust[Ex] function. Furthermore, wintrust.dll itself is not cryptographically signed. If wintrust.dll is copied from the System32 directory to the dawn of war directory and then modified so that WinVerifyTrust always returns ERROR_SUCCESS, then the cryptographic checks are sidestepped (ERROR_SUCCESS is conveniently the value zero, so this simply means replacing the first five bytes of WinVerifyTrust with xor eax, eax; retn 12; or 33 C0 C2 0C 00 in machine code).

If content.bin is now modified so that the unlock masks are all zero, then when the game is run, it'll load wintrust.dll from the game directory rather than the System32 directory (DLLs in the 'current' directory override those in the system directory by default), and when it comes to verify content.bin, the patched WinVerifyTrust returns ERROR_SUCCESS, and so the game believes that the file is still cryptographically signed. It then comes to see what content the GFWL account has unlocked by doing 'does UnlockMask bitwise-and UnlockBits equal UnlockMask' for each unlock set, and as zero bitwise-and anything does equal zero, it'll unlock all of the content.

For further reading, see part 2.