Point Of Interest

Yesterday I asked in Twitter, who can convert byte array to a formatted hex string in PowerShell in 5 minutes. I got one solution with the reference to Format-Hex function. Then I asked opposite question: can you convert formatted hex dump with address and ASCII panes back to byte array in PowerShell in 5 minutes? Didn’t get any response.

This subject is interesting and sometimes is necessary. Due to my specialization (cryptography), I have to deal with these formats often. And not only hex, Base64 with and without headers as well.

Common formatting examples

To make the subject clear, I provide some formatted examples:

• Unformatted raw hex string:

3081bd308198300906052b0e03021a0500304731133011060a0992268993f22c6401191603636f
6d31173015060a0992268993f22c6401191607636f6e746f736f311730150603550403130e636f
6e746f736f2d4443322d4341170d3130303330363131313033315a170d31353033303531313130
33315a302430220203123456170d3132303732343134323730355a300c300a0603551d1504030a
0103300906052b0e03021a05000315000e0df85638f094d170d905744f045d1287dff3df

• Formatted hex string, 16 octets per row:

30 81 bd 30 81 98 30 09  06 05 2b 0e 03 02 1a 05
00 30 47 31 13 30 11 06  0a 09 92 26 89 93 f2 2c
64 01 19 16 03 63 6f 6d  31 17 30 15 06 0a 09 92

• Hex string with address and ASCII values:

0000	30 81 bd 30 81 98 30 09  06 05 2b 0e 03 02 1a 05   0..0..0...+.....
0010	00 30 47 31 13 30 11 06  0a 09 92 26 89 93 f2 2c   .0G1.0.....&...,
0020	64 01 19 16 03 63 6f 6d  31 17 30 15 06 0a 09 92   d....com1.0.....

• Raw Base64

MIG9MIGYMAkGBSsOAwIaBQAwRzETMBEGCgmSJomT8ixkARkWA2NvbTEXMBUGCgmS
JomT8ixkARkWB2NvbnRvc28xFzAVBgNVBAMTDmNvbnRvc28tREMyLUNBFw0xMDAz

• Base64 with header:

-----BEGIN XYZ-----
MIG9MIGYMAkGBSsOAwIaBQAwRzETMBEGCgmSJomT8ixkARkWA2NvbTEXMBUGCgmS
JomT8ixkARkWB2NvbnRvc28xFzAVBgNVBAMTDmNvbnRvc28tREMyLUNBFw0xMDAz
-----END XYZ-----

Converting the byte array to one of these formats may be quite challenging. But real challenge is to convert formatted data back to binary copy.

CryptoAPI or cheat sheet

To work this out, you have to check and, most likely, significantly upgrade your string parsing skills in PowerShell. But it is worth to mention Windows built-in solutions which can be utilized in PowerShell.

Two things to know:

1. Bad thing: we will deal with Win32 native functions
2. Good thing: it will take just few lines of code :)

We will need two functions:

• CryptStringToBinary – to convert string to a byte array;
• CryptBinaryToString – to format byte array to a string: base64 (with various options) or to hex (with various options).

At first, let’s write unmanaged function signatures:

$signature = @"
[DllImport("Crypt32.dll", CharSet = CharSet.Auto, SetLastError = true)]
public static extern bool CryptStringToBinary(
    string pszString,
    int cchString,
    int dwFlags,
    byte[] pbBinary,
    ref int pcbBinary,
    int pdwSkip,
    ref int pdwFlags
);
[DllImport("Crypt32.dll", CharSet = CharSet.Auto, SetLastError = true)]
public static extern bool CryptBinaryToString(
    byte[] pbBinary,
    int cbBinary,
    int dwFlags,
    StringBuilder pszString,
    ref int pcchString
);
"@
Add-Type -MemberDefinition $signature -Namespace PKI -Name Crypt32 -UsingNamespace "System.Text"
$encoding = @{'Base64Header' = 0; 'Base64' = 1; 'HexRaw' = 12; 'Hex' = 4; 'HexAddr' = 10;
    'HexAscii' = 5; 'HexAddrAscii' = 11}

I used a simplified encoding set for brevity.

The signatures are fairly simple. Key notes about these functions:

• Functions returns Boolean which tells whether the input string is in the same format being requested in the dwFlags parameter.
• These functions require two calls, where first call returns the size for return value.
• Return value size and actual value is returned in the pcbBinary and pbBinary parameters, respectively, in the CryptStringToBinary function.
• Return value size and actual value is returned in the pszString and pcchString parameters, respectively, in the CryptBinaryToString function.
• flags CRYPT_STRING_ANY, CRYPT_STRING_HEX_ANY and CRYPT_STRING_BASE64_ANY are not supported in CryptBinaryToString functions.

Binary to string. Examples

Now we will look into some examples. The full sample code with comments:

# say, it is our byte array
[Byte[]]$array = 0..49
# initialize variable to receive resulting string length
$pcchString = 0
# call the CryptBinaryToString function to get string length. pszString is $null for the first call
if ([PKI.Crypt32]::CryptBinaryToString($array,$array.Length,$encoding['Base64'],$null,[ref]$pcchString)) {
    # instantiate a StringBuilder object with required size
    $SB = New-Object Text.StringBuilder $pcchString
    # call the function again and pass StringBuilder into pszString parameter
    [void][PKI.Crypt32]::CryptBinaryToString($array,$array.Length,$encoding['Base64'],$sb,[ref]$pcchString)
    # display produced output
    $SB.ToString()
} else {
    Write-Warning $((New-Object ComponentModel.Win32Exception ([Runtime.InteropServices.Marshal]::GetLastWin32Error())).Message)
}

When using the code for different formats, only dwFlags parameter is changed in both function calls. And here is the output for different formats:

• To Base64

AAECAwQFBgcICQoLDA0ODxAREhMUFRYXGBkaGxwdHh8gISIjJCUmJygpKissLS4v
MDE=

• To raw hex string

000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f3031

• To hex with 16 octets per row

00 01 02 03 04 05 06 07  08 09 0a 0b 0c 0d 0e 0f
10 11 12 13 14 15 16 17  18 19 1a 1b 1c 1d 1e 1f
20 21 22 23 24 25 26 27  28 29 2a 2b 2c 2d 2e 2f
30 31

• To hex with address pane

0000    00 01 02 03 04 05 06 07  08 09 0a 0b 0c 0d 0e 0f
0010    10 11 12 13 14 15 16 17  18 19 1a 1b 1c 1d 1e 1f
0020    20 21 22 23 24 25 26 27  28 29 2a 2b 2c 2d 2e 2f
0030    30 31

• To hex with ASCII pane

00 01 02 03 04 05 06 07  08 09 0a 0b 0c 0d 0e 0f   ................
10 11 12 13 14 15 16 17  18 19 1a 1b 1c 1d 1e 1f   ................
20 21 22 23 24 25 26 27  28 29 2a 2b 2c 2d 2e 2f    !"#$%&'()*+,-./
30 31                                              01

• To hex with both, address and ASCII panes

0000    00 01 02 03 04 05 06 07  08 09 0a 0b 0c 0d 0e 0f   ................
0010    10 11 12 13 14 15 16 17  18 19 1a 1b 1c 1d 1e 1f   ................
0020    20 21 22 23 24 25 26 27  28 29 2a 2b 2c 2d 2e 2f    !"#$%&'()*+,-./
0030    30 31                                              01

fast, quick, elegant!

String to binary. Examples

You can say that binary to hex is very simple, but opposite conversion is more sophisticated. You may need to determine whether the hex dump contains address and/or ASCII panes and ignore them during conversion. Hopefully, CryptStringToBinary function will take care of them. Here is another function to convert formatted string to a byte array:

function Convert-HexToBinary ([string]$hex) {
    # decoding hashtable contains universal flags: Base64Any and HexAny. The function attempts to
    # get the correct input string format and then decode it
    $decoding = @{'Base64Header' = 0; 'Base64' = 1; 'HexRaw' = 12; 'Hex' = 4; 'HexAddr' = 10;
        'HexAscii' = 5; 'HexAddrAscii' = 11; 'Base64Any' = 6; 'HexAny' = 8}
    # initialize variables to receive resulting byte array size and actual input string format
    $pcbBinary = 0
    $pdwFlags = 0
    # call CryptStringToBinary to get resulting byte array size and actual input string format
    if ([PKI.Crypt32]::CryptStringToBinary($hex,$hex.Length,$decoding['HexAny'],$null,[ref]$pcbBinary,0,[ref]$pdwFlags)) {
        # create enough large byte array
        $array = New-Object byte[] -ArgumentList $pcbBinary
        # call the function again to write converted bytes to a byte array
        [void][PKI.Crypt32]::CryptStringToBinary($hex,$hex.Length,$decoding['HexAny'],$array,[ref]$pcbBinary,0,[ref]$pdwFlags)
    } else {
        Write-Warning $((New-Object ComponentModel.Win32Exception ([Runtime.InteropServices.Marshal]::GetLastWin32Error())).Message)
    }
}

Look at this:

PS C:\> $string = @"
>> 0000    00 01 02 03 04 05 06 07  08 09 0a 0b 0c 0d 0e 0f
>> 0010    10 11 12 13 14 15 16 17  18 19 1a 1b 1c 1d 1e 1f
>> 0020    20 21 22 23 24 25 26 27  28 29 2a 2b 2c 2d 2e 2f
>> 0030    30 31
>> "@
>>
PS C:\> $array = Convert-HexToBinary $string
Input string encoding: 10
PS C:\> $array[0..10]
0
1
2
3
4
5
6
7
8
9
10
PS C:\> $array.Length
50
PS C:\>

As you can see, we got our original byte array! Exactly 50 bytes! I showed the $pdwFlags variable which stores actual format type of input string. 10 stands for CRYPT_STRING_HEXADDR. Don’t believe? Check CryptStringToBinary function description page.

You think it is all? Hell no! If we put comments, will it work?

PS C:\> $string = @"
>> 0000    00 01 02 03 04 05 06 07  08 09 0a 0b 0c 0d 0e 0f ;line one
>> 0010    10 11 12 13 14 15 16 17  18 19 1a 1b 1c 1d 1e 1f ; line two
>> 0020    20 21 22 23 24 25 26 27  28 29 2a 2b 2c 2d 2e 2f ; FF BB
>> 0030    30 31
>> "@
>>
PS C:\>  $array = Convert-HexToBinary $string
Input string encoding: 11
PS C:\> $array[0..10]
0
1
2
3
4
5
6
7
8
9
10
PS C:\> $array.Length
50
PS C:\>

The function is enough smart. Exactly 50 bytes. Though, the function identified the string as hex string with address and ASCII panes. But the result is achieved!

Final word

Today we explored another powerful and useful feature from Win32 repository. Decent google/bing skills and an ability to write native function signature definitions in PowerShell allows you to solve this task in 5 minutes! Not all love Win32, but sometimes it simply saves many hours of your work. Spend it wisely!

HTH

---