Posts on this page:

Simple JSON lexical parser in PowerShell (academic version)

Hey guys! I was silent for a while due to a lack of good topics to discuss. Today I want to present another piece of my class work at university for “Compiler Development” course. The task is to write a manual lexical parser for a language of my choice. I decided to take JSON language, because its syntax is relatively simple and requires most common techniques to parse. In addition, it has well-looking BNF grammar for custom parser implementations.

The purpose

The purpose of lexical analysis is to read the source code and convert them to a sequence of tokens (lexemes) which are minimal parts of each language. It is important understand that lexical analysis doesn’t perform semantic (meaning) validation. That is, lexical analysis determines whether the source code can be written in a specific language’s alphabet. It doesn’t mean that the code will be executed successfully. Source code semantic is validated only after lexical analysis and uses its product (a set of tables, keywords, operators, literals, identifiers, etc.).

You can think that there is no need to write your own lexical parser, because there are LOTS of them. For example, PowerShell contains built-in JSON encoder and decoder via ConvertTo-JSON and ConvertFrom-JSON cmdlets. Though, these cmdlets completely hide parsing result and perform object conversion. You can’t access internal parser to look at exact results of the parsing. But results of lexical parsers are actively used in web. For example, JS-based syntax highlighters use lexical parser to split the source code into tokens and colorize or highlight them for better readability. And my website does it as well (though, not via JS). For example, all XML and PowerShell code snippets on my blog are colorized by using lexical parsers. For PowerShell code I’m using Tokenize method in System.Management.Automation.PSParser class. For XML strings I’m using custom XML tokenizer. And cororize them according to token types.

Read more →

Array search: hash tables, behind the scenes

Hello blog readers!

Here is another tl;dr; blog post! Yesterday I completed my winter exam session at university and want to recall one interesting work I had year ago at the course called “Data structures and algorithms” where we learned various data structures and manipulation algorithms. During the course we developed them in programming languages with further analysis. In array search class work I had to implement, analyze and compare two search methods: sentinel search and hash table search.

Most search algorithms have complexity. This means that their performance depends on array size. Larger is array, more time is required to find element in array. There is binary search that gives which better than linear, but still depends on array size and requires sorted array. Binary search is impossible for unsorted arrays. What next? Next is search algorithm that would give us constant complexity. This means that regardless of array size, search will be completed in constant time. This algorithm (actually, data structure) is hash table.

What is hash table? It is an associative array that maps keys to data values. Unlike classic arrays, there is no such term as array index, instead there used term key value. Key is an identification information about data value. During class work I learned a lot about hash tables and faced a number of very interesting challenges while attempting to develop a reliable implementation of hash table. And this blog post will reveal all of them!

Read more →

X.509 Name Constraints certificate extension – all you should know

Hello S-1-1-0, PowerShell CryptoGuy (aka @Crypt32) is here again. Today I want to discuss about X.509 Name Constraints certificate extension. It is not widely used, but sometimes it is necessary. As extension name depicts, it is used to provide constraints or restrictions to certificate subject and subject alternative names (SAN) extension.

Brief Description

Name Constraints extension is defined and described in RFC 5280 § Extension presence in an end-entity certificate does not have any effect and is applied only to CA certificates that issue certificates to end entities. Once defined, the extension applies restrictions on any certificates that appear below that CA in the tree. Name Constraints may appear further in the certification path to set more restrictive constraints. It is not possible to set less restrictive constraints at lower levels. This prevents low-level (in the certification path meaning) CAs to violate restrictions applied at higher levels.

PKI Hierarchy

Figure 1 - sample certificate chain

Here we see a 3-tier PKI hierarchy with applied Name Constraints extension at 2nd level (below root). This is indicated by a yellow triangle. Name Constraints restrictions are applied to all directly and indirectly issued certificates. CA-2 doesn’t define Name Constraints extension in its own certificate, but restrictions still apply to certificates issued by CA-2 indirectly.

Read more →

Working with certificate revocation lists (CRL) in PowerShell (part 2)

Hello S-1-1-0!

In previous post we gave an introduction into techniques to work with certificate revocation lists in PowerShell. We explored common steps to read CRL’s basic information, CRL extensions and revoked certificate collection. Today I will discuss about CRL handy shortcuts and signature validation.

Get CRL next publication date and number

In some environments, it is impossible to automatically copy CRLs from CA server to CRL distribution points or there is a scenario when PKI administrators run custom scripts to monitor CRL health status at CRL distribution points and update them if they are about to expire. For such purposes I maintain two shortcut methods to quickly identify required values.

CRL validity is determined by a NextUpdate field. If the current time passes that timestamp, the CRL is considered expired. To provide better validity handling, Microsoft use their own Next CRL Publish CRL extension. This extension contains a date/time value at which CA will issue new CRL. This value (when present) is always set prior to value in NextUpdate field to provide a time window to replicate newly published CRL across all distribution points prior existing CRLs expire. I have a good article on this subject: How ThisUpdate, NextUpdate and NextCRLPublish are calculated (v2). However, Next CRL Publish extension is presented in CRLs issued by Microsoft CAs and is absent in 3rd party CAs, as the result, next CRL publication date is determined solely by Next Update field. Moreover, there might be a case when CA is in the decommission process and issues its last CRL which is supposed to be valid infinitely.

To simplify all these validations, I developed a GetNextPublish method. The logic of this method is as follows:

  1. If Next CRL Publish extension is presented, extension’s value is returned.
  2. If Next CRL Publish extension is absent: a value in the NextUpdate field is returned. If return value is null, the CRL is valid infinitely and no updated are expected.
PS C:\> $crl.GetNextPublish()

otrdiena, 2016. gada 25. oktobri 20:00:00

PS C:\>

For monitoring purposes you may want to compare CRL numbers (they are numbered sequentially) across all CRL distribution points with a base value (for example, retrieved from CA database). This will ensure that there are no stale CRLs especially when CRL was published manually. For such purposes, I maintain a GetCRLNumber method:

PS C:\> $crl.GetCRLNumber()

If the CRL Number extension is absent, then zero value is returned, because such CRLs doesn’t have CRL numbering capabilities. This is the case for Version 1 CRLs, or non-conformant CAs.

Signature Validation

Let’s take a look into our CRL (from DigiCert):

PS C:\> $crl

Version             : 2
Type                : Base CRL
IssuerName          : System.Security.Cryptography.X509Certificates.X500DistinguishedName
Issuer              : CN=DigiCert SHA2 Secure Server CA, O=DigiCert Inc, C=US
ThisUpdate          : 2016.10.18. 20:14:33
NextUpdate          : 2016.10.25. 20:00:00
SignatureAlgorithm  : 1.2.840.113549.1.1.11 (sha256RSA)
Extensions          : { (Authority Key Identifier), (CRL Number), (Issuing Distribution Po
RevokedCertificates : {Serial number: 0c587cfa9bf443daeab70526d4bc009f revoked at: 2015.11.06. 21:57:32, Serial number:
                       0b3ba5097ac6f59b551a1338357a0981 revoked at: 2015.11.09. 11:22:51, Serial number: 0e213e45ff44bd
                      975d6d22cbb8a40f2d revoked at: 2015.11.09. 19:21:01, Serial number: 08d64d9f888feee694b32d06bba9f
                      b83 revoked at: 2015.11.09. 19:22:03...}
RawData             : {48, 131, 4, 111...}
Handle              : 0

PS C:\>

Here we see a SignatureAlgorithm property that stores the information about signature algorithm (captain obvious is here too). This information is used to validate the signature against issuer. Let’s take proper and invalid issuer certificates:

PS C:\> $issuer = New-Object Security.Cryptography.X509Certificates.X509Certificate2 C:\Certs\DigiCertSHA2SecureServerCA
PS C:\> $nonissuer = New-Object Security.Cryptography.X509Certificates.X509Certificate2 C:\Certs\verisign.cer
PS C:\> $issuer

Thumbprint                                Subject
----------                                -------
1FB86B1168EC743154062E8C9CC5B171A4B7CCB4  CN=DigiCert SHA2 Secure Server CA, O=DigiCert Inc, C=US

PS C:\> $nonissuer

Thumbprint                                Subject
----------                                -------
A5EC73D48C34FCBEF1005AEB85843524BBFAB727  OU=Class 2 Public Primary Certification Authority, O="VeriSign, Inc.", C=US

PS C:\>

And try to use both certificates to validate signature:

PS C:\> $crl.VerifySignature($issuer,$true)
PS C:\> $crl.VerifySignature($nonissuer,$true)
PS C:\>

We can observe that DigiCert is the issuer of this CRL, while VeriSign is not. Plain and simple. VerifySignature method accepts two arguments: an instance of X509Certificate2 class and boolean value that indicates whether to perform CRL issuer name validation along with raw signature validation. In this case, SubjectName field of the issuer (or candidate at that point) is compared with IssuerName property of the CRL. Moreover, binary comparison is performed. This was made to be compatible with RFC3280 (although, obsolete, but still in use) which requires binary match. RFC5280 allows case-insensitive string match.

When using CRLs in your code to validate revocation status, do not forget to validate CRL signature as this will make your code more reliable and trustworthy.

Currently, VerifySignature supports RSA, DSA and ECDSA signatures. Alternate signature algorithms (RSASSA-PSS) are not yet supported, but I do have plans to add support of alternate signature formats.

Interoperability with CryptoAPI

In the past, I used CryptoAPI native functions (through p/invoke) to decode CRL objects. Since I have a reliable and very robust ASN.1 parser, I moved CRL decoder to a fully managed .NET code without using a piece of unmanaged code. As the result, normally you don’t need to dispose CRL object in order to release unmanaged resources and prevent memory leaks. However, there might be a need to get an unmanaged reference to CRL object. For such purposes I maintain a Handle property that contains a reference to a CRL object in unmanaged memory. Normally, it is zero:

PS C:\> $crl.Handle

If you need an unmanaged reference, call GetSafeContext method. This method will copy CRL object into unmanaged memory and retrieve a pointer to that memory block:

PS C:\> $crl.GetSafeContext()

                                                  IsInvalid                                                    IsClosed
                                                  ---------                                                    --------
                                                      False                                                       False

PS C:\> $crl.Handle

Now, you can use this handle to pass into unmanaged functions. For example, into CertAddCRLContextToStore function to add CRL into Windows Certificate Store. When CRL handle is populated, it must be freed after you finish working with CRL object by calling ReleaseContext method:

PS C:\> $crl.ReleaseContext()
PS C:\> $crl.Handle
PS C:\>

The difference between ReleaseContext and Reset methods is simple: ReleaseContext releases only unmanaged resources, while keeping managed object alive. If there are no unmanaged resources associated with the current object, the method does nothing (no errors are thrown). Reset method releases unmanaged resources (if necessary) and clears managed object by clearing all fields and setting them to default null/empty values.

That’s all for today. In next post, I’ll talk about CRL generation in PowerShell, so stay tuned!

Working with certificate revocation lists (CRL) in PowerShell (part 1)

Hello everyone!

Today I would like to summarize techniques on working with X.509 certificate revocation lists (CRL) in PowerShell. There are a lot of examples in my weblog, but most of this information is provided as context-specific addition to work in a given article’s context. Before talking about the subject, I’d like to put few words about the reason of this blog post and why it is written in that way.


As PowerShell evolves, it starts to cover more and new areas. And this process continues since PowerShell birth (in 2006). Systems administrators become more critical to script functionality. Previously, if something was not doable at all, we skipped that thing. With PowerShell we are able to do much more things. Maybe, not natively, maybe not in an elegant manner (say, through complex parsing), maybe very ugly, but we can do that right now. If community is interested in some area, they will develop a framework to make things easier and available to everyone.

One big area I’m interesting in is public key infrastructure, CryptoAPI, certificates and everything related to them. This happened several years ago (I recall it was in 2009) when I already was a passionate PowerShell enthusiast. Unfortunately, I realized that PowerShell has very-very basic support of certificate-related stuff. Mostly, this is due to poor support from underlying .NET platform. As the result, I attempted to start my big project to integrate cryptography into PowerShell through PS module and standalone scripts. Apparently, I was one of the first PS enthusiasts who started cryptography integration into PowerShell. I made huge work during these 7 years, went through tons of mistakes, bad choices, misunderstandings, trials and probes. It took 7 years and still continues. As of now, I have developed one of the best PowerShell module to work with PKI I’m aware of: PowerShell PKI. It is not just a module, it is a whole framework, which offers additional functionality through .NET-style objects and methods you can call from PS console. All this makes me confident in PowerShell PKI area.

Read more →