Posts on this page:

Simple JSON lexical parser in PowerShell (academic version)

Hey guys! I was silent for a while due to a lack of good topics to discuss. Today I want to present another piece of my class work at university for “Compiler Development” course. The task is to write a manual lexical parser for a language of my choice. I decided to take JSON language, because its syntax is relatively simple and requires most common techniques to parse. In addition, it has well-looking BNF grammar for custom parser implementations.

The purpose

The purpose of lexical analysis is to read the source code and convert them to a sequence of tokens (lexemes) which are minimal parts of each language. It is important understand that lexical analysis doesn’t perform semantic (meaning) validation. That is, lexical analysis determines whether the source code can be written in a specific language’s alphabet. It doesn’t mean that the code will be executed successfully. Source code semantic is validated only after lexical analysis and uses its product (a set of tables, keywords, operators, literals, identifiers, etc.).

You can think that there is no need to write your own lexical parser, because there are LOTS of them. For example, PowerShell contains built-in JSON encoder and decoder via ConvertTo-JSON and ConvertFrom-JSON cmdlets. Though, these cmdlets completely hide parsing result and perform object conversion. You can’t access internal parser to look at exact results of the parsing. But results of lexical parsers are actively used in web. For example, JS-based syntax highlighters use lexical parser to split the source code into tokens and colorize or highlight them for better readability. And my website does it as well (though, not via JS). For example, all XML and PowerShell code snippets on my blog are colorized by using lexical parsers. For PowerShell code I’m using Tokenize method in System.Management.Automation.PSParser class. For XML strings I’m using custom XML tokenizer. And cororize them according to token types.


Read more →

Working with certificate revocation lists (CRL) in PowerShell (part 2)

Hello S-1-1-0!

In previous post we gave an introduction into techniques to work with certificate revocation lists in PowerShell. We explored common steps to read CRL’s basic information, CRL extensions and revoked certificate collection. Today I will discuss about CRL handy shortcuts and signature validation.

Get CRL next publication date and number

In some environments, it is impossible to automatically copy CRLs from CA server to CRL distribution points or there is a scenario when PKI administrators run custom scripts to monitor CRL health status at CRL distribution points and update them if they are about to expire. For such purposes I maintain two shortcut methods to quickly identify required values.

CRL validity is determined by a NextUpdate field. If the current time passes that timestamp, the CRL is considered expired. To provide better validity handling, Microsoft use their own Next CRL Publish CRL extension. This extension contains a date/time value at which CA will issue new CRL. This value (when present) is always set prior to value in NextUpdate field to provide a time window to replicate newly published CRL across all distribution points prior existing CRLs expire. I have a good article on this subject: How ThisUpdate, NextUpdate and NextCRLPublish are calculated (v2). However, Next CRL Publish extension is presented in CRLs issued by Microsoft CAs and is absent in 3rd party CAs, as the result, next CRL publication date is determined solely by Next Update field. Moreover, there might be a case when CA is in the decommission process and issues its last CRL which is supposed to be valid infinitely.


Read more →

Working with certificate revocation lists (CRL) in PowerShell (part 1)

Hello everyone!

Today I would like to summarize techniques on working with X.509 certificate revocation lists (CRL) in PowerShell. There are a lot of examples in my weblog, but most of this information is provided as context-specific addition to work in a given article’s context. Before talking about the subject, I’d like to put few words about the reason of this blog post and why it is written in that way.

Motivation

As PowerShell evolves, it starts to cover more and new areas. And this process continues since PowerShell birth (in 2006). Systems administrators become more critical to script functionality. Previously, if something was not doable at all, we skipped that thing. With PowerShell we are able to do much more things. Maybe, not natively, maybe not in an elegant manner (say, through complex parsing), maybe very ugly, but we can do that right now. If community is interested in some area, they will develop a framework to make things easier and available to everyone.

One big area I’m interesting in is public key infrastructure, CryptoAPI, certificates and everything related to them. This happened several years ago (I recall it was in 2009) when I already was a passionate PowerShell enthusiast. Unfortunately, I realized that PowerShell has very-very basic support of certificate-related stuff. Mostly, this is due to poor support from underlying .NET platform. As the result, I attempted to start my big project to integrate cryptography into PowerShell through PS module and standalone scripts. Apparently, I was one of the first PS enthusiasts who started cryptography integration into PowerShell. I made huge work during these 7 years, went through tons of mistakes, bad choices, misunderstandings, trials and probes. It took 7 years and still continues. As of now, I have developed one of the best PowerShell module to work with PKI I’m aware of: PowerShell PKI. It is not just a module, it is a whole framework, which offers additional functionality through .NET-style objects and methods you can call from PS console. All this makes me confident in PowerShell PKI area.


Read more →

Efficient way to get AD user membership recursively with PowerShell

The other day, one customer asked for a solution to get full user membership in Active Directory for audit purposes. The solution should retrieve not only direct group membership, but indirect (through group nesting) too. Although, the question is plain and simple, solution is very interesting from various perspectives.

At first, let illustrate a sample user and group membership diagram:

Active Directory group membership graph

Quick diagram observation suggests us that we have a directed graph (it is not a tree), where users and groups are vertexes and membership relations are directed edges. Arrows identify relationship direction.

Our graph contains two users, User1 and User2 and eight groups: G1G8. In a given case, User1 is direct member of groups G1, G2 and G3, User2 is direct member of G8 only. Group G1 is member of G4, G2 is direct member of G4 and G5 and so on. For description purposes I labeled all edges. This should be clear.


Read more →

Convert data between binary, hex and Base64 in PowerShell – Managed Edition

Some time ago I wrote a blog post about converting data between Hex, Base64 and binary in PowerShell by using CryptoAPI functionality: Convert data between binary, hex and Base64 in PowerShell. I was impressed by those functions, because CryptStringToBinary function is magical and is able to convert “messy” hex string sequence (with or without address and ASCII columns which are not part of the data) to a pure byte array. I wish they have a bit more flexibility and extensibility. For example, when supporting certificate issues, I receive dumps from 3rd party tools (OpenSSL and similar) and browsers. They use different delimiters to separate hex octets. One tool use minus signs, other use colons to separate hex octets:

0000 - 20 ab 34 00 ff 87 50 1e-de fb c9 3d 10 2f 7b fd    .4...P....=./{.
0010 - 99 a1 61 e0 3d 5f 93 82-63 e9 0a 6f 1a 22 4f 04   ..a.=_..c..o."O.

or this:

26:C0:29:E9:8C:AB:C3:9E:95:38:74:8A:87:D3:86:8D
5C:5A:BA:47:44:83:7E:CB:48:BE:DD:E5:39:51:24:42:C6:C5:60:8B
DA:26:B8:C8:F4:04:3E:62:F3:7F:3B:EC:1D:9F:85:66:28:00:45:55:66:
15:FF:BB:37:77:97:59:F0:EC:0B:B6

Unfortunately, CryptStringToBinary supports only whitespace characters as delimiter and I have to manually remove them from dump before converting to a byte array. So I decided to get my own converter with blackjack and hookers in managed language.


Read more →