DataPower Latency Regular Expressions

DPLatencyRegex.md

This Gist is a small collection of regular expressions used to parse DataPower latency log entries. It only works for basic latency entries, not extended latency entries. Also, it may only work for MPGS; this hasn't been tested for WSPs.

reg = re.compile(r'\w\w\w (\w\w\w \d\d \d\d\d\d \d\d:\d\d:\d\d) \[0x80e00073\]\[latency\]\[info\] \w+\(([^)]+)\): tid\((\d+)\).+ Latency:\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+\[(https?://[^/]+)([^?]+)?(\?.+)?\]')

Here is a description by groupings.

1. \w\w\w (\w\w\w \d\d \d\d\d\d \d\d:\d\d:\d\d) - This is the first grouping. It makes the date (not including the week day) the first element.
2. \[0x80e00073\]\[latency\]\[info\] \w+\(([^)]+)\) - This skips a bunch of constants and captures the name of the MPGW. It expects it to be between the matching parenthesis.
3. : tid\((\d+)\) - This captures the Transaction ID.
4. .+ Latency: - This makes sure we're matching a Latency record. The constant in Step 2 probably did this already. The space before the capital L makes sure this is not an extended latency record.
5. \s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+) - These are the 16 numbers separated by spaces.
6. \s+\[(https?://[^/]+) - this matches the host and port.
7. ([^?]+)? - this matches the optional URI after the host.
8. (\?.+)? - this matches an optional parameter list.
9. \] - The expression should end with a square bracket.