JSON is currently one of the most common format protocols for Internet communication. Data is also parsed and processed primarily in the JSON format.
JSONPath is a message query syntax specification introduced for the JSON format. In data processing, simple JSONPath syntax can be used to quickly obtain the value of a specific member in a complex nested JSON struct. In addition, extension functions of the JayWay library can be used to aggregate or operate member fields of a certain type.
Basic Features
Basic Syntaxes
The $ root node operator represents the root node of the current JSON struct.
The .<childName> dot operator or the ['<childName>'] bracket operator represents selecting the child member named childName of the current object.
The .. recursive operator represents obtaining all child members of the current object in a recursive manner.
The [<index>] subscript operator represents obtaining the index-th child member of the current iterable object.
Obtaining a Specific Member Variable in Nested JSON
The following figure shows the container stdout log structure collected by Tencent Kubernetes Engine (TKE):
To obtain the name of the current Pod, that is, the qcloud-app member field, you can use the JSONPath syntax $.kubernetes.labels.qcloud-app or $.['kubernetes'].['labels'].['qcloud-app'] in data processing.
The execution result is as follows. It can be seen that the log corresponding to JSONPath has been successfully read in the test result.
Note
When you use JSONPath to process parameters, if a JSON variable name contains special characters such as the period (.), the bracket operator must be used to wrap the variable name.
For example, to obtain the corresponding member field in the JSON struct {"key1.key2":"value1"}, you need to use $.['key1.key2'].
Advanced Features
Advanced Syntaxes
The * wildcard operator represents obtaining all child members of the current object.
The *~ built-in function represents obtaining the names of all child objects of the current iterable object.
The min() built-in function represents obtaining the minimum value among the child objects of the current iterable object.
The max() built-in function represents obtaining the maximum value among the child objects of the current iterable object.
The sum() built-in function represents obtaining the sum of the child objects of the current iterable object.
The concat() built-in function represents concatenating multiple objects and generating a string.
Aggregating Specific Field Data
When a JSON struct contains a list of objects, the list is typically of variable length. Take the request return log in the figure below as an example:
Processing chains cannot be used to aggregate member properties of a list with variable length, but the * syntax in JSONPath can be used to match all elements in the list.
For example, to obtain all VIP addresses in the VipList, you can use the JSONPath syntax $.data.Response.Result.Routers[*].VipList[0].Vip.
The execution result is as follows. It can be seen that the VIP addresses in all structs have been successfully obtained in the test result.
Merging and Modifying Struct Members
In certain scenarios, you need to merge multiple objects in a JSON struct during data processing to deliver them to downstream systems for further operations. Consider the following format:
When downstream systems lack computing capability, to aggregate the properties of Virtual Private Cloud (VPC) and subnets during data processing, you can use the concat() function in JSONPath to aggregate multiple fields, and further modify the strings on this basis.
For example, you can use the $.concat($.data.Response.SubnetSet[0].VpcId,"#",$.data.Response.SubnetSet[0].SubnetId,"#",$.data.Response.SubnetSet[0].CidrBlock)) syntax to concatenate the properties of VPC and subnets, and separate them with the # character.
The execution result is as follows. It can be seen that the resource information related to VPC has been successfully obtained and consolidated in the test result:
