eAPI Benchmark program for Java

This document describes about Benchmark.java, it is the benchmark program used at eAPI Performance Test.

Yutaka Yasuda (Arista Square)

24/June/2013 : First release. Test had been done by 7048T switch with EOS 4.12.1.

1. Introduction
1.1 Data Format
1.2 HTTP Header Format
2. Getting Started
3. Code

1. Introduction

eAPI executes EOS CLI commands on the switch remotely by using JSONRPC over HTTP/HTTPS. That means the eAPI client program needs to be adaptive to JSONRPC formed data handling and communication by HTTP/HTTPS. And also the Basic Authentication feature would be required for HTTP/HTTPS communication.

1.1 Data Format

Here is the JSON formatted data to push the "show version" command to the switch.

The request { "id":0, "method":"runCmds", "params":{ "cmds":[ "show version" ], "format":"json", "version":1 }, "jsonrpc":"2.0" } The response { "jsonrpc": "2.0", "result": [ { "modelName": "DCS-7048T-4S-F", "internalVersion": "4.12.1-1275950.EOS4121", "systemMacAddress": "00:1c:73:0c:0c:0c", "serialNumber": "JSH09494949", "memTotal": 2009472, "bootupTimestamp": 1371631462.6494012, "memFree": 121376, "version": "4.12.1", "architecture": "i386", "internalBuildId": "688851f9-4aac-4f78-81ca-69a8114f172a", "hardwareRevision": "02.02" } ], "id": 0 }

1.2 HTTP Header Format

Here is the actual HTTP headers which had been exchanged at this program. It includes the Basic Authentication header.

The POST request POST /command-api HTTP/1.1 Accept-Charset: UTF-8 Content-Type: application/json User-Agent: C/YasuTest Host: 192.168.11.111:80 Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2 Connection: keep-alive Content-Length: 118 Authorization: Basic WjdWu7DwjD2xhw9Di The response HTTP/1.0 200 OK Server: BaseHTTP/0.3 Python/2.7 Date: Fri, 21 Jun 2013 09:58:51 GMT Cache-control: no-store Cache-control: no-cache Cache-control: must-revalidate Cache-control: max-age=0 Cache-control: pre-check=0 Cache-control: post-check=0 Pragma: no-cache Set-Cookie: COMMAND_API_SESSION=0; Path=/ Content-type: application/json Content-length: 542 Connection: close

2. Getting Started

[ Excuse my limited ability about Java. The code would be miserable. I have almost nothing about Java but my student who is trying to touch eAPI prefers to use Java. ]

Setup and Execute the Sample Program

A necessary thing is correspondence to JSONRPC over HTTP / HTTPS and there are some libraries can make it. This time, I tried JSON-RPC 2.0. Additionally, it requires json-smart too.

The required files are served following sites. Get them and place in the same directory.

JSON-RPC 2.0: jsonrpc2-base-1.35.jar, jsonrpc2-client-1.14.3.jar, jsonrpc2-server-1.10.1.jar
json-smart: json-smart-1.1.1.jar

Here is a Benchmark.java which executes "show version" and "show vlan" command. You can get / compile / execute it as follows;

Compiling $ javac -cp .:json-smart-1.1.1.jar:jsonrpc2-base-1.35.jar:jsonrpc2-client-1.14.3.jar:jsonrpc2-server-1.10.1.jar Benchmark.java Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8 Note: Benchmark.java uses unchecked or unsafe operations. Note: Recompile with -Xlint:unchecked for details. $ Executing $ java -cp .:json-smart-1.1.1.jar:jsonrpc2-base-1.35.jar:jsonrpc2-client-1.14.3.jar:jsonrpc2-server-1.10.1.jar Benchmark Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8 Time(ns) 344672000 <<< the response time to send a command by JSONRPC over HTTP then get the response of it 4.12.1 <<< extracted from the response of "show version" by key "version" {"dynamic":false,"status":"active","name":"default","interfaces":{"Ethernet49":{"privatePromoted":false}}} <<< extracted from the response of "show vlan" by key "1" $

3. Code

Here is a brief explanation of the benchmark program Benchmark.java. It simply sends "show version" and "show vlan" command.

Basic Authentication

Unfortunately, JSONRPC 2.0 library cannot handle Basic Authentication. The library does not accept "http://username@password:192.168.11.111/command-api" style URL. Hence I interpose Authenticator class of Java standard classes to treat Basic Authentication. In the code, there are hardcoded username and password. (It is not the best way, definitely. Better solution needed. Fortunately, it is adaptive for HTTPS to specify the protocol https: simply.)

// Force Basic authorization final String rpcuser ="admin"; // change to your own username final String rpcpassword ="xxxxxxx"; // change to your own password Authenticator.setDefault( new Authenticator() { protected PasswordAuthentication getPasswordAuthentication() { return new PasswordAuthentication (rpcuser, rpcpassword.toCharArray()); } } );

Preparation

Following part looks to prepare the session, but it takes 50ms and it does not send any packet. (I didn't count this delay as a response time of eAPI in the experiment, of course.)

I cannot understand why it takes such long time. The address of the target is also hardcoded into here. When you change the protocol part of the URL to https, the communication will be done by HTTPS.

// The JSON-RPC 2.0 server URL URL serverURL = null; try { serverURL = new URL("http://192.168.11.111/command-api"); // change to your own address } catch (MalformedURLException e) { e.printStackTrace(); // need better information } // Create new JSON-RPC 2.0 client session JSONRPC2Session mySession = new JSONRPC2Session(serverURL);

Making JSONRPC Data

Following part makes JSONRPC formed data. EOS CLI commands will be embedded in it. The sample code puts "show version" and "show vlan" command there.

// Construct new request String method = "runCmds"; Map cmdParam = new HashMap(); cmdParam.put("version", 1); List cmds = new LinkedList(); cmds.add("show version"); // 1st request cmds.add("show vlan"); // 2nd request cmdParam.put("cmds", cmds); cmdParam.put("format", "json"); int requestID = 0; JSONRPC2Request request = new JSONRPC2Request(method, cmdParam, requestID);

As a result, the following JSON data will be formed.

{"id":0,"method":"runCmds","params":{"cmds":["show version","show vlan"],"format":"json","version":1},"jsonrpc":"2.0"}

The document of JSONRPC2Request has the example to build a JSON formatted string by yourself. (But it needs to be parsed even it is the complete JSON string, then re-formatted JSON string will be sent, probably.)

Creating HTTP Connection and Sending Request

The TCP connection request will be sent by calling send() as follows. Then the result will be stored to the variable response. I have measured the time of this process as the response time of eAPI.

At this time, the Basic Authentication process starts internally. It sends POST request without any Authorization header at first, then it would be refused as 401 authorization fail, after that, it re-sends the same POST request with Authorization header automatically.

long t1 = System.nanoTime(); // timer set JSONRPC2Response response = null; try { response = mySession.send(request); ... long t2 = System.nanoTime(); // timer set

Following HTTP data would be exchanged.

The request POST /command-api HTTP/1.1 Accept-Charset: UTF-8 Content-Type: application/json User-Agent: C/YasuTest Host: 192.168.11.111:80 Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2 Connection: keep-alive Content-Length: 118 Authorization: Basic WjdWu7DwjD2xhw9Di {"id":0,"method":"runCmds","params":{"cmds":["show version","show vlan"],"format":"json","version":1},"jsonrpc":"2.0"} The response HTTP/1.0 200 OK Server: BaseHTTP/0.3 Python/2.7 Date: Fri, 21 Jun 2013 09:58:51 GMT Cache-control: no-store Cache-control: no-cache Cache-control: must-revalidate Cache-control: max-age=0 Cache-control: pre-check=0 Cache-control: post-check=0 Pragma: no-cache Set-Cookie: COMMAND_API_SESSION=0; Path=/ Content-type: application/json Content-length: 542 Connection: close { "jsonrpc": "2.0", "result": [ { "modelName": "DCS-7048T-4S-F", "internalVersion": "4.12.1-1275950.EOS4121", "systemMacAddress": "00:1c:73:0c:0c:0c", "serialNumber": "JSH09494949", "memTotal": 2009472, "bootupTimestamp": 1371631462.6494012, "memFree": 121376, "version": "4.12.1", "architecture": "i386", "internalBuildId": "688851f9-4aac-4f78-81ca-69a8114f172a", "hardwareRevision": "02.02" } { "sourceDetail": "", "vlans": { "1": { "status": "active", "name": "default", "interfaces": { "Ethernet49": { "privatePromoted": false } }, "dynamic": false }, ..... (snip) .... ], "id": 0 }

Extracting the Element from the Message

When it answers the correct response, the variable result has the list of the command responses. The code to extract them out is as follows. Check up the following code and the above HTTP conversation.

if (response.indicatesSuccess()) { JSONArray resp = (JSONArray)response.getResult(); JSONObject obj = (JSONObject)resp.get(0); System.out.println(obj.get("version")); obj = (JSONObject)resp.get(1); JSONObject vlans = (JSONObject)obj.get("vlans"); JSONObject vlan = (JSONObject)vlans.get("1"); System.out.println(vlan); ...

As the result, the following string is provided.

4.12.1 {"dynamic":false,"status":"active","name":"default","interfaces":{"Ethernet49":{"privatePromoted":false}}}

Yutaka Yasuda (yasuda [ at ] ylb.jp)