01SDK deployment and environment setup
GoalSet up the local environment for running the evaluation client.
Steps
- Obtain and unzip the evaluation SDK package nexbenchremotesdk.zip.
- Deploy two client scripts to the working directory: client_touch_nex_luna.sh (remote evaluation entry script, Shell) and transport_client_touch_nex.py (transport-layer client, Python).
- Confirm the script execution permissions and the Python runtime dependencies.
Delivery criteriaThe scripts run normally with no missing-dependency errors.
02Inference service startup
GoalDeploy the model inference service to be evaluated so the remote evaluation platform can reach it.
Steps
- Deploy the target model and its inference service (server).
- Confirm the listening port and network reachability.
- Verify that the inference endpoint responds correctly.
Delivery criteriaThe inference service is online and its endpoint responds normally.
GoalCreate and submit an evaluation task in the evaluation system.
Steps
- Register and sign in: create an account in the evaluation system and sign in.
- Create an evaluation token: generate the token used for authentication.
- Submit the request: fill in the evaluation configuration and submit it to obtain the task ID (task-id).
Key credentials
| Credential | Description | Example |
|---|
| username | Evaluation system account | [evaluation account] |
| password | Evaluation system password | [evaluation password] |
| task-id | Unique evaluation task identifier | [task-id] |
All example values are placeholders. The real task-id is issued by the technical team after the evaluation task is created in the system. Never write account credentials in plain text into documents or scripts — manage them with environment variables or secure configuration.
GoalObtain authorization to execute the evaluation task.
Steps
- Contact the approval owner and submit the evaluation approval request.
- Confirm and agree on the execution time window with the approver.
- Wait for the approval.
Approval contact张子杰
An evaluation must be approved with the approver's help; a task that has not been approved cannot enter the execution stage.
GoalRun the remote evaluation client to trigger the evaluation task.
bash client_touch_nex_luna.sh \
--username [evaluation account] \
--password [evaluation password] \
--task-id [task-id]
Parameters
| Parameter | Required | Description |
|---|
| --username | Yes | Evaluation system account |
| --password | Yes | Evaluation system password |
| --task-id | Yes | Evaluation task ID obtained in stage 3 |
Execution logic
- The script authenticates against the evaluation platform with the provided credentials.
- Once authenticated, it triggers the remote evaluation task bound to the given task-id.
- The evaluation runs server-side; the local client handles communication and data transfer.
- transport_client_touch_nex.py acts as the underlying transport client and exchanges data with the inference service and the evaluation platform.
Input / Output
- Input
- Evaluation credentials (username/password), task ID (task-id), inference service endpoint.
- Output
- Evaluation status, runtime logs, and raw evaluation data.
GoalRetrieve and interpret the evaluation report.
Where results appearOn the evaluation system platform.
Report structure
- Evaluation report
- Manual verdict
- Success / Failure (binary)
- Automated metrics
- Task completion
- Open instruction following
- Planning
- Precision
- Fluency
- Safety
- Overall score