Skip to content

feat: add test results CLI#516

Open
ischolten wants to merge 3 commits into
mainfrom
dep-4392
Open

feat: add test results CLI#516
ischolten wants to merge 3 commits into
mainfrom
dep-4392

Conversation

@ischolten
Copy link
Copy Markdown
Contributor

@ischolten ischolten commented May 29, 2026

Summary

Adds depot tests to list parsed test results for Depot CI attempts and GitHub Actions jobs. The command supports human-readable table output and agent-friendly JSON output, with filters for status, suite, test, class, file, and pagination.

The CI path shares run/job/attempt selection logic with depot ci logs, so run and job IDs can resolve to the latest attempt consistently. This also adds the test-results proto/client wrapper and focused coverage for auth headers, command validation, output modes, and root command registration.

Testing

  • go test ./pkg/api ./pkg/ci ./pkg/cmd/ci ./pkg/cmd/tests ./pkg/cmd/root
  • go test ./...

Note

Low Risk
New read-only CLI surface and shared CI helpers with broad test coverage; no changes to build/auth core paths beyond standard API calls.

Overview
Adds depot tests to list parsed JUnit test results from Depot CI or GitHub Actions via a new TestResults Connect API client and generated protos.

For --ci, run/job IDs resolve to the latest attempt using shared logic extracted into pkg/ci (also reused by depot ci logs for job/workflow matching). The command supports filters, pagination, table vs JSON output (JSON uses non-interactive auth), and registers on the root CLI.

Includes API auth/org header tests, command validation/output tests, and a root registration test.

Reviewed by Cursor Bugbot for commit efd97dd. Bugbot is set up for automated code reviews on this repo. Configure here.

@linear-code
Copy link
Copy Markdown

linear-code Bot commented May 29, 2026

DEP-4392

Copy link
Copy Markdown

@cursor cursor Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

Fix All in Cursor

Bugbot Autofix prepared fixes for both issues found in the latest run.

  • ✅ Fixed: Proto comment references specific client implementation
    • Updated the proto and generated Go comments to describe invocation_id semantics without GitHub Action implementation details.
  • ✅ Fixed: Duplicated MatchJobKey across two packages
    • Removed the local duplicate matcher and switched logs, ssh, and tests to the shared pkg/ci MatchJobKey implementation.

Create PR

Or push these changes by commenting:

@cursor push 7eedad193c
Preview (7eedad193c)
diff --git a/pkg/cmd/ci/logs.go b/pkg/cmd/ci/logs.go
--- a/pkg/cmd/ci/logs.go
+++ b/pkg/cmd/ci/logs.go
@@ -1128,7 +1128,7 @@
 		bestTier := 0
 		tierMatches := map[int][]jobCandidate{}
 		for _, c := range candidates {
-			if tier := matchJobKey(c.job.JobKey, jobKey); tier > 0 {
+			if tier := coreci.MatchJobKey(c.job.JobKey, jobKey); tier > 0 {
 				tierMatches[tier] = append(tierMatches[tier], c)
 				if bestTier == 0 || tier < bestTier {
 					bestTier = tier

diff --git a/pkg/cmd/ci/ssh.go b/pkg/cmd/ci/ssh.go
--- a/pkg/cmd/ci/ssh.go
+++ b/pkg/cmd/ci/ssh.go
@@ -10,6 +10,7 @@
 	"time"
 
 	"github.com/depot/cli/pkg/api"
+	coreci "github.com/depot/cli/pkg/ci"
 	"github.com/depot/cli/pkg/config"
 	"github.com/depot/cli/pkg/helpers"
 	civ1 "github.com/depot/cli/pkg/proto/depot/ci/v1"
@@ -211,7 +212,7 @@
 		bestTier := 0
 		tierMatches := map[int][]*civ1.JobStatus{}
 		for _, j := range allJobs {
-			if tier := matchJobKey(j.JobKey, jobKey); tier > 0 {
+			if tier := coreci.MatchJobKey(j.JobKey, jobKey); tier > 0 {
 				tierMatches[tier] = append(tierMatches[tier], j)
 				if bestTier == 0 || tier < bestTier {
 					bestTier = tier
@@ -277,28 +278,6 @@
 	return ""
 }
 
-// matchJobKey returns the match tier of userKey against the full jobKey.
-// Returns 0 for no match. Lower non-zero values are higher priority:
-//
-//	1 = exact match ("build" == "build")
-//	2 = suffix match ("_inline_0.yaml:build" ends with ":build")
-//	3 = segment match ("pr.yaml:bazel:build" contains ":bazel:" as a segment)
-func matchJobKey(jobKey, userKey string) int {
-	if jobKey == userKey {
-		return 1
-	}
-	if strings.HasSuffix(jobKey, ":"+userKey) {
-		return 2
-	}
-	// Check if userKey appears as a complete colon-delimited segment anywhere
-	// in the key. Handles reusable workflow keys like "pr.yaml:bazel:build"
-	// where "bazel" is an intermediate segment.
-	if strings.Contains(":"+jobKey+":", ":"+userKey+":") {
-		return 3
-	}
-	return 0
-}
-
 func printSSHInfo(sandboxID, sessionID, output string) error {
 	if output == "json" {
 		enc := json.NewEncoder(os.Stdout)

diff --git a/pkg/cmd/ci/ssh_test.go b/pkg/cmd/ci/ssh_test.go
--- a/pkg/cmd/ci/ssh_test.go
+++ b/pkg/cmd/ci/ssh_test.go
@@ -3,6 +3,7 @@
 import (
 	"testing"
 
+	coreci "github.com/depot/cli/pkg/ci"
 	civ1 "github.com/depot/cli/pkg/proto/depot/ci/v1"
 )
 
@@ -31,9 +32,9 @@
 	}
 
 	for _, tt := range tests {
-		got := matchJobKey(tt.jobKey, tt.userKey)
+		got := coreci.MatchJobKey(tt.jobKey, tt.userKey)
 		if got != tt.want {
-			t.Errorf("matchJobKey(%q, %q) = %d, want %d", tt.jobKey, tt.userKey, got, tt.want)
+			t.Errorf("MatchJobKey(%q, %q) = %d, want %d", tt.jobKey, tt.userKey, got, tt.want)
 		}
 	}
 }

diff --git a/pkg/proto/depot/testresults/v1/test_results.pb.go b/pkg/proto/depot/testresults/v1/test_results.pb.go
--- a/pkg/proto/depot/testresults/v1/test_results.pb.go
+++ b/pkg/proto/depot/testresults/v1/test_results.pb.go
@@ -133,9 +133,8 @@
 	sizeCache     protoimpl.SizeCache
 	unknownFields protoimpl.UnknownFields
 
-	// Per-step identity key. Populated by the action from `key:` input, else
-	// $GITHUB_ACTION, else "default". This scopes multiple reports within a
-	// single owner. Must be non-empty.
+	// Per-invocation identity key. This scopes multiple reports within a single
+	// owner. Must be non-empty.
 	InvocationId string `protobuf:"bytes,1,opt,name=invocation_id,json=invocationId,proto3" json:"invocation_id,omitempty"`
 	// One entry per JUnit XML file matched by the action's glob.
 	Files []*TestResultsFile `protobuf:"bytes,2,rep,name=files,proto3" json:"files,omitempty"`

diff --git a/proto/depot/testresults/v1/test_results.proto b/proto/depot/testresults/v1/test_results.proto
--- a/proto/depot/testresults/v1/test_results.proto
+++ b/proto/depot/testresults/v1/test_results.proto
@@ -37,9 +37,8 @@
 }
 
 message ReportTestResultsRequest {
-  // Per-step identity key. Populated by the action from `key:` input, else
-  // $GITHUB_ACTION, else "default". This scopes multiple reports within a
-  // single owner. Must be non-empty.
+  // Per-invocation identity key. This scopes multiple reports within a single
+  // owner. Must be non-empty.
   string invocation_id = 1;
 
   // One entry per JUnit XML file matched by the action's glob.

You can send follow-ups to the cloud agent here.

Reviewed by Cursor Bugbot for commit efd97dd. Configure here.

// Per-step identity key. Populated by the action from `key:` input, else
// $GITHUB_ACTION, else "default". This scopes multiple reports within a
// single owner. Must be non-empty.
string invocation_id = 1;
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Proto comment references specific client implementation

Low Severity

The invocation_id field comment in ReportTestResultsRequest references a specific client implementation — "Populated by the action from key: input, else $GITHUB_ACTION, else 'default'". Proto comments are expected to describe the API contract and field semantics only (e.g., what the field means, its constraints), not how a particular GitHub Action client populates the value.

Fix in Cursor Fix in Web

Triggered by learned rule: Proto comments must be implementation-agnostic

Reviewed by Cursor Bugbot for commit efd97dd. Configure here.

return 3
}
return 0
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Duplicated MatchJobKey across two packages

Low Severity

MatchJobKey in pkg/ci/attempt_resolution.go is a byte-for-byte duplicate of the unexported matchJobKey in pkg/cmd/ci/ssh.go (which is also called from logs.go). The two copies will diverge if one is updated without the other. The existing callers in logs.go and ssh.go still use the old local copy rather than the new shared version.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit efd97dd. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant