Allow Code Quality only analysis #3064

mbg · 2025-08-28T16:19:02Z

This is a first stab at enabling Code Quality analysis to be able to run on its own, while also maintaining the ability for it to run in addition to a Code Scanning analysis.

Fundamentally, this means that there are two separate ways in which Code Quality analysis can run:

The existing "add-on" implementation, where the database is initialised for Code Scanning, and we inject the code-quality suite into the run-queries call. The Code Quality SARIF is then generated by an extra call to interpret-results.
The new "standalone" implementation, where the database is initialised for Code Quality by disabling default queries and configuring the database with code-quality queries.

43d9bc8 and 5d95d46 implement the bulk of the changes needed for the "add-on" implementation that result from needing to check whether Code Quality is enabled in addition to or instead of Code Scanning.

1746aed implements the bulk of the changes needed for the "standalone" implementation. Concretely, this checks whether Code Quality is enabled on its own, and then mutates the configuration as necessary.

Probably best reviewed commit-by-commit.

Risk assessment

For internal use only. Please select the risk level of this change:

High risk: Changes are not fully under feature flags, have limited visibility and/or cannot be tested outside of production.

Merge / deployment checklist

Confirm this change is backwards compatible with existing workflows.
Consider adding a changelog entry for this change.
Confirm the readme and docs have been updated if necessary.

src/analyze.ts

src/config-utils.ts

henrymercer · 2025-09-01T16:00:38Z

src/config-utils.ts

+      logger.warning(
+        "Query customizations will be ignored, because only `code-quality` analysis is enabled.",
+      );


Should we actually error in these cases? It's easy to miss Actions warnings, and users could think they're analysing something that doesn't actually work. That also avoids us the slight confusion of then having to override config.originalUserInput.

I wasn't sure about this, but decided to go with a warning because in Default Setup we provide inputs to queries and config. We can probably change that there for when Code Quality is enabled, but Code Scanning isn't, but I didn't want to make this an error just yet in case it causes us problems during the transition there.

I appreciate wanting to make the transition smooth, but I have some concerns about modifying config.originalUserInput, particularly since we use this to generate telemetry.

My understanding is today we have analyses with a quality-queries input and without an analysis-kinds input. This means we'll run code scanning, and therefore shouldn't hit this case. Then the transition can move to an explicit analysis-kinds input with the config and other customization inputs removed.

Is there anything I'm missing here?

I think there are two separate points here:

Whether we can swap over from using quality-queries to analysis-kinds and remove the queries input in CQ-only mode in one go. (Probably "yes" and this being a warning is just overly defensive.)

That modifying config.originalUserInput is an issue for telemetry.

Regarding 2., I agree that this isn't a very nice approach. I'll have a look at refactoring this tomorrow so that originalUserInput can remain as-is and we set up the configuration for a CQ-only database differently.

src/upload-lib.ts

henrymercer · 2025-09-01T16:30:22Z

src/config-utils.ts

+ *
+ * @returns Returns `CodeScanning` if `AnalysisKind.CodeScanning` is enabled; otherwise `CodeQuality`.
+ */
+export function getDbAnalysisConfig(config: Config): AnalysisConfig {


This seems like a detail of how saving SARIF results currently works, in particular the fact that when both modes are on, we currently produce a code scanning file, then filter that to produce a code quality file. I don't really think it's a property of the database, since we'll ask the database to analyse both code scanning and code quality results in that situation.

I'd consider removing this in favour of some more explicit logic concerning the SARIF extension in analyze, unless there are more situations where this notion of "primary consumer" matters.

This seems like a detail of how saving SARIF results currently works, in particular the fact that when both modes are on, we currently produce a code scanning file, then filter that to produce a code quality file. I don't really think it's a property of the database, since we'll ask the database to analyse both code scanning and code quality results in that situation.

I don't think this is true, since in CQ-only mode the database is specifically initialised with code-quality queries. What getDbAnalysisConfig tells us is whether the database is initialised with a Code Scanning configuration (in CS only or CS+CQ mode) or a Code Quality configuration (in CQ only mode).

It's true though that this is currently only relevant when we are generating the "primary" SARIF file.

I don't think this is true, since in CQ-only mode the database is specifically initialised with code-quality queries. What getDbAnalysisConfig tells us is whether the database is initialised with a Code Scanning configuration (in CS only or CS+CQ mode) or a Code Quality configuration (in CQ only mode).

I'm unsure how much meaning this has in practice. For instance, if I really wanted to, I could create a code scanning configuration that runs the code quality queries only by disabling the default queries and running a custom query pack. I'd expect the database to be identical to a database with analysis-kinds: code-quality. The only difference I can see is the name of the SARIF file and the upload API.

…bled

…ised with

…queCategory`

Copilot

Pull Request Overview

This PR implements the ability to run Code Quality analysis independently from Code Scanning analysis, while maintaining backwards compatibility for combined analysis. The main purpose is to provide a "standalone" implementation where the database can be initialized specifically for Code Quality analysis by disabling default queries and configuring quality-specific queries.

Key changes include:

Refactored upload targets into a new analyses.ts module with dedicated configurations for both Code Scanning and Code Quality
Added logic to detect when only Code Quality analysis is enabled and configure the database accordingly
Updated upload functions to use the new analysis configuration objects

Reviewed Changes

Copilot reviewed 27 out of 27 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
src/analyses.ts	New module defining analysis configurations and endpoints for Code Scanning and Code Quality
src/config-utils.ts	Enhanced with functions to determine database analysis kind and handle Code Quality-only configurations
src/util.ts	Added `isDefined` utility function for type narrowing
src/upload-lib.ts	Refactored to use new analysis configurations instead of hardcoded upload targets
src/analyze.ts	Updated SARIF file generation logic to handle different analysis configurations
Various test files	Updated test cases to use new analysis configuration objects
Workflow files	Updated to use `analysis-kinds` input instead of deprecated `quality-queries`

src/analyses.ts

…st in the event of future changes

…lts`

…y`-only analysis

github-advanced-security bot found potential problems Aug 28, 2025

View reviewed changes

src/analyze.ts Fixed Show fixed Hide fixed

mbg force-pushed the mbg/cq/allow-cq-only-analysis branch 4 times, most recently from e28ecb0 to 603a572 Compare September 1, 2025 15:17

mbg marked this pull request as ready for review September 1, 2025 15:17

mbg requested a review from a team as a code owner September 1, 2025 15:17

mbg requested review from Copilot and henrymercer September 1, 2025 15:17

This comment was marked as outdated.

Sign in to view

mbg force-pushed the mbg/cq/allow-cq-only-analysis branch from 5b9c46f to a99145b Compare September 1, 2025 15:47

henrymercer reviewed Sep 1, 2025

View reviewed changes

This was referenced Sep 2, 2025

Fix missing/wrong properties in createTestConfig #3071

Merged

Overlay databases: Use Config instead of AugmentationProperties #3072

Merged

Hard-code quality queries #3074

Merged

mbg added 15 commits September 4, 2025 11:38

Update checks to use analysis-kinds instead of quality-queries

434df8e

Add utility function to check if code scanning is enabled

85a4853

Move UploadTarget definitions to analyses.ts

ca7dd4a

Rename UploadTarget to AnalysisConfig

0162708

Add kind property to AnalysisConfig and documentation

cb8f28f

Only specify queries for run-queries if both analysis kinds are ena…

e4ffe6f

…bled

Handle different permutations of analysis kinds in analyze action

ff57bbf

Add note for getPerQueryAlertCounts

6d2d204

Add sarifExtension field to AnalysisConfig

04bb074

Add and use helpers for determining analysis config the DB is initial…

8ea50b8

…ised with

Add test with analysisKindsInput: code-quality

244e04c

Move isDefined from start-proxy to util

811aef8

Override query configuration for Code Quality only analyses

01fe6a1

Fix addSarifExtension and add tests

86275f6

Check if Code Scanning is enabled before uploading Code Scanning SARIF

7baedbc

mbg added 2 commits September 4, 2025 11:44

Update condition and comment for CQ-only config

3ee9287

Remove default arguments from uploadSpecifiedFiles and `validateUni…

51d74ac

…queCategory`

Copilot AI review requested due to automatic review settings September 4, 2025 10:48

mbg force-pushed the mbg/cq/allow-cq-only-analysis branch from f8915bb to d2e5cc8 Compare September 4, 2025 10:49

Do not mutate originalUserInput

f4fca70

Copilot AI reviewed Sep 4, 2025

View reviewed changes

src/analyses.ts Show resolved Hide resolved

mbg added 3 commits September 4, 2025 12:03

Make code that builds the list of queries for run-queries more robu…

ae2df70

…st in the event of future changes

Make conditions for interpret-results more robust

d08e9a2

Add runInterpretResultsFor to de-duplicate code for `interpret-resu…

38f1a70

…lts`

mbg force-pushed the mbg/cq/allow-cq-only-analysis branch from d2e5cc8 to 38f1a70 Compare September 4, 2025 11:26

mbg requested a review from henrymercer September 4, 2025 16:44

mbg added 3 commits September 5, 2025 12:35

Matrix over analysis-kinds in quality-queries check

6d0bcea

Rename getDbAnalysisKind and getDbAnalysisConfig

5d822f1

Throw an error if query customisations are enabled for a `code-qualit…

918e792

…y`-only analysis

mbg force-pushed the mbg/cq/allow-cq-only-analysis branch from 1ff29bf to 7916994 Compare September 5, 2025 13:04

Fix: Include matrix.analysis-kinds in artifact names

e75b5d3

mbg force-pushed the mbg/cq/allow-cq-only-analysis branch from 7916994 to e75b5d3 Compare September 5, 2025 13:27

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Allow Code Quality only analysis #3064

Allow Code Quality only analysis #3064

Uh oh!

mbg commented Aug 28, 2025 •

edited

Loading

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

Uh oh!

henrymercer Sep 1, 2025

Uh oh!

mbg Sep 1, 2025

Uh oh!

henrymercer Sep 1, 2025

Uh oh!

mbg Sep 1, 2025

Uh oh!

Uh oh!

henrymercer Sep 1, 2025

Uh oh!

mbg Sep 1, 2025

Uh oh!

henrymercer Sep 1, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Allow Code Quality only analysis #3064

Are you sure you want to change the base?

Allow Code Quality only analysis #3064

Uh oh!

Conversation

mbg commented Aug 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Risk assessment

Merge / deployment checklist

Uh oh!

Uh oh!

This comment was marked as outdated.

Uh oh!

Uh oh!

henrymercer Sep 1, 2025

Choose a reason for hiding this comment

Uh oh!

mbg Sep 1, 2025

Choose a reason for hiding this comment

Uh oh!

henrymercer Sep 1, 2025

Choose a reason for hiding this comment

Uh oh!

mbg Sep 1, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

henrymercer Sep 1, 2025

Choose a reason for hiding this comment

Uh oh!

mbg Sep 1, 2025

Choose a reason for hiding this comment

Uh oh!

henrymercer Sep 1, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

mbg commented Aug 28, 2025 •

edited

Loading