Allow using new CodeQL languages without updating the CodeQL Action #2914

henrymercer · 2025-05-30T17:05:55Z

This PR revives #779, adding a few new features:

Correctly handle language aliases (these didn't exist at the time)
Determine whether or not a language is traced by looking for its tracing-config.lua file — this is what we do in the CLI too
Simplify the parsing logic in config-utils.ts

Commit-by-commit review recommended as one commit has a large diff from renaming the Language enum to KnownLanguage.

Caveats:

We don't attempt to make language handling in the start-proxy Action generic, because this Action can't assume access to the CodeQL CLI, which is the source of truth. However this is probably OK because the default behaviour of this Action, i.e. not to proxy anything, is OK for new languages.

Merge / deployment checklist

Confirm this change is backwards compatible with existing workflows.
Confirm the readme has been updated if necessary.
Confirm the changelog has been updated if necessary.

Co-authored-by: Copilot <[email protected]>

mbg

Thanks for working on this and getting it all up-to-date after the other recent changes! Sorry for the premature review earlier as well. This broadly look good. I have a few suggestions and questions.

Should all the .includes checks for languages get the same treatment as that one check for rust?

src/languages.ts

mbg · 2025-08-05T12:17:49Z

src/start-proxy.ts

+ * In general, the CodeQL CLI is the source of truth for language aliases, and to
+ * allow us to more easily support new languages, we want to avoid hardcoding these
+ * aliases in the Action itself.  However this is difficult to do in the start-proxy
+ * Action since this Action does not use CodeQL, so we're accepting some hardcoding
+ * for this Action.


This doesn't have to be addressed as part of this PR, but we only use the start-proxy in CodeQL (default setup) workflows. The start-proxy steps follows the init step, so we could rely on that here if we wanted to.

I agree with your italicisation :) We could rely on that ordering, but I'd prefer not to.

src/start-proxy.ts

src/config-utils.ts

src/analyze-action.ts

src/autobuild.ts

mbg · 2025-08-05T12:41:04Z

src/init-action.ts

@@ -342,6 +342,31 @@ async function run() {
    }
    core.endGroup();

+    // Set CODEQL_ENABLE_EXPERIMENTAL_FEATURES for rust


How do the changes here interact with #2960?

I don't see any problematic interactions, though there might be a couple of merge conflicts to address.

Copilot

Pull Request Overview

This PR allows the CodeQL Action to work with new CodeQL languages without requiring updates to the Action itself. The changes generalize language handling by querying the CodeQL CLI as the source of truth for supported languages and language aliases, instead of hardcoding them in the Action.

Key Changes

Replaces the hardcoded Language enum with a generic Language type and a KnownLanguage enum for languages requiring special handling
Implements dynamic language detection using CodeQL CLI's betterResolveLanguages() and isTracedLanguage() methods
Simplifies language parsing logic by delegating alias resolution to the CLI

Reviewed Changes

Copilot reviewed 45 out of 67 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
src/languages.ts	Refactors from hardcoded enum to generic type with known language subset
src/config-utils.ts	Implements dynamic language support using CLI as source of truth
src/codeql.ts	Adds methods to determine if languages are traced/scanned
src/start-proxy.ts	Maintains hardcoded aliases for proxy action (CLI not available)
src/util.ts	Adds async utility functions and fixes String() wrapper usage
Multiple test files	Updates all references from Language to KnownLanguage enum
lib/ files	Generated JavaScript code reflecting TypeScript changes

Copilot · 2025-08-05T16:58:30Z

src/util.ts

+  core.exportVariable(EnvVar.FEATURE_MULTI_LANGUAGE, "false");
+  core.exportVariable(EnvVar.FEATURE_SANDWICH, "false");
+  core.exportVariable(EnvVar.FEATURE_SARIF_COMBINE, "true");
+  core.exportVariable(EnvVar.FEATURE_WILL_UPLOAD, "true");


[nitpick] The removal of String() wrapper is good for consistency, but these environment variable values should use boolean strings consistently. Consider using a constant or helper function to ensure consistency across the codebase.

Suggested change

core.exportVariable(EnvVar.FEATURE_WILL_UPLOAD, "true");

core.exportVariable(EnvVar.FEATURE_MULTI_LANGUAGE, BOOLEAN_FALSE);

core.exportVariable(EnvVar.FEATURE_SANDWICH, BOOLEAN_FALSE);

core.exportVariable(EnvVar.FEATURE_SARIF_COMBINE, BOOLEAN_TRUE);

core.exportVariable(EnvVar.FEATURE_WILL_UPLOAD, BOOLEAN_TRUE);

Copilot · 2025-08-05T16:58:30Z

src/upload-lib.ts

@@ -541,7 +541,7 @@ export function validateSarifFileSchema(
    );
  }

-  if (errors.length) {
+  if (errors.length > 0) {


The change from errors.length to errors.length > 0 is more explicit and consistent with the similar change in other files, improving code readability.

Copilot · 2025-08-05T16:58:31Z

src/config-utils.ts

+  return (languagesInput || "")
+    .split(",")
+    .map((x) => x.trim().toLowerCase())
+    .filter((x) => x.length > 0);
 }


This function duplicates logic that was previously inline in getRawLanguages. Consider extracting this to a shared utility function to avoid code duplication and improve maintainability.

Copilot · 2025-08-05T16:58:32Z

src/config-utils.ts

+    }
+  }
+  return supportedLanguages;
+}


This function is called within language detection logic that could be called frequently. Consider caching the result to avoid repeated calls to betterResolveLanguages() within the same execution context.

Copilot · 2025-08-05T16:58:33Z

src/init-action.ts

@@ -240,7 +240,10 @@ async function sendCompletedStatusReport(
      packs: JSON.stringify(packs),
      trap_cache_languages: Object.keys(config.trapCaches).join(","),
      trap_cache_download_size_bytes: Math.round(
-        await getTotalCacheSize(Object.values(config.trapCaches), logger),
+        await getTotalCacheSize(


The filter for undefined values suggests that config.trapCaches may contain undefined entries. Consider updating the type definition or initialization logic to prevent undefined values from being added to trapCaches in the first place.

Note: See the diff below for a potential fix:

@@ -238,9 +238,13 @@ paths_ignore: pathsIgnore, queries: queries.join(","), packs: JSON.stringify(packs), - trap_cache_languages: Object.keys(config.trapCaches).join(","), + // Only include languages for which the trap cache is defined + trap_cache_languages: Object.keys(config.trapCaches) + .filter((lang) => config.trapCaches[lang] !== undefined) + .join(","), trap_cache_download_size_bytes: Math.round( await getTotalCacheSize( + // Only include defined trap cache entries Object.values(config.trapCaches).filter((x) => x !== undefined), logger, ),

mbg

Thanks for addressing the feedback! I think the change of type for the trap caches is good.

mbg · 2025-08-05T17:37:22Z

src/languages.ts

+/** A language supported by CodeQL. */
+export type Language = string;


Minor: Technically this now represents any language identifier, whether supported or not

henrymercer added 4 commits May 30, 2025 17:58

Remove per-language hardcoded traced/scanned info

3b57965

Clarify meaning of Language enum

fa0b6ff

Use CodeQL CLI to parse languages

e303175

Rename Language enum and use generic Language where possible

932be8f

Copilot AI review requested due to automatic review settings May 30, 2025 17:05

henrymercer requested a review from a team as a code owner May 30, 2025 17:05

This comment was marked as outdated.

Sign in to view

henrymercer added 2 commits May 30, 2025 18:09

Rust: Set experimental features variable before language parsing

1d22e83

Fix early return for no autobuild languages

a28197c

Base automatically changed from henrymercer/bump-minimum-codeql-2.16.6 to main June 9, 2025 21:43

mbg mentioned this pull request Jul 2, 2025

Basic support for overlay PR analysis #2945

Merged

3 tasks

Merge branch 'main' into henrymercer/language-extensibility

c481481

henrymercer requested review from Copilot and mbg August 5, 2025 10:07

This comment was marked as outdated.

Sign in to view

Copilot AI review requested due to automatic review settings August 5, 2025 10:13

henrymercer added the Rebuild Re-transpile JS & re-generate workflows label Aug 5, 2025

github-actions bot removed the Rebuild Re-transpile JS & re-generate workflows label Aug 5, 2025

This comment was marked as outdated.

Sign in to view

Remove unnecessary String initializations

cfb8d07

Co-authored-by: Copilot <[email protected]>

henrymercer force-pushed the henrymercer/language-extensibility branch from 4d8db50 to cfb8d07 Compare August 5, 2025 10:19

Improve detection of Rust in languages input

83e92ed

Copilot AI review requested due to automatic review settings August 5, 2025 10:43

This comment was marked as outdated.

Sign in to view

mbg self-requested a review August 5, 2025 12:08

Merge branch 'main' into henrymercer/language-extensibility

bf692c0

henrymercer dismissed mbg’s stale review via bf692c0 August 5, 2025 12:21

mbg reviewed Aug 5, 2025

View reviewed changes

Improve docstrings for language types

fa18cc9

henrymercer added 2 commits August 5, 2025 17:51

Fix docstring in start-proxy

e682065

Use more explicit checks on .length for readability

ea05bf2

Copilot AI review requested due to automatic review settings August 5, 2025 16:57

Copilot AI reviewed Aug 5, 2025

View reviewed changes

Improve type of trapCaches now Language is non-exhaustive

377976a

henrymercer requested a review from mbg August 5, 2025 17:11

mbg previously approved these changes Aug 5, 2025

View reviewed changes

Update Language doc

f30d00f

henrymercer dismissed mbg’s stale review via f30d00f August 6, 2025 08:23

henrymercer requested a review from mbg August 6, 2025 08:24

henrymercer enabled auto-merge August 6, 2025 08:24

mbg approved these changes Aug 6, 2025

View reviewed changes

henrymercer merged commit 60bf7df into main Aug 6, 2025
282 checks passed

henrymercer deleted the henrymercer/language-extensibility branch August 6, 2025 08:38

henrymercer mentioned this pull request Aug 6, 2025

Do not artificially restrict the set of supported languages #779

Closed

3 tasks

-  core.exportVariable(EnvVar.FEATURE_WILL_UPLOAD, "true");
+  core.exportVariable(EnvVar.FEATURE_MULTI_LANGUAGE, BOOLEAN_FALSE);
+  core.exportVariable(EnvVar.FEATURE_SANDWICH, BOOLEAN_FALSE);
+  core.exportVariable(EnvVar.FEATURE_SARIF_COMBINE, BOOLEAN_TRUE);
+  core.exportVariable(EnvVar.FEATURE_WILL_UPLOAD, BOOLEAN_TRUE);

		/** A language supported by CodeQL. */
		export type Language = string;

Allow using new CodeQL languages without updating the CodeQL Action #2914

Allow using new CodeQL languages without updating the CodeQL Action #2914

Uh oh!

Conversation

henrymercer commented May 30, 2025

Merge / deployment checklist

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

This comment was marked as outdated.

Uh oh!

mbg left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mbg Aug 5, 2025

Choose a reason for hiding this comment

Uh oh!

henrymercer Aug 5, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mbg Aug 5, 2025

Choose a reason for hiding this comment

Uh oh!

henrymercer Aug 5, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Key Changes

Reviewed Changes

Uh oh!

Copilot AI Aug 5, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 5, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 5, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 5, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Aug 5, 2025

Choose a reason for hiding this comment

Uh oh!

mbg left a comment

Choose a reason for hiding this comment

Uh oh!

mbg Aug 5, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!