| Devon Loehr | 3d6770f | 2024-08-19 15:33:59 | [diff] [blame] | 1 | This folder contains scripts for discovering and cataloguing instances of |
| 2 | compiler warnings in Chromium builds. |
| 3 | |
| 4 | ### Background |
| 5 | There are currently many clang warnings which we suppress when building |
| 6 | Chromium, but intend to one day re-enable (try searching for `-Wno` in |
| 7 | [build/config/compiler.BUILD.gn](https://source.chromium.org/chromium/chromium/src/+/main:build/config/compiler/BUILD.gn) |
| 8 | for some examples). Unfortunately, actually turning these warnings back on is a |
| 9 | daunting task due to the sheer variety of platforms on which chromium needs to |
| 10 | build. Each of these compiles a different subset of the codebase, meaning that |
| 11 | certain warnings may only show up when building on a single, obscure system. |
| 12 | |
| 13 | The Chromium CQ lets us check for this by building on many different systems |
| 14 | simultaneously. If we disable the `-Werror` flag, the build logs for each system |
| 15 | will contain a list of all the warnings that occurred during the build. However, |
| 16 | the CQ is quite large. It's not feasible to manually inspect the logs for every |
| 17 | single build to ensure all possible warning sites are caught. |
| 18 | |
| 19 | The scripts in this directory serve to automate that process. `pull_logs.py` |
| 20 | uses the bitbucket cli tool `bb` (which must be in the PATH) to pull the build |
| 21 | logs for all trybots from a given CQ run, then `collect_warnings.py` can process |
| 22 | them to determine all the places in the codebase where a warning occurs. With |
| 23 | that information, it's feasible to begin fixing the occurrences. |
| 24 | |
| 25 | ### Intended use |
| 26 | The scripts are flexible, but they are intended for use as part of the following |
| 27 | workflow: |
| 28 | 1. Enable the warning of interest by commenting out the relevant line in the |
| 29 | BUILD file. |
| 30 | 2. Disable `-Werror` by setting `treat_warnings_as_errors` to `false` in |
| 31 | [build/config/compiler/compiler.gni](https://source.chromium.org/chromium/chromium/src/+/main:build/config/compiler/compiler.gni;l=50;drc=e1c0ef1369e527d7027782d1285df483e29a200a). |
| 32 | 3. Create a CL with the above changes, and start a CQ Dry Run. |
| 33 | 4. When the dry run finishes, run `pull_logs.py` with the CL and patchset number |
| 34 | the CQ ran on. |
| 35 | 5. When the logs have been downloaded, run `collect_warnings.py` with the |
| 36 | warning flag you're interested in. |
| 37 | |
| 38 | ### Notable script options |
| 39 | The section describes particularly notable behavior. See the scripts themselves |
| 40 | for a full list of options. Almost all options have an equivalent single-letter |
| 41 | form. |
| 42 | |
| 43 | `pull_logs.py` |
| 44 | * The script pulls the build logs for a single trybot step (by default, the |
| 45 | compilation step). The same step can have different names on different builders, |
| 46 | so the `--step` argument may be specified multiple times. If so, the scripts |
| 47 | tries to pull logs from each step name in order until one succeeds. |
| 48 | * By default, the logs are written to a temporary folder whose name is printed |
| 49 | to stdout. You can specify the destination with the `--log-dir` argument. |
| 50 | * The resulting build logs can be extremely large, but most of the lines are |
| 51 | uninteresting boilerplate. The `--filter` flag may be used to automatically |
| 52 | remove uninteresting lines before saving the log to disk. The definition of |
| 53 | "interesting" is controlled by a lambda defined in the script file; by default, |
| 54 | all lines beginning with `[` are pruned. Feel free to edit the lambda locally |
| 55 | if you want a different filter. |
| 56 | * If the `--delete-logs` flag is passed, all `.txt` files in the log |
| 57 | directory will be deleted before the new ones are downloaded. Not using this |
| 58 | argument can result in having stale files in the log directory, preventing |
| 59 | accurate collection of warnings. The argument is disabled by default to prevent |
| 60 | unintentional deletion of data. |
| 61 | |
| 62 | `collect_warnings.py` |
| 63 | * The `--warning` flag is required to determine which lines correspond to the |
| 64 | warning of interest. The value of the flag should be the ending text on the line |
| 65 | where the compiler announces the warning, e.g. |
| 66 | `--warning [-Wthread-safety-reference-return]`. |
| 67 | A convenient shortcut to collect all warnings in the file (frequently there will |
| 68 | only be one type of warning anyway) is to simply pass `--warning ]`. |
| 69 | * By default, the collected info is written to a temporary file whose name is |
| 70 | printed to stdout. You can specify your own output file using the `--output` |
| 71 | argument. If the value is `stdout` or `-`, the text will be printed to |
| 72 | stdout instead of a file. |
| 73 | * The scripts has two "modes": by default, it outputs a `.json` file with |
| 74 | detailed information about each warning emitted. If the `--summarize` flag is |
| 75 | passed, it will instead output a more human-readable `.txt` file which simply |
| 76 | lists every warning site, along with a count of number of files and warnings |
| 77 | emitted total. |
| 78 | * The detailed information consists of a list of warnings for each source file. |
| 79 | Each warning entry contains the line and character number, as well as the line |
| 80 | of code which triggered the warning. Finally, it contains a list of builders on |
| 81 | which the warning occurred. |
| 82 | |
| 83 | ### Example invocations |
| 84 | Pull logs for cl 1234567, patchset 8. Logs will be stored in the default |
| 85 | location |
| 86 | (`tools/warning_analysis/build_logs`). |
| 87 | |
| 88 | `python3 tools/warning_analysis/pull_logs.py -cl 12345678 -p 8` |
| 89 | |
| 90 | Pull logs for cl 1234567, patchset 8. Delete any old logs, filter out |
| 91 | uninteresting lines from the new logs, and print progress to the console as the |
| 92 | script is running. |
| 93 | |
| 94 | `python3 tools/warning_analysis/pull_logs.py -cl 12345678 -p 8 -d --filter -v` |
| 95 | |
| 96 | Collect instances of clang's C++ extension warning from the default |
| 97 | log location, and store a summary in `thread_safety.txt`. |
| 98 | |
| 99 | `python3 tools/warning_analysis/collect_warnings.py -w [-Wvla-cxx-extension] -o cxx_extension --summarize` |
| 100 | |
| 101 | Collect instances of any warning, and store the detailed output in `out.json`. |
| 102 | |
| 103 | `python3 tools/warning_analysis/collect_warnings.py -w ]` |
