Skip to content

Fix flaky tests for TestQueryLog #16930

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 10 commits into
base: main
Choose a base branch
from
Open

Fix flaky tests for TestQueryLog #16930

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
c2b5334
Mitigate port allocation race conditions in tests
kapillamba4 Jul 27, 2025
0b2663d
Revert "Mitigate port allocation race conditions in tests"
kapillamba4 Jul 27, 2025
f8d3fd6
add go-test in ci.yml
kapillamba4 Jul 27, 2025
39707ad
update ci.yml
kapillamba4 Jul 27, 2025
d87ab63
update ci.yml
kapillamba4 Jul 27, 2025
d887be2
rebuild
kapillamba4 Jul 27, 2025
16fdba1
rebuild
kapillamba4 Jul 27, 2025
5503dc5
rebuild
kapillamba4 Jul 27, 2025
a44a87e
update ci.yml
kapillamba4 Jul 27, 2025
e5d0180
update ci.yml
kapillamba4 Jul 27, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
61 changes: 61 additions & 0 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,67 @@
- run: make -C documentation/examples/remote_storage
- run: make -C documentation/examples

go-test:
name: Go tests - ${{ matrix.config.name }} - inst${{ matrix.instance }}
runs-on: ubuntu-latest
continue-on-error: true
strategy:
fail-fast: false
matrix:
config:
- name: "long-timeout"
tcp_fin_timeout: 1200
tcp_tw_reuse: 0
port_range: "60000 60999"
- name: "short-timeout"
tcp_fin_timeout: 0
tcp_tw_reuse: 0
port_range: "32768 60999"
instance: [1, 2, 3]
container:
image: quay.io/prometheus/golang-builder:1.24-base
options: >-
--sysctl net.ipv4.tcp_fin_timeout=${{ matrix.config.tcp_fin_timeout }}
--sysctl net.ipv4.tcp_tw_reuse=${{ matrix.config.tcp_tw_reuse }}
--sysctl net.ipv4.ip_local_port_range="${{ matrix.config.port_range }}"
--privileged
steps:
- uses: actions/checkout@v4

Check warning

Code scanning / Scorecard

Pinned-Dependencies Medium

score is 7: GitHub-owned GitHubAction not pinned by hash
Click Remediation section below to solve this issue

- name: Cache Go modules
uses: actions/cache@v4

Check warning

Code scanning / Scorecard

Pinned-Dependencies Medium

score is 7: GitHub-owned GitHubAction not pinned by hash
Click Remediation section below to solve this issue
with:
path: |
~/.cache/go-build
/go/pkg/mod
key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-

- name: Download Go modules
run: go mod download

- name: Run Go tests - ${{ matrix.config.name }} - inst${{ matrix.instance }}
shell: bash
run: |
start_time=$(date +%s)
log_file="/tmp/test_output_${{ matrix.config.name }}_inst${{ matrix.instance }}.log"
touch "$log_file"
echo "Writing test logs to $log_file"
echo "TCP settings applied via Docker options:"
echo " FIN timeout: ${{ matrix.config.tcp_fin_timeout }}"
echo " TW reuse: ${{ matrix.config.tcp_tw_reuse }}"
echo " Port range: ${{ matrix.config.port_range }}"

for i in {1..10}; do
echo "Run #$i with ${{ matrix.config.name }}" | tee -a "$log_file"
go test --tags=dedupelabels -failfast -count=1 ./cmd/prometheus/... 2>&1 | tee -a "$log_file"
done

end_time=$(date +%s)
duration=$((end_time - start_time))
echo "Total test duration for ${{ matrix.config.name }}: ${duration} seconds" | tee -a "$log_file"

test_go_more:
name: More Go tests
runs-on: ubuntu-latest
Expand Down
9 changes: 9 additions & 0 deletions util/testutil/port.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,15 @@ func portWasUsed(port int) bool {
}

func getPort() (int, error) {
// lc := net.ListenConfig{
// Control: func(network, address string, c syscall.RawConn) error {
// return c.Control(func(fd uintptr) {
// syscall.SetsockoptInt(int(fd), syscall.SOL_SOCKET, syscall.SO_REUSEADDR, 1)
// })
// },
// }

// listener, err := lc.Listen(context.Background(), "tcp", ":0")
listener, err := net.Listen("tcp", ":0")
if err != nil {
return 0, err
Expand Down
Loading