GH-49435: [CI][Python] Fix macOS protobuf@33 keg-only discovery

[tadeja]

Rationale for this change

Homebrew upgrading protobuf beyond v33 (as described in #49435) also causes new runtime Bus error.

What changes are included in this PR?

a) Pin protobuf@33 in cpp/Brewfile (as discussed in #49436)
b) Add keg-only env setup in python.yml so CMake/pkg-config can find protobuf@33

This resolves Fatal Python error: Bus error on macOS 15 Intel when importing pyarrow.flight. Homebrew's latest protobuf pulls in Abseil 20260107
-- Found abslAlt: 1 (found version "20260107")
which is ABI-incompatible with Arrow's bundled gRPC v1.46.3

One of earliest Python CI logs with the Bus error:
https://github.com/apache/arrow/actions/runs/22722817262/job/65889451081#step:10:103

Note: Other workflows using cpp/Brewfile will prob need similar keg-only setup. (There's failing C++ job log here, Ruby etc.)

Are these changes tested?

Yes, fork workflow completed the Python macOS 15-intel job with success.

Are there any user-facing changes?

No.

• GitHub Issue: [C++][FlightRPC] Flight fails building with protobuf v34.0+ due to functions updated with nodiscard #49435

[tadeja]

AMD64 macOS 15-intel Python 3 and ARM64 macOS 14 Python 3 both successfully completed with the protobuf@33 change.

Now attempting similar change for cpp.yml and ruby.yml.

[tadeja]

@raulcd, @rok what do you say about this PR? it can be moved into existing draft #49436 if you'd prefer.

• unrelated job 'test_write_dataset_s3_put_only' timeout failure could use a re-run: https://github.com/apache/arrow/actions/runs/22914401035/job/66495531287?pr=49491#step:10:266

[rok]

Will users building for source need to use keg only? Should we update docs?

[kou]

+1

This is a workaround. We should remove @33 eventually after the gRPC problem is fixed.
See also:

• https://github.com/Homebrew/homebrew-core/blob/8b55249c502449f0c5a4c667d52af8eb7cd34010/Formula/g/grpc.rb#L37C30-L37C71
• grpc/grpc#41755

Could you add a comment when we can remove this workaround?

[tadeja]

Ruby job is failing anew (no changes here except for newline)
https://github.com/apache/arrow/actions/runs/22944683842/job/66602842927?pr=49491#step:8:1689
/Users/runner/work/arrow/arrow/build/cpp/src/arrow/flight/Flight.pb.h:16:2: error: "Protobuf C++ gencode is built with an incompatible version of"

I see homebrew merged this last night so grpc changed "protobuf@33" to "protobuf" and is on version 34.0 now;

• Homebrew/homebrew-core@552efca#diff-1bd6970eef0fe40eabd3ee6d2f57b3dfec00fc5579e0a13856c45a0381082150R37
• https://formulae.brew.sh/formula/grpc

As it appears grpc formula now depends on protobuf (v34), grpc_cpp_plugin generates v34 incompatible code that can't compile against protobuf@33 headers

[tadeja]

( Unrelated issue of Windows test segfault is being worked on in #49462. )

[tadeja]

@raulcd what are your thoughts on the next steps here? - CI jobs are passing again today, also for ARM64 macOS 14 GLib & Ruby (pull_request)
This time using solution from your draft PR + protobuf@33 workaround only for cpp.yml and python.yml + reverted env setup for ruby.yml as that doesn't seem to be necessary there anymore I suppose due to brew update last night? (I'll remove unneeded newline with my next commit)

[kou]

Can we use brew update workaround instead of this?
We can remove the brew update once GitHub Actions runner ships new Homebrew by default.