I wish I'd known about the search issue.

I do see a new opportunity for Ballista. By leveraging all of the Spark-compatible operators and expressions being built in Comet, it would be able to support a wider range of queries much more quickly.

Ballista already uses protobuf for sending plans to executors and Comet accepts protobuf plans (in a similar, but different format).