Convert from using Arrow Flight SQL (gRPC) to Websockets

[prmoore77]

While Arrow Flight SQL (gRPC) has opened up a large amount of compatibility - the gRPC framework has some downsides:

1. It does NOT play well with browser-based / javascript clients. We must have a nodeJS back-end server library to have front-end javascript connectivity to GizmoSQL - or use a websocket proxy.
2. It takes a long time to compile - making builds somewhat painful.

We should convert the GizmoSQL server to use modern websockets with Arrow IPC for data transfer - for low-latency connectivity, high throughput, and maximum compatibility with browsers / javascript clients.

This would of course require that we update the client libraries for ADBC, JDBC, and ODBC - but it would be well worth it. We would also need to update the GizmoSQL UI, as well as any other popular clients/connectors (Grafana, dbt, SQLMesh, Metabase, Superset, Tableau, etc.).

Attached is a plan doc.
websocket-migration-plan.md

[TFMV]

Phil — I think you’re zeroing in on a real problem. The friction around gRPC and the lack of clean browser-native access is a legitimate constraint, and a WebSocket + Arrow IPC path is a strong way to simplify that and improve developer experience.

That said, if the goal is to build an enterprise platform, I don’t think we can treat this as a pure replacement decision.

Apache Arrow Flight SQL is doing more than moving bytes; it’s our bridge into the standard data ecosystem via ADBC, JDBC, and ODBC. That interoperability is table stakes in enterprise environments, where tools like Grafana, dbt, and Tableau are not optional, they’re expected to work out of the box.

If we remove Flight entirely, we’re effectively taking on the burden of recreating and maintaining that compatibility surface ourselves. That’s a large, ongoing investment and a potential barrier to adoption.

My suggestion would be:
• Keep Flight as the standards-based interface for ecosystem integration
• Introduce the WebSocket + Arrow IPC layer as a first-class interface for browser and modern clients

This gives us the best of both worlds:
• Immediate unlock for browser-native and JS use cases
• Continued compatibility with the broader enterprise tooling ecosystem
• Space to evolve the WebSocket protocol into something differentiated (e.g., streaming, push, interactive workflows)

If, over time, the WebSocket layer proves it can fully cover those integration needs, we can revisit consolidation with real data. Until then, preserving interoperability while improving developer experience feels like the more durable path.

[TFMV]

I added WebSocket transport to Porter and got it working end-to-end, but under load it’s already showing some instability. It feels like I’m fighting the protocol a bit once throughput and concurrency ramp up.

Rusty suggested looking at QUIC, which makes sense given the transport characteristics. I’m considering it, though I’m not entirely confident yet in handling the lower-level mechanics and edge cases it introduces.