Hi HN!I built *pyproc* to let Go services call Python like a local function — *no CGO and no separate microservice*. It runs a pool of Python worker processes and talks over *Unix Domain Sockets* on the same host/pod, so you get low overhead, process isolation, and parallelism beyond the GIL.
*Why this exists*
* Keep your Go service, reuse Python/NumPy/pandas/PyTorch/scikit-learn.
* Avoid network hops, service discovery, and ops burden of a separate Python service.
*Quick try (~5 minutes)*
Go (app):
“`
go get github.com/YuminosukeSato/pyproc@latest
“`
Python (worker):
“`
pip install pyproc-worker
“`
Minimal worker (Python):
“`
from pyproc_worker import expose, run_worker
@expose
def predict(req):
return {“result”: req[“value”] * 2}
if __name__ == “__main__”:
run_worker()
“`
Call from Go:
“`
import (
“context”
“fmt”
“github.com/YuminosukeSato/pyproc/pkg/pyproc”
)
func main() {
pool, _ := pyproc.NewPool(pyproc.PoolOptions{
Config: pyproc.PoolConfig{Workers: 4, MaxInFlight: 10},
WorkerConfig: pyproc.WorkerConfig{SocketPath: “/tmp/pyproc.sock”, PythonExec: “python3”, WorkerScript: “worker.py”},
}, nil)
_ = pool.Start(context.Background())
defer pool.Shutdown(context.Background())
var out map[string]any
_ = pool.Call(context.Background(), “predict”, map[string]any{“value”: 42}, &out)
fmt.Println(out[“result”]) // 84
}
“`
*Scope / limits*
* Same-host/pod only (UDS). Linux/macOS supported; Windows named pipes not yet.
* Best for request/response payloads ≲ ~100 KB JSON; GPU orchestration and cross-host serving are out of scope.
*Benchmarks (indicative)*
* Local M1, simple JSON: ~*45µs p50* and *~200k req/s* with 8 workers. Your numbers will vary.
*What’s included*
* Pure Go client (no CGO), Python worker lib, pool, health checks, graceful restarts, and examples.
*Docs & code*
* README, design/ops/security docs, pkg.go.dev: [https://github.com/YuminosukeSato/pyproc](https://github.com/YuminosukeSato/pyproc)
*License*
* Apache-2.0. Current release: v0.2.x.
*Feedback welcome*
* API ergonomics, failure modes under load, and priorities for codecs/transports (e.g., Arrow IPC, gRPC-over-UDS).
—
Source for details: project README and docs. ([github.com][1])
[1]: https://github.com/YuminosukeSato/pyproc “GitHub – YuminosukeSato/pyproc: Call Python from Go without CGO or microservices – Unix domain socket based IPC for ML inference and data processin”
Comments URL: https://news.ycombinator.com/item?id=45257929
Points: 1
# Comments: 0
Source: github.com
