Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 23 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,36 +1,39 @@
# Weir public protocol

Shared public schemas, generated Go types, validation, and owned DNS helpers for
Weir servers and clients. Requires Go 1.27.1.
Shared public schemas, generated Go types and validation for Weir servers and
clients. Requires Go 1.27.1.

```sh
go get github.com/batchstream/weir-protocol@v0.14.0
go get github.com/batchstream/weir-protocol@v0.15.0
```

The dependency direction is one-way: Weir and the Go SDK depend on this module.
This module depends on neither repository, including through test dependencies.
It contains no peer discovery protocol, server lifecycle, backend adapter, or SDK.
It contains no server lifecycle, backend adapter, or SDK.

- `api/weir/v1`: unary `weir.v1.StoreService.ResolveStore` and bidirectional
`Execute` for Read, Mutate, and Native.
- `api/protocol`: common envelope, endpoint, resource, record, and Native validation.
- `api/netlimit`: standard Go DNS transport with joined cancellation.
- `api/weir/v1`: the bidirectional `weir.v1.StoreService.Execute` RPC for Read,
Mutate, and Native. Clients send it to a Weir server that hosts the Store; any
other server fails the RPC with NOT_FOUND.
- `api/protocol`: common envelope, Store name, resource, record, and Native
validation.

The public schema lives in `api/weir/v1/store.proto`. Servers and SDKs consume
the module's generated Go types.

## Streaming semantics

Execute selects one Store and operation kind for its lifetime. Each Read or Mutate
request carries one record. Servers aggregate admitted records into database
batches; clients send and consume incrementally without collecting a wire batch.
There is no Weir-imposed item or payload-size limit on business calls or individual
records. Backend protocols govern accepted documents and commands; gRPC carries
one request or event per message. Native carries a required opaque Document.
NativeHead carries optional opaque metadata and a content type for subsequent
body chunks. Request and metadata bytes are opaque to the public protocol. Their content types select adapter-owned
formats; new adapters need no protobuf fields or schema changes. Empty request
data is permitted. Unknown valid content types are left to the selected adapter.
Execute selects one Store and operation kind for its lifetime. Each Read or
Mutate request carries one record. Servers aggregate admitted records into
database batches; clients send and consume incrementally without collecting a
wire batch. The protocol imposes no item or payload-size limit on business calls
or individual records; servers may bound the size of one message. Backend
protocols govern accepted documents and commands; gRPC carries one request or
event per message. Native carries a required opaque Document. NativeHead carries
optional opaque metadata and a content type for subsequent body chunks. Request
and metadata bytes are opaque to the public protocol. Their content types select
adapter-owned formats; new adapters need no protobuf fields or schema changes.
Empty request data is permitted. Unknown valid content types are left to the
selected adapter.

Each request is validated before execution. Earlier requests may have effects
when a later request fails validation. The first record index is 1; every later
Expand Down Expand Up @@ -142,8 +145,8 @@ Prepare the complete module cache in a temporary module copy before disabling
network access. `download all` includes graph/test metadata that lazy runtime
downloads omit; its extra ZIP checksums stay outside the committed `go.sum`.

Default tests use task-owned loopback DNS and in-memory protocol data. They do not
contact external endpoints, start databases, or change host DNS. Descriptor tests
Default tests use in-memory protocol data. They do not contact external
endpoints or start databases. Descriptor tests
protect the public RPC and field contract.
The dependency check examines every selected module, including transitive and
test requirements, rejects Server/SDK dependencies and replacements, examines
Expand Down
125 changes: 0 additions & 125 deletions api/netlimit/dns.go

This file was deleted.

27 changes: 0 additions & 27 deletions api/netlimit/dns_test.go

This file was deleted.

73 changes: 0 additions & 73 deletions api/protocol/endpoints.go

This file was deleted.

61 changes: 0 additions & 61 deletions api/protocol/endpoints_test.go

This file was deleted.

6 changes: 6 additions & 0 deletions api/protocol/protocol.go
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,12 @@ const MaxTransportMessageBytes = int(^uint(0) >> 1)
var storePattern = regexp.MustCompile(`^[a-z](?:[a-z0-9]|-[a-z0-9])*$`)
var mediaPattern = regexp.MustCompile(`^[a-z0-9!#$&^_.+-]+/[a-z0-9!#$&^_.+-]+$`)

// ValidStoreName reports whether name is a well-formed Store name: a lowercase
// letter followed by lowercase letters, digits and single internal hyphens.
func ValidStoreName(name string) bool {
return storePattern.MatchString(name)
}

// ParseRelativeResource validates a canonical Store-relative path and returns its
// decoded segments. Store identity is supplied separately by the request envelope.
func ParseRelativeResource(resource string) ([]string, error) {
Expand Down
13 changes: 13 additions & 0 deletions api/protocol/protocol_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,19 @@ import (
"google.golang.org/protobuf/reflect/protoreflect"
)

func TestStoreNames(t *testing.T) {
for _, name := range []string{"data", "mongo-1", strings.Repeat("a", 128)} {
if !ValidStoreName(name) {
t.Errorf("valid Store name rejected: %q", name)
}
}
for _, name := range []string{"", "Data", "1data", "a--b", "data/path", "data?filter", "data:7447", "a-", "weir://data"} {
if ValidStoreName(name) {
t.Errorf("invalid Store name accepted: %q", name)
}
}
}

func TestCanonicalRelativeResource(t *testing.T) {
valid := []string{"db/c/s:a%2Fb", "a/s:%E4%B8%AD", "a/i:-3", "records"}
for _, resource := range valid {
Expand Down
Loading