diff --git a/docs/extend/ai-usage.mdx b/docs/extend/ai-usage.mdx new file mode 100644 index 0000000..cb5db0e --- /dev/null +++ b/docs/extend/ai-usage.mdx @@ -0,0 +1,47 @@ +--- +sidebar_position: 5 +title: AI Usage +description: How artificial intelligence tools are used in the KTestify project. +--- + +# AI Usage in KTestify + +KTestify uses artificial intelligence (AI) tools to support development. This page explains what AI is used for and the safeguards that keep quality high. + +--- + +## What AI is used for + +AI assists with three specific tasks: + +| Task | What AI does | +|---|---| +| **Code suggestions** | AI suggests individual pieces of code, such as small snippets or method implementations. A developer reviews every suggestion before it is accepted. No agentic or autonomous usage is involved. | +| **Documentation** | AI helps generate user-facing documentation, including the pages you are reading now. | +| **Javadoc** | AI helps generate Javadoc comments for Java source files. | + +--- + +## What AI is not used for + +- AI does not write code on its own or make changes without human involvement. +- AI does not run tests, merge pull requests, or make decisions about the codebase. +- AI is never used in an agentic mode where it acts independently. + +--- + +## Review process + +Everything AI generates is reviewed by a developer before it becomes part of the project: + +1. **Code suggestions** are read, tested, and adjusted as needed before being committed. +2. **Documentation** is checked for accuracy and clarity. +3. **Javadoc** is verified to make sure it correctly describes the behaviour of the code. + +If an AI-generated contribution is inaccurate or incomplete, it is corrected or discarded. No content is accepted without human review. + +--- + +## Summary + +AI is a supporting tool in KTestify, not an autonomous contributor. It helps with code suggestions, documentation, and Javadoc, and every output is reviewed before it is used. diff --git a/docs/extend/architecture.mdx b/docs/extend/architecture.mdx index 9d34ef8..56bf8ad 100644 --- a/docs/extend/architecture.mdx +++ b/docs/extend/architecture.mdx @@ -62,14 +62,19 @@ Synchronous transports additionally populate a new `attributes` map on `Consumed --- +## Why the separation matters + +The three layers are connected by exactly one type, `ConsumedRecord`. Because the transport layer only ever emits that type and the assertion layer only ever consumes it, the two are fully decoupled. The practical payoff is threefold: + +- **Swap transports without touching assertions.** A new broker is a new `RecordFetcher` (or `RequestResponseClient`) implementation. Every `RecordMatcher` keeps working because it only sees `ConsumedRecord`. +- **Reuse matchers across transports.** The same `FileRecordMatcher` asserts a Kafka `String` body and an HTTP response body, because both arrive as `ConsumedRecord`. +- **Stay test-framework agnostic.** `ktestify-cucumber` (or any future adapter) drives the engine through `ConsumerContext` / `ProducerContext` and reads back `ConsumedRecord`. It never imports a broker client. + ## Layer responsibilities ### Transport - `RecordFetcher` -Knows: Kafka broker, partitions, offsets, deduplication. -Does NOT know: matchers, files, test frameworks. - -The contract is a single interface: +The transport layer turns a concrete message broker into a stream of `ConsumedRecord`. The contract itself is deliberately tiny, a single `fetch()` method plus `close()`: ```java public interface RecordFetcher extends AutoCloseable { @@ -78,7 +83,9 @@ public interface RecordFetcher extends AutoCloseable { } ``` -Swapping Kafka for IBM MQ means writing a new `IbmMqRecordFetcher`, nothing else changes. +`fetch()` blocks until at least one record that passes the configured filters is available, or the read timeout expires. It returns a non-empty, unmodifiable list, or throws `FetchException`. + +The only implementation today is `KafkaRecordFetcher`, which knows Kafka brokers, partitions, offsets, and the deduplication registry. Swapping Kafka for IBM MQ means writing a new `IbmMqRecordFetcher` that implements the same interface. Nothing in the layers above changes. --- @@ -105,16 +112,13 @@ Does NOT know: Kafka internals, comparison algorithms. ```java // AbstractKafkaConsumer.call(), simplified -var fetcher = new KafkaRecordFetcher(context); -try { +try (KafkaRecordFetcher fetcher = new KafkaRecordFetcher<>(context)) { List> records = fetcher.fetch(); // transport MatchContext matchCtx = buildMatchContext(); MatchResult result = matcher.match(records, matchCtx); // assertion return result.isPassed(); } catch (FetchException e) { - throw new ConsumerException(e.getMessage(), e); -} finally { - fetcher.close(); + throw new ConsumerException(e.getMessage()); } ``` @@ -163,8 +167,7 @@ ktestify-core ktestify-cucumber RecordFetcher BackgroundStepDefinition RequestResponseClient ValidationStepDefinition KafkaRecordFetcher ◄──────── ConsumerContext (config only) -AbstractKafkaConsumer ConsumerValidationService -AbstractSynchronousConsumer +AbstractKafkaConsumer ConsumerValidationService.** It uses `ConsumerContext` / `ProducerContext` (ktestify-core abstractions) to configure the engine and receives only `ConsumedRecord` back. This is what keeps the Cucumber layer free of broker specifics and lets the same step definitions drive Kafka, HTTP, or any future transport without modification PollingRequestResponseClient RecordMatcher MatchContext / MatchResult diff --git a/docs/extend/core-concepts.mdx b/docs/extend/core-concepts.mdx index de679e8..4282b5e 100644 --- a/docs/extend/core-concepts.mdx +++ b/docs/extend/core-concepts.mdx @@ -8,7 +8,7 @@ description: Key domain types in ktestify-core, ConsumedRecord, MatchContext, Ma ## `ConsumedRecord` -The **only** data type that crosses layer boundaries. It is the universal output of the transport layer (`RecordFetcher` for asynchronous transports, `RequestResponseClient` for synchronous ones since `1.1.1`) and the universal input of the assertion layer. +The **only** data type that crosses layer boundaries. It is the universal output of the transport layer (`RecordFetcher` for asynchronous transports, `RequestResponseClient` for synchronous ones since `1.1.1`) and the universal input of the assertion layer. The class itself has existed since `0.3.0`; the `attributes` map was added in `1.1.1`. ```java @Value @@ -20,7 +20,7 @@ public class ConsumedRecord { V value; // String for raw, GenericRecord for Avro Instant timestamp; Map headers; // protocol headers (Kafka headers, HTTP response headers, ...) - Map attributes; // NEW in 1.1.1, transport metadata, never null, defaults to emptyMap() + Map attributes; // since 1.1.1, transport metadata, never null, defaults to emptyMap() static ConsumedRecord fromKafkaRecord(ConsumerRecord record) { ... } MatchedRecord toMatchedRecord() { ... } @@ -29,15 +29,17 @@ public class ConsumedRecord { `attributes` holds structured transport metadata that does not belong under `headers`, for example an HTTP status code and elapsed time, a future gRPC status code, or an MQ reason code. Kafka and Azure Blob leave it empty. A synchronous transport plugin populates it, and `AttributeRecordMatcher` (see [Built-in matchers →](matchers/built-in-matchers)) asserts against it. -`ConsumedRecord` ships both a full constructor (accepting `attributes`) and a backward-compatible overload without it (defaults to `Collections.emptyMap()`), plus a `@Builder`, so existing transports keep compiling unchanged. +`ConsumedRecord` ships both a full constructor (accepting `attributes`) and a backward-compatible overload without it (defaults to `Collections.emptyMap()`), plus a `@Builder`, so existing transports keep compiling unchanged. The `attributes` field is documented as never null and defaults to an empty map, so matchers can safely call `isEmpty()` rather than checking for null. --- ## `MatchedRecord` -Deduplication token, represents a record that has already been claimed by a consumer step. It holds `topic + partition + offset + key + timestamp` and is stored in the static deduplication registry. +Deduplication token, represents a record that has already been claimed by a consumer step. It holds `topic + partition + offset + key + timestamp` and is stored in the static deduplication registry maintained by `KafkaRecordFetcher`. -`MatchedRecord` deliberately **excludes `processedTime`** from `equals`/`hashCode` so that two records from the same Kafka partition+offset are always considered the same, regardless of when they were processed. +`MatchedRecord` deliberately **excludes `processedTime`** from `equals`/`hashCode` so that two records from the same Kafka partition+offset are always considered the same, regardless of when they were processed. The `processedTime` field is annotated with Lombok `@With`, meaning it is a mutable-on-copy timestamp used for reporting, not for identity. + +The conversion is one way and cheap: `ConsumedRecord.toMatchedRecord()` builds the token, and `KafkaRecordFetcher` calls `MATCHED_RECORDS.contains(record.toMatchedRecord())` before claiming a record and `MATCHED_RECORDS.add(...)` after. The registry is a `ConcurrentHashMap.newKeySet()` shared across all fetcher instances in the JVM, and `KafkaRecordFetcher.clearMatchedRecords()` wipes it. In the Cucumber flow that clear happens in the `@Before` hook at the start of every scenario, which is why two steps in the same scenario never double-claim a record. If you use `KafkaRecordFetcher` directly outside Cucumber, you must call `clearMatchedRecords()` yourself before each test. --- @@ -54,17 +56,17 @@ public class MatchContext { boolean strictMatching; String matchKey; String matchValue; - Map expectedAttributes; // NEW in 1.1.1, defaults to emptyMap(), used by AttributeRecordMatcher + Map expectedAttributes; // since 1.1.1, defaults to emptyMap(), used by AttributeRecordMatcher // Convenience, for single-record matchers public String getMatchFilePath() { return matchFilePaths != null && !matchFilePaths.isEmpty() - ? matchFilePaths.get(0) : null; + ? matchFilePaths.get(0): null; } } ``` -`expectedAttributes` holds the key/value pairs to assert against a record's `attributes` map, for example `{"statusCode": "200"}`. It follows the same convention as `excludedFields`, always non-null, defaults to an empty map, and matchers check `isEmpty()` rather than `null`. +`expectedAttributes` holds the key/value pairs to assert against a record's `attributes` map, for example `{"statusCode": "200"}`. It follows the same convention as `excludedFields`: both default to an empty collection rather than null, and matchers check `isEmpty()` instead of null. The default `buildMatchContext()` in `AbstractKafkaConsumer` sets `strictMatching` to `false` and leaves `matchKey`, `matchValue`, and `expectedAttributes` unset. --- @@ -81,7 +83,9 @@ public class MatchResult { String actual; static MatchResult pass() { ... } + static MatchResult pass(String expected, String actual) { ... } static MatchResult fail(String diff, String expected, String actual) { ... } + static MatchResult fail(String message) { ... } } ``` @@ -95,11 +99,11 @@ Key fields: | Field | Type | Description | |---|---|---| -| `topic` | `Topic` | The output topic (must be OUTPUT type — validated in builder) | -| `matchMethod` | `String` | One of the `ConfigConstants.method*` values | +| `topic` | `Topic` | The output topic, must be OUTPUT type, validated in the builder | +| `matchMethod` | `String` | One of the `RecordMatcherFactory.METHOD_*` constants | | `matchFilePaths` | `List` | Expected file paths (single or batch) | | `excludedFields` | `List` | Field names to ignore in comparison | -| `expectedRecordKey` | `String` | Key filter — record must match this key | +| `expectedRecordKey` | `String` | Key filter, the record must match this key | | `readTimeout` | `long` | Milliseconds | | `consumerDeltaTime` | `long` | Milliseconds (DataTable seconds × 1000) | | `isBatchConsumer` | `boolean` | Enables batch fetch mode | @@ -112,10 +116,10 @@ Key fields: ```java @Data @Builder public class Topic { - String topicName; - String topicAlias; - String topicNamespace; - Topic.Type topicType; // INPUT or OUTPUT + String topicName; + String topicAlias; + TopicNamespace topicNamespace; // nested, holds namespace + namespaceAlias + Topic.Type topicType; // INPUT or OUTPUT // Returns "namespace.topicName" or just "topicName" if no namespace String getNamespacedTopic() { ... } @@ -124,19 +128,34 @@ public class Topic { --- +## `RecordMatcher` + +The assertion contract. A matcher receives the records fetched by a `RecordFetcher` (or returned by a `RequestResponseClient`) and asserts them against the expected state carried by a `MatchContext`. + +```java +@FunctionalInterface +public interface RecordMatcher { + MatchResult match(List> records, MatchContext context); +} +``` + +Implementations have zero dependency on Kafka, HTTP, or any transport. They only know about `ConsumedRecord`, which is exactly why the same matcher works for every transport. Concrete implementations live in `io.github.ktestify.match.impl`: `NoOpRecordMatcher` (always passes, for consume-only scenarios), `FileRecordMatcher`, `XmlRecordMatcher`, `XPathRecordMatcher`, `FieldsRecordMatcher`, `FileKeyRecordMatcher`, `KeyRecordMatcher`, `AttributeRecordMatcher`, and their `Avro*` counterparts. + +--- + ## `RecordMatcherFactory` -Pure static factory, no DI, no singleton. Resolves the right `RecordMatcher` implementation based on `matchMethod` and whether the consumer is raw or Avro. +Pure static factory, no DI, no singleton. Resolves the right `RecordMatcher` implementation based on `matchMethod` and whether the consumer is raw (`String`) or Avro (`GenericRecord`). Two typed entry points, `forRaw(String)` and `forAvro(String)`, let the compiler enforce the value type. ```java -RecordMatcherFactory.forRaw("matchFile") → FileRecordMatcher -RecordMatcherFactory.forAvro("matchFile") → AvroFileRecordMatcher -RecordMatcherFactory.forRaw("matchXML") → XmlRecordMatcher -RecordMatcherFactory.forAvro("matchXML") → throws ConsumerException ← not supported -RecordMatcherFactory.forRaw("methodMatchAttributes") → AttributeRecordMatcher<>() // NEW in 1.1.1, raw only +RecordMatcherFactory.forRaw("methodMatchFile") → FileRecordMatcher +RecordMatcherFactory.forAvro("methodMatchFile") → AvroFileRecordMatcher +RecordMatcherFactory.forRaw("methodMatchXML") → XmlRecordMatcher +RecordMatcherFactory.forAvro("methodMatchXML") → throws ConsumerException (not supported for Avro) +RecordMatcherFactory.forRaw("methodMatchAttributes") → AttributeRecordMatcher<>() // since 1.1.1, raw only ``` -See [Built-in matchers →](matchers/built-in-matchers) for the full mapping table. +When `matchMethod` is `null` or blank, both methods return a `NoOpRecordMatcher`, which makes consume-only scenarios a first-class use case. The Avro path supports only `methodMatchFile`, `methodMatchKeyValue`, `methodFieldsToMatch`, and `methodRecordKeyMatch`. XML, XPath, and attribute matching are raw-only, so requesting them for an Avro topic throws `ConsumerException`. See [Built-in matchers →](matchers/built-in-matchers) for the full mapping table. --- diff --git a/docs/extend/plugins/create-plugin.mdx b/docs/extend/plugins/create-plugin.mdx index e3fcb0c..82d9e9a 100644 --- a/docs/extend/plugins/create-plugin.mdx +++ b/docs/extend/plugins/create-plugin.mdx @@ -6,7 +6,7 @@ description: Generate a complete ktestify plugin project in seconds using the Ma # Creating a KTestify Plugin -This guide walks you through creating a ktestify plugin from start to finish using the **ktestify-plugin-archetype** Maven archetype. The archetype generates a complete, compilable project with all the boilerplate — plugin SPI class, config reader, transport layer, Cucumber step definitions, services, unit tests, and build configuration. +This guide walks you through creating a ktestify plugin from start to finish using the **ktestify-plugin-archetype** Maven archetype. The archetype generates a complete, compilable project with all the boilerplate: plugin SPI class, config reader, transport layer, Cucumber step definitions, services, unit tests, and build configuration. :::tip The archetype is the **recommended way** to start a new plugin. It ensures your project follows the correct structure, naming conventions, and build setup from day one. @@ -73,15 +73,15 @@ The generated project follows the same three-layer separation as ktestify-core: └──────────────────────────────────────────────────────────────┘ ``` -- **Transport** — `MyPluginRecordFetcher` implements `RecordFetcher` from ktestify-core. Returns `List>` — the only type crossing layer boundaries. -- **Orchestration** — `MyPluginConsumer` wires fetch → match → result. Uses `RecordMatcherFactory.forRaw()` to select the correct matcher. -- **Assertion** — all standard `RecordMatcher` implementations from ktestify-core are reused as-is. No matcher code needed in your plugin. +- **Transport**: `MyPluginRecordFetcher` implements `RecordFetcher` from ktestify-core. It returns `List>`, the only type crossing layer boundaries. +- **Orchestration**: `MyPluginConsumer` wires fetch to match to result. It uses `RecordMatcherFactory.forRaw()` to select the correct matcher. +- **Assertion**: all standard `RecordMatcher` implementations from ktestify-core are reused as-is. No matcher code is needed in your plugin. --- ## Phase 1: Generate the Project -### Step 1.1 — Run the archetype +### Step 1.1. Run the archetype ```bash mvn archetype:generate \ @@ -102,7 +102,7 @@ mvn archetype:generate \ -DinteractiveMode=false ``` -### Step 1.2 — Archetype parameters +### Step 1.2. Archetype parameters | Parameter | Description | Example | |---|---|---| @@ -118,16 +118,16 @@ mvn archetype:generate \ | `githubOrg` | GitHub username or organisation | `your-github-username` | | `githubRepoName` | GitHub repository name | `ktestify-plugin-s3` | -> **ktestify-core version**: The generated project depends on the latest released version of `ktestify-core`. This version is managed by Dependabot in the archetype repository — when a new version of ktestify-core is released, Dependabot automatically opens a PR to bump it in the template POM. +> **ktestify-core version**: The generated project depends on the latest released version of `ktestify-core`. This version is managed by Dependabot in the archetype repository. When a new version of ktestify-core is released, Dependabot automatically opens a PR to bump it in the template POM. -### Step 1.3 — Verify it compiles +### Step 1.3. Verify it compiles ```bash cd ktestify-plugin-s3 mvn compile ``` -If this succeeds, you have a valid plugin skeleton. The generated code compiles out of the box — all TODO stubs are syntactically valid. +If this succeeds, you have a valid plugin skeleton. The generated code compiles out of the box, and all TODO stubs are syntactically valid. --- @@ -135,7 +135,7 @@ If this succeeds, you have a valid plugin skeleton. The generated code compiles The generated code contains TODO markers where you need to add your transport-specific logic. Here's what to implement: -### Step 2.1 — Add your transport SDK dependency +### Step 2.1. Add your transport SDK dependency Add your transport SDK to `pom.xml`: @@ -148,7 +148,7 @@ Add your transport SDK to `pom.xml`: ``` -### Step 2.2 — Implement the RecordFetcher +### Step 2.2. Implement the RecordFetcher Open `src/main/java/.../io/S3RecordFetcher.java` and implement the `fetch()` method: @@ -180,7 +180,7 @@ public List> fetch() throws FetchException { } ``` -### Step 2.3 — Implement the Action Service +### Step 2.3. Implement the Action Service Open `src/main/java/.../services/S3ActionService.java` and implement the `send()` method: @@ -199,7 +199,7 @@ public void send(KtestifyS3Entity resource, String recordId, String sourceFile) } ``` -### Step 2.4 — Customize step wording (optional) +### Step 2.4. Customize step wording (optional) The generated steps use generic wording like `"S3 resource"` and `"expected S3 record from file"`. You can rename these to be more natural for your transport: @@ -215,7 +215,7 @@ The generated steps use generic wording like `"S3 resource"` and `"expected S3 r ## Phase 3: Configuration -### Step 3.1 — Update reference.conf +### Step 3.1. Update reference.conf The generated `reference.conf` has a basic connection-string pattern. Update it to match your transport's authentication model: @@ -239,7 +239,7 @@ ktestify.plugins.s3 { } ``` -### Step 3.2 — Update the Config class +### Step 3.2. Update the Config class Add fields to `S3Config.java` to match your new config keys: @@ -258,7 +258,7 @@ private S3Config(Config cfg) { ## Phase 4: Testing -### Step 4.1 — Unit tests +### Step 4.1. Unit tests The archetype generates unit tests for the plugin lifecycle and config loading. Run them: @@ -266,7 +266,7 @@ The archetype generates unit tests for the plugin lifecycle and config loading. mvn test ``` -### Step 4.2 — Integration tests +### Step 4.2. Integration tests Add integration tests using Testcontainers. Create `src/test/java/.../S3PluginIT.java`: @@ -300,7 +300,7 @@ mvn verify ## Phase 5: Use Your Plugin -### Option A — As a Maven dependency +### Option A. As a Maven dependency ```xml @@ -311,7 +311,7 @@ mvn verify ``` -### Option B — Drop the JAR into `/workspace/plugins` +### Option B. Drop the JAR into `/workspace/plugins` ```bash docker run --rm \ @@ -376,10 +376,10 @@ Looking for real-world examples? Check out the first-party plugins: ## Resources -- [ktestify-plugin-archetype](https://github.com/ktestify/ktestify-plugin-archetype) — the archetype repository -- [Plugin System Documentation](plugin-system) — how the plugin loading works -- [ktestify-core API](https://github.com/ktestify/ktestify-core) — base classes and interfaces -- [Adding a Transport](../transports/adding-a-transport) — transport layer deep dive +- [ktestify-plugin-archetype](https://github.com/ktestify/ktestify-plugin-archetype): the archetype repository +- [Plugin System Documentation](plugin-system): how the plugin loading works +- [ktestify-core API](https://github.com/ktestify/ktestify-core): base classes and interfaces +- [Adding a Transport](../transports/adding-a-transport): transport layer deep dive --- @@ -387,10 +387,10 @@ Looking for real-world examples? Check out the first-party plugins: Once your plugin is complete and tested: -1. **Publish to Maven Central** — follow [Sonatype's guide](https://central.sonatype.org/publishing/publish-maven/) -2. **Create a GitHub repository** — use `ktestify-plugin-*` naming convention -3. **Document on docs.ktestify.xyz** — add a page next to the Azure Blob example -4. **Announce in the ktestify community** — open a discussion thread +1. **Publish to Maven Central**: follow [Sonatype's guide](https://central.sonatype.org/publishing/publish-maven/) +2. **Create a GitHub repository**: use `ktestify-plugin-*` naming convention +3. **Document on docs.ktestify.xyz**: add a page next to the Azure Blob example +4. **Announce in the ktestify community**: open a discussion thread Happy plugin building! diff --git a/docs/extend/plugins/http.mdx b/docs/extend/plugins/http.mdx new file mode 100644 index 0000000..d733d79 --- /dev/null +++ b/docs/extend/plugins/http.mdx @@ -0,0 +1,249 @@ +--- +sidebar_position: 1 +title: HTTP Plugin +description: ktestify-plugin-http, send synchronous HTTP requests and assert responses as part of your test scenarios. +--- + +# HTTP Plugin + +`ktestify-plugin-http` is a first-party KTestify plugin that adds a synchronous HTTP transport to your test scenarios. It lets you send HTTP requests and assert on the response status, body, and headers, including polling until an expected status is returned. + +The plugin implements the `RequestResponseClient` contract from `ktestify-core`. Unlike the Kafka and Azure Blob transports, which block until a record appears, the HTTP transport sends a request right now and returns the answer immediately. + +:::caution[Work in progress] +This plugin is actively being developed. The API surface, step definitions, and configuration keys may evolve before the first stable release. +::: + +--- + +## When to Use + +Use the HTTP plugin when your system, + +- Exposes a synchronous REST API that you want to call from a scenario +- Needs request and response assertions on status, body, or headers +- Requires polling a endpoint until it returns an expected status +- Combines HTTP calls with Kafka produce and consume steps in one flow + +--- + +## Installation + +### Maven Dependency + +```xml + + io.github.ktestify + ktestify-plugin-http + 0.0.1-SNAPSHOT + test + +``` + +### With ktestify-cucumber (Docker) + +The HTTP plugin is a first-party plugin and is bundled in the `ktestify-cucumber` fat JAR. No additional installation is required. + +--- + +## Configuration + +The plugin reads settings from `ktestify.plugins.http` in your HOCON configuration file. Place your configuration in `application.conf` or override via environment variables. + +### Configuration Keys + +```hocon +ktestify.plugins.http { + connect-timeout = 10s + read-timeout = 30s + poll-interval = 500ms + follow-redirects = true + tls { + trust-all = false + } +} +``` + +| Key | Type | Default | Description | +|---|---|---|---| +| `connect-timeout` | duration | `10s` | Maximum time to establish a connection | +| `read-timeout` | duration | `30s` | Maximum time to read a response | +| `poll-interval` | duration | `500ms` | Delay between polling attempts | +| `follow-redirects` | boolean | `true` | Whether to follow HTTP redirects | +| `tls.trust-all` | boolean | `false` | Trust all TLS certificates (disable verification) | + +--- + +## Background Steps + +### `Given HTTP endpoint` + +Declares a single HTTP endpoint with a base URL. + +```gherkin +Given HTTP endpoint + | endpointAlias | baseUrl | + | orders-api | http://localhost:8080/api | +``` + +| Column | Type | Required | Default | Description | +|---|---|---|---|---| +| `endpointAlias` | `string` | yes | - | Alias referenced by request steps | +| `baseUrl` | `string` | yes | - | Base URL for the endpoint | + +### `Given HTTP endpoints` + +Declares multiple HTTP endpoints at once. + +```gherkin +Given HTTP endpoints + | endpointAlias | baseUrl | + | orders-api | http://localhost:8080/api | + | users-api | http://localhost:8081 | +``` + +| Column | Type | Required | Default | Description | +|---|---|---|---|---| +| `endpointAlias` | `string` | yes | - | Alias referenced by request steps | +| `baseUrl` | `string` | yes | - | Base URL for the endpoint | + +### `Given HTTP bearer token` + +Attaches a bearer token to requests sent to an endpoint. + +```gherkin +Given HTTP bearer token + | endpointAlias | token | + | orders-api | {{ENV:API_TOKEN}} | +``` + +| Column | Type | Required | Default | Description | +|---|---|---|---|---| +| `endpointAlias` | `string` | yes | - | Endpoint the token applies to | +| `token` | `string` | yes | - | Bearer token value, supports dynamic variables | + +### `Given HTTP assets directory` + +Declares the base directory for request and expected response files. + +```gherkin +Given HTTP assets directory + | absolutePath | + | ./src/test/resources/data | +``` + +| Column | Type | Required | Default | Description | +|---|---|---|---|---| +| `absolutePath` | `string` | yes | - | Absolute or relative path to asset files | + +--- + +## Action Steps + +### `When HTTP request is sent` + +Sends an HTTP request and stores the response under an alias for later assertions. + +```gherkin +When HTTP request is sent + | endpointAlias | method | path | file | responseAlias | + | orders-api | POST | /orders/validate | order.json | validate-resp | + +When HTTP request is sent + | endpointAlias | method | path | queryParams | responseAlias | + | orders-api | GET | /orders/{id} | id=ORD-001 | fetch-resp | +``` + +| Column | Type | Required | Default | Description | +|---|---|---|---|---| +| `endpointAlias` | `string` | yes | - | Endpoint to send the request to | +| `method` | `string` | yes | - | HTTP method (GET, POST, PUT, DELETE, ...) | +| `path` | `string` | yes | - | Path appended to the base URL, supports `{placeholder}` substitution | +| `file` | `string` | no | - | Request body file, resolved against the assets directory | +| `queryParams` | `string` | no | - | Query parameters as `key=value` pairs, comma separated | +| `responseAlias` | `string` | yes | - | Alias used to reference the response in assertions | + +--- + +## Validation Steps + +### `Then expected HTTP response status` + +Asserts the HTTP status code of a stored response. + +```gherkin +Then expected HTTP response status + | responseAlias | statusCode | + | validate-resp | 200 | +``` + +| Column | Type | Required | Default | Description | +|---|---|---|---|---| +| `responseAlias` | `string` | yes | - | Response to assert | +| `statusCode` | `int` | yes | - | Expected HTTP status code | + +### `Then expected HTTP response body from file` + +Asserts the response body against a file, with optional excluded keys. + +```gherkin +Then expected HTTP response body from file + | responseAlias | file | excludedKeys | + | validate-resp | expected.json | timestamp,id | +``` + +| Column | Type | Required | Default | Description | +|---|---|---|---|---| +| `responseAlias` | `string` | yes | - | Response to assert | +| `file` | `string` | yes | - | Expected body file | +| `excludedKeys` | `string` | no | - | Comma separated keys to ignore during comparison | + +### `Then expected HTTP response XML body from file` + +Asserts an XML response body against a file, with optional excluded elements. + +```gherkin +Then expected HTTP response XML body from file + | responseAlias | file | excludedElements | + | validate-resp | expected.xml | ns:CreationDateTime | +``` + +| Column | Type | Required | Default | Description | +|---|---|---|---|---| +| `responseAlias` | `string` | yes | - | Response to assert | +| `file` | `string` | yes | - | Expected XML body file | +| `excludedElements` | `string` | no | - | Comma separated elements to ignore during comparison | + +### `And HTTP response header should match` + +Asserts a response header value. + +```gherkin +And HTTP response header should match + | responseAlias | header | value | + | validate-resp | Content-Type | application/json | +``` + +| Column | Type | Required | Default | Description | +|---|---|---|---|---| +| `responseAlias` | `string` | yes | - | Response to assert | +| `header` | `string` | yes | - | Header name | +| `value` | `string` | yes | - | Expected header value | + +### `Then HTTP endpoint should eventually return` + +Polls an endpoint until it returns an expected status or the read timeout elapses. + +```gherkin +Then HTTP endpoint should eventually return + | endpointAlias | method | path | expectedStatus | readTimeout | + | orders-api | GET | /orders/ORD-001/status | 200 | 30 | +``` + +| Column | Type | Required | Default | Description | +|---|---|---|---|---| +| `endpointAlias` | `string` | yes | - | Endpoint to poll | +| `method` | `string` | yes | - | HTTP method | +| `path` | `string` | yes | - | Path appended to the base URL | +| `expectedStatus` | `int` | yes | - | Status code to wait for | +| `readTimeout` | `int` | yes | - | Maximum seconds to keep polling | diff --git a/docs/extend/plugins/index.mdx b/docs/extend/plugins/index.mdx index 27996b4..779978e 100644 --- a/docs/extend/plugins/index.mdx +++ b/docs/extend/plugins/index.mdx @@ -118,7 +118,7 @@ Then expected blob from file ### Build Your Own Plugin -Start with the Maven archetype — it generates a complete, compilable project in seconds: +Start with the Maven archetype, which generates a complete, compilable project in seconds: ```bash mvn archetype:generate \ diff --git a/docs/getting-started/ci-environment.mdx b/docs/getting-started/ci-environment.mdx index 18ee3a6..df4c7f0 100644 --- a/docs/getting-started/ci-environment.mdx +++ b/docs/getting-started/ci-environment.mdx @@ -4,7 +4,7 @@ title: CI Environment description: Why KTestify needs a dedicated test environment, a production mirror, and how to schedule it as a daily regression gate. --- -# CI Environment : Test Against a Production Mirror +# CI Environment: Test Against a Production Mirror ## The purpose of KTestify @@ -48,8 +48,8 @@ flowchart TB ``` Key properties of this environment: -- **Same topology as production** : same topic names, same number of partitions, same schema subjects. -- **Latest code** : your application is deployed here first, before any release to production. +- **Same topology as production**: same topic names, same number of partitions, same schema subjects. +- **Latest code**: your application is deployed here first, before any release to production. - **KTestify is the only external consumer** on output topics. - **No other team, no other application** writes to the same topics during the test run. @@ -78,7 +78,7 @@ The recommended pattern is a **nightly scheduled pipeline** that: ``` 00:00 ──► Clean test environment (topics, statestores) 00:05 ──► Run KTestify suite -09:00 ──► ✅ Green : ready to release OR ❌ Red : regression found, notify team +09:00 ──► ✅ Green: ready to release OR ❌ Red: regression found, notify team ``` This gives you a **daily regression gate**: any breaking change merged yesterday is caught overnight, before it reaches production. @@ -92,23 +92,17 @@ import TabItem from '@theme/TabItem'; ```yaml title=".gitlab-ci.yml" regression-tests: stage: test - image: docker:27 - services: - - docker:27-dind + image: ghcr.io/ktestify/ktestify-cucumber:latest + variables: + KTESTIFY_CONFIG_FILE: $CI_PROJECT_DIR/.config/ktestify.conf rules: - if: '$CI_PIPELINE_SOURCE == "schedule"' script: - - docker run --rm - -v "$CI_PROJECT_DIR/workspace/features:/workspace/features" - -v "$CI_PROJECT_DIR/workspace/reports:/workspace/reports" - -e KTESTIFY_BOOTSTRAP_SERVERS="$TEST_KAFKA_BROKERS" - -e KTESTIFY_TOPIC_NAMESPACE="$KTESTIFY_TOPIC_NAMESPACE" - ghcr.io/ktestify/ktestify-cucumber:latest - /workspace/features + - java -cp @/app/jib-classpath-file @/app/jib-main-class-file artifacts: when: always paths: - - workspace/reports/ + - $CI_PROJECT_DIR/workspace/reports ``` Create a schedule in **CI/CD → Schedules** pointing at `main`, set to run nightly. @@ -178,5 +172,5 @@ Before starting a run, ensure: ## See also - [Installation →](installation), pulling the Docker image and CI pipeline examples -- [Configuration →](../write-tests/configuration), `topic-namespace` and all environment variable overrides +- [Configuration →](configuration), `topic-namespace` and all environment variable overrides - [Timeout tuning →](../write-tests/advanced/timeout-tuning), adjust timeouts for slower environments diff --git a/docs/getting-started/configuration.mdx b/docs/getting-started/configuration.mdx index ec942fb..07ddc84 100644 --- a/docs/getting-started/configuration.mdx +++ b/docs/getting-started/configuration.mdx @@ -107,13 +107,13 @@ For Avro tests, also add: | HOCON key | Environment variable | Default | Description | |---|---|---|---| | `ktestify.kafka.consumer.group-id` | `KAFKA_CONSUMER_GROUP_ID` | `ktestify-consumer-group` | Kafka consumer group ID | -| `ktestify.kafka.consumer.enable-auto-commit` | — | `false` | Auto-commit offsets. Keep `false` — KTestify manages offsets manually. | -| `ktestify.kafka.consumer.auto-offset-reset` | — | `earliest` | What to do when no offset exists: `earliest` or `latest` | -| `ktestify.kafka.consumer.session-timeout` | — | `30s` | Kafka consumer session timeout | -| `ktestify.kafka.consumer.heartbeat-interval` | — | `10s` | Heartbeat interval (must be < `session-timeout / 3`) | -| `ktestify.kafka.consumer.max-poll-records` | — | `500` | Max records returned per `poll()` call | -| `ktestify.kafka.consumer.key-deserializer` | — | `StringDeserializer` | Key deserializer class (fully qualified) | -| `ktestify.kafka.consumer.value-deserializer` | — | `StringDeserializer` | Value deserializer class. Avro consumers override this automatically. | +| `ktestify.kafka.consumer.enable-auto-commit` | none | `false` | Auto-commit offsets. Keep `false` because KTestify manages offsets manually. | +| `ktestify.kafka.consumer.auto-offset-reset` | none | `earliest` | What to do when no offset exists: `earliest` or `latest` | +| `ktestify.kafka.consumer.session-timeout` | none | `30s` | Kafka consumer session timeout | +| `ktestify.kafka.consumer.heartbeat-interval` | none | `10s` | Heartbeat interval (must be < `session-timeout / 3`) | +| `ktestify.kafka.consumer.max-poll-records` | none | `500` | Max records returned per `poll()` call | +| `ktestify.kafka.consumer.key-deserializer` | none | `StringDeserializer` | Key deserializer class (fully qualified) | +| `ktestify.kafka.consumer.value-deserializer` | none | `StringDeserializer` | Value deserializer class. Avro consumers override this automatically. | #### Producer @@ -146,9 +146,9 @@ For Avro tests, also add: | HOCON key | Environment variable | Default | Description | |---|---|---|---| | `ktestify.schema-registry.url` | `SCHEMA_REGISTRY_URL` | `http://localhost:8081` | Schema Registry base URL | -| `ktestify.schema-registry.cache-capacity` | — | `50` | Number of schemas to cache in memory | -| `ktestify.schema-registry.auto-register-schemas` | — | `true` | Automatically register new schemas when producing Avro | -| `ktestify.schema-registry.compatibility-level` | — | `BACKWARD` | Schema compatibility level: `NONE`, `BACKWARD`, `FORWARD`, `FULL` | +| `ktestify.schema-registry.cache-capacity` | none | `50` | Number of schemas to cache in memory | +| `ktestify.schema-registry.auto-register-schemas` | none | `true` | Automatically register new schemas when producing Avro | +| `ktestify.schema-registry.compatibility-level` | none | `BACKWARD` | Schema compatibility level: `NONE`, `BACKWARD`, `FORWARD`, `FULL` | | `ktestify.schema-registry.auth.basic-auth-credentials-source` | `SCHEMA_REGISTRY_BASIC_AUTH_CREDENTIALS_SOURCE` | `""` | Credentials source: `USER_INFO`, `URL`, `SASL_INHERIT` | | `ktestify.schema-registry.auth.basic-auth-user-info` | `SCHEMA_REGISTRY_BASIC_AUTH_USER_INFO` | `""` | Credentials in `username:password` format | | `ktestify.schema-registry.ssl.truststore-location` | `SCHEMA_REGISTRY_SSL_TRUSTSTORE_LOCATION` | `""` | Path to SSL truststore for Schema Registry | @@ -168,8 +168,8 @@ For Avro tests, also add: |---|---|---|---|---| | `ktestify.framework.timeouts.default-read-timeout` | `KTESTIFY_DEFAULT_READ_TIMEOUT` | `10s` | **`30s`** | How long to wait for a matching record on the output topic | | `ktestify.framework.timeouts.consumer-delta-time` | `KTESTIFY_CONSUMER_DELTA_TIME` | `20s` | **`60s`** | How far back in time to seek before consuming (`now − delta`) | -| `ktestify.framework.timeouts.poll-interval` | — | `100ms` | `100ms` | Kafka poll loop interval | -| `ktestify.framework.timeouts.buffer-time` | — | `5s` | `5s` | Safety buffer added to the outer `ExecutorService` timeout on top of `default-read-timeout` | +| `ktestify.framework.timeouts.poll-interval` | none | `100ms` | `100ms` | Kafka poll loop interval | +| `ktestify.framework.timeouts.buffer-time` | none | `5s` | `5s` | Safety buffer added to the outer `ExecutorService` timeout on top of `default-read-timeout` | Both `default-read-timeout` and `consumer-delta-time` can be overridden **per scenario** in the validation step DataTable (values in **seconds**): @@ -195,8 +195,8 @@ Then expected record from file | HOCON key | Environment variable | Default | Description | |---|---|---|---| | `ktestify.framework.execution.snapshot-mode` | `KTESTIFY_SNAPSHOT_MODE` | `false` | When `true`, updates expected files instead of asserting against them | -| `ktestify.framework.execution.strict-matching` | — | `true` | Fail on any mismatch. Always `false` internally in KTestify's matchers — not user-configurable at this time. | -| `ktestify.framework.execution.max-retries` | — | `3` | Maximum retries for transient failures | +| `ktestify.framework.execution.strict-matching` | none | `true` | Fail on any mismatch. Always `false` internally in KTestify's matchers, not user-configurable at this time. | +| `ktestify.framework.execution.max-retries` | none | `3` | Maximum retries for transient failures | --- @@ -204,8 +204,8 @@ Then expected record from file | HOCON key | Environment variable | Default | Description | |---|---|---|---| -| `ktestify.framework.reporting.enabled` | — | `true` | Enable generation of the test report | -| `ktestify.framework.reporting.format` | — | `html` | Report format: `html`, `json`, `cucumber` | +| `ktestify.framework.reporting.enabled` | none | `true` | Enable generation of the test report | +| `ktestify.framework.reporting.format` | none | `html` | Report format: `html`, `json`, `cucumber` | | `ktestify.framework.reporting.output` | `KTESTIFY_OUTPUT_DIR` | `src/target/reports/report.html` | Output path for the generated report | --- diff --git a/docs/getting-started/installation.mdx b/docs/getting-started/installation.mdx index 4da0aef..e60f602 100644 --- a/docs/getting-started/installation.mdx +++ b/docs/getting-started/installation.mdx @@ -71,7 +71,7 @@ ktestify { } ``` -All values can be overridden by environment variables, see [Configuration →](../write-tests/configuration). +All values can be overridden by environment variables, see [Configuration →](configuration). --- @@ -212,4 +212,4 @@ jobs: - [Quick start →](quick-start), write your first feature file - ⚠️ [CI environment →](ci-environment), why a dedicated test environment is non-negotiable -- ⚙️ [Configuration →](../write-tests/configuration), all HOCON keys and environment variable overrides +- ⚙️ [Configuration →](configuration), all HOCON keys and environment variable overrides diff --git a/docs/getting-started/quick-start.mdx b/docs/getting-started/quick-start.mdx index 8282915..959145f 100644 --- a/docs/getting-started/quick-start.mdx +++ b/docs/getting-started/quick-start.mdx @@ -114,6 +114,6 @@ KTestify will: ## Next steps - [Step reference →](../write-tests/step-reference), every step and every DataTable column -- ⚙️ [Configuration →](../write-tests/configuration), HOCON keys and environment variable overrides +- ⚙️ [Configuration →](configuration), HOCON keys and environment variable overrides - [Dynamic variables →](../write-tests/dynamic-variables), inject dates, UUIDs, env vars into payloads - ⚠️ [CI environment →](ci-environment), scheduling daily regression tests diff --git a/docs/intro.mdx b/docs/intro.mdx index 3b53ad3..3331154 100644 --- a/docs/intro.mdx +++ b/docs/intro.mdx @@ -1,16 +1,16 @@ --- sidebar_position: 1 title: Introduction -description: KTestify, modular, open-source Kafka Streams integration testing framework with a Gherkin DSL. +description: KTestify, a modular, open-source Kafka integration testing framework with a Gherkin DSL. --- # What is KTestify? -**KTestify** is a modular, open-source framework for integration-testing Kafka/Kafka Streams data pipelines. +**KTestify** is a modular, open-source framework for integration-testing Kafka and Kafka Streams data pipelines. -At its core sits a transport-agnostic engine that separates concerns cleanly into three layers, **Transport**, **Orchestration**, and **Assertion**. +At its core sits a transport-agnostic engine that separates concerns cleanly into three layers: **Transport**, **Orchestration**, and **Assertion**. -Today, a **Cucumber/Gherkin adapter** (`ktestify-cucumber`) lets teams write plain-English test scenarios that produce messages, consume from output topics, and assert record content without touching a single line of Kafka client code. Other tests engines such as Playwright or RobotFramework are planned in the future. +A **Cucumber/Gherkin adapter** (`ktestify-cucumber`) lets teams write plain-English test scenarios that produce messages, consume from output topics, and assert record content without touching a single line of Kafka client code. ```gherkin Feature: Order stream validation @@ -35,20 +35,22 @@ Feature: Order stream validation | enriched | order-enriched-output.json | ``` -But KTestify does not stop at Kafka clients and Gherkin steps. Its modular architecture allows you to extend the framework with custom transports, matchers thanks to plugins. +KTestify does not stop at Kafka clients and Gherkin steps. Its modular architecture lets you extend the framework with custom transports and matchers through plugins. -For example, there is an Azure Blob Storage plugin available to lets you interact with the Azure Blob Storage to check that your Sink/Source Kafka connectors are working as expected. +For example, the Azure Blob Storage plugin lets you interact with Azure Blob Storage to check that your Sink or Source Kafka connectors are working as expected. The HTTP plugin lets you test synchronous request and response flows. --- -## Two tracks, one framework +## How this documentation is organized -KTestify's documentation is split into two reader tracks: +The documentation follows four purpose-based sections. Pick the one that matches what you are trying to do, not who you are. -| Track | Who is it for? | Where to start | +| Section | What it is for | Start here | |---|---|---| -| **‍🧪 Write Tests** | QA engineers & test authors who want to write Gherkin scenarios | [Getting Started →](getting-started/installation) | -| **🔧 Extend the Framework** | Java developers who want to add transports, matchers, or plugins | [Architecture →](extend/architecture) | +| **Tutorials** | Learning-oriented walkthroughs that take you from zero to a running test | [Installation](getting-started/installation) | +| **How-to Guides** | Task recipes for writing steps, using plugins, and extending the framework | [Writing tests overview](write-tests/overview) | +| **Reference** | Lookup tables: every step, config key, matcher, and plugin | [Step reference](write-tests/step-reference) | +| **Explanation** | Understanding-oriented material on architecture and core concepts | [Architecture](extend/architecture) | --- @@ -58,14 +60,16 @@ KTestify's documentation is split into two reader tracks: |---|---| | [`ktestify-core`](https://github.com/ktestify/ktestify-core) | Transport-agnostic library, Kafka clients, matchers, config, models | | [`ktestify-cucumber`](https://github.com/ktestify/ktestify-cucumber) | Standalone Cucumber/Gherkin application, step definitions, hooks, services | -| [`ktestify-plugin-azureblob`](https://github.com/ktestify/ktestify-plugin-azureblob) | Azure Blob Storage plugin _(in progress)_ | +| [`ktestify-plugin-azureblob`](https://github.com/ktestify/ktestify-plugin-azureblob) | Azure Blob Storage plugin for validating connector output | +| [`ktestify-plugin-http`](https://github.com/ktestify/ktestify-plugin-http) | HTTP request and response transport plugin | +| [`ktestify-plugin-notifications`](https://github.com/ktestify/ktestify-plugin-notifications) | Suite-level notification plugin for Teams, Slack, and webhooks | --- ## Quick links -- 🚀 **[Install & run](getting-started/installation)**, Docker image or standalone JAR -- ⚠️ **[CI environment requirements](getting-started/ci-environment)**, dedicated Kafka stack is mandatory -- 📖 **[Full step reference](write-tests/step-reference)** -- 🔧 **[Architecture deep-dive](extend/architecture)** -- 📋 **[CHANGELOG](https://github.com/ktestify/ktestify-core/blob/main/CHANGELOG.md)** +- Install and run, Docker image or standalone JAR: [Tutorials](getting-started/installation) +- CI environment requirements, a dedicated Kafka stack is mandatory: [CI environment](getting-started/ci-environment) +- Full step reference: [Step reference](write-tests/step-reference) +- Architecture deep-dive: [Explanation](extend/architecture) +- CHANGELOG: [ktestify-core](https://github.com/ktestify/ktestify-core/blob/main/CHANGELOG.md) diff --git a/docs/write-tests/actions/send-avro-record.mdx b/docs/write-tests/actions/send-avro-record.mdx index 23bfe13..913fe38 100644 --- a/docs/write-tests/actions/send-avro-record.mdx +++ b/docs/write-tests/actions/send-avro-record.mdx @@ -4,7 +4,7 @@ title: Send Avro Record description: Produce an Avro-serialised message to a Kafka input topic from a JSON file. --- -# Send Avro Record : `When record from file based on schema is sent` +# Send Avro Record: `When record from file based on schema is sent` Produces an Avro-serialised message to a Kafka **input** topic. KTestify reads the payload from a JSON file, serialises it using the specified Avro schema, and registers the schema in Schema Registry if needed. @@ -87,6 +87,5 @@ Every row must resolve to the **same physical topic**. Mixing topics in one Data - [Avro assertions →](../assertions/avro-matchers) - [Schemas background step →](../background/schemas) -- [Configuration →](../configuration) +- [Configuration →](../../getting-started/configuration) - [Multi-row DataTables →](../advanced/multi-row-datatables) - diff --git a/docs/write-tests/actions/send-raw-record.mdx b/docs/write-tests/actions/send-raw-record.mdx index e972961..af8b6b7 100644 --- a/docs/write-tests/actions/send-raw-record.mdx +++ b/docs/write-tests/actions/send-raw-record.mdx @@ -4,7 +4,7 @@ title: Send Raw Record description: Produce a raw (String/JSON/text/XML) message to a Kafka input topic from a file. --- -# Send Raw Record : `When record from file is sent` +# Send Raw Record: `When record from file is sent` Produces a raw (String-serialised) message to a Kafka **input** topic. The file content is read as-is, with [dynamic variables](../dynamic-variables) resolved before sending. diff --git a/docs/write-tests/actions/wait-and-script.mdx b/docs/write-tests/actions/wait-and-script.mdx index d9f7308..3b4a8d6 100644 --- a/docs/write-tests/actions/wait-and-script.mdx +++ b/docs/write-tests/actions/wait-and-script.mdx @@ -50,7 +50,7 @@ When script is executed There is no topic concept for script steps, so no same-topic guard rail applies here (unlike producer/assertion steps, see [Multi-row DataTables](../advanced/multi-row-datatables)). -### Example : trigger a downstream reset before consuming +### Example: trigger a downstream reset before consuming ```gherkin Scenario: Processed orders appear after reset diff --git a/docs/write-tests/advanced/multi-row-datatables.mdx b/docs/write-tests/advanced/multi-row-datatables.mdx index 8e6f463..ad4dc2c 100644 --- a/docs/write-tests/advanced/multi-row-datatables.mdx +++ b/docs/write-tests/advanced/multi-row-datatables.mdx @@ -17,11 +17,11 @@ Most KTestify steps accept a Cucumber `DataTable` with **more than one data row* | Producer (`When record from file is sent`, `...based on schema is sent`) | ✅ Yes | ✅ Yes | Rows sent sequentially, in order | | Script execution (`When script is executed`, `And execute script`) | ✅ Yes | N/A | Rows run sequentially, fail-fast on first non-zero exit | | Single-record assertions (`Then expected record from file`, XML/XPath, Avro file/field match, key-only, key+value, watchers) | ✅ Yes | ✅ Yes | Rows validated sequentially, in order, sharing one pinned "now" | -| Batch assertions (`expected records from files`, `...based on schema`) | ❌ Single row only | — | The one row already describes an any-to-all match across N files | +| Batch assertions (`expected records from files`, `...based on schema`) | ❌ Single row only | none | The one row already describes an any-to-all match across N files | --- -## Producer steps : always multi-row +## Producer steps: always multi-row Producing a Kafka record is a stateless operation, there's no offset/seek math involved, so every row of the DataTable is sent, **in row order**, without restriction other than the same-topic guard rail below. @@ -36,7 +36,7 @@ Both records are produced, in that order, before the step completes. --- -## Single-record assertion steps : multi-row, same topic only +## Single-record assertion steps: multi-row, same topic only Every single-record assertion step (`Then expected record from file`, XML/XPath matchers, Avro file/field match, key-only/key+value assertions, and the `record should (not) appear in topic` watchers) drives **one Kafka consumer call per row**. Supporting arbitrary multi-topic rows here would reopen the offset-skew problem the framework works hard to avoid (see below), so KTestify only allows multiple rows **when they all target the same physical topic**: @@ -62,7 +62,7 @@ To avoid this, KTestify captures `System.currentTimeMillis()` **once**, at the t Both producer steps and single-record assertion steps resolve the topic of every DataTable row and assert they all point to the **same physical topic** (namespaced topic name + type) before doing anything else, no partial sends, no partial validation. ```gherkin -# ❌ This throws immediately — mixed topics in one DataTable +# ❌ This throws immediately, mixed topics in one DataTable When record from file is sent | topicName | file | recordKey | | orders-in | order-001.json | order-001 | @@ -89,7 +89,7 @@ When record from file is sent --- -## Batch assertion steps : still single-row only +## Batch assertion steps: still single-row only `expected records from files` and `expected records from files based on schema` remain **strictly single-row**. Their one row already describes an any-to-all match across `expectedRecordsCount` records collected from a single consumer call, adding more DataTable rows on top of that would be ambiguous (multiple independent batches? one bigger batch?) so KTestify rejects it loudly instead of guessing: diff --git a/docs/write-tests/advanced/timeout-tuning.mdx b/docs/write-tests/advanced/timeout-tuning.mdx index 82570cf..5c3a3ba 100644 --- a/docs/write-tests/advanced/timeout-tuning.mdx +++ b/docs/write-tests/advanced/timeout-tuning.mdx @@ -11,7 +11,7 @@ KTestify has a custom Kafka Consumer that has a time-to-live (TTL) to prevent te --- -## Layer 1 : Inner timeout (`KafkaRecordFetcher`) +## Layer 1: Inner timeout (`KafkaRecordFetcher`) The inner timeout controls how long the fetcher blocks waiting for records from Kafka. It maps to the Kafka `poll` loop. @@ -27,7 +27,7 @@ ktestify.framework.timeouts.default-read-timeout = 30s --- -## Layer 2 : Outer safety net (`ExecutorService`) +## Layer 2: Outer safety net (`ExecutorService`) The orchestration layer submits the consumer to an `ExecutorService` and calls `.get(readTimeout + BUFFER_TIME)`. This catches JVM-level hangs that slip past the inner Kafka timeout. @@ -71,7 +71,7 @@ So if `readTimeout = 30s`, the outer guard fires at `35s`. This is intentional, --- -## Example : per-step timeout override +## Example: per-step timeout override ```gherkin Then expected record from file @@ -81,7 +81,7 @@ Then expected record from file --- -## Example : global config for slow environment +## Example: global config for slow environment ```hocon ktestify { diff --git a/docs/write-tests/assertions/avro-matchers.mdx b/docs/write-tests/assertions/avro-matchers.mdx index 0616ee0..5a3b9ed 100644 --- a/docs/write-tests/assertions/avro-matchers.mdx +++ b/docs/write-tests/assertions/avro-matchers.mdx @@ -10,7 +10,7 @@ Avro matchers deserialise `GenericRecord` values (via Schema Registry) and compa --- -## File match : `Then expected record from file based on schema` +## File match: `Then expected record from file based on schema` Fetches an Avro record, converts it to JSON, and compares it against an expected JSON file. Excludes specified top-level keys from comparison. @@ -31,7 +31,7 @@ Then expected record from file based on schema --- -## Key + file match : `Then expected record with key and value from file based on schema` +## Key + file match: `Then expected record with key and value from file based on schema` Asserts both the record **key** and **value** (as Avro → JSON) against a file. @@ -43,25 +43,49 @@ Then expected record with key and value from file based on schema --- -## Field value match : `Then expected record based on schema should have fields matching from given value` +## Field value match: `Then expected record based on schema should have fields matching from given value` Asserts one or more specific Avro field values without a file. +### Single field + +Use the `key` / `value` columns to assert a single field: + ```gherkin Then expected record based on schema should have fields matching from given value | topicAlias | key | value | | orders-out | status | PROCESSED | ``` +### Multiple fields + +Use the `keys` / `values` columns to assert multiple fields in a single step. Separate field names and expected values +with semicolons (`;`). Every pair must match for the assertion to pass. + +```gherkin +Then expected record based on schema should have fields matching from given value + | topicAlias | keys | values | + | orders-out | status;amount | PENDING;99 | +``` + +The above validates that `status == PENDING` **and** `amount == 99`. + | Column | Type | Required | Description | |---|---|---|---| | `topicAlias` | `string` | ✅ | Alias of a declared output topic | -| `key` | `string` | ✅ | Avro field name | -| `value` | `string` | ✅ | Expected field value (string-compared after JSON serialisation) | +| `key` | `string` | ❌* | Avro field name (single-field mode) | +| `value` | `string` | ❌* | Expected field value (single-field mode) | +| `keys` | `string` | ❌* | Semicolon-separated Avro field names (multi-field mode) | +| `values` | `string` | ❌* | Semicolon-separated expected values (multi-field mode) | +| `consumerReadTimeout` | `integer` | ❌ | Max seconds to wait for a record | +| `consumerDeltaTime` | `integer` | ❌ | How far back (seconds) to seek | + +\* Either `key` + `value` (single field) **or** `keys` + `values` (multiple fields) must be provided. The two modes +are mutually exclusive. --- -## Key match : `Then expected Avro record should match key` +## Key match: `Then expected Avro record should match key` Asserts only the **record key**; the Avro value is ignored. @@ -73,7 +97,7 @@ Then expected record based on schema should match key --- -## Positional fields : `Then expected record based on schema should have fields matching from file` +## Positional fields: `Then expected record based on schema should have fields matching from file` Assert a specific character range of the JSON-serialised Avro value. @@ -97,6 +121,5 @@ All rows share one pinned `referenceTimestamp` so their delta-time seek windows - [Batch Avro assertions →](batch-assertions) - [Schemas background step →](../background/schemas) -- [Configuration →](../configuration) +- [Configuration →](../../getting-started/configuration) - [Multi-row DataTables →](../advanced/multi-row-datatables) - diff --git a/docs/write-tests/assertions/batch-assertions.mdx b/docs/write-tests/assertions/batch-assertions.mdx index be5c4a1..a38d7cb 100644 --- a/docs/write-tests/assertions/batch-assertions.mdx +++ b/docs/write-tests/assertions/batch-assertions.mdx @@ -10,7 +10,7 @@ Batch steps consume multiple records from an output topic in a single step and m --- -## Raw batch : `Then expected records from files` +## Raw batch: `Then expected records from files` ```gherkin Then expected records from files @@ -28,7 +28,7 @@ Then expected records from files --- -## Avro batch : `Then expected records from files based on schema` +## Avro batch: `Then expected records from files based on schema` ```gherkin Then expected records from files based on schema diff --git a/docs/write-tests/assertions/raw-matchers.mdx b/docs/write-tests/assertions/raw-matchers.mdx index 4c3a926..5d94f22 100644 --- a/docs/write-tests/assertions/raw-matchers.mdx +++ b/docs/write-tests/assertions/raw-matchers.mdx @@ -10,7 +10,7 @@ Raw matchers compare String-serialised Kafka record values against an expected f --- -## File match : `Then expected record from file` +## File match: `Then expected record from file` Fetches the next record from the output topic and asserts its value matches the content of an expected file character-for-character (with coloured diff on failure). @@ -22,15 +22,15 @@ Then expected record from file | Column | Type | Required | Default | Description | |---|---|---|---|---| -| `topicAlias` | `string` | ✅ | — | Alias of a declared output topic | -| `file` | `string` | ✅ | — | Expected file path, relative to assets directory | -| `expectedRecordKey` | `string` | ❌ | — | Filter: only accept records with this exact key | +| `topicAlias` | `string` | ✅ | none | Alias of a declared output topic | +| `file` | `string` | ✅ | none | Expected file path, relative to assets directory | +| `expectedRecordKey` | `string` | ❌ | none | Filter: only accept records with this exact key | | `consumerReadTimeout` | `integer` | ❌ | From config (30s) | Max seconds to wait for a record | | `consumerDeltaTime` | `integer` | ❌ | From config (60s) | How far back (seconds) to seek before consuming | --- -## Key + value match : `Then expected record with key and value from file` +## Key + value match: `Then expected record with key and value from file` Asserts both the record **key** and **value** match the content of an expected file. @@ -44,7 +44,7 @@ The expected file must contain the key on the first line and the value starting --- -## Positional fields : `Then expected record should have fields matching from file` +## Positional fields: `Then expected record should have fields matching from file` Asserts a specific character range within a specific line of the record value. @@ -64,7 +64,7 @@ Then expected record should have fields matching from file --- -## Record key match : `Then expected record should match key` +## Record key match: `Then expected record should match key` Asserts only the **record key** (value is ignored). diff --git a/docs/write-tests/assertions/watcher.mdx b/docs/write-tests/assertions/watcher.mdx index 71b65f4..330c35a 100644 --- a/docs/write-tests/assertions/watcher.mdx +++ b/docs/write-tests/assertions/watcher.mdx @@ -4,7 +4,7 @@ title: Watcher (Presence / Absence) description: Assert that a record appears or does not appear on an output topic within the timeout window. --- -# Watcher : `And record should (not) appear in topic` +# Watcher: `And record should (not) appear in topic` The watcher steps don't compare record content, they simply verify that **at least one record** (optionally matching a key) does or does not appear on a topic within the read timeout window. @@ -39,19 +39,19 @@ Passes only if **no** record appears within `consumerReadTimeout` seconds. Use t | Column | Type | Required | Description | |---|---|---|---| | `topicAlias` | `string` | ✅ | Alias of a declared output topic | -| `topicType` | `string` | ✅ | `raw` or `avro` — determines deserialiser | +| `topicType` | `string` | ✅ | `raw` or `avro`, determines the deserialiser | | `consumerReadTimeout` | `integer` | ❌ | Max seconds to wait (or confirm absence) | | `consumerDeltaTime` | `integer` | ❌ | How far back (seconds) to seek | :::tip[Choosing the right timeout for negative assertions] -For `should not appear`, set `consumerReadTimeout` to a value that is long enough to be confident nothing will arrive, but short enough to not slow your suite down unnecessarily. 5–10 seconds is usually a good starting point. +For `should not appear`, set `consumerReadTimeout` to a value that is long enough to be confident nothing will arrive, but short enough to not slow your suite down unnecessarily. A value between 5 and 10 seconds is usually a good starting point. ::: --- ## Multiple rows, same topic -Both watcher steps support multiple rows — useful for checking presence/absence of several distinct keys on the same topic in one step. Rows are validated sequentially, in order, and share one pinned `referenceTimestamp` so their seek windows don't drift. Every row must target the **same physical topic**; mixing topics throws `TopicMismatchException` before any consumer call is made: +Both watcher steps support multiple rows, which is useful for checking presence or absence of several distinct keys on the same topic in one step. Rows are validated sequentially, in order, and share one pinned `referenceTimestamp` so their seek windows don't drift. Every row must target the **same physical topic**. Mixing topics throws `TopicMismatchException` before any consumer call is made: ```gherkin And record should not appear in topic @@ -64,7 +64,7 @@ See [Multi-row DataTables](../advanced/multi-row-datatables) for the full rules. --- -## Example : filtering scenario +## Example: filtering scenario ```gherkin Scenario: Invalid orders are filtered out diff --git a/docs/write-tests/assertions/xml-matchers.mdx b/docs/write-tests/assertions/xml-matchers.mdx index 3cd101c..03816f8 100644 --- a/docs/write-tests/assertions/xml-matchers.mdx +++ b/docs/write-tests/assertions/xml-matchers.mdx @@ -10,7 +10,7 @@ XML matchers perform a **structural** XML comparison (element order is irrelevan --- -## Structural XML match : `Then expected record from file based on XML` +## Structural XML match: `Then expected record from file based on XML` Fetches a record and compares its value as XML. Attribute order, whitespace, and element order are normalised before comparison. @@ -42,7 +42,7 @@ Elements listed here are removed from both the expected and actual XML before co --- -## XPath match : `Then expected record based on XML should have fields matching from file` +## XPath match: `Then expected record based on XML should have fields matching from file` Extracts one or more values from the record using XPath expressions and compares them against the corresponding expected file content line-by-line. @@ -55,7 +55,7 @@ Then expected record based on XML should have fields matching from file | Column | Type | Required | Description | |---|---|---|---| | `topicAlias` | `string` | ✅ | Alias of a declared output topic | -| `file` | `string` | ✅ | Expected values file — one expected value per line matching the XPath order | +| `file` | `string` | ✅ | Expected values file, one expected value per line matching the XPath order | | `xpathExpressions` | `string` | ✅ | Comma-separated XPath expressions | ### Expected file format for XPath diff --git a/docs/write-tests/background/assets-directory.mdx b/docs/write-tests/background/assets-directory.mdx index 4dedbe0..9bb7a88 100644 --- a/docs/write-tests/background/assets-directory.mdx +++ b/docs/write-tests/background/assets-directory.mdx @@ -4,7 +4,7 @@ title: Assets Directory description: Declare the base directory from which KTestify resolves all payload and expected files. --- -# Assets Directory : `Given assets directory` +# Assets Directory: `Given assets directory` The **assets directory** is the base path from which all file references in DataTable columns are resolved. Declare it once in the Background and all `When` / `Then` file paths become relative to it. diff --git a/docs/write-tests/background/namespaces.mdx b/docs/write-tests/background/namespaces.mdx index 686d83f..b301f70 100644 --- a/docs/write-tests/background/namespaces.mdx +++ b/docs/write-tests/background/namespaces.mdx @@ -4,7 +4,7 @@ title: Namespaces description: Declare a topic namespace (prefix) that is transparently prepended to all topic names. --- -# Namespaces : `Given namespace` / `Given namespaces` +# Namespaces: `Given namespace` / `Given namespaces` A **namespace** is a prefix that KTestify prepends to every topic name to form the physical Kafka topic name: `namespace.topicName`. This lets you share a single Kafka cluster across environments by varying the prefix. diff --git a/docs/write-tests/background/schemas.mdx b/docs/write-tests/background/schemas.mdx index 85d72b9..3faa411 100644 --- a/docs/write-tests/background/schemas.mdx +++ b/docs/write-tests/background/schemas.mdx @@ -4,7 +4,7 @@ title: Schemas description: Declare Avro schemas for use in Avro producer . --- -# Schemas : `Given schema` +# Schemas: `Given schema` :::info[Schemas are only required for producers] Schemas are only required if you use Avro producers, for consumers values are already deserialized by Kafka, so KTestify does not need to know the schema. @@ -37,7 +37,7 @@ When you set `schemaName = Order` in a producer step, KTestify looks up the sche --- -## Example : Avro producer step +## Example: Avro producer step ```gherkin When record from file based on schema is sent @@ -61,5 +61,4 @@ ktestify { } ``` -See [Configuration →](../configuration) for full Schema Registry security options (basic auth, SSL). - +See [Configuration →](../../getting-started/configuration) for full Schema Registry security options (basic auth, SSL). diff --git a/docs/write-tests/background/topics.mdx b/docs/write-tests/background/topics.mdx index bcd0559..0f7d40a 100644 --- a/docs/write-tests/background/topics.mdx +++ b/docs/write-tests/background/topics.mdx @@ -4,7 +4,7 @@ title: Topics description: Declare input and output Kafka topics for your scenario. --- -# Topics : `Given input topic` / `Given output topic` +# Topics: `Given input topic` / `Given output topic` Topics are the entry points for producers (`input`) and consumers (`output`). You must declare a topic before referencing it in `When` or `Then` steps. diff --git a/docs/write-tests/configuration.mdx b/docs/write-tests/configuration.mdx deleted file mode 100644 index 69690ac..0000000 --- a/docs/write-tests/configuration.mdx +++ /dev/null @@ -1,17 +0,0 @@ ---- -sidebar_position: 2 -title: Configuration -description: All HOCON keys, environment variables, and defaults, see the Getting Started Configuration page. ---- - -# Configuration - -The full configuration reference has moved to Getting Started. - -➡️ **[Configuration reference →](../getting-started/configuration)** - -It covers every HOCON key and environment variable for: -- Kafka brokers, security, consumers and producers -- Schema Registry auth and SSL -- Timeouts and per-scenario overrides -- Directories, logging, and execution flags diff --git a/docs/write-tests/step-reference.mdx b/docs/write-tests/step-reference.mdx index ad15aaa..222581a 100644 --- a/docs/write-tests/step-reference.mdx +++ b/docs/write-tests/step-reference.mdx @@ -10,7 +10,7 @@ Complete catalogue of every KTestify step. All DataTable columns are listed with --- -## Background : `Given` steps +## Background: `Given` steps ### `Given namespace` @@ -24,7 +24,7 @@ Given namespace | Column | Type | Required | Default | Description | |---|---|---|---|---| -| `namespace` | `string` | ✅ | — | Namespace prefix (e.g. `myapp`, `com.example`) | +| `namespace` | `string` | ✅ | none | Namespace prefix (e.g. `myapp`, `com.example`) | --- @@ -41,8 +41,8 @@ Given namespaces | Column | Type | Required | Default | Description | |---|---|---|---|---| -| `namespace` | `string` | ✅ | — | Namespace prefix | -| `namespaceAlias` | `string` | ✅ | — | Alias referenced in topic declarations | +| `namespace` | `string` | ✅ | none | Namespace prefix | +| `namespaceAlias` | `string` | ✅ | none | Alias referenced in topic declarations | --- @@ -58,10 +58,10 @@ Given input topic | Column | Type | Required | Default | Description | |---|---|---|---|---| -| `topicName` | `string` | ✅ | — | Physical topic name (namespace prepended) | -| `topicAlias` | `string` | ✅ | — | Alias for use in `When` steps | +| `topicName` | `string` | ✅ | none | Physical topic name (namespace prepended) | +| `topicAlias` | `string` | ✅ | none | Alias for use in `When` steps | | `namespace` | `string` | ❌ | Config default | Override namespace for this topic | -| `namespaceAlias` | `string` | ❌ | — | Reference a `Given namespaces` alias | +| `namespaceAlias` | `string` | ❌ | none | Reference a `Given namespaces` alias | --- @@ -77,10 +77,10 @@ Given output topic | Column | Type | Required | Default | Description | |---|---|---|---|---| -| `topicName` | `string` | ✅ | — | Physical topic name (namespace prepended) | -| `topicAlias` | `string` | ✅ | — | Alias for use in `Then` steps | +| `topicName` | `string` | ✅ | none | Physical topic name (namespace prepended) | +| `topicAlias` | `string` | ✅ | none | Alias for use in `Then` steps | | `namespace` | `string` | ❌ | Config default | Override namespace for this topic | -| `namespaceAlias` | `string` | ❌ | — | Reference a `Given namespaces` alias | +| `namespaceAlias` | `string` | ❌ | none | Reference a `Given namespaces` alias | --- @@ -112,13 +112,13 @@ Given schema | Column | Type | Required | Default | Description | |---|---|---|---|---| -| `schemaName` | `string` | ✅ | — | Schema Registry subject name (prefix for `schemaName-value`) | -| `schemaAlias` | `string` | ✅ | — | Short alias | +| `schemaName` | `string` | ✅ | none | Schema Registry subject name (prefix for `schemaName-value`) | +| `schemaAlias` | `string` | ✅ | none | Short alias | | `schemaVersion` | `integer` | ❌ | latest | Schema version to use | --- -## Actions — `When` steps +## Actions: `When` steps :::info[Multi-row DataTables] Producer and script steps below accept multiple DataTable rows, executed sequentially in order. Producer steps additionally require every row to resolve to the **same physical topic** (`TopicMismatchException` otherwise). See [Multi-row DataTables](advanced/multi-row-datatables) for full details. @@ -136,10 +136,10 @@ When record from file is sent | Column | Type | Required | Default | Description | |---|---|---|---|---| -| `topicName` | `string` | ✅ | — | Topic name or alias (must be declared as INPUT) | -| `file` | `string` | ✅ | — | Payload file, relative to assets directory | +| `topicName` | `string` | ✅ | none | Topic name or alias (must be declared as INPUT) | +| `file` | `string` | ✅ | none | Payload file, relative to assets directory | | `recordKey` | `string` | ❌ | `null` | Kafka record key | -| `headerFile` | `string` | ❌ | — | JSON file of Kafka headers `{"name":"value"}` | +| `headerFile` | `string` | ❌ | none | JSON file of Kafka headers `{"name":"value"}` | --- @@ -155,8 +155,8 @@ When record from file based on schema is sent | Column | Type | Required | Default | Description | |---|---|---|---|---| -| `topicName` | `string` | ✅ | — | Input topic name or alias | -| `file` | `string` | ✅ | — | JSON payload file | +| `topicName` | `string` | ✅ | none | Input topic name or alias | +| `file` | `string` | ✅ | none | JSON payload file | | `schemaName` | `string` | ❌ | TopicNameStrategy | Schema alias or name declared via `Given schema` | | `recordKey` | `string` | ❌ | `null` | Kafka record key | @@ -188,14 +188,14 @@ When script is executed | Column | Type | Required | Default | Description | |---|---|---|---|---| -| `scriptPath` | `string` | ✅ | — | Path to the script file | -| `scriptArgs` | `string` | ❌ | — | Comma-separated arguments | +| `scriptPath` | `string` | ✅ | none | Path to the script file | +| `scriptArgs` | `string` | ❌ | none | Comma-separated arguments | Fails if exit code ≠ 0. --- -## Assertions : `Then` / `And` steps +## Assertions: `Then` / `And` steps :::info[Multi-row DataTables] Every single-record assertion step below (all except the two batch steps) accepts multiple DataTable rows, validated sequentially in order, **provided every row targets the same physical topic** (`TopicMismatchException` otherwise). All rows in the same step share one pinned `referenceTimestamp` so delta-time seek windows don't drift row-to-row. The batch steps (`expected records from files` / `...based on schema`) remain strictly single-row. See [Multi-row DataTables →](advanced/multi-row-datatables) for full details. @@ -213,9 +213,9 @@ Then expected record from file | Column | Type | Required | Default | Description | |---|---|---|---|---| -| `topicAlias` | `string` | ✅ | — | Output topic alias | -| `file` | `string` | ✅ | — | Expected file path | -| `expectedRecordKey` | `string` | ❌ | — | Filter by record key | +| `topicAlias` | `string` | ✅ | none | Output topic alias | +| `file` | `string` | ✅ | none | Expected file path | +| `expectedRecordKey` | `string` | ❌ | none | Filter by record key | | `consumerReadTimeout` | `integer` | ❌ | Config (30) | Seconds to wait | | `consumerDeltaTime` | `integer` | ❌ | Config (60) | Seconds to seek back | @@ -233,11 +233,11 @@ Then expected record should have fields matching from file | Column | Type | Required | Default | Description | |---|---|---|---|---| -| `topicAlias` | `string` | ✅ | — | Output topic alias | -| `file` | `string` | ✅ | — | Expected file | -| `line` | `integer` | ✅ | — | 1-based line number | -| `from` | `integer` | ✅ | — | Start char index (inclusive) | -| `to` | `integer` | ✅ | — | End char index (exclusive) | +| `topicAlias` | `string` | ✅ | none | Output topic alias | +| `file` | `string` | ✅ | none | Expected file | +| `line` | `integer` | ✅ | none | 1-based line number | +| `from` | `integer` | ✅ | none | Start char index (inclusive) | +| `to` | `integer` | ✅ | none | End char index (exclusive) | --- @@ -253,10 +253,10 @@ Then expected record from file based on XML | Column | Type | Required | Default | Description | |---|---|---|---|---| -| `topicAlias` | `string` | ✅ | — | Output topic alias | -| `file` | `string` | ✅ | — | Expected XML file | -| `excludedElements` | `string` | ❌ | — | Comma-separated element names to skip | -| `expectedRecordKey` | `string` | ❌ | — | Filter by record key | +| `topicAlias` | `string` | ✅ | none | Output topic alias | +| `file` | `string` | ✅ | none | Expected XML file | +| `excludedElements` | `string` | ❌ | none | Comma-separated element names to skip | +| `expectedRecordKey` | `string` | ❌ | none | Filter by record key | | `consumerReadTimeout` | `integer` | ❌ | Config (30) | Seconds to wait | | `consumerDeltaTime` | `integer` | ❌ | Config (60) | Seconds to seek back | @@ -274,9 +274,9 @@ Then expected record based on XML should have fields matching from file | Column | Type | Required | Default | Description | |---|---|---|---|---| -| `topicAlias` | `string` | ✅ | — | Output topic alias | -| `file` | `string` | ✅ | — | Expected values file (one per line) | -| `xpathExpressions` | `string` | ✅ | — | Comma-separated XPath expressions | +| `topicAlias` | `string` | ✅ | none | Output topic alias | +| `file` | `string` | ✅ | none | Expected values file (one per line) | +| `xpathExpressions` | `string` | ✅ | none | Comma-separated XPath expressions | --- @@ -292,10 +292,10 @@ Then expected record from file based on schema | Column | Type | Required | Default | Description | |---|---|---|---|---| -| `topicAlias` | `string` | ✅ | — | Output topic alias | -| `file` | `string` | ✅ | — | Expected JSON file | -| `excludedKeys` | `string` | ❌ | — | Comma-separated Avro field names to exclude | -| `expectedRecordKey` | `string` | ❌ | — | Filter by record key | +| `topicAlias` | `string` | ✅ | none | Output topic alias | +| `file` | `string` | ✅ | none | Expected JSON file | +| `excludedKeys` | `string` | ❌ | none | Comma-separated Avro field names to exclude | +| `expectedRecordKey` | `string` | ❌ | none | Filter by record key | | `consumerReadTimeout` | `integer` | ❌ | Config (30) | Seconds to wait | | `consumerDeltaTime` | `integer` | ❌ | Config (60) | Seconds to seek back | @@ -305,17 +305,33 @@ Then expected record from file based on schema Assert specific Avro field values without a file. +**Single field** (use `key` / `value` columns): + ```gherkin Then expected record based on schema should have fields matching from given value | topicAlias | key | value | | orders-out | status | PROCESSED | ``` +**Multiple fields** (use `keys` / `values` columns, semicolon-separated): + +```gherkin +Then expected record based on schema should have fields matching from given value + | topicAlias | keys | values | + | orders-out | status;amount | PENDING;99 | +``` + | Column | Type | Required | Default | Description | |---|---|---|---|---| -| `topicAlias` | `string` | ✅ | — | Output topic alias | -| `key` | `string` | ✅ | — | Avro field name | -| `value` | `string` | ✅ | — | Expected field value | +| `topicAlias` | `string` | ✅ | none | Output topic alias | +| `key` | `string` | ❌* | none | Avro field name (single-field mode) | +| `value` | `string` | ❌* | none | Expected field value (single-field mode) | +| `keys` | `string` | ❌* | none | Semicolon-separated Avro field names (multi-field mode) | +| `values` | `string` | ❌* | none | Semicolon-separated expected values (multi-field mode) | +| `consumerReadTimeout` | `integer` | ❌ | Config (30) | Seconds to wait | +| `consumerDeltaTime` | `integer` | ❌ | Config (60) | Seconds to seek back | + +\* Either `key` + `value` or `keys` + `values` must be provided. --- @@ -331,9 +347,9 @@ Then expected records from files | Column | Type | Required | Default | Description | |---|---|---|---|---| -| `topicAlias` | `string` | ✅ | — | Output topic alias | -| `expectedRecordsCount` | `integer` | ✅ | — | Exact number of records to collect | -| `files` | `string` | ✅ | — | Comma-separated expected file paths | +| `topicAlias` | `string` | ✅ | none | Output topic alias | +| `expectedRecordsCount` | `integer` | ✅ | none | Exact number of records to collect | +| `files` | `string` | ✅ | none | Comma-separated expected file paths | | `consumerReadTimeout` | `integer` | ❌ | Config (30) | Seconds to wait for all records | | `consumerDeltaTime` | `integer` | ❌ | Config (60) | Seconds to seek back | @@ -351,10 +367,10 @@ Then expected records from files based on schema | Column | Type | Required | Default | Description | |---|---|---|---|---| -| `topicAlias` | `string` | ✅ | — | Output topic alias | -| `expectedRecordsCount` | `integer` | ✅ | — | Exact number of records to collect | -| `files` | `string` | ✅ | — | Comma-separated expected JSON file paths | -| `excludedKeys` | `string` | ❌ | — | Avro field names to exclude from all records | +| `topicAlias` | `string` | ✅ | none | Output topic alias | +| `expectedRecordsCount` | `integer` | ✅ | none | Exact number of records to collect | +| `files` | `string` | ✅ | none | Comma-separated expected JSON file paths | +| `excludedKeys` | `string` | ❌ | none | Avro field names to exclude from all records | | `consumerReadTimeout` | `integer` | ❌ | Config (30) | Seconds to wait for all records | | `consumerDeltaTime` | `integer` | ❌ | Config (60) | Seconds to seek back | @@ -372,8 +388,8 @@ And record should appear in topic | Column | Type | Required | Default | Description | |---|---|---|---|---| -| `topicAlias` | `string` | ✅ | — | Output topic alias | -| `topicType` | `string` | ✅ | — | `raw` or `avro` | +| `topicAlias` | `string` | ✅ | none | Output topic alias | +| `topicType` | `string` | ✅ | none | `raw` or `avro` | | `consumerReadTimeout` | `integer` | ❌ | Config (30) | Seconds to wait | | `consumerDeltaTime` | `integer` | ❌ | Config (60) | Seconds to seek back | diff --git a/sidebars.ts b/sidebars.ts index e7518c7..07ec4a0 100644 --- a/sidebars.ts +++ b/sidebars.ts @@ -4,114 +4,122 @@ const sidebars: SidebarsConfig = { docsSidebar: [ 'intro', - // ── Getting Started ────────────────────────────────────────────────────── + // Tutorials (learning-oriented) { type: 'category', - label: 'Getting Started', - link: {type: 'generated-index', description: 'Everything you need to run your first KTestify test.'}, + label: 'Tutorials', + link: {type: 'generated-index', description: 'Get KTestify running and write your first test, step by step.'}, items: [ 'getting-started/installation', - 'getting-started/quick-start', 'getting-started/ci-environment', - 'getting-started/configuration', + 'getting-started/quick-start', ], }, - // ── Write Tests (QA / Test-Engineer track) ─────────────────────────────── + // How-to Guides (task-oriented) { type: 'category', - label: '‍🧪 Write Tests', - link: {type: 'doc', id: 'write-tests/overview'}, + label: 'How-to Guides', + link: {type: 'generated-index', description: 'Task recipes: write steps, use plugins, extend the framework.'}, items: [ 'write-tests/overview', - 'write-tests/configuration', { type: 'category', - label: 'Background — Given', - link: {type: 'generated-index', description: 'Declare topics, namespaces, schemas, and directories for your scenario.'}, + label: 'Write Kafka tests', items: [ - 'write-tests/background/namespaces', - 'write-tests/background/topics', - 'write-tests/background/assets-directory', - 'write-tests/background/schemas', + { + type: 'category', + label: 'Background (Given)', + items: [ + 'write-tests/background/namespaces', + 'write-tests/background/topics', + 'write-tests/background/assets-directory', + 'write-tests/background/schemas', + ], + }, + { + type: 'category', + label: 'Actions (When)', + items: [ + 'write-tests/actions/send-raw-record', + 'write-tests/actions/send-avro-record', + 'write-tests/actions/wait-and-script', + ], + }, + { + type: 'category', + label: 'Assertions (Then)', + items: [ + 'write-tests/assertions/raw-matchers', + 'write-tests/assertions/xml-matchers', + 'write-tests/assertions/avro-matchers', + 'write-tests/assertions/batch-assertions', + 'write-tests/assertions/watcher', + ], + }, + 'write-tests/dynamic-variables', + { + type: 'category', + label: 'Advanced patterns', + items: [ + 'write-tests/advanced/timeout-tuning', + 'write-tests/advanced/batch-testing', + 'write-tests/advanced/multi-row-datatables', + ], + }, ], }, { type: 'category', - label: 'Actions — When', - link: {type: 'generated-index', description: 'Produce messages, send files, trigger scripts.'}, + label: 'Use plugins', items: [ - 'write-tests/actions/send-raw-record', - 'write-tests/actions/send-avro-record', - 'write-tests/actions/wait-and-script', - ], - }, - { - type: 'category', - label: 'Assertions — Then', - link: {type: 'generated-index', description: 'Consume and assert output records.'}, - items: [ - 'write-tests/assertions/raw-matchers', - 'write-tests/assertions/xml-matchers', - 'write-tests/assertions/avro-matchers', - 'write-tests/assertions/batch-assertions', - 'write-tests/assertions/watcher', + 'extend/plugins/http', + 'extend/plugins/azureblob', + 'extend/plugins/notifications', ], }, - 'write-tests/dynamic-variables', { type: 'category', - label: 'Advanced', - link: {type: 'generated-index', description: 'Batch mode, timeout tuning, and roundtrip test patterns.'}, + label: 'Extend the framework', items: [ - 'write-tests/advanced/timeout-tuning', - 'write-tests/advanced/batch-testing', - 'write-tests/advanced/multi-row-datatables', + 'extend/transports/adding-a-transport', + 'extend/matchers/custom-matcher', + 'extend/plugins/create-plugin', ], }, + ], + }, + + // Reference (information-oriented) + { + type: 'category', + label: 'Reference', + link: {type: 'generated-index', description: 'Lookup tables: every step, config key, matcher, and plugin.'}, + items: [ 'write-tests/step-reference', + 'getting-started/configuration', + 'extend/matchers/built-in-matchers', + 'extend/plugins/index', ], }, - // ── Extend (Java-Developer track) ──────────────────────────────────────── + // Explanation (understanding-oriented) { type: 'category', - label: '🔧 Extend the Framework', - link: {type: 'doc', id: 'extend/architecture'}, + label: 'Explanation', + link: {type: 'generated-index', description: 'Why KTestify is built the way it is: architecture and core concepts.'}, items: [ 'extend/architecture', 'extend/core-concepts', + 'extend/ai-usage', { type: 'category', label: 'Transports', - link: {type: 'generated-index', description: 'How KTestify fetches records and how to add a new transport.'}, items: [ 'extend/transports/kafka', - 'extend/transports/adding-a-transport', 'extend/transports/synchronous-transports', ], }, - { - type: 'category', - label: 'Matchers', - link: {type: 'generated-index', description: 'Built-in assertion strategies and how to write your own.'}, - items: [ - 'extend/matchers/built-in-matchers', - 'extend/matchers/custom-matcher', - ], - }, - { - type: 'category', - label: 'Plugins', - link: {type: 'doc', id: 'extend/plugins/index'}, - items: [ - 'extend/plugins/index', - 'extend/plugins/plugin-system', - 'extend/plugins/create-plugin', - 'extend/plugins/azureblob', - 'extend/plugins/notifications' - ], - }, ], }, ], diff --git a/src/components/HomepageFeatures/index.tsx b/src/components/HomepageFeatures/index.tsx index d6f05e8..3bc8936 100644 --- a/src/components/HomepageFeatures/index.tsx +++ b/src/components/HomepageFeatures/index.tsx @@ -79,10 +79,28 @@ const features: Feature[] = [ tag: 'ktestify-plugin-azureblob', title: 'Azure Blob plugin', description: - 'Extend your test scenarios with Azure Blob Storage steps. Upload files, validate blob contents, and chain blob checks with Kafka assertions — all from the same feature file.', + 'Extend your test scenarios with Azure Blob Storage steps. Upload files, validate blob contents, and chain blob checks with Kafka assertions, all from the same feature file.', href: 'https://github.com/ktestify/ktestify-plugin-azureblob', badge: 'Plugin', }, + { + icon: , + tag: 'ktestify-plugin-http', + title: 'HTTP plugin', + description: + 'Add synchronous HTTP request and response steps to your scenarios. Send requests, assert on status, body, and headers, and poll an endpoint until it returns an expected status.', + href: 'https://github.com/ktestify/ktestify-plugin-http', + badge: 'Plugin', + }, + { + icon: , + tag: 'ktestify-plugin-notifications', + title: 'Notifications plugin', + description: + 'Send suite-level notifications to Teams, Slack, or a webhook when your test run finishes. Keep your team informed of regressions without leaving the pipeline.', + href: 'https://github.com/ktestify/ktestify-plugin-notifications', + badge: 'Plugin', + }, ]; // ── Component ─────────────────────────────────────────────────────────────────