Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions docs/extend/ai-usage.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
---
sidebar_position: 5
title: AI Usage
description: How artificial intelligence tools are used in the KTestify project.
---

# AI Usage in KTestify

KTestify uses artificial intelligence (AI) tools to support development. This page explains what AI is used for and the safeguards that keep quality high.

---

## What AI is used for

AI assists with three specific tasks:

| Task | What AI does |
|---|---|
| **Code suggestions** | AI suggests individual pieces of code, such as small snippets or method implementations. A developer reviews every suggestion before it is accepted. No agentic or autonomous usage is involved. |
| **Documentation** | AI helps generate user-facing documentation, including the pages you are reading now. |
| **Javadoc** | AI helps generate Javadoc comments for Java source files. |

---

## What AI is not used for

- AI does not write code on its own or make changes without human involvement.
- AI does not run tests, merge pull requests, or make decisions about the codebase.
- AI is never used in an agentic mode where it acts independently.

---

## Review process

Everything AI generates is reviewed by a developer before it becomes part of the project:

1. **Code suggestions** are read, tested, and adjusted as needed before being committed.
2. **Documentation** is checked for accuracy and clarity.
3. **Javadoc** is verified to make sure it correctly describes the behaviour of the code.

If an AI-generated contribution is inaccurate or incomplete, it is corrected or discarded. No content is accepted without human review.

---

## Summary

AI is a supporting tool in KTestify, not an autonomous contributor. It helps with code suggestions, documentation, and Javadoc, and every output is reviewed before it is used.
27 changes: 15 additions & 12 deletions docs/extend/architecture.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -62,14 +62,19 @@ Synchronous transports additionally populate a new `attributes` map on `Consumed

---

## Why the separation matters

The three layers are connected by exactly one type, `ConsumedRecord<V>`. Because the transport layer only ever emits that type and the assertion layer only ever consumes it, the two are fully decoupled. The practical payoff is threefold:

- **Swap transports without touching assertions.** A new broker is a new `RecordFetcher` (or `RequestResponseClient`) implementation. Every `RecordMatcher` keeps working because it only sees `ConsumedRecord`.
- **Reuse matchers across transports.** The same `FileRecordMatcher` asserts a Kafka `String` body and an HTTP response body, because both arrive as `ConsumedRecord<String>`.
- **Stay test-framework agnostic.** `ktestify-cucumber` (or any future adapter) drives the engine through `ConsumerContext` / `ProducerContext` and reads back `ConsumedRecord`. It never imports a broker client.

## Layer responsibilities

### Transport - `RecordFetcher<V>`

Knows: Kafka broker, partitions, offsets, deduplication.
Does NOT know: matchers, files, test frameworks.

The contract is a single interface:
The transport layer turns a concrete message broker into a stream of `ConsumedRecord<V>`. The contract itself is deliberately tiny, a single `fetch()` method plus `close()`:

```java
public interface RecordFetcher<V> extends AutoCloseable {
Expand All @@ -78,7 +83,9 @@ public interface RecordFetcher<V> extends AutoCloseable {
}
```

Swapping Kafka for IBM MQ means writing a new `IbmMqRecordFetcher<V>`, nothing else changes.
`fetch()` blocks until at least one record that passes the configured filters is available, or the read timeout expires. It returns a non-empty, unmodifiable list, or throws `FetchException`.

The only implementation today is `KafkaRecordFetcher`, which knows Kafka brokers, partitions, offsets, and the deduplication registry. Swapping Kafka for IBM MQ means writing a new `IbmMqRecordFetcher<V>` that implements the same interface. Nothing in the layers above changes.

---

Expand All @@ -105,16 +112,13 @@ Does NOT know: Kafka internals, comparison algorithms.

```java
// AbstractKafkaConsumer.call(), simplified
var fetcher = new KafkaRecordFetcher(context);
try {
try (KafkaRecordFetcher<K, V> fetcher = new KafkaRecordFetcher<>(context)) {
List<ConsumedRecord<V>> records = fetcher.fetch(); // transport
MatchContext matchCtx = buildMatchContext();
MatchResult result = matcher.match(records, matchCtx); // assertion
return result.isPassed();
} catch (FetchException e) {
throw new ConsumerException(e.getMessage(), e);
} finally {
fetcher.close();
throw new ConsumerException(e.getMessage());
}
```

Expand Down Expand Up @@ -163,8 +167,7 @@ ktestify-core ktestify-cucumber
RecordFetcher<V> BackgroundStepDefinition
RequestResponseClient<Req,V> ValidationStepDefinition
KafkaRecordFetcher ◄──────── ConsumerContext (config only)
AbstractKafkaConsumer ConsumerValidationService
AbstractSynchronousConsumer
AbstractKafkaConsumer ConsumerValidationService.** It uses `ConsumerContext` / `ProducerContext` (ktestify-core abstractions) to configure the engine and receives only `ConsumedRecord<V>` back. This is what keeps the Cucumber layer free of broker specifics and lets the same step definitions drive Kafka, HTTP, or any future transport without modification
PollingRequestResponseClient
RecordMatcher<V>
MatchContext / MatchResult
Expand Down
63 changes: 41 additions & 22 deletions docs/extend/core-concepts.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ description: Key domain types in ktestify-core, ConsumedRecord, MatchContext, Ma

## `ConsumedRecord<V>`

The **only** data type that crosses layer boundaries. It is the universal output of the transport layer (`RecordFetcher<V>` for asynchronous transports, `RequestResponseClient<Req, V>` for synchronous ones since `1.1.1`) and the universal input of the assertion layer.
The **only** data type that crosses layer boundaries. It is the universal output of the transport layer (`RecordFetcher<V>` for asynchronous transports, `RequestResponseClient<Req, V>` for synchronous ones since `1.1.1`) and the universal input of the assertion layer. The class itself has existed since `0.3.0`; the `attributes` map was added in `1.1.1`.

```java
@Value
Expand All @@ -20,7 +20,7 @@ public class ConsumedRecord<V> {
V value; // String for raw, GenericRecord for Avro
Instant timestamp;
Map<String, String> headers; // protocol headers (Kafka headers, HTTP response headers, ...)
Map<String, String> attributes; // NEW in 1.1.1, transport metadata, never null, defaults to emptyMap()
Map<String, String> attributes; // since 1.1.1, transport metadata, never null, defaults to emptyMap()

static <V> ConsumedRecord<V> fromKafkaRecord(ConsumerRecord<String, V> record) { ... }
MatchedRecord toMatchedRecord() { ... }
Expand All @@ -29,15 +29,17 @@ public class ConsumedRecord<V> {

`attributes` holds structured transport metadata that does not belong under `headers`, for example an HTTP status code and elapsed time, a future gRPC status code, or an MQ reason code. Kafka and Azure Blob leave it empty. A synchronous transport plugin populates it, and `AttributeRecordMatcher` (see [Built-in matchers →](matchers/built-in-matchers)) asserts against it.

`ConsumedRecord` ships both a full constructor (accepting `attributes`) and a backward-compatible overload without it (defaults to `Collections.emptyMap()`), plus a `@Builder`, so existing transports keep compiling unchanged.
`ConsumedRecord` ships both a full constructor (accepting `attributes`) and a backward-compatible overload without it (defaults to `Collections.emptyMap()`), plus a `@Builder`, so existing transports keep compiling unchanged. The `attributes` field is documented as never null and defaults to an empty map, so matchers can safely call `isEmpty()` rather than checking for null.

---

## `MatchedRecord`

Deduplication token, represents a record that has already been claimed by a consumer step. It holds `topic + partition + offset + key + timestamp` and is stored in the static deduplication registry.
Deduplication token, represents a record that has already been claimed by a consumer step. It holds `topic + partition + offset + key + timestamp` and is stored in the static deduplication registry maintained by `KafkaRecordFetcher`.

`MatchedRecord` deliberately **excludes `processedTime`** from `equals`/`hashCode` so that two records from the same Kafka partition+offset are always considered the same, regardless of when they were processed.
`MatchedRecord` deliberately **excludes `processedTime`** from `equals`/`hashCode` so that two records from the same Kafka partition+offset are always considered the same, regardless of when they were processed. The `processedTime` field is annotated with Lombok `@With`, meaning it is a mutable-on-copy timestamp used for reporting, not for identity.

The conversion is one way and cheap: `ConsumedRecord.toMatchedRecord()` builds the token, and `KafkaRecordFetcher` calls `MATCHED_RECORDS.contains(record.toMatchedRecord())` before claiming a record and `MATCHED_RECORDS.add(...)` after. The registry is a `ConcurrentHashMap.newKeySet()` shared across all fetcher instances in the JVM, and `KafkaRecordFetcher.clearMatchedRecords()` wipes it. In the Cucumber flow that clear happens in the `@Before` hook at the start of every scenario, which is why two steps in the same scenario never double-claim a record. If you use `KafkaRecordFetcher` directly outside Cucumber, you must call `clearMatchedRecords()` yourself before each test.

---

Expand All @@ -54,17 +56,17 @@ public class MatchContext {
boolean strictMatching;
String matchKey;
String matchValue;
Map<String,String> expectedAttributes; // NEW in 1.1.1, defaults to emptyMap(), used by AttributeRecordMatcher
Map<String,String> expectedAttributes; // since 1.1.1, defaults to emptyMap(), used by AttributeRecordMatcher

// Convenience, for single-record matchers
public String getMatchFilePath() {
return matchFilePaths != null && !matchFilePaths.isEmpty()
? matchFilePaths.get(0) : null;
? matchFilePaths.get(0): null;
}
}
```

`expectedAttributes` holds the key/value pairs to assert against a record's `attributes` map, for example `{"statusCode": "200"}`. It follows the same convention as `excludedFields`, always non-null, defaults to an empty map, and matchers check `isEmpty()` rather than `null`.
`expectedAttributes` holds the key/value pairs to assert against a record's `attributes` map, for example `{"statusCode": "200"}`. It follows the same convention as `excludedFields`: both default to an empty collection rather than null, and matchers check `isEmpty()` instead of null. The default `buildMatchContext()` in `AbstractKafkaConsumer` sets `strictMatching` to `false` and leaves `matchKey`, `matchValue`, and `expectedAttributes` unset.

---

Expand All @@ -81,7 +83,9 @@ public class MatchResult {
String actual;

static MatchResult pass() { ... }
static MatchResult pass(String expected, String actual) { ... }
static MatchResult fail(String diff, String expected, String actual) { ... }
static MatchResult fail(String message) { ... }
}
```

Expand All @@ -95,11 +99,11 @@ Key fields:

| Field | Type | Description |
|---|---|---|
| `topic` | `Topic` | The output topic (must be OUTPUT type — validated in builder) |
| `matchMethod` | `String` | One of the `ConfigConstants.method*` values |
| `topic` | `Topic` | The output topic, must be OUTPUT type, validated in the builder |
| `matchMethod` | `String` | One of the `RecordMatcherFactory.METHOD_*` constants |
| `matchFilePaths` | `List<String>` | Expected file paths (single or batch) |
| `excludedFields` | `List<String>` | Field names to ignore in comparison |
| `expectedRecordKey` | `String` | Key filter — record must match this key |
| `expectedRecordKey` | `String` | Key filter, the record must match this key |
| `readTimeout` | `long` | Milliseconds |
| `consumerDeltaTime` | `long` | Milliseconds (DataTable seconds × 1000) |
| `isBatchConsumer` | `boolean` | Enables batch fetch mode |
Expand All @@ -112,10 +116,10 @@ Key fields:
```java
@Data @Builder
public class Topic {
String topicName;
String topicAlias;
String topicNamespace;
Topic.Type topicType; // INPUT or OUTPUT
String topicName;
String topicAlias;
TopicNamespace topicNamespace; // nested, holds namespace + namespaceAlias
Topic.Type topicType; // INPUT or OUTPUT

// Returns "namespace.topicName" or just "topicName" if no namespace
String getNamespacedTopic() { ... }
Expand All @@ -124,19 +128,34 @@ public class Topic {

---

## `RecordMatcher<V>`

The assertion contract. A matcher receives the records fetched by a `RecordFetcher` (or returned by a `RequestResponseClient`) and asserts them against the expected state carried by a `MatchContext`.

```java
@FunctionalInterface
public interface RecordMatcher<V> {
MatchResult match(List<ConsumedRecord<V>> records, MatchContext context);
}
```

Implementations have zero dependency on Kafka, HTTP, or any transport. They only know about `ConsumedRecord`, which is exactly why the same matcher works for every transport. Concrete implementations live in `io.github.ktestify.match.impl`: `NoOpRecordMatcher` (always passes, for consume-only scenarios), `FileRecordMatcher`, `XmlRecordMatcher`, `XPathRecordMatcher`, `FieldsRecordMatcher`, `FileKeyRecordMatcher`, `KeyRecordMatcher`, `AttributeRecordMatcher`, and their `Avro*` counterparts.

---

## `RecordMatcherFactory`

Pure static factory, no DI, no singleton. Resolves the right `RecordMatcher` implementation based on `matchMethod` and whether the consumer is raw or Avro.
Pure static factory, no DI, no singleton. Resolves the right `RecordMatcher` implementation based on `matchMethod` and whether the consumer is raw (`String`) or Avro (`GenericRecord`). Two typed entry points, `forRaw(String)` and `forAvro(String)`, let the compiler enforce the value type.

```java
RecordMatcherFactory.forRaw("matchFile") → FileRecordMatcher
RecordMatcherFactory.forAvro("matchFile") → AvroFileRecordMatcher
RecordMatcherFactory.forRaw("matchXML") → XmlRecordMatcher
RecordMatcherFactory.forAvro("matchXML") → throws ConsumerException ← not supported
RecordMatcherFactory.forRaw("methodMatchAttributes") → AttributeRecordMatcher<>() // NEW in 1.1.1, raw only
RecordMatcherFactory.forRaw("methodMatchFile") → FileRecordMatcher
RecordMatcherFactory.forAvro("methodMatchFile") → AvroFileRecordMatcher
RecordMatcherFactory.forRaw("methodMatchXML") → XmlRecordMatcher
RecordMatcherFactory.forAvro("methodMatchXML") → throws ConsumerException (not supported for Avro)
RecordMatcherFactory.forRaw("methodMatchAttributes") → AttributeRecordMatcher<>() // since 1.1.1, raw only
```

See [Built-in matchers →](matchers/built-in-matchers) for the full mapping table.
When `matchMethod` is `null` or blank, both methods return a `NoOpRecordMatcher`, which makes consume-only scenarios a first-class use case. The Avro path supports only `methodMatchFile`, `methodMatchKeyValue`, `methodFieldsToMatch`, and `methodRecordKeyMatch`. XML, XPath, and attribute matching are raw-only, so requesting them for an Avro topic throws `ConsumerException`. See [Built-in matchers →](matchers/built-in-matchers) for the full mapping table.

---

Expand Down
Loading
Loading